Jekyll2021-01-28T14:07:19+00:00https://build3r.github.io/fastai//fastai/feed.xmlShabaz blog for Fast.ai CourseMy notes fast ai course, I started with v3 but have shifted to v4MNIST number classification from Scratch2021-01-28T00:00:00+00:002021-01-28T00:00:00+00:00https://build3r.github.io/fastai//fastai/2021/01/28/MNIST<p>These notes are from Course-v4 of fastai and it differs a lot from the course-v3. In this course the have published a book call fastBook (available for free as Jupyter Notebook).</p> <p>In this chapter we try to first diffrentiate nbetween 3 and 7. By cvarious techniques</p> <ol> <li>Using pixel similarity (gives around 85% accuracy)</li> <li>Train a Linear Model from scratch (95% acc) <ol> <li>Optimisize this using inbuilt fastai and Pytorch classes and fns</li> </ol> </li> <li>Create Simple neural (non-liner = ReLU) net with 3 layers (97% acc)</li> <li>Use cnn_learner along resnet18 as base model (9% acc)</li> </ol> <h3 id="opening-and-viewing-a-image-as-tensor">Opening and viewing a image as tensor</h3> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>im3_path = threes im3 = Image.open(im3_path) tesnor(img) #creates img to array of pixels df = pd.DataFrame(im3_t[4:15,4:22]) df.style.set_properties(**{'font-size':'6pt'}).background_gradient('Greys') #plots image as table of greys </code></pre></div></div> <ul> <li>Learn to invisiona a simple base model which you think will perform reasonably well and there compare you models model with it.</li> <li>Other was is to search around for similiar problems solved by people and using those solutions with our dataset.</li> </ul> <p>A list comprehension looks like this:<br /> <code class="language-plaintext highlighter-rouge">new_list = [f(o) for o in a_list if o&gt;0]</code><br /> This will return every element of <code class="language-plaintext highlighter-rouge">a_list</code> that is greater than 0, after passing it to the function f.</p> <p><code class="language-plaintext highlighter-rouge">stack</code> : stacks up individual tensors in a collection into a single tensor.</p> <p><strong>rank</strong> = the number of axes or dimensions of a tensor<br /> <strong>shape</strong> = the size of each axes of a tensor</p> <p>PyTorch already provides mean absolute differenc(L1 norm) root mean squared error (RMSE) (L2Norm) as loss functions. You’ll find these inside torch.nn.functional, which the PyTorch team recommends importing as F (and is available by default under that name in fastai): </p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>F.l1_loss(a_3.float(),mean7) F.mse_loss(a_3,mean7).sqrt() </code></pre></div></div> <p>NumPy is the most widely used library for scientific and numeric programming in Python. It provides very similar functionality and a very similar API to that provided by PyTorch; however, it does <strong>not support</strong> using the <strong>GPU or calculating gradients</strong>, which are both critical for deep learning.<br /> This can be jagged = can be arrays of arrays, with the innermost arrays potentially being different sizes</p> <p>Pytorch Tensor restriction is that a tensor cannot use just any old type—it has to use a single basic numeric type for all components. For example, a PyTorch tensor <strong>cannot be jagged</strong>.</p> <h3 id="crash-course-on-using-tensors">Crash Course on Using Tensors</h3> <ol> <li> <p>Create a Tesnor: pass a list (or list of lists, or list of lists of lists, etc.)</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>data = [[1,2,3],[4,5,6]] arr = array (data) #numpy tns = tensor(data) #pytorch tensor </code></pre></div> </div> </li> <li> <p>Access:Tensors are 0 indexed<br /> Use indexes<br /> tns is [4,5,6] #access row<br /> tns[:,1] is [2,5] #acces colum<br /> tns[1,1:3] is [5,6] #slice access 3 is excluded</p> </li> <li> <p>You can use the standard operators such as +, -, *, /:<br /> tns+1 = [[2,3,4],[5,6,7]]</p> </li> <li> <p>Tensors have a type: and wil automatically change type ex multiplies with float<br /> tns.type() = ‘torch.LongTensor’</p> </li> </ol> <p><strong>Broadcasting</strong>. Pytorch will expand the tensor with the smaller rank to have the same size as the one with the larger rank. Broadcasting is an important capability that makes tensor code much easier to write.</p> <h3 id="stochastic-gradient-descent-sgd">Stochastic Gradient Descent (SGD)</h3> <p>To minise a function we have to towards negative slope = negative gradient.<br /> In other words, the gradients will tell us how much we have to change each weight to make our model better.<br /> To plot a fn use (fast_ai)<br /> <code class="language-plaintext highlighter-rouge">plot_function(fn, x_axis, ya_xis)</code><br /> to mark a point<br /> <code class="language-plaintext highlighter-rouge">plt.scatter(-1.5, f(-1.5), color='black');</code></p> <p><strong>Calculating gradient aka derivative value</strong><br /> <code class="language-plaintext highlighter-rouge">xt = tensor(3.).requires_grad_()</code><br /> ‘_’ at end of fn indicates inplace changes ( tell PyTorch that we want to calculate gradients with respect to that variable at that value. It is essentially tagging the variable, so PyTorch will remember to keep track of how to compute gradients of the other, direct calculations on it that you will ask for.)</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def f(x): return x**2 xt = tensor(3.).requires_grad_() yt = f(xt) yt # = tensor(9., grad_fn=&lt;PowBackward0&gt;) yt.backward() #backward propagation = calculate_gradient xt.grad # = tensor(6.) </code></pre></div></div> <p>Tthe derivative of <code class="language-plaintext highlighter-rouge">x**2</code> is <code class="language-plaintext highlighter-rouge">2*x</code>, and we have <code class="language-plaintext highlighter-rouge">x=3</code>, so the gradients should be <code class="language-plaintext highlighter-rouge">2*3=6</code>, which is what PyTorch calculated for us!</p> <p>We don’t directly use the derivative adjust our weigts instead we need move slowly. The rate of moving is given by <strong>learning rate</strong>. generally it’s between 0.001 and 0.1 . Picking it is more on art than math.<br /> The weight adjustment is given by</p> <blockquote> <p>w -= gradient(w) * lr</p> </blockquote> <p><img src="https://build3r.github.io/fastai//images/a757c7226737448f88df2b84fadb4139.png" alt="small_learning_rate" /><br /> Picking a low learning rate is better because if you pick too high the the loss is get worse<br /> <img src="https://build3r.github.io/fastai//images/ead0ffd6cc9b4fe3a647c725bdea34cd.png" alt="large__learning_rate" /></p> <p>params.grad.data will get the grad but wont calculate it</p> <h3 id="example-to-calculate-gradient-descent">Example to calculate Gradient descent</h3> <p>A example os a role coster going <strong>over a hill</strong></p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>time = torch.arange(0,20).float() speed = torch.randn(20)*3 + 0.75*(time-9.5)**2 + 1 #some radom quadratic function plt.scatter(time,speed); </code></pre></div></div> <p><img src="https://build3r.github.io/fastai//images/a0b8231d16a44efe8a2e1e378228697e.png" alt="5cf6e8f18638d903f8e1543c751675aa.png" /></p> <p>Steps of a gradient descent:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def f(t, params): a,b,c = params return a*(t**2) + (b*t) + c #some quadratic fn our guess params = torch.randn(3).requires_grad_() #random weigths preds = f(time, params) </code></pre></div></div> <p>Initial : loss = 152.6150, random weights (got lucky)<br /> <img src="https://build3r.github.io/fastai//images/5de3380c54364280bd55c1ef8470cd9a.png" alt="initial prediction" /></p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>loss.backward() lr = 1e-5 params.data -= lr * params.grad.data params.grad = None preds = f(time,params) mse(preds, speed) </code></pre></div></div> <p>1st iteration: loss = 152.3409<br /> <img src="https://build3r.github.io/fastai//images/4bfce942a156430da9dea45466b560af.png" alt="after 1st iteration.png" /></p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> params = torch.randn(3).requires_grad_() #random weigths lr = 1e-5 def apply_step(params, prn=True): preds = f(time, params) loss = mse(preds, speed) loss.backward() params.data -= lr * params.grad.data params.grad = None if prn: print(loss.item()) return preds </code></pre></div></div> <p>for i in range(100): apply_step(params)</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>After apply step for 100 iteration: loss =125.3798828125 ![aafc7974bd070abd287fb723ee1447d1.png](https://build3r.github.io/fastai//images/42a3098589994bb499bacd81204940d3.png) On every stem the guessed function changes as we are updating the params ### Summarizing Gradient descent: ![136f40a920e639139ada0cb589bb163d.png](https://build3r.github.io/fastai//images/2cb9308593f74007854e375869fd3d6b.png) ## MNIST Example 1. Create training data set python train_x = torch.cat([stacked_threes, stacked_sevens]).view(-1, 28*28) #converting matrix to list #train_x.shape = torch.Size([12396, 784]) = 6131+6265, 28*28 train_y = tensor(*len(threes) + *len(sevens)).unsqueeze(1) #train_x.shape,train_y.shape = (torch.Size([12396, 784]), torch.Size([12396, 1])) dset = list(zip(train_x,train_y)) x,y = dset #x.shape,y =. (torch.Size(), tensor())  view(-1,28*28) = change a list of matrices (a rank-3 tensor) to a list of vectors (a rank-2 tensor). -1 is a special parameter to view that means "make this axis as big as necessary to fit all the data" unsqueeze = Returns a new tensor with a dimension of size one inserted at the specified position. 2. Create Training Data set python valid_x = torch.cat([valid_3_tens, valid_7_tens]).view(-1, 28*28) valid_y = tensor(*len(valid_3_tens) + *len(valid_7_tens)).unsqueeze(1) valid_dset = list(zip(valid_x,valid_y))  3. Initialize weights  def init_params(size, std=1.0): return (torch.randn(size)*std).requires_grad_() #random weights withe gradient required weights = init_params((28*28,1)) #data of 28*28, 1  Note: **Bias**: The function weights*pixels won't be flexible enough—it is always equal to 0 when the pixels are equal to 0 (i.e., its intercept is 0). You might remember from high school math that the formula for a line is y=w*x+b; we still need the b. We'll initialize it to a random number too:  bias = init_params(1) #test one data (train_x*weights.T).sum() + bias #.T = transpose is because we are doing normal multiplicatin in testing not matrix mul # = tensor([-6.2330], grad_fn=&lt;AddBackward0&gt;)  4. Do Prediction ### @ = matrix multiply python def linear1(xb): return xb@weights + bias preds = linear1(train_x) """ preds = tensor([[ -6.2330], [-10.6388], [-20.8865], [-15.9176], [ -1.6866], [-11.3568]], grad_fn=&lt;AddBackward0&gt;)"""  ### Why we don't use accuracy as loss function TLDR: small change in x ont change the prediction leading to a 0 gradient We have a significant technical problem here. The gradient of a function is its slope, or its steepness, which can be defined as rise over run—that is, how much the value of the function goes up or down, divided by how much we changed the input. We can write this in mathematically as: (y\_new - y\_old) / (x\_new - x\_old). This gives us a good approximation of the gradient when x\_new is very similar to x\_old, meaning that their difference is very small. But accuracy only changes at all when a prediction changes from a 3 to a 7, or vice versa. The problem is that a small change in weights from x\_old to x\_new isn't likely to cause any prediction to change, so (y\_new - y\_old) will almost always be 0. In other words, the gradient is 0 almost everywhere. &gt; S: In mathematical terms, accuracy is a function that is constant almost everywhere (except at the threshold, 0.5), so its derivative is nil almost everywhere (and infinity at the threshold). This then gives gradients that are 0 or infinite, which are useless for updating the model. torch.where(a,b,c) == [b[i] if a[i] else c[i] for i in range(len(a))] ### Sigmoid fn mnist_loss as currently defined such that it assumes that predictions are always between 0 and 1. We need to ensure, then, that this is actually the case! sigmoid(x) = 1/(1+e^-x) </code></pre></div></div> <p>def sigmoid(x): return 1/(1+torch.exp(-x))</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Metric = what we care about Loss function = Similar to metric but behaves properly with gradients 5. Defines loss function python def mnist_loss(predictions, targets): predictions = predictions.sigmoid() return torch.where(targets==1, 1-predictions, predictions).mean() </code></pre></div></div> <h3 id="mini-batches">Mini Batches</h3> <p>We can do optimzation for single data set or the whole data set and take average in every step but it for whole dataset it take lot of time and for single it wouldn’t give much info.</p> <p>So we compromise between two can calculate avg loss for few data items at a time. Which is called mini batch. Large the batch size more accurate the predictions but longer it will take to process. This also helps in parallelizingthe work in gpu.</p> <p>We get better generalization if we can vary things during training. One simple and effective thing we can vary is what data items we put in each mini-batch. Rather than simply enumerating our dataset in order for every epoch, instead what we normally do is randomly shuffle it on every epoch, before we create mini-batches. PyTorch and fastai provide a class that will do the shuffling and mini-batch collation for you, called DataLoader.</p> <p>When we pass a Dataset to a DataLoader we will get back many batches which are themselves tuples of tensors representing batches of independent and dependent variables</p> <ol> <li>Gradient</li> </ol> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> for x,y in dl: pred = model(x) loss = loss_func(pred, y) loss.backward() #auto gradient calculation parameters -= parameters.grad * lr </code></pre></div></div> <ol> <li>Step<br /> Puttling it all together <code class="language-plaintext highlighter-rouge">weights = init_params((28*28,1)) bias = init_params(1) #create traning data loader dl = DataLoader(dset, batch_size=256) xb,yb = first(dl) #xb.shape,yb.shape = (torch.Size($256, 784$), torch.Size($256, 1$)) #create validation dataloader valid\_dl = DataLoader(valid\_dset, batch_size=256)</code><br /> def calc_grad(xb, yb, model):<br /> preds = model(xb)<br /> loss = mnist_loss(preds, yb)<br /> loss.backward()<br /> calc_grad(batch, train_y[:4], linear1)<br /> #weights.grad.mean(),bias.grad = (tensor(-0.0415), tensor([-0.2826]))<br /> weights.grad.zero_()<br /> bias.grad.zero_();</li> </ol> <p>loss.backward actually adds the gradients of loss to any gradients that are currently stored. So, we have to set the current gradients to 0 first.<br /> Train:<br /> <code class="language-plaintext highlighter-rouge">.data</code>: we have to tell PyTorch not to take the gradient of this step too—otherwise things will get very confusing when we try to compute the derivative at the next batch! If we assign to the data attribute of a tensor then PyTorch will not take the gradient of that step.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def train_epoch(model, lr, params): for xb,yb in dl: calc_grad(xb, yb, model) for p in params: p.data -= p.grad*lr p.grad.zero_() </code></pre></div></div> <p>Batch Accuracy</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def batch_accuracy(xb, yb): preds = xb.sigmoid() correct = (preds&gt;0.5) == yb return correct.float().mean() </code></pre></div></div> <p>Validate Epoch</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">validate_epoch</span><span class="p">(</span><span class="n">model</span><span class="p">):</span> <span class="n">accs</span> <span class="o">=</span> <span class="p">[</span><span class="n">batch_accuracy</span><span class="p">(</span><span class="n">model</span><span class="p">(</span><span class="n">xb</span><span class="p">),</span> <span class="n">yb</span><span class="p">)</span> <span class="k">for</span> <span class="n">xb</span><span class="p">,</span><span class="n">yb</span> <span class="ow">in</span> <span class="n">valid_dl</span><span class="p">]</span> <span class="k">return</span> <span class="nb">round</span><span class="p">(</span><span class="n">torch</span><span class="p">.</span><span class="n">stack</span><span class="p">(</span><span class="n">accs</span><span class="p">).</span><span class="n">mean</span><span class="p">().</span><span class="n">item</span><span class="p">(),</span> <span class="mi">4</span><span class="p">)</span> <span class="sb"></span><span class="n">python</span> <span class="n">validate_epoch</span><span class="p">(</span><span class="n">linear1</span><span class="p">)</span> <span class="c1"># 0.7175 </span><span class="n">lr</span> <span class="o">=</span> <span class="mf">1.</span> <span class="n">params</span> <span class="o">=</span> <span class="n">weights</span><span class="p">,</span><span class="n">bias</span> <span class="n">train_epoch</span><span class="p">(</span><span class="n">linear1</span><span class="p">,</span> <span class="n">lr</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span> <span class="n">validate_epoch</span><span class="p">(</span><span class="n">linear1</span><span class="p">)</span> <span class="c1">#0.7313 </span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">20</span><span class="p">):</span> <span class="n">train_epoch</span><span class="p">(</span><span class="n">linear1</span><span class="p">,</span> <span class="n">lr</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span> <span class="k">print</span><span class="p">(</span><span class="n">validate_epoch</span><span class="p">(</span><span class="n">linear1</span><span class="p">),</span> <span class="n">end</span><span class="o">=</span><span class="s">' '</span><span class="p">)</span> <span class="c1">#0.891 0.9354 0.9491 0.9569 0.9608 0.9627 0.9637 0.9651 0.9661 0.9661 0.9671 0.9681 0.9695 0.9705 0.971 0.972 0.9735 0.9735 0.9735 0.9744 </span></code></pre></div></div> <h3 id="creating-optimizer">Creating Optimizer</h3> <p>We now do what we did manually above with Pytorch inbuilt functionality the function <code class="language-plaintext highlighter-rouge">linear1 = x@w + b</code> is give by pytorch as nn.Linear</p> <p><code class="language-plaintext highlighter-rouge">nn.Linear</code> does the same thing as our <code class="language-plaintext highlighter-rouge">init_params</code> and <code class="language-plaintext highlighter-rouge">linear</code> together. It contains both the weights and biases in a single class.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>linear_model = nn.Linear(28*28,1) </code></pre></div></div> <p>Data setup</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import fastbook fastbook.setup_book() from fastai.vision.all import * from fastbook import * matplotlib.rc('image', cmap='Greys') lr = 1e-5 path =Path( "../data/mnist_sample")#untar_data(URLs.MNIST_SAMPLE) Path.BASE_PATH = path threes = (path/'train'/'3').ls().sorted() sevens = (path/'train'/'7').ls().sorted() seven_tensors = [tensor(Image.open(o)) for o in sevens] three_tensors = [tensor(Image.open(o)) for o in threes] stacked_sevens = torch.stack(seven_tensors).float()/255 stacked_threes = torch.stack(three_tensors).float()/255 #validation Set valid_3_tens = torch.stack([tensor(Image.open(o)) for o in (path/'valid'/'3').ls()]) valid_3_tens = valid_3_tens.float()/255 valid_7_tens = torch.stack([tensor(Image.open(o)) for o in (path/'valid'/'7').ls()]) valid_7_tens = valid_7_tens.float()/255 #create training data train_x = torch.cat([stacked_threes, stacked_sevens]).view(-1, 28*28) #converting matric to list train_y = tensor(*len(threes) + *len(sevens)).unsqueeze(1) #We need a label for each image. Well use 1 for 3s and 0 for 7s dset = list(zip(train_x,train_y)) dl = DataLoader(dset, batch_size=256) #create validation data set valid_x = torch.cat([valid_3_tens, valid_7_tens]).view(-1, 28*28) valid_y = tensor(*len(valid_3_tens) + *len(valid_7_tens)).unsqueeze(1) valid_dset = list(zip(valid_x,valid_y)) valid_dl = DataLoader(valid_dset, batch_size=256) #Loss function with sigmoid def mnist_loss(predictions, targets): predictions = predictions.sigmoid() return torch.where(targets==1, 1-predictions, predictions).mean() #gradient calculation def calc_grad(xb, yb, model): preds = model(xb) loss = mnist_loss(preds, yb) loss.backward() #calculate batch accuracy def batch_accuracy(xb, yb): preds = xb.sigmoid() correct = (preds&gt;0.5) == yb return correct.float().mean() #validating one epoch def validate_epoch(model): accs = [batch_accuracy(model(xb), yb) for xb,yb in valid_dl] return round(torch.stack(accs).mean().item(), 4) </code></pre></div></div> <p>Basic Optimizer</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class BasicOptim: def __init__(self,params,lr): self.params,self.lr = list(params),lr def step(self, *args, **kwargs): for p in self.params: p.data -= p.grad.data * self.lr def zero_grad(self, *args, **kwargs): for p in self.params: p.grad = None lr = 1e-5 opt = BasicOptim(linear_model.parameters(), lr) </code></pre></div></div> <p>Train Epoch and model</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def train_epoch(model): for xb,yb in dl: calc_grad(xb, yb, model) opt.step() opt.zero_grad() def train_model(model, epochs): for i in range(epochs): train_epoch(model) print(validate_epoch(model), end=' ') train_model(linear_model, 20) </code></pre></div></div> <p>fastai provides the <code class="language-plaintext highlighter-rouge">SGD</code> class which, by default, does the same thing as our <code class="language-plaintext highlighter-rouge">BasicOptim</code>:</p> <p>With inbuilt optimizer SGD</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">linear_model</span> <span class="o">=</span> <span class="n">nn</span><span class="p">.</span><span class="n">Linear</span><span class="p">(</span><span class="mi">28</span><span class="o">*</span><span class="mi">28</span><span class="p">,</span><span class="mi">1</span><span class="p">)</span> <span class="n">opt</span> <span class="o">=</span> <span class="n">SGD</span><span class="p">(</span><span class="n">linear_model</span><span class="p">.</span><span class="n">parameters</span><span class="p">(),</span> <span class="n">lr</span><span class="p">)</span> <span class="n">train_model</span><span class="p">(</span><span class="n">linear_model</span><span class="p">,</span> <span class="mi">20</span><span class="p">)</span> </code></pre></div></div> <h4 id="using-learner">Using Learner:</h4> <p>fastai also provides <code class="language-plaintext highlighter-rouge">Learner.fit</code>, which we can use instead of <code class="language-plaintext highlighter-rouge">train_model</code>. To create a Learner we first need to create a DataLoaders, by passing in our training and validation DataLoaders:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>dls = DataLoaders(dl, valid_dl) learn = Learner(dls, nn.Linear(28*28,1), opt_func=SGD, loss_func=mnist_loss, metrics=batch_accuracy) learn.fit(10, lr=lr) </code></pre></div></div> <h3 id="adding-a-nonlinearity">Adding a Nonlinearity</h3> <p>So we made a simple linear classifier. A linear classifier is very constrained in terms of what it can do.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def simple_net(xb): res = xb@w1 + b1 res = res.max(tensor(0.0)) #ReLU res = res@w2 + b2 return res w1 = init_params((28*28,30)) b1 = init_params(30) w2 = init_params((30,1)) b2 = init_params(1) </code></pre></div></div> <p>w1 and w2 are weight tensors, and b1 and b2 are bias tensors;</p> <p>plot_function(F.relu)</p> <p><img src="https://build3r.github.io/fastai//images/3680e4826c064d2b9b93e11ff1a3e4a5.png" alt="ReLU" /></p> <p>Using Inbuilt fns</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>simple_net = nn.Sequential( nn.Linear(28*28,30), nn.ReLU(), nn.Linear(30,1) ) #weights and biases are auto initialised </code></pre></div></div> <p>Using Simple net</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>learn = Learner(dls, simple_net, opt_func=SGD, loss_func=mnist_loss, metrics=batch_accuracy) learn.fit(40, 0.1) </code></pre></div></div> <p>|epoch | train_loss | valid_loss | batch_accuracy | time |—|—|—|—|–| |0| 0.313009| 0.408768| 0.507360| 00:00 |1| 0.146016| 0.232771| 0.798822| 00:00</p> <p>Final Accuracy = <strong>0.982826292514801 after 40 epochs</strong></p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>learn.recorder.values [(#3) [0.31300875544548035,0.4087675213813782,0.5073601603507996], (#3) [0.14601589739322662,0.23277133703231812,0.7988224029541016], </code></pre></div></div> <p>the training process is recorded in learn.recorder, with the table of output stored in the values attributes</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>plt.plot(L(learn.recorder.values).itemgot(2)); #get batch accuracy </code></pre></div></div> <p>Visualize the weights</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>w,b = learn.model.parameters() show_image(w.view(28,28)) </code></pre></div></div> <p><img src="https://build3r.github.io/fastai//images/16e3997547a6430bb022c1aeebf2f287.png" alt="weight 1 of 30" /></p> <h2 id="using-full-fastai-tool-kit">Using full FastAi tool kit</h2> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>dls = ImageDataLoaders.from_folder(path) learn = cnn_learner(dls, resnet18, pretrained=False, loss_func=F.cross_entropy, metrics=accuracy) learn.fit_one_cycle(1, 0.1) ---------------------------------------------------------- epoch train_loss valid_loss accuracy time 0 0.182440 0.030915 0.996075 00:06 </code></pre></div></div> <p><strong>In Just 1 epoch we got accuracy of 0.996075</strong> with power of inbuit learners and transfer learning from resnet 18</p> <h3 id="deep-learning-vocabulary">Deep learning vocabulary</h3> <table> <thead> <tr> <th>Term</th> <th>Meaning</th> </tr> </thead> <tbody> <tr> <td>ReLU</td> <td>Function that returns 0 for negative numbers and doesn’t change positive numbers.</td> </tr> <tr> <td>Mini-batch</td> <td>A small group of inputs and labels gathered together in two arrays. A gradient descent step is updated on this batch (rather than a whole epoch).</td> </tr> <tr> <td>Forward pass</td> <td>Applying the model to some input and computing the predictions.</td> </tr> <tr> <td>Loss</td> <td>A value that represents how well (or badly) our model is doing.</td> </tr> <tr> <td>Gradient</td> <td>The derivative of the loss with respect to some parameter of the model.</td> </tr> <tr> <td>Backward pass</td> <td>Computing the gradients of the loss with respect to all model parameters.</td> </tr> <tr> <td>Gradient descent</td> <td>Taking a step in the directions opposite to the gradients to make the model parameters a little bit better.</td> </tr> <tr> <td>Learning rate</td> <td>The size of the step we take when applying SGD to update the parameters of the model.</td> </tr> </tbody> </table> <p><strong>Activations:: Numbers that are calculated (both by linear and nonlinear layers) Parameters:: Numbers that are randomly initialized, and optimized (that is, the numbers that define the model)</strong></p>These notes are from Course-v4 of fastai and it differs a lot from the course-v3. In this course the have published a book call fastBook (available for free as Jupyter Notebook).Lesson 3 Data blocks and Activation Functions2020-07-20T00:00:00+00:002020-07-20T00:00:00+00:00https://build3r.github.io/fastai//fastai/2020/07/20/lesson3<p>Try out Andrew Ns Bottom up style Machine learning course on Coursera <a href="https://www.coursera.org/learn/machine-learning">link</a></p> <p>Kaggle has a Python based tool to download its data <code class="language-plaintext highlighter-rouge">pip install kaggle --upgrade</code></p> <p><strong>Pandas</strong></p> <p>It is the standard to accessing tabular data</p> <p>Ex : This shows the header and first few rows of a csv</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="p">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">path</span><span class="o">/</span><span class="s">'train_v2.csv'</span><span class="p">)</span> <span class="n">df</span><span class="p">.</span><span class="n">head</span><span class="p">()</span> </code></pre></div></div> <h3 id="data-block-api">Data Block API</h3> <p><a href="https://docs.fast.ai/data_block.html">https://docs.fast.ai/data_block.html</a></p> <p>Dataset just defines <code class="language-plaintext highlighter-rouge">getItem</code> and <code class="language-plaintext highlighter-rouge">len</code> which has to be implemented by all types of datasets</p> <p>Creating data set ex: here src: DataSet</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">src</span> <span class="o">=</span> <span class="p">(</span><span class="n">ImageList</span><span class="p">.</span><span class="n">from_csv</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="s">'train_v2.csv'</span><span class="p">,</span> <span class="n">folder</span><span class="o">=</span><span class="s">'train-jpg'</span><span class="p">,</span> <span class="n">suffix</span><span class="o">=</span><span class="s">'.jpg'</span><span class="p">)</span> <span class="p">.</span><span class="n">split_by_rand_pct</span><span class="p">(</span><span class="mf">0.2</span><span class="p">)</span> <span class="p">.</span><span class="n">label_from_df</span><span class="p">(</span><span class="n">label_delim</span><span class="o">=</span><span class="s">' '</span><span class="p">))</span> </code></pre></div></div> <p>DataLoader is used to create <a href=":/9cb7ed9fe5b34fe19e6c7d71ac0820d0">minibatches</a> where you specify the batch size and dataset.</p> <p>DataBunch binds together a training_data_loader and a valid_data_loader with optionally a test_data_loader.</p> <p>Creating dataloader follwed by data bunch in one go ex:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">data</span> <span class="o">=</span> <span class="p">(</span><span class="n">src</span><span class="p">.</span><span class="n">transform</span><span class="p">(</span><span class="n">tfms</span><span class="p">,</span> <span class="n">size</span><span class="o">=</span><span class="mi">128</span><span class="p">)</span> <span class="p">.</span><span class="n">databunch</span><span class="p">().</span><span class="n">normalize</span><span class="p">(</span><span class="n">imagenet_stats</span><span class="p">))</span> </code></pre></div></div> <p><em>COCO</em> is the famous dataset for object detection.</p> <p>Transforms will flip the image horizontally at randomly so make data better by default but some you ight need to flip them vertically too.</p> <p>Ex <code class="language-plaintext highlighter-rouge">tfms = get_transforms(flip_vert=True, max_lighting=0.1, max_zoom=1.05, max_warp=0.)</code></p> <p>f_score: it is a single number which is used measure false positive, false negative etc, there are diffrent types like f1, f2. Kaggle uses f2 score to judge.</p> <p>Arg_max = the class with highest accuracy.</p> <p>Accuracy_thresh is used when th classification can lead to more than one class. we can pass minimum threshold to it.</p> <p>Partial function: a same function with ceratin parameter is fixed.</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">somefn</span><span class="p">(</span><span class="n">x1</span><span class="p">,</span> <span class="n">x2</span><span class="p">)</span> <span class="n">somefn_x2_5</span> <span class="o">=</span> <span class="n">somefn</span><span class="p">(</span><span class="n">x2</span><span class="p">,</span> <span class="n">x2</span> <span class="o">=</span> <span class="mi">5</span><span class="p">)</span> <span class="c1">#is same as </span><span class="n">partial</span><span class="p">(</span><span class="n">somefn</span><span class="p">,</span> <span class="n">x2</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span> </code></pre></div></div> <p>Transfer learning: In the planet ex we initially used 128x128 now we can use the same learner and create a new data bunch with 256x256 and learn on existing learner which will keep things which were already learned and will add more to it.</p> <p>Choosing learning rate:<br /> Most of the initial lr looks the same, after the first set of fitting we try to chose lr which is 10x back from the point when it is shooting up and run it from taht to the the initial lr.</p> <p>Collection of data sets: <a href="https://course.fast.ai/datasets">https://course.fast.ai/datasets</a></p> <h3 id="segmentation">Segmentation</h3> <p>we use <code class="language-plaintext highlighter-rouge">Learner.create_unet</code> which us better model for segmentation than cnn.</p> <p><code class="language-plaintext highlighter-rouge">Learner.create_unet(****).to_fp16</code> mixed flosting point precision. By deafult everything uses 32 bit precision for Floating points. But fastai library supports mixed precision which can lead to faster training time and sometimes to better results. Given that you have latest graphics card that supports it and also the latest drivers.</p> <h3 id="face-center-example">Face Center example</h3> <p>regression = when output is some continous number.</p> <h3 id="nlp-classification">NLP Classification</h3> <p>We directly process text as it is not number so we need to convert it. This is done in two differents steps: <strong>tokenization</strong> and <strong>numericalization</strong>.</p> <h5 id="tokenization">Tokenization</h5> <p>The first step of processing we make the texts go through is to split the raw sentences into words, or more exactly tokens. The easiest way to do this would be to split the string on spaces, but we can be smarter:</p> <ul> <li>we need to take care of punctuation</li> <li>some words are contractions of two different words, like isn’t or don’t</li> <li>we may need to clean some parts of our texts, if there’s HTML code for instance</li> </ul> <p>In text classification we do two steps</p> <ol> <li> <p>Language model</p> </li> <li> <p>Classification model</p> </li> </ol> <h3 id="what-is-deep-learning">What is Deep Learning</h3> <p>Basically</p> <p>(Input Matrix X Weigth Matrix -&gt; Activation Fn) -&gt; (Input Matrix X Weigth Matrix -&gt; Activation Fn) -&gt;(Input Matrix X Weigth Matrix -&gt; Activation Fn)…… See how close it is to target (minimize loss fn)</p> <p>Update Weight matrix using gradient descent the <strong>Repeat</strong> thats it.</p> <p>Use this learn more <a href="http://neuralnetworksanddeeplearning.com">http://neuralnetworksanddeeplearning.com</a></p> <h4 id="activation-fn">Activation fn</h4> <p>It the function used on the output after matrix multipliction.</p> <p>Initially we used to use Binary step, logistic, TanH, ArcTan etc</p> <p>But nowadays everyone uses <strong>Rectified Liner Unit (ReLU)</strong> which is nothing but max(X,0) that zeroing out all negatives.</p> <h2 id="from-course-v4">From Course v4</h2> <p>Script to Run a Train a Model and Predict End to End</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">fastbook</span> <span class="kn">import</span> <span class="o">*</span> <span class="kn">from</span> <span class="nn">fastai.vision.widgets</span> <span class="kn">import</span> <span class="o">*</span> <span class="c1">#Assuming Data is downloaded </span><span class="n">bear_types</span> <span class="o">=</span> <span class="s">'grizzly'</span><span class="p">,</span><span class="s">'black'</span><span class="p">,</span><span class="s">'teddys'</span> <span class="n">path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="s">'../data/bears'</span><span class="p">)</span> <span class="c1">#setup a data block valid_pct=0.2 means use 20 % data as validation </span><span class="n">bears</span> <span class="o">=</span> <span class="n">DataBlock</span><span class="p">(</span> <span class="n">blocks</span><span class="o">=</span><span class="p">(</span><span class="n">ImageBlock</span><span class="p">,</span> <span class="n">CategoryBlock</span><span class="p">),</span> <span class="n">get_items</span><span class="o">=</span><span class="n">get_image_files</span><span class="p">,</span> <span class="n">splitter</span><span class="o">=</span><span class="n">RandomSplitter</span><span class="p">(</span><span class="n">valid_pct</span><span class="o">=</span><span class="mf">0.2</span><span class="p">,</span> <span class="n">seed</span><span class="o">=</span><span class="mi">42</span><span class="p">),</span> <span class="n">get_y</span><span class="o">=</span><span class="n">parent_label</span><span class="p">,</span> <span class="n">item_tfms</span><span class="o">=</span><span class="n">Resize</span><span class="p">(</span><span class="mi">128</span><span class="p">))</span> <span class="c1">#modify the data block to use RandomResizedCrop and augmentations </span><span class="n">bears</span> <span class="o">=</span> <span class="n">bears</span><span class="p">.</span><span class="n">new</span><span class="p">(</span><span class="n">item_tfms</span><span class="o">=</span><span class="n">RandomResizedCrop</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="n">nrows</span><span class="o">=</span><span class="mi">1</span><span class="p">)(</span><span class="mi">128</span><span class="p">),</span> <span class="n">batch_tfms</span><span class="o">=</span><span class="n">aug_transforms</span><span class="p">())</span> <span class="c1">#load data </span><span class="n">dls</span> <span class="o">=</span> <span class="n">bears</span><span class="p">.</span><span class="n">dataloaders</span><span class="p">(</span><span class="n">path</span><span class="p">)</span> <span class="n">dls</span><span class="p">.</span><span class="n">train</span><span class="p">.</span><span class="n">show_batch</span><span class="p">(</span><span class="n">max_n</span><span class="o">=</span><span class="mi">8</span><span class="p">,</span> <span class="n">nrows</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">unique</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span> <span class="c1">#create convolution neural network with resnet18 as the base model </span><span class="n">learn</span> <span class="o">=</span> <span class="n">cnn_learner</span><span class="p">(</span><span class="n">dls</span><span class="p">,</span> <span class="n">resnet18</span><span class="p">,</span> <span class="n">metrics</span><span class="o">=</span><span class="n">error_rate</span><span class="p">)</span> <span class="c1">#fine tune/use transfer learning for 4 epochs </span><span class="n">learn</span><span class="p">.</span><span class="n">fine_tune</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span> <span class="c1">#generate a interpretation </span><span class="n">interp</span> <span class="o">=</span> <span class="n">ClassificationInterpretation</span><span class="p">.</span><span class="n">from_learner</span><span class="p">(</span><span class="n">learn</span><span class="p">)</span> <span class="n">interp</span><span class="p">.</span><span class="n">plot_confusion_matrix</span><span class="p">()</span> <span class="c1">#plot top 5 losses or high confident wrong answers </span><span class="n">interp</span><span class="p">.</span><span class="n">plot_top_losses</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="n">nrows</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="c1">#save data as pkl file </span><span class="n">learn</span><span class="p">.</span><span class="n">export</span><span class="p">(</span><span class="s">'../data/bears/new_bear.pkl'</span><span class="p">)</span> <span class="c1">#load saved model for inference </span><span class="n">learn_inf</span> <span class="o">=</span> <span class="n">load_learner</span><span class="p">(</span><span class="n">path</span><span class="o">/</span><span class="s">'new_bear.pkl'</span><span class="p">)</span> <span class="c1">#predic a given image </span><span class="n">learn_inf</span><span class="p">.</span><span class="n">predict</span><span class="p">(</span><span class="s">'../data/bears/grizzly/00000068.jpg'</span><span class="p">)</span> </code></pre></div></div> <h2 id="ethics">Ethics</h2> <p>Recommendation systems are predicting what content people will like, but they also have a lot of power in determining what content people even see.</p>Try out Andrew Ns Bottom up style Machine learning course on Coursera linkLesson 2 Running you first classifier2020-07-16T00:00:00+00:002020-07-16T00:00:00+00:00https://build3r.github.io/fastai//fastai/2020/07/16/lesson2<h2 id="old-models-wont-work-as-fastai-introduced-a-breaking-change-pin-lib-in-diffrent-env">Old models wont work as Fastai introduced a breaking change pin lib in diffrent env</h2> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>fastai == 2.0.3 fastcore == 1.0.4 pytorch == 1.4.0 </code></pre></div></div> <ol> <li>Downloading data from Google Images: use the script go get all URLS run it in browser console</li> </ol> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> urls=Array.from(document.querySelectorAll('.rg_i')).map(el=&gt; el.hasAttribute('data-src')?el.getAttribute('data-src'):el.getAttribute('data-iurl')); window.open('data:text/csv;charset=utf-8,' + escape(urls.join('\n'))); </code></pre></div></div> <ol> <li>Creating validation set automatically from data<br /> new API is DataBlock = <em>template</em> for creating a DataLoaderspython<br /> bears = DataBlock(<br /> blocks=(ImageBlock, CategoryBlock),<br /> get_items=get_image_files,<br /> splitter=RandomSplitter(valid_pct=0.2, seed=42),<br /> get_y=parent_label,<br /> item_tfms=Resize(128))<br /> dls = bears.dataloaders(path)</li> </ol> <pre><code class="language-np.random.seed(42)">data = ImageDataBunch.from_folder(path, train=".", valid_pct=0.2,ds_tfms=get_transforms(), size=224, num_workers=4).normalize(imagenet_stats) </code></pre> <p><img src="https://build3r.github.io/fastai//images/logo.png" alt="dfbc43add93317ed13ce59b6da31f673.png" width="924" height="354" class="jop-noMdConv" /></p> <p><code class="language-plaintext highlighter-rouge">np.random.seed(42)</code> is always set to same number here so that validation set remains same and improve our model independant of data changes.</p> <p>We also use RandomResizedCrop to use a n random crop in every epoch</p> <ol> <li>Tips for choosing learning rate: Look for the range where the accuracy is imroving steadily (not very steep).<br /> ex here the error rate decrese is steady between &amp;10^-5&amp; and &amp;10^-4&amp; so we use<br /> <code class="language-plaintext highlighter-rouge">learn.fit_one_cycle(2, max_lr=slice(3e-5,3e-4))</code><br /> ![learn_rate]</li> </ol> <p><img src="https://build3r.github.io/fastai//images/learning_rate.png" alt="learning_rate" /></p> <ol> <li>All data sets have x and y<br /> x = file name<br /> y = labels</li> <li>You can create application with UI inside Jupyter notebooks.</li> <li><code class="language-plaintext highlighter-rouge">open_image(path)</code> fn to open any image</li> <li><code class="language-plaintext highlighter-rouge">learn.export()</code> exports the trained model to export.pkl its ~99MB</li> <li><a href="https://www.starlette.io/">Starlette</a> is web app toolkit similar to flas which has async/await style</li> <li>Trainining loss &gt; validation loss means data is not properly fitted</li> <li>Too many epochs leads to overfitting. It happens when error rate seems to be going down tthen in subsequents epochs start going up.</li> </ol> <h2 id="lesson-2-sgd">Lesson 2: SGD</h2> <p>Stochastic Gradient Descent (SGD)</p> <p>Tensor in deeplearning it is nothing but a array ( 1 or more dimensions) which is of regular shape (all rows are of equal size)<br /> Rank: number of dimensions</p> <p>x = matrix/ vector = tensor<br /> a = matrix/ vector = tensor<br /> <code class="language-plaintext highlighter-rouge">y = x@a</code> = matrix product</p> <p>Mean squared Error MSE is the common error fn for regression<br /> <code class="language-plaintext highlighter-rouge">def mse(y_hat, y): return ((y_hat-y)**2).mean()</code><br /> y_hat or $\hat{y}$ is predicted value and y is the actual</p> <p><code class="language-plaintext highlighter-rouge">tensor(-1.,1)</code> is eqivalent to <code class="language-plaintext highlighter-rouge">tensor(-1.0,1.0)</code> we do this because we want everything to be float</p> <p>In this we are trying to fit a line of the form<br /> y = mx +c which we convert to<br /> $y_i$ = $a_1x_{1i} + a_2x_{2i}$ where $x_2$ = 1<br /> We first try to gues $a_1,a_2$ and calculate MSE. we then try minimise MSE<br /> We can do that either try to keep guessing by changing $a_1,a_2$ up and down and calculate MSE and see where it reduces or we can use Maths in which a derivative calculates the same thing<br /> Gradient/Derivative~ tells us whether moving it up or down $a_2$ or tilting it up or down $a_1$ make it better i.e change the MSE.</p> <blockquote> <p>Gradient descent is an algorithm that minimizes functions. Given a function defined by a set of parameters, gradient descent starts with an initial set of parameter values and iteratively moves toward a set of parameter values that minimize the function. This iterative minimization is achieved by taking steps in the negative direction of the function gradient.</p> </blockquote> <h4 id="mini-batches">mini-batches</h4> <p>In this ex we calculated loss for all the points in practice it willnot possible as tehre will be too mnay points when the rank/complexity increases.<br /> So we use Mini batches for every epoch/iteration we calculate loss from a random subset of points</p> <p>Model or Architecture is nothing but the mathematical fn here it is <code class="language-plaintext highlighter-rouge">y = mx + c</code> or $y = a_1x_1 + a_2x_2$</p> <p>Parameter are the coefficients or weights are the number which we are trying to find here they are $a_1 and\ a_2$</p> <p>How (and why) to create a good validation set? <a href="https://www.fast.ai/2017/11/13/validation-sets/">link</a></p>Old models wont work as Fastai introduced a breaking change pin lib in diffrent envLesson 1 Using Jupyter Notebook2020-07-15T00:00:00+00:002020-07-15T00:00:00+00:00https://build3r.github.io/fastai//fastai/2020/07/15/lesson1<p>Basic Jupyter note book command and fast.ai library functions</p> <h2 id="lesson-1">Lesson 1</h2> <p>​ To auto reload ​</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>%reload_ext autoreload %autoreload 2 %matplotlib inline </code></pre></div></div> <p>​ <code class="language-plaintext highlighter-rouge">?function-name</code>: Shows the definition and docstring for that function<br /> <code class="language-plaintext highlighter-rouge">??function-name</code>: Shows the source code for that function ​ <code class="language-plaintext highlighter-rouge">help(fn_name)</code> <a href="http://fast.ai">fast.ai</a> helper fn like man<br /> <code class="language-plaintext highlighter-rouge">doc(fn_name)</code> gives the details informations and links to docs ​ Line magic star with <code class="language-plaintext highlighter-rouge">‘%’%timeit [i+1 for i in range(1000)]</code> : Runs a line ten thousand times and displays the average time it took to run it.<br /> <code class="language-plaintext highlighter-rouge">%debug</code>: Allows to inspect a function which is showing an error using the Python debugger. ​ <code class="language-plaintext highlighter-rouge">path.ls()</code> Helper to do ‘ls’ on Path(python data type) object<br /> You can append to path object like path/’images’ ​</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>data = ImageDataBunch.from_name_re(path_img, fnames, pat, ds_tfms=get_transforms(), size=224, bs=bs).normalize(imagenet_stats) </code></pre></div></div> <p>​ size=224 : standard image size because of GPU limitation<br /> Return a data bunch object which will used in most of <a href="http://fast.ai">fast.ai</a> course ​ <code class="language-plaintext highlighter-rouge">get_transforms</code><br /> Tranforms the image to square and also does centre cropping and a lot more ​ <code class="language-plaintext highlighter-rouge">normalize(imagenet_stats)</code><br /> Normalises the color RGB so that model can train well, if model is not working well try to normalise. ​ <code class="language-plaintext highlighter-rouge">data.classes</code> = lists all the classes (ex different breeds) of data<br /> <code class="language-plaintext highlighter-rouge">data.c</code> = lists the number of classes = len(data.classes) ​ <code class="language-plaintext highlighter-rouge">learn = cnn_learner(data, models.resnet34, metrics=error_rate)</code><br /> cnn_learner = a type of convolution neural network<br /> Resnet34 is type of model/architecture. Works almost every time two major ones resnet_34 and resnet_50 number denotes the layer size ​ metrics = things to print can be error_rate or accuracy ​ <kbd>Tab</kbd> to auto complete ​</p> <p>machine_learning</p>Basic Jupyter note book command and fast.ai library functionsLesson 0 - Setting up fastai course and gotchas2020-07-14T00:00:00+00:002020-07-14T00:00:00+00:00https://build3r.github.io/fastai//fastai/2020/07/14/lesson0<p>Setting up conda, gothchas if jupyter notebook and basic of markdown</p> <h2 id="updating">Updating</h2> <p>We use Conda to manage the Jupyter notebook and Fastai libraries</p> <ul> <li>Update conda <code class="language-plaintext highlighter-rouge">conda update conda</code></li> <li>List all the libs installed <code class="language-plaintext highlighter-rouge">conda list | grep "fastai"</code></li> <li>Update fastai <code class="language-plaintext highlighter-rouge">conda install -c fastai fastai</code></li> </ul> <h2 id="lesson-0-jupyter-notebooks">Lesson 0 Jupyter Notebooks</h2> <p>Command mode blue :blue_book:<br /> Edit mode green :green_book:</p> <p><kbd>B</kbd> to create new cell<br /> <kbd>S</kbd> to save<br /> <kbd>m</kbd> convert cell to mark down in command mode<br /> <kbd>y</kbd> convert cell to code mode<br /> <kbd>A</kbd> to insert above<br /> <kbd>b</kbd> to insert below<br /> <kbd>c</kbd> copy<br /> <kbd>X</kbd> cut<br /> <kbd>Z</kbd> undo</p> <p><kbd>Crtl + shift + -</kbd> spilt cell at cursor<br /> <kbd>Shift + enter</kbd> to run<br /> There is a inbuilt terminal which you can launch from man directory (laggy with tunnelling)</p> <ul> <li>Markdown <ul> <li>‘*’ italics</li> <li>‘**’ bold</li> <li>‘`’ inline</li> <li>‘&gt;’ block quote</li> <li>[text](link) hyperlink</li> </ul> </li> </ul> <p>link with ‘[]’ and place the link adjacent to the text, surrounded with ‘()’</p>Setting up conda, gothchas if jupyter notebook and basic of markdown