My neural network overfits very badly. Please help!

username: Ali_Mashoud
cooked: <p>Hello, I hope everyone is fine. I am trying to train a simple neural network on a FASHION MNIST data-set. However, although its a simple neural network it overly over-fits the data-set giving an accuracy of almost 630%. I don’t know what is the problem. Every time I try to train a simple neural net. I get accuracy like this. Here is the code below. Please do have a look at it and give your valuable feedback.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/b/fbc7a7bd183e6afcfeec248d97fa7dccbda4edc1.png" data-download-href="https://discuss.pytorch.org/uploads/default/fbc7a7bd183e6afcfeec248d97fa7dccbda4edc1" title="Capture"><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/b/fbc7a7bd183e6afcfeec248d97fa7dccbda4edc1.png" alt="Capture" data-base62-sha1="zVlBtsyVzgQ5HfHZTU9QC0mHzb3" width="690" height="219" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture</span><span class="informations">988×314 13.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/c/1ceeb8182496064b229ce5defe4866e3c1549d66.png" data-download-href="https://discuss.pytorch.org/uploads/default/1ceeb8182496064b229ce5defe4866e3c1549d66" title="Capture_1"><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/c/1ceeb8182496064b229ce5defe4866e3c1549d66.png" alt="Capture_1" data-base62-sha1="47WOZMIJwluGlrKni2wrBd7t1nE" width="690" height="335" data-dominant-color="F4F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture_1</span><span class="informations">982×478 14.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/4/c40a83ba16da2c6947bc367a94a4542c8d380b3b.png" data-download-href="https://discuss.pytorch.org/uploads/default/c40a83ba16da2c6947bc367a94a4542c8d380b3b" title="Capture_2"><img src="https://discuss.pytorch.org/uploads/default/original/3X/c/4/c40a83ba16da2c6947bc367a94a4542c8d380b3b.png" alt="Capture_2" data-base62-sha1="rYg6UMkOm0sAxQiIzQM7ZAInG9t" width="690" height="262" data-dominant-color="F7F7F7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture_2</span><span class="informations">819×312 6.59 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/2/f2022a96ea9854d44d773dba88167a49b0f75f8c.png" data-download-href="https://discuss.pytorch.org/uploads/default/f2022a96ea9854d44d773dba88167a49b0f75f8c" title="Capture_3"><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/2/f2022a96ea9854d44d773dba88167a49b0f75f8c.png" alt="Capture_3" data-base62-sha1="ywUbJx85N47qK3nJEB4huu7itB2" width="690" height="259" data-dominant-color="F4F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture_3</span><span class="informations">855×321 8.26 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/f/7fbb47f98b23d62f699a9081ad5af934c6f007a3.png" data-download-href="https://discuss.pytorch.org/uploads/default/7fbb47f98b23d62f699a9081ad5af934c6f007a3" title="Capture_4"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/f/7fbb47f98b23d62f699a9081ad5af934c6f007a3.png" alt="Capture_4" data-base62-sha1="idXSYIA4wQ95VTby0zSsfDPoSTF" width="690" height="373" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture_4</span><span class="informations">921×498 16.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/3/534c64c312f008643e34b8edf89c3d4a539664aa.png" data-download-href="https://discuss.pytorch.org/uploads/default/534c64c312f008643e34b8edf89c3d4a539664aa" title="Capture_5"><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/3/534c64c312f008643e34b8edf89c3d4a539664aa.png" alt="Capture_5" data-base62-sha1="bSTiL8coJKzMVrQmSqyGJKn5CdA" width="690" height="398" data-dominant-color="EFEFEF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture_5</span><span class="informations">868×501 30.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Imahn
cooked: <p>I cannot see any behavior of <a href="https://duckduckgo.com/?q=Overfitting+in+deep+learning&amp;atb=v1-1&amp;ia=web" rel="noopener nofollow ugc">overfitting</a> in your post. Overfitting means that the training loss, accuracy or whatever is higher than the validation loss, accuracy.</p>
<p>I cannot see your entire code in the screenshot, but an accuracy greater than 1 is obviously not possible. A perfect classifier would achieve an accuracy of 1, because then every sample would be classified correctly. Also take a look at <a href="https://en.wikipedia.org/wiki/Accuracy_and_precision#In_binary_classification" rel="noopener nofollow ugc">this Wikipedia entry</a>.</p>
<p>Maybe you can take a look online at other tutorials how the accuracy can be calculated? Or you try it yourself given the referenced Wikipedia article, where the definition is listed.</p>

username: Ali_Mashoud
cooked: <p>but accuracy of 697% doesn’t this makes my neural net overfit the data-set?</p>

username: Ali_Mashoud
cooked: <p>The code line that prints the accuracy is as follows :</p>
<blockquote>
<p>print(f’epoch: {i:2}  batch: {b:4} [{10*b:6}/50000] loss:{loss.item():10.8f}\ accuracy: {trn_corr.item()<em>100/(10</em>b):7.3f}%')</p>
</blockquote>
<p>Please identify if there is any error in this code? Thanks</p>

username: Ali_Mashoud
cooked: <p>Here is the code for training the neural network; if you cannot see it properly in the above images:</p>
<blockquote>
<p>import time</p>
<p>start_time = time.time()</p>
<p>epochs = 5</p>
<p>train_losses = <span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<p>test_losses = <span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<p>train_correct = <span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<p>test_correct = <span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<p>for i in range(epochs):</p>
<p>trn_corr = 0</p>
<p>tst_corr = 0</p>
<p><span class="hashtag-raw">#Running</span> the batches</p>
<p>for b,(X_train,y_train) in enumerate(train_dataloader):</p>
<pre><code>b+=1

#Apply the model

y_pred = model(X_train)

loss = criterion(y_pred,y_train)

#Tally the number of correct predictions

predicted = torch.max(y_pred.data,1)[1]

batch_corr = (predicted==y_train).sum()

trn_corr += batch_corr

#Update paramters

optimizer.zero_grad()

loss.backward()

optimizer.step()

#Print interim results

if b%100 == 0:

  print(f'epoch: {i:2}  batch: {b:4} [{10*b:6}/50000] loss:{loss.item():10.8f}\ accuracy: {trn_corr.item()*100/(10*b):7.3f}%')
</code></pre>
<p>train_losses.append(loss)</p>
<p>train_correct.append(trn_corr)</p>
</blockquote>

username: Imahn
cooked: <aside class="quote no-group" data-username="Ali_Mashoud" data-post="3" data-topic="141588" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ali_mashoud/48/33224_2.png" class="avatar"> Ali_Mashoud:</div>
<blockquote>
<p>but accuracy of 697% doesn’t this makes my neural net overfit the data-set?</p>
</blockquote>
</aside>
<p>You cannot read off from this value (which must be wrong, because the accuracy has an upper bound of 1) anything about overfitting. Please reread my last post and feel free to visit the references in there.</p>
<aside class="quote no-group" data-username="Ali_Mashoud" data-post="4" data-topic="141588">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ali_mashoud/48/33224_2.png" class="avatar"> Ali_Mashoud:</div>
<blockquote>
<p>Please identify if there is any error in this code? Thanks</p>
</blockquote>
</aside>
<p>Could you please provide this print-statement in another form? So instead of an f-string (I am unfamiliar with them), could you please write it in this way:</p>
<pre data-code-wrap="python"><code class="lang-python">print('Random float between 0 and 1: {:.2f}'.format(torch.rand(1).item())
</code></pre>
<p>When you write <code>{:.2f}</code>, Python will only print the first two decimal places.</p>
<p>And just to make sure: What are the shapes of <code>X_train</code>, <code>Y_train</code> and <code>y_pred</code>? So when you do</p>
<pre data-code-wrap="python"><code class="lang-python">print('Shapes of training samples, predictions and labels: {} {} {}'.format(X_train.shape, y_pred.shape, y_train.shape))
</code></pre>
<p>what is the output?</p>

username: Ali_Mashoud
cooked: <p>Thank-you for your help, I actually detected the error. It was syntax error I had to write 10 instead of 100.</p>