Learning Curves and behaviour

brunoj: <p>I trained Alexnet on ecg dataset which is imbalance. My learning curves are these. 10 epochs, 0.001 Lr and Adam is optimizer. Batch size is 16.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/6/469176778d50f1e602197cc281b7e30c19965b0b.png" data-download-href="https://discuss.pytorch.org/uploads/default/469176778d50f1e602197cc281b7e30c19965b0b" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/469176778d50f1e602197cc281b7e30c19965b0b_2_690x388.png" alt="image" data-base62-sha1="a4h4G4uqEobYDEwwBSNFL8AjibN" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/469176778d50f1e602197cc281b7e30c19965b0b_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/469176778d50f1e602197cc281b7e30c19965b0b_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/469176778d50f1e602197cc281b7e30c19965b0b_2_1380x776.png 2x" data-dominant-color="F3F3F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1920×1080 273 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
It also skips some classes as here<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/4/1463c0662aacde0330f7d9126a4011a7a0e38d02.png" data-download-href="https://discuss.pytorch.org/uploads/default/1463c0662aacde0330f7d9126a4011a7a0e38d02" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/1463c0662aacde0330f7d9126a4011a7a0e38d02_2_690x388.png" alt="image" data-base62-sha1="2Ungv2dOOoHAxKxDQ76jYE13iSK" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/1463c0662aacde0330f7d9126a4011a7a0e38d02_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/1463c0662aacde0330f7d9126a4011a7a0e38d02_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/1463c0662aacde0330f7d9126a4011a7a0e38d02_2_1380x776.png 2x" data-dominant-color="D3D3D6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1920×1080 210 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Where did I go wrong?<br>
I also wanna know what impact batch size has and lr on this. I dont understand this weird learning curve.</p>
<p>I trained vgg16 and got these<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/c/2cac5ab62d81914d169d10cb726223abb3155b5d.png" data-download-href="https://discuss.pytorch.org/uploads/default/2cac5ab62d81914d169d10cb726223abb3155b5d" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/c/2cac5ab62d81914d169d10cb726223abb3155b5d_2_690x388.png" alt="image" data-base62-sha1="6nch7CglnzHZiOgGYNmGOqS27zT" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/c/2cac5ab62d81914d169d10cb726223abb3155b5d_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/c/2cac5ab62d81914d169d10cb726223abb3155b5d_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/c/2cac5ab62d81914d169d10cb726223abb3155b5d_2_1380x776.png 2x" data-dominant-color="F4F3F0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1920×1080 230 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Your model seems to overfit the majority class and the accuracy might be misleading due to the <a href="https://en.wikipedia.org/wiki/Accuracy_paradox">Accuracy Paradox</a> so you might want to check the metric to plot.<br>
To counter overfitting in an imbalanced class setup you could use a weighted loss or could use a <code>WeightedRandomSampler</code> to oversample the minority classes.</p>

brunoj: <p>Alright I see this article<br>
<a href="https://naadispeaks.blog/2021/07/31/handling-imbalanced-classes-with-weighted-loss-in-pytorch/" class="inline-onebox" rel="noopener nofollow ugc">Handling Imbalanced Classes with Weighted Loss in PyTorch | NaadiSpeaks</a>  I<br>
but how do that class_weights are calculated? It says 1- (<span class="hashtag-raw">#samples</span> in class) which class??? like is it for the minimum sampled class???  my dataset is like this<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/6/4696bca25dcc8e201dd07680c72184a8e3119cdb.png" data-download-href="https://discuss.pytorch.org/uploads/default/4696bca25dcc8e201dd07680c72184a8e3119cdb" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/4696bca25dcc8e201dd07680c72184a8e3119cdb_2_690x388.png" alt="image" data-base62-sha1="a4sng4onVR1snzS01fJgnqI4jLd" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/4696bca25dcc8e201dd07680c72184a8e3119cdb_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/4696bca25dcc8e201dd07680c72184a8e3119cdb_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/4696bca25dcc8e201dd07680c72184a8e3119cdb_2_1380x776.png 2x" data-dominant-color="E6D2DC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1920×1080 124 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>You could define the weights via <code>weight = 1.0 / count</code> where <code>count</code> represents the number of samples for each class.</p>

brunoj: <p>Hey <a class="mention" href="/u/ptrblck">@ptrblck</a>  I m facing an issue while doing transfer learning on ECG dataset using alexnet architecture. My classification report came like this:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/d/1d2d99544733799bc5a90dcf9a97427bd21f1eed.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/1d2d99544733799bc5a90dcf9a97427bd21f1eed" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/d/1d2d99544733799bc5a90dcf9a97427bd21f1eed_2_690x388.jpeg" alt="image" data-base62-sha1="4a7xBabsrPl0JacRMXSxK3Wfy6F" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/d/1d2d99544733799bc5a90dcf9a97427bd21f1eed_2_690x388.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/1/d/1d2d99544733799bc5a90dcf9a97427bd21f1eed_2_1035x582.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/d/1d2d99544733799bc5a90dcf9a97427bd21f1eed_2_1380x776.jpeg 2x" data-dominant-color="E7E7E7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1920×1080 113 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Why the classes have 0 precision and recall. I dont understand.</p>

ptrblck: <p>The metrics would indicate a zero value e.g. if your model never predicts these classes.<br>
Based on the results your model is only predicting the two classes with the most <code>support</code>.</p>

brunoj: <p>so you mean i should balance the whole dataset to equal samples and check? or is there any other possibility.<br>
I previously had F ‘0’ class with 161 samples i augmented so now 322 but still its imbalance but previously I have seen my model without augmentation predicted the last class as well and skips ist 4 and 5th. but after augmnetation which is still imblance my model skips all except 2 and 3 class.<br>
Moreover if i plot the classification report it shows different results like here.<br>
The without augmentation<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/8/68f278b31ac8a10ebe667f4b147b231c40bfc0bd.png" data-download-href="https://discuss.pytorch.org/uploads/default/68f278b31ac8a10ebe667f4b147b231c40bfc0bd" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/68f278b31ac8a10ebe667f4b147b231c40bfc0bd_2_690x388.png" alt="image" data-base62-sha1="eYp9hnE2dMIwshgj0cBT0cghSH3" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/68f278b31ac8a10ebe667f4b147b231c40bfc0bd_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/68f278b31ac8a10ebe667f4b147b231c40bfc0bd_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/68f278b31ac8a10ebe667f4b147b231c40bfc0bd_2_1380x776.png 2x" data-dominant-color="DFE0E4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1920×1080 266 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/0/50eae3ddb377edb174d787326aeef47b4b8ba8fe.png" data-download-href="https://discuss.pytorch.org/uploads/default/50eae3ddb377edb174d787326aeef47b4b8ba8fe" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/0/50eae3ddb377edb174d787326aeef47b4b8ba8fe_2_690x388.png" alt="image" data-base62-sha1="bxPrJDS2IOPGuH7RSWidZjc5kQu" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/0/50eae3ddb377edb174d787326aeef47b4b8ba8fe_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/0/50eae3ddb377edb174d787326aeef47b4b8ba8fe_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/5/0/50eae3ddb377edb174d787326aeef47b4b8ba8fe_2_1380x776.png 2x" data-dominant-color="DBDBE0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1920×1080 297 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/8/68bb1252807598634788944027bb94c4c8eaa98a.png" data-download-href="https://discuss.pytorch.org/uploads/default/68bb1252807598634788944027bb94c4c8eaa98a" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/68bb1252807598634788944027bb94c4c8eaa98a_2_690x388.png" alt="image" data-base62-sha1="eWusgytOIrUVs2SGpchqnyvaF3c" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/68bb1252807598634788944027bb94c4c8eaa98a_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/68bb1252807598634788944027bb94c4c8eaa98a_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/68bb1252807598634788944027bb94c4c8eaa98a_2_1380x776.png 2x" data-dominant-color="C9C0C5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1920×1080 207 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>