Pretrained ResNet151 model for Recurrent Plot Classification?

Ivan_A: <p>I’m trying to do a binary classification between people with ADHD and Non-ADHD, I converted the EEG signals into images with Recurrent Plot, It’s a good approach use a pretrained ResNet151 model for this task?.</p>

Preetham_R_Patlolla: <p>Using the pretrained model by replacing the last layers and training on your dataset will be a good choice. This can improve the performance.</p>

Ivan_A: <p>Thank you very much, one more question, my neural network is not able to learn from my images, I am using around 4,560 images with ADHD and 5,200 with non-ADHD and I am getting this kind of results, any advice?</p>
<pre><code class="lang-auto">Epoch 0/99
----------
train Loss: 0.6730 Acc: 0.6294

Epoch 1/99
----------
train Loss: 0.6253 Acc: 0.6662

Epoch 2/99
----------
train Loss: 0.6153 Acc: 0.6746

Epoch 3/99
----------
train Loss: 0.6099 Acc: 0.6818

Epoch 4/99
----------
train Loss: 0.6031 Acc: 0.6880

Epoch 5/99
----------
train Loss: 0.5983 Acc: 0.6909

Epoch 6/99
----------
train Loss: 0.5950 Acc: 0.6956

Epoch 7/99
----------
train Loss: 0.5838 Acc: 0.7079

Epoch 8/99
----------
train Loss: 0.5779 Acc: 0.7073

Epoch 9/99
----------
train Loss: 0.5782 Acc: 0.7094

Epoch 10/99
----------
train Loss: 0.5763 Acc: 0.7123

Epoch 11/99
----------
train Loss: 0.5752 Acc: 0.7116

Epoch 12/99
----------
train Loss: 0.5751 Acc: 0.7163

Epoch 13/99
----------
train Loss: 0.5753 Acc: 0.7167

Epoch 14/99
----------
train Loss: 0.5754 Acc: 0.7128

Epoch 15/99
----------
train Loss: 0.5713 Acc: 0.7173

Epoch 16/99
----------
train Loss: 0.5761 Acc: 0.7116

Epoch 17/99
----------
train Loss: 0.5736 Acc: 0.7105

Epoch 18/99
----------
train Loss: 0.5721 Acc: 0.7141

Epoch 19/99
----------
train Loss: 0.5713 Acc: 0.7153

Epoch 20/99
----------
train Loss: 0.5736 Acc: 0.7137

Epoch 21/99
----------
train Loss: 0.5713 Acc: 0.7144

Epoch 22/99
----------
train Loss: 0.5713 Acc: 0.7164

Epoch 23/99
----------
train Loss: 0.5722 Acc: 0.7146

Epoch 24/99
----------
train Loss: 0.5734 Acc: 0.7145

Epoch 25/99
----------
train Loss: 0.5729 Acc: 0.7130

Epoch 26/99
----------
train Loss: 0.5723 Acc: 0.7173

Epoch 27/99
----------
train Loss: 0.5737 Acc: 0.7104

Epoch 28/99
----------


</code></pre>
<p>These are the images<br>
ADHD</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/9/d991e002b84303f90e02c9e882e966a974abaceb.png" data-download-href="https://discuss.pytorch.org/uploads/default/d991e002b84303f90e02c9e882e966a974abaceb" title="646ADHD24recurrence_plot"><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/9/d991e002b84303f90e02c9e882e966a974abaceb.png" alt="646ADHD24recurrence_plot" data-base62-sha1="v2I96LJCLNPFqbli02eMALfAgvV" width="499" height="499" data-dominant-color="DEDEDE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">646ADHD24recurrence_plot</span><span class="informations">998×998 35.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Control<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/8/e882c85c19c4faeb974b7ac61c40591a1484a0d5.png" data-download-href="https://discuss.pytorch.org/uploads/default/e882c85c19c4faeb974b7ac61c40591a1484a0d5" title="418Control22recurrence_plot"><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/8/e882c85c19c4faeb974b7ac61c40591a1484a0d5.png" alt="418Control22recurrence_plot" data-base62-sha1="xaSYmRP9xBbFXF3J3SqC6UfonUV" width="499" height="499" data-dominant-color="CCCCCC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">418Control22recurrence_plot</span><span class="informations">998×998 40.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Ivan_A: <p>In the ResNet model, the more the number of images increased, the worse its performance was. Is it a good idea to apply Principal Components Analysis to the signals and then convert the most significant Principals Components into images?</p>

Preetham_R_Patlolla: <p>May I know what type of loss function and optimizer you are using? What is the batch size, and the learning rate? Are you normalizing your data before passing it to the network?</p>

Ivan_A: <p>The batch_size =4<br>
criterion = nn.CrossEntropyLoss()<br>
optimizer_ft = optim.SGD(model_ft.parameters(), lr=0.001, momentum=0.9)</p>
<p>Yes I’m normalize the data.</p>
<p>So, what do you think, i need increase the batch_size?</p>

Ivan_A: <p>I decided to increase the batch_size = 6, if batch_size = 10, Google Colab takes you out  after some epochs.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/e/3e5e9bc10b900720f8dafa7c26b01e05c8ae911a.png" alt="Screenshot from 2020-11-17 17-04-33" data-base62-sha1="8TKiE7J5KXv1OLQ21d16DKhWlPI" width="474" height="200"></p>
<p>Has someone been in a similar situation?</p>

Preetham_R_Patlolla: <p>Everything’s seems to be fine to me. Have you tried increasing the learning rate ? If not try doing it. Also, PCA before ResNet might not do any good.</p>

Ivan_A: <p>Thank you for your time, I really appreciate it, I will  keep trying</p>

Ivan_A: <p>It is necessary to be careful with the use of the GPU, when it is not running, it is necessary to disconnect from the GPU, to avoid this type of limitation.</p>

Ivan_A: <p>1-The model started to increase the accuracy with increasing the  batch_size = 32.</p>
<p>2-It is necessary to be careful with the use of the GPU, when it is not running, it is necessary to disconnect from the GPU, to avoid this type of limitation.</p>
<p>3-The model is ResNet152(pretrained=True).</p>

Ivan_A: <p>4.-Finally, the model takes a long time in the training phase (3 - 4 days), because it is not really managing to learn, it is necessary to make an extraction of characteristics and a selection of them, then when you already have the most significant ones, convert them to images I finally decided to use Continuous Wavelet Transform to get a Power Spectrum Analysis.</p>