Overfitting in resnet-34 vs vgg-16

username: Divya_B
cooked: <p>Hi All,</p>
<p>I trained a dataset (grey-scale ultrasound images. with 15 classes) on <code>vgg-16</code> (using batch norm version) and <code>resnet-34</code>.<br>
<code>vgg-16</code> gives me a validation accuracy of 92% where  as I can only hit 83% with <code>resnet-34</code> .</p>
<p>I handled overfitting in both architectures with dropout in FC layer and regularization in optimizer.<br>
I don’t know why there would be minimal overfitting with <code>vgg</code> and not <code>resnet</code> . The <code>resnet</code> model train loss is 0.02 vs valid loss 0.67. Moreover <code>resnet</code> model doesn’t seem to improve beyond this loss range. I have tried hyper-parameter tuning on weight decay, learning rate, momentum, dropout.</p>
<p>The purpose of this exercise to improve our predictions on the dataset. Hence I started trying <code>resnet-34</code> after <code>vgg-16</code>.</p>
<p>I would love some suggestions on</p>
<ol>
<li>How to improve <code>resnet-34</code> performance and address its overfitting.</li>
<li>Any other architecture suggestions that would work for this type of dataset.</li>
</ol>

username: Divya_B
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> <a class="mention" href="/u/innovarul">@InnovArul</a> would love to hear your thoughts.</p>

username: ptrblck
cooked: <p>I assume you used the same training code and just swapped the VGG model for the ResNet?<br>
Are you using both pretrained models or are you training from scratch?</p>
<p>Also, did you make sure to change the <code>fc</code> attribute in <code>resnet34</code> for your output linear layer and the <code>classifier</code> attribute in your <code>vgg16_bn</code> model?</p>

username: Divya_B
cooked: <p>Thanks a bunch for the reply. I started with pre-trained model, tuning the classifier/FC layer first and then sequentially unfroze the <code>resnet-34</code> and <code>vgg16_bn</code> for training. As I unfroze <code>resnet-34</code> layers the model seems to overfit on training data. Not sure if this is a symptom that I am not accounting for. The proportion of images across all class are constant across train and validation.</p>

username: ptrblck
cooked: <p>Did the validation accuracy get worse after unfreezing the ResNet?<br>
If you are dealing with overfitting, you could try to increase the regularization or add some data augmentation.</p>

username: Divya_B
cooked: <p>When I started unfreezing the layers of resnet the accuracy went from 50% to 83% using SGD optimizer, but it saturates at this stage. I am using data augmentation that was suggested for these type of images. Yes, I should try to spend more quality time on regularization. I have-not experimented with different optimizers yet, not sure if they would make a difference.</p>

username: GiridharVP
cooked: <p>Hello,</p>
<p>I am facing a similar problem. I wanted to compare VGG16, Resnet34 and mobilenetV2 on the same data-set, I obtain a 90% accuracy while using VGG, with the feature extractor being frozen, but Resnet34 and MobillenetV2 tend not to converge at similar levels(- max accuracy was 80%) with feature extractor being frozen.<br>
I use the same learning rates( staring at 1e-4) with annealing , regularizes(L2-1e-4 and 1e-5), and  Adam.<br>
Should I try SGD or other parameters.</p>