I am trying to use Resnet50 for regression

username: algeriapy
cooked: <p>Hi,<br>
I am trying to use pretrained Resnet50 for regression task<br>
I changed the output of the fc to 1<br>
model = torchvision.models.resnet50(pretrained=True)<br>
model.fc.out_features = 1</p>
<p>and I am using the MSELoss  as loss function and my batch_size = 15<br>
criterion = nn.MSELoss()<br>
preds = network(images)<br>
loss = criterion(preds, labels)</p>
<p>I am getting the following error:</p>
<pre><code>return torch._C._VariableFunctions.broadcast_tensors(tensors)
</code></pre>
<p>RuntimeError: The size of tensor a (15000) must match the size of tensor b (15) at non-singleton dimension 0</p>

username: futscdav
cooked: <p>You need to replace the entire fc layer, likely something like<br>
<code>model.fc = nn.Linear(2048, 1)</code><br>
should be what you want.</p>

username: algeriapy
cooked: <p>Thanks a lot<br>
I tried this solution before, but I think it did not work because I had not restarted the kernel</p>
<p>Now it is working</p>

username: algeriapy
cooked: <p>I have another problem<br>
when I have used the same notwork as classification for 8 classes and my batch_size = 15<br>
but when I used it for regression output = 1 with even batch_size = 8<br>
it tells</p>
<pre><code>training, momentum, eps, torch.backends.cudnn.enabled
</code></pre>
<p>RuntimeError: CUDA out of memory. Tried to allocate 2.00 MiB (GPU 0; 6.00 GiB total capacity; 4.43 GiB already allocated; 1.14 MiB free; 161.30 MiB cached)</p>
<p>what do you think cause this issue ?</p>

username: Anshumaan_Dash
cooked: <p>Try to restart the kernel again before loading the model.</p>

username: algeriapy
cooked: <p>Yes, I did that<br>
but did not solve this issue</p>

username: Anshumaan_Dash
cooked: <p>Maybe your training data is huge. I mean, each sample of your training data is huge. By the way, could you share the nature of the input (I am assuming images) and labels that you are feeding to the model? Is it the same for both classification and regression?</p>

username: algeriapy
cooked: <p>yes<br>
they are the same, the same data I feed to the network input images (3,224,224)<br>
the only difference is the labels<br>
in the first case I replaced the regression 0-8 by classes 0-1, 1-2, …<br>
and in the second case I used the true regression value</p>

username: Anshumaan_Dash
cooked: <p>Could you check if a smaller bath_size works in this case? If that works, you can accumulate the gradients and run the weight update every n batches.</p>

username: algeriapy
cooked: <p>In the whole training process:<br>
In the training phase, it works for 209 iterations it finished without any problem<br>
the running was stopped in the validation evaluation (validation has 1100 samples)<br>
after 4% the running was interrupted (in the validation process I used batch_size = 1)</p>
<p>Is that make any sense !!!</p>

username: Anshumaan_Dash
cooked: <p>Sure. Can you just check one more thing - see if between the training and validation phase,  you are moving the model to and from gpu to cpu.</p>
<p>And, if possible try to delete the batches at the end of training and validation like <code>del images; del labels</code></p>

username: algeriapy
cooked: <p>thanks bro,<br>
I have not used move model to gpu than cpu between training and validation</p>
<p>I solved the problem first by adding del images; del labels<br>
and I added in the validation phase<br>
model.eval()<br>
with torch.no_grad():<br>
preds = model(images)</p>
<p>previously, it was</p>
<p>model.eval()<br>
preds = model(images)</p>

username: Anshumaan_Dash
cooked: <p>Ah! I see. So you were keeping the gradients as well.</p>

username: algeriapy
cooked: <aside class="quote no-group" data-username="algeriapy" data-post="12" data-topic="81121">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/34f0e0/48.png" class="avatar"> algeriapy:</div>
<blockquote>
<p>model.eval()</p>
</blockquote>
</aside>
<p>I thought it would be enough because no<br>
loss.backward()<br>
optimizer.step()<br>
in the validation phase</p>