How can I run vgg16 (transfer learning) on my GPU which has 4GB of memory?

username: laro
cooked: <p>I want to use <code>VGG16</code> (transfer learning), but I don’t have enough memory:</p>
<ul>
<li>According to <code>nvidia-smi</code> I have 4GB of memory</li>
<li>Model:</li>
</ul>
<pre><code class="lang-auto">  model = torchvision.models.vgg16(pretrained=True)          
  for p in model.parameters():
      p.requires_grad = False

  sin                 = model.classifier[0].in_features    
  model.classifier = nn.Sequential(
      nn.Linear(sin, 128), nn.ReLU(),
      nn.Linear(128, 2)
  )
</code></pre>
<p>According to: <code>torchinfo.summary(model, (64, 3, 224, 224))</code></p>
<pre><code class="lang-auto">Estimated Total Size (MB): 7046.64
</code></pre>
<p>And I’m trying to train it with:</p>
<pre><code class="lang-auto">DEVICE    = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") 
model     = model.to(DEVICE)
adam      = optim.AdamW  (model.parameters(), lr=1e-4, betas=(0.9, 0.99), weight_decay=2e-4)
loss      = nn.CrossEntropyLoss()

model.train(True) 
for ii, (x, y) in enumerate(trainDL):
    x = x.to(DEVICE)
    y = y.to(DEVICE)
    
    z   = model(x) 
    ...
</code></pre>
<p>Where <code>trainDL</code> is <code>torch.utils.data.DataLoader</code></p>
<ul>
<li>versions:torch==1.10.0torchinfo==1.5.3torchvision==0.11.1</li>
</ul>
<p>I’m getting <code>CUDA out of memory</code> (which seems true because I have less GPU memory than the model size)</p>
<p>I read this old post (2 years ago…):</p><aside class="quote quote-modified" data-post="1" data-topic="35662">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/4af34b/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/cuda-out-of-memory-in-transfer-learning-vgg16/35662">CUDA out of memory in transfer learning VGG16</a> 
  </div>
  <blockquote>
    Hello, I tried to run the code from this website, about transfer learning using VGG16. 
<a href="https://www.kaggle.com/carloalbertobarbano/vgg16-transfer-learning-pytorch" class="onebox" target="_blank" rel="noopener nofollow ugc">https://www.kaggle.com/carloalbertobarbano/vgg16-transfer-learning-pytorch</a> 
But I have trouble with cuda out of memory, in the train process. 
I already tried to minimize this error by change the batch_size and num_workers in torch.utils.data.DataLoader into the small value but it still out of memory. 
Is there anyone has any suggestion how to get the proper batch size and num_workers??? 
Thank you very much 
…
  </blockquote>
</aside>
<p>
and maybee there is new solution to that issue ?</p>
<ol>
<li>How can I use <code>vgg16</code> (transfer learning) and run it on my GPU ?</li>
<li>What do I need to change in my code above to make it runs on my GPU ?</li>
</ol>

username: ptrblck
cooked: <p>If the model parameters already take more memory than your GPU has I don’t think there is a way to make it work besides using only a subset of the model on the GPU and the rest on the CPU.<br>
I don’t know if you would expect any speedup due to the transfer of tensors between the GPU and the host, so you would have to profile it against a CPU-only run.</p>