About the ram issue in detection task in PyTorch?

username: Sam_d
cooked: <p>I made my code in TensorFlow and it works well for training and validation with 16 GB ram T4 gpu but then i convert it in pytorch and now its showing that 22 GB L4 gpu is still not enough for training in PyTorch, Does anyone have any idea whats going on?</p>

username: ptrblck
cooked: <p>Without any information it’s impossible to know what’s going on.</p>

username: Sam_d
cooked: <p>Hi,<br>
Here my training and validation setup</p>
<p>“”"<br>
device = torch.device(“cuda” if torch.cuda.is_available() else “cpu”)</p>
<h1><a name="p-463655-initialize-model-optimizer-and-scheduler-1" class="anchor" href="#p-463655-initialize-model-optimizer-and-scheduler-1"></a>Initialize model, optimizer, and scheduler</h1>
<p>model = YOLOv3Small().to(device)  # Move model to GPU<br>
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)<br>
scheduler = CyclicLR(optimizer, base_lr=1e-5, max_lr=1e-2, step_size_up=108, mode=‘triangular2’)</p>
<h1><a name="p-463655-initialize-mixed-precision-scaler-2" class="anchor" href="#p-463655-initialize-mixed-precision-scaler-2"></a>Initialize mixed-precision scaler</h1>
<p>scaler = GradScaler()</p>
<p>num_epochs = 100<br>
train_epoch_losses = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
val_epoch_losses = <span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<h1><a name="p-463655-train_loaders-dataloadertensordatasettrain_images-train_labels-batch_sizebatch_size-shuffletrue-3" class="anchor" href="#p-463655-train_loaders-dataloadertensordatasettrain_images-train_labels-batch_sizebatch_size-shuffletrue-3"></a>Train_loaders = [DataLoader(TensorDataset(Train_Images, Train_Labels), batch_size=batch_size, shuffle=True)]</h1>
<h1><a name="p-463655-train_loaders-dataloadertensordatasetval_images-val_labels_1_s-batch_sizebatch_size-shuffletrue-4" class="anchor" href="#p-463655-train_loaders-dataloadertensordatasetval_images-val_labels_1_s-batch_sizebatch_size-shuffletrue-4"></a>Train_loaders = [DataLoader(TensorDataset(Val_Images, Val_Labels_1_s), batch_size=batch_size, shuffle=True)]</h1>
<p>for epoch in range(2):<br>
model.train()<br>
total_train_loss = 0.0<br>
num_train_batches = 0</p>
<pre><code>for Train_loader in Train_loaders:
    for images, labels in Train_loader:
        images, labels = images.to(device), labels.to(device)  # Ensure data is on the GPU

        optimizer.zero_grad()

        with autocast():  # Use mixed-precision
            output = model(images)
            conf_loss, ciou_loss = YoloLoss_s_pt(Y_true=labels, Y_Pred=output)
            total_loss = conf_loss + ciou_loss

        # Gradient accumulation (if applicable)
        total_loss.backward()
        if (num_train_batches + 1) % accumulation_steps == 0:
            optimizer.step()

        # Accumulate loss and batch count
        total_train_loss += total_loss.item()
        num_train_batches += 1

        # Clear GPU cache to avoid fragmentation
        torch.cuda.empty_cache()

avg_train_loss = total_train_loss / num_train_batches
train_epoch_losses.append(avg_train_loss)

model.eval()
total_val_losses = 0.0
num_val_batches = 0

with torch.no_grad():
    for Val_loader in Valid_loaders:
        for images, labels in Val_loader:
            images, labels = images.to(device), labels.to(device)  # Ensure data is on the GPU

            with autocast():  # Use mixed-precision during validation as well
                output = model(images)
                conf_loss, ciou_loss = YoloLoss_s_pt(Y_true=labels, Y_Pred=output)
                total_loss = conf_loss + ciou_loss

            total_val_losses += total_loss.item()
            num_val_batches += 1

avg_val_loss = total_val_losses / num_val_batches
val_epoch_losses.append(avg_val_loss)

# Save best model based on validation loss
if avg_val_loss == min(val_epoch_losses):
    torch.save(model.state_dict(), "best_model_1st.pth")
    print(f"Best Loss at epoch {epoch + 1} with valid loss: {avg_val_loss:4f}")

# Update learning rate
current_lr = scheduler.get_last_lr()[0]
print(f"Epoch {epoch + 1}/{num_epochs} - Train Loss: {avg_train_loss:4f}, Val Loss: {avg_val_loss}, LR: {current_lr:8f}")

# Update scheduler
scheduler.step()

# Clear the GPU cache at the end of each epoch
torch.cuda.empty_cache()
</code></pre>
<p>“”"</p>
<p>and this is the output:</p>
<p>:13: FutureWarning: <code>torch.cuda.amp.GradScaler(args...)</code> is deprecated. Please use <code>torch.amp.GradScaler('cuda', args...)</code> instead.<br>
scaler = GradScaler()<br>
:30: FutureWarning: <code>torch.cuda.amp.autocast(args...)</code> is deprecated. Please use <code>torch.amp.autocast('cuda', args...)</code> instead.<br>
with autocast():  # Use mixed-precision</p>
<hr>
<p>OutOfMemoryError                          Traceback (most recent call last)</p>
<p> in &lt;cell line: 0&gt;()<br>
29<br>
30             with autocast():  # Use mixed-precision<br>
—&gt; 31                 output = model(images)<br>
32                 conf_loss, ciou_loss = YoloLoss_s_pt(Y_true=labels, Y_Pred=output)<br>
33                 total_loss = conf_loss + ciou_loss</p>
<p>12 frames</p>
<p>/usr/local/lib/python3.11/dist-packages/torch/nn/functional.py in leaky_relu(input, negative_slope, inplace)<br>
1900         result = torch._C.<em>nn.leaky_relu</em>(input, negative_slope)<br>
1901     else:<br>
 → 1902         result = torch._C._nn.leaky_relu(input, negative_slope)<br>
1903     return result<br>
1904</p>
<p>OutOfMemoryError: CUDA out of memory. Tried to allocate 64.00 MiB. GPU 0 has a total capacity of 14.75 GiB of which 49.06 MiB is free. Process 34100 has 14.70 GiB memory in use. Of the allocated memory 14.54 GiB is allocated by PyTorch, and 36.12 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (<a href="https://pytorch.org/docs/stable/notes/cuda.html#environment-variables" class="inline-onebox" rel="noopener nofollow ugc">CUDA semantics — PyTorch 2.5 documentation</a>)</p>

username: Soumya_Kundu
cooked: <p>Are all your hyperparameters the same? for example the batch size?</p>