Clamping leads to cuda out of memory but tensor.data.clamp() works. why?

ElToto: <p>I have a standard dataloader which loads images.<br>
On top of every image I want to add a static tensor.<br>
But I  want to clamp this to (0,1).<br>
This new image is used to train a model.</p>
<p>The following code roughly show the important steps.<br>
(everything is on gpu)</p>
<pre><code>static_tensor = torch.load(path)
for img in dataloader:
  img = img.cuda()
  addition_tensor = img + static_tensor
  clamped_tensor = addition_tensor.clamp(0,1)
  eval = model(clamped_tensor)
  loss = criterion (eval, label)
  loss.backward()
  optimizer.step()
</code></pre>
<p>This creates out of memory errors.</p>
<p>But if I change the clamping into</p>
<pre><code>clamped_tensor = additon_tensor.data.clamp(0,1)
</code></pre>
<p>it no longer creates this error.</p>
<p>What is the reason behind this?</p>
<p>Edit: I noticed that my static tensor has requires_grad = True.</p>
<p>Am I correct to assume that this tensor ‘saves’ the autograd with the model and the graph of this gets bigger in every instance of the for loop?</p>

albanD: <p>Hi,</p>
<p>Your static Tensor should not have requires_grad=True I guess.<br>
And in that case, doing any op here will increase the memory usage as we need to save some values for the backward.</p>
<p>Note that you should never use <code>.data</code> in general!<br>
Here you can use <code>.detach()</code> which will have similar behavior but will avoid weird side effects!</p>

NovGee: <p>Hi, thank you for your advice!<br>
I got another question about clamping, if static Tensor do have requires_grad=True and clamping tensor cause errors like “CUDA error: an illegal memory access was encountered”.<br>
Do you have any ideas about this?  Thank you ! I’m sorry if this is an unrelated issue! <img src="https://discuss.pytorch.org/images/emoji/apple/sob.png?v=10" title=":sob:" class="emoji" alt=":sob:"></p>

ptrblck: <p>Could you update to the latest PyTorch release and check, if you are still hitting this issue?<br>
If so, could you post a minimal executable code snippet to reproduce the issue, please?</p>

NovGee: <p>Thank you for your reply!<br>
Here is code:</p>
<pre><code class="lang-auto">import torch.multiprocessing as mp
import torch.nn.functional as F
from torch.cuda.amp import autocast, GradScaler

def task():
    for epoch in range(0, 100):
        trainer = Trainer()
        trainer.training()

class Trainer():
    def training(self, model, dataloader):
        train_loss = 0.0
        for i, (images, target) in enumerate(dataloader):
            with autocast():
                output = model(images)
                loss = criterion(output, target)
                train_loss += loss.item()
            GradScaler.scale(loss).backward()
            GradScaler.step(optimizer)
            GradScaler.update()

def criterion(predict, target):
    loss_0 = torch.tensor(0.0, device='0, 1')
    n, c, h, w = predict.size()
    for i in range(n):
        loss = F.nll_loss(predict, target, reduction='None').view(-1)
        loss = loss.clamp(min=0.0, max=1000.0) #RuntimeError: CUDA error: an illegal memory access was encountered
        loss_hard, _ = loss.topk(int(0.1 * h * w))
        loss_0 += loss_hard
    return loss_0 / n

if __name__ == '__main__':
    mp.spawn(task, nprocs, args)
</code></pre>
<p>My pytorch version is 1.7 and cuda version is 11.0.The reason I don’t update to the latest release is because loss function used to work fine until I trained with another larger dataset and this error shows accidently during training.<br>
So is this about the scale of dataset? If not, may be other parts of  my codes is wrong.<br>
Thank you!</p>