TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first

WayneAng: <pre><code>for epoch in range(start_epoch, start_epoch + epochs):

    print('\n\n\nEpoch: {}\n&lt;Train&gt;'.format(epoch))
    net.train(True)
    loss = 0
    learning_rate = learning_rate * (0.5 ** (epoch // 4))
    for param_group in optimizer.param_groups:
        param_group["learning_rate"] = learning_rate
    torch.set_grad_enabled(True)
    for idx, (inputs, targets, paths) in enumerate(trainloader):
        inputs, targets = inputs.to(device), targets.to(device)
        outputs = net(inputs)
        if type(outputs) == tuple:
            outputs = outputs[0]
        batch_loss = dice_coef(outputs, targets)
        optimizer.zero_grad()
        batch_loss.backward()
        optimizer.step()
        loss += float(batch_loss)
        progress_bar(idx, len(trainloader), 'Loss: %.5f, Dice-Coef: %.5f'
                     % ((loss / (idx + 1)), (1 - (loss / (idx + 1)))))
    log_msg = '\n'.join(['Epoch: %d  Loss: %.5f,  Dice-Coef:  %.5f' \
                         % (epoch, loss / (idx + 1), 1 - (loss / (idx + 1)))])
    logging.info(log_msg)
</code></pre>
<p>def dice_coef(preds, targets, backprop=True):<br>
smooth = 1.0<br>
class_num = 2<br>
if backprop:<br>
for i in range(class_num):<br>
pred = preds[:,i,:,:]<br>
target = targets[:,i,:,:]<br>
intersection = (pred * target).sum()<br>
loss_ = 1 - ((2.0 * intersection + smooth) / (pred.sum() + target.sum() + smooth))<br>
if i == 0:<br>
loss = loss_<br>
else:<br>
loss = loss + loss_<br>
loss = loss/class_num<br>
return loss<br>
else:<br>
# Need to generalize<br>
targets = np.array(targets.argmax(1))<br>
if len(preds.shape) &gt; 3:<br>
preds = np.array(preds).argmax(1)<br>
for i in range(class_num):<br>
pred = (preds==i).astype(np.uint8)<br>
target= (targets==i).astype(np.uint8)<br>
intersection = (pred * target).sum()<br>
loss_ = 1 - ((2.0 * intersection + smooth) / (pred.sum() + target.sum() + smooth))<br>
if i == 0:<br>
loss = loss_<br>
else:<br>
loss = loss + loss_<br>
loss = loss/class_num<br>
return loss<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/4/14e3e9488bb390215450e2c84ae3f471f5dddefc.png" data-download-href="https://discuss.pytorch.org/uploads/default/14e3e9488bb390215450e2c84ae3f471f5dddefc" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/14e3e9488bb390215450e2c84ae3f471f5dddefc_2_690x183.png" alt="image" data-base62-sha1="2YNQw0Y3chTBPktZq9xe724zVTu" width="690" height="183" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/14e3e9488bb390215450e2c84ae3f471f5dddefc_2_690x183.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/14e3e9488bb390215450e2c84ae3f471f5dddefc_2_1035x274.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/14e3e9488bb390215450e2c84ae3f471f5dddefc_2_1380x366.png 2x" data-dominant-color="303030"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1383×367 39.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I facing this type of error anyone knows how to solve it?</p>

ptrblck: <p>As the error message suggests, you would have to push the tensor to the CPU first before converting it to a numpy array via <code>tensor.cpu()</code>.<br>
In particular <code>np.array(targets.argmax(1))</code> seems to raise the error to use:</p>
<pre><code class="lang-python">targets = targets.argmax(1).cpu().numpy()
</code></pre>
<p>instead.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier.</p>

WayneAng: <p>Thanks for your help.</p>
<p>Oic, thanks for your advise I will try to use it ^^</p>

EvanZ: <p>I have a similar error that I think is arising when I try to use sklearn’s <code>roc_auc_score</code>:</p>
<pre><code class="lang-auto">Exception has occurred: TypeError
Caught TypeError in replica 0 on device 0.
Original Traceback (most recent call last):
  File "/home/ubuntu/deep-behavior-embedding/.virtualenv/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
    output = module(*input, **kwargs)
  File "/home/ubuntu/deep-behavior-embedding/.virtualenv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ubuntu/deep-behavior-embedding/.virtualenv/lib/python3.7/site-packages/pytorch_lightning/overrides/data_parallel.py", line 77, in forward
    output = super().forward(*inputs, **kwargs)
  File "/home/ubuntu/deep-behavior-embedding/.virtualenv/lib/python3.7/site-packages/pytorch_lightning/overrides/base.py", line 57, in forward
    output = self.module.validation_step(*inputs, **kwargs)
  File "/home/ubuntu/deep-behavior-embedding/src/model/lightning_model.py", line 144, in validation_step
    auc = roc_auc_score(y_true=y, y_score=torch.sigmoid(x))
  File "/home/ubuntu/deep-behavior-embedding/.virtualenv/lib/python3.7/site-packages/sklearn/utils/validation.py", line 63, in inner_f
    return f(*args, **kwargs)
  File "/home/ubuntu/deep-behavior-embedding/.virtualenv/lib/python3.7/site-packages/sklearn/metrics/_ranking.py", line 522, in roc_auc_score
    y_type = type_of_target(y_true)
  File "/home/ubuntu/deep-behavior-embedding/.virtualenv/lib/python3.7/site-packages/sklearn/utils/multiclass.py", line 261, in type_of_target
    if is_multilabel(y):
  File "/home/ubuntu/deep-behavior-embedding/.virtualenv/lib/python3.7/site-packages/sklearn/utils/multiclass.py", line 147, in is_multilabel
    y = np.asarray(y)
  File "/home/ubuntu/deep-behavior-embedding/.virtualenv/lib/python3.7/site-packages/numpy/core/_asarray.py", line 102, in asarray
    return array(a, dtype, copy=False, order=order)
  File "/home/ubuntu/deep-behavior-embedding/.virtualenv/lib/python3.7/site-packages/torch/_tensor.py", line 643, in __array__
    return self.numpy()
TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
  File "/home/ubuntu/deep-behavior-embedding/finetune.py", line 108, in &lt;module&gt;
    trainer.fit(model, datamodule=dm)
</code></pre>
<p>The code is here:</p>
<pre><code class="lang-auto">    def validation_step(self, batch, batch_idx):
        x = batch['src']
        y = batch['label']
        mask = batch['mask']

        x = self.base_model(x, mask)
        x = self.linear(x).mean(axis=1).squeeze(1)
        
        loss = F.binary_cross_entropy_with_logits(input=x,
                               target=y)
        self.log('val_loss', loss)
        try:
            auc = roc_auc_score(y_true=y, y_score=torch.sigmoid(x))
            self.log('auc', auc)
        except ValueError:
            self.log('auc', 0)
        return loss

</code></pre>
<p>Should I be converting to numpy arrays? I’m afraid this might slow things down.</p>

Parskatt: <p>Sorry for reviving an old thread. Is there any reason that pytorch explictly raises an exception instead of simply calling .cpu() internally inside .numpy().</p>
<p>I personally find it annoying to have to write .cpu().numpy() every time. For attached tensors this is even more annoying with .detach().cpu().numpy() which is very verbose.</p>
<p>Have you considered integrating these steps into the .numpy() call? Is there some risk?</p>

Kenzo: <p>Just use <code>.tolist()</code> instead of <code>.cpu().numpy()</code></p>

ptrblck: <aside class="quote no-group" data-username="Parskatt" data-post="5" data-topic="126585">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/parskatt/48/29053_2.png" class="avatar"> Parskatt:</div>
<blockquote>
<p>Have you considered integrating these steps into the .numpy() call? Is there some risk?</p>
</blockquote>
</aside>
<p>Yes, I think the explicit nature of detaching the tensor and moving it to the CPU makes clear that:</p>
<ul>
<li>Autograd won’t be able to track the operations performed on the result array,</li>
<li>all operations will be performed on the CPU on the result array.</li>
</ul>
<p>While some users could find it annoying, I think it can avoid a lot of confusion especially for novice users as the detaching operation is explicit.<br>
You can find a lot of topics here where a user was wondering why the training got stuck after using a threshold operation (which implicitly detaches the result). Of course it makes sense once you know that a threshold is not differentiable, but a user might still spend some time in debugging it.</p>

Parskatt: <p>Not sure what you mean? My tensors are typically images or feature maps, and I want to e.g. show them as images, or further process in numpy. tolist() doesnt make sense there.</p>

KanZa: <p>Any suggestions for my code? I am also facing the same error. I have tried argmax(1).cpu().numpy()</p>
<pre><code class="lang-auto">losses = []

def train_epoch(models, criterion, optimizers, dataloaders):
    models.train()  
    global iters
    train_loss = []
    for data in tqdm(dataloaders['train'], leave=False, total=len(dataloaders['train'])):
        with torch.cuda.device(CUDA_VISIBLE_DEVICES):
            inputs = data[0].cuda()
            labels = data[1].cuda()
        iters += 1
        optimizers.zero_grad()       
        scores, _, features = models(inputs) 
        
        target_loss = criterion(scores, labels)
        target_loss.detach()
        losses.append(target_loss)
        
        m_backbone_loss = torch.sum(target_loss) / target_loss.size(0)    
        loss = m_backbone_loss
        loss.backward()
        optimizers.step()  
        #train_loss.append(loss.item())
    return loss
</code></pre>

srishti-git1110: <p>Which line in this code is giving you the error?</p>
<p>Also, could you please paste the error message as it shows?</p>

KanZa: <p>Error: can’t convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.</p>
<p>I want to plot losses</p>

srishti-git1110: <p>By any chance, are you using the list <code>losses</code> to get the values to plot?</p>
<p>If yes, then the problem lies in these two lines -</p>
<aside class="quote no-group" data-username="KanZa" data-post="9" data-topic="126585">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kanza/48/1529_2.png" class="avatar"> KanZa:</div>
<blockquote>
<pre><code class="lang-auto">target_loss.detach()
losses.append(target_loss)
</code></pre>
</blockquote>
</aside>
<p>Replace both of them with this single line -</p>
<pre><code class="lang-auto">losses.append(target_loss.detach().cpu())
</code></pre>
<p>Let me know if you still face the error.</p>

KanZa: <p>Thank you so much, Srishti!<br>
But I am still facing the same error.</p>

srishti-git1110: <p>Please share the code that gives you the error if this doesn’t work -</p>
<pre><code class="lang-auto">losses.append(target_loss.detach().cpu())
</code></pre>
<p>Or this -</p>
<pre><code class="lang-auto">losses.append(target_loss.detach().cpu().numpy())
</code></pre>
<p>(And sorry, I forgot to include .cpu() in my previous reply - have edited it now.)</p>

KanZa: <p>yeah this .cpu() helped to resolve.<br>
Thank you so much!</p>

KanZa: <p>I am facing the same error for the following code:</p>
<pre><code class="lang-auto"> with torch.no_grad():
        for idx, (inputs, targets) in enumerate(ploader):
            inputs, targets = inputs.to(device), targets.to(device)
            outputs = net(inputs)
            scores, predicted = outputs.max(1)
            # save top1 confidence score 
            outputs = F.normalize(outputs, dim=1)
            probs = F.softmax(outputs, dim=1)
            top1_scores.append(probs[0][predicted.item()])
            progress_bar(idx, len(ploader))
            #top1_scores.detach().cpu().numpy()
    
    idx = np.argsort(top1_scores)                 #Error is for this line
    samples = np.array(samples)
    return samples[idx[:1000]]

</code></pre>

srishti-git1110: <p>If the following line gives the error,</p>
<aside class="quote no-group" data-username="KanZa" data-post="16" data-topic="126585">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kanza/48/1529_2.png" class="avatar"> KanZa:</div>
<blockquote>
<p><code>inputs, targets = inputs.to(device), targets.to(device)</code></p>
</blockquote>
</aside>
<p>make sure PyTorch tensors are being returned in the <code>__getitem__</code> method of the Dataset class whose instance is being used to create <code>ploader</code>.</p>

KanZa: <p>I just tried to run the original repository, didn’t read and understand the code fully. But any comments how to fix getitem() here?</p>
<pre><code class="lang-auto">sub5k = Loader2(is_train=False,  transform=transform_test, path_list=samples)
ploader = torch.utils.data.DataLoader(sub5k, batch_size=1, shuffle=False, num_workers=2)

class Loader2(Dataset):
    def __init__(self, is_train=True, transform=None, path='./DATA', path_list=None):
        self.is_train = is_train
        self.transform = transform
        self.path_list = path_list

        if self.is_train: # train
            self.img_path = path_list
        else:
            if path_list is None:
                self.img_path = glob.glob('./DATA/train/*/*') # for loss extraction
            else:
                self.img_path = path_list
    def __len__(self):
        return len(self.img_path)

    def __getitem__(self, idx):
        if self.is_train:
            img = cv2.imread(self.img_path[idx][:-1])
        else:
            if self.path_list is None:
                img = cv2.imread(self.img_path[idx])
            else:
                img = cv2.imread(self.img_path[idx][:-1])
        img = Image.fromarray(img)
        img = self.transform(img)
        label = int(self.img_path[idx].split('/')[-2])

        return img, label

</code></pre>

srishti-git1110: <p>Try adding the following before the return statement:</p>
<pre><code class="lang-auto">label = torch.tensor([label])
</code></pre>
<p>Check out <a href="https://pytorch.org/vision/stable/generated/torchvision.transforms.ToTensor.html" rel="noopener nofollow ugc">this</a> that you could use in your transforms object to convert your <code>img</code> to a pytorch tensor.</p>

KanZa: <p>Thank you for your help, I resolved the issue</p>