RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. Can't use GPU with tacotron2

username: Hasan_Khan
cooked: <pre><code class="lang-auto">Shifting CUDA to CPU for Inferencing
</code></pre>
<p>I am trying to generate inference results of my trained <strong>Text-to-Speech Tacotron2 model</strong> on <strong>CPU</strong>. However, initially the model provide inferencing on <strong>GPU</strong> but due to the non-availability of <strong>GPU</strong> I am transferring to <code>CPU device</code>. I have made the required changes like <code>map_location = torch.device('cpu')</code></p>
<pre><code class="lang-auto">CUDA to CPU Inferencing
</code></pre>
<p>I am trying to generate inference results of my trained <strong>Text-to-Speech Tacotron2 model</strong> on <strong>CPU</strong>. However, initially the model provide inferencing on <strong>GPU</strong> but due to the non-availability of <strong>GPU</strong> I am transferring to <code>CPU device</code>. I have made the required changes like <code>map_location = torch.device('cpu')</code>.</p>
<p>Still the error is not resolved. Please help me understand the issue and get the error resolved. Thanks!!</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/4/741447bef077b55a98b196a96acc89123d3e4110.png" data-download-href="https://discuss.pytorch.org/uploads/default/741447bef077b55a98b196a96acc89123d3e4110" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/4/741447bef077b55a98b196a96acc89123d3e4110.png" alt="image" data-base62-sha1="gySPSKjOlW0Fj7xUSQXMVC8iFZS" width="690" height="339" data-dominant-color="EEEFEF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1079×531 38.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/b/8bb7052552e185fe8c01840595281fa0285e0ac4.png" data-download-href="https://discuss.pytorch.org/uploads/default/8bb7052552e185fe8c01840595281fa0285e0ac4" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/8/b/8bb7052552e185fe8c01840595281fa0285e0ac4.png" alt="image" data-base62-sha1="jVYucBsfcHNShY6CgrkXy27GK8Y" width="690" height="252" data-dominant-color="F7D9D9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1296×475 26.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>I cannot reproduce the issue, as the <code>torch.hub.load</code> method would already load the model as described in the <a href="https://pytorch.org/hub/nvidia_deeplearningexamples_waveglow/">docs</a>:</p>
<pre><code class="lang-python">&gt;&gt;&gt; import torch
&gt;&gt;&gt; waveglow = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_waveglow', model_math='fp32')
Downloading: "https://github.com/NVIDIA/DeepLearningExamples/archive/torchhub.zip" to /root/.cache/torch/hub/torchhub.zip
Downloading checkpoint from https://api.ngc.nvidia.com/v2/models/nvidia/waveglow_ckpt_fp32/versions/19.09.0/files/nvidia_waveglowpyt_fp32_20190427
&gt;&gt;&gt; waveglow
WaveGlow(
  (upsample): ConvTranspose1d(80, 80, kernel_size=(1024,), stride=(256,))
  (WN): ModuleList(
    (0): WN(
      (in_layers): ModuleList(
        (0): Conv1d(512, 1024, kernel_size=(3,), stride=(1,), padding=(1,))
        (1): Conv1d(512, 1024, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
   [...]
</code></pre>
<p>Calling <code>torch.load</code> afterwards on it yields the expected <code>seek</code> error:</p>
<pre><code class="lang-python">&gt;&gt;&gt; torch.load(waveglow, map_location='cpu')

AttributeError: 'WaveGlow' object has no attribute 'seek'
</code></pre>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier and allows to index the code for a better search.</p>

username: Hasan_Khan
cooked: <p>Is it capable of shifting to CPU from GPU ? Also, what is this seek error indicating?</p>

username: ptrblck
cooked: <p><code>torch.hub.load</code> loads the model onto the CPU so there is no need to push it to the CPU again.</p>
<p>The <code>seek</code> error is raised, since <code>waveglow</code> is an object of the <code>nn.Module</code> type and not a file (which is seek’able).</p>

username: stupid-doge
cooked: <p>actually, I meet the same issues. I am using pretrained model from slow_r50. In the training process, I fine tune the pre-trained model, then save the paramters as ‘ResNet18_best.pth’. Thus, in the testing file, I am trying to load the model structure, then use <code>torch.load</code> to load the best parameters. However, I meet the same issue, Could you help me out? Thanks!</p>
<pre><code class="lang-auto">import os
os.environ['CUDA_VISIBLE_DEVICES'] = '1'
import torch
import torch.nn as nn
import torch.nn.functional as F
import pandas as pd
from torch.utils.data import Dataset, DataLoader
import torchvision.transforms as transforms
from PIL import Image
from dataset import MyDataset
from models import resnet18


device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')


transforms = transforms.Compose([
        transforms.Resize((224, 224)),
        transforms.ToTensor(),
        transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)) 
])


# test_dataset = MyDataset("/hw3_16fpv", "test_for_student.csv", stage="test",ratio=0.2,transform=transforms)
test_dataset = MyDataset("/hw3_16fpv", "test_for_student.csv", stage="test",ratio=0.2,transform=transforms)
test_loader = DataLoader(test_dataset, batch_size=32, shuffle=False, num_workers=16)
print(len(test_loader))


# net = resnet18(num_classes=10, sample_size=224, sample_duration=16).to(device)
# use the pretrained model
net = torch.hub.load('facebookresearch/pytorchvideo', 'slow_r50', pretrained=True).to(device)
# replace the last layers
net.proj = nn.Linear(in_features=2048, out_features=10, bias=True)
net.load_state_dict(torch.load('ResNet18_best.pth'))



net.eval()
result = []
with torch.no_grad():
    for data in test_loader:
        inputs, labels = data
        inputs, labels = inputs.to(device), labels.to(device)
        outputs = net(inputs)
        _, predicted = torch.max(outputs.data, 1)
        result.extend(predicted.cpu().numpy())
        
fread = open("test_for_student.label", "r")
video_ids = []
for line in fread.readlines():
    video_id = os.path.splitext(line.strip())[0]
    video_ids.append(video_id)



with open('result_ResNet18_3D.csv', "w") as f:
    f.writelines("Id,Category\n")
    for i, pred_class in enumerate(result):
        f.writelines("%s,%d\n" % (video_ids[i], pred_class))

</code></pre>
<p>The error is:</p>
<pre><code class="lang-auto">Using cache found in /home/kaiz/.cache/torch/hub/facebookresearch_pytorchvideo_main
Traceback (most recent call last):
  File "/home/kaiz/hw3/test2csv.py", line 35, in &lt;module&gt;
    net.load_state_dict(torch.load('ResNet18_best.pth'))
  File "/opt/miniconda3/envs/5032/lib/python3.9/site-packages/torch/serialization.py", line 789, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/opt/miniconda3/envs/5032/lib/python3.9/site-packages/torch/serialization.py", line 1131, in _load
    result = unpickler.load()
  File "/opt/miniconda3/envs/5032/lib/python3.9/site-packages/torch/serialization.py", line 1101, in persistent_load
    load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location))
  File "/opt/miniconda3/envs/5032/lib/python3.9/site-packages/torch/serialization.py", line 1083, in load_tensor
    wrap_storage=restore_location(storage, location),
  File "/opt/miniconda3/envs/5032/lib/python3.9/site-packages/torch/serialization.py", line 215, in default_restore_location
    result = fn(storage, location)
  File "/opt/miniconda3/envs/5032/lib/python3.9/site-packages/torch/serialization.py", line 182, in _cuda_deserialize
    device = validate_cuda_device(location)
  File "/opt/miniconda3/envs/5032/lib/python3.9/site-packages/torch/serialization.py", line 166, in validate_cuda_device
    raise RuntimeError('Attempting to deserialize object on a CUDA '
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
</code></pre>

username: ptrblck
cooked: <p>Did you try to use the suggested <code>map_location=torch.device('cpu')</code> in <code>torch.load</code> to map the <code>state_dict</code> to the CPU as it seems your system doesn’t have a GPU to move the data to?</p>

username: stupid-doge
cooked: <p>Definitely, the cud’s in the training process’s is true… however, in this test file, sometimes Duda is false.<br>
Thus, when I try to use cpu directly for the whole python file, still not work</p>

username: ptrblck
cooked: <p>It’s unexpected that CUDA is sometimes available and sometimes not. Did you see this issue before and does your GPU work afterwards? Also, is this specific to one environment or a general issue on your machine?</p>

username: stupid-doge
cooked: <p>I always used one environment… actually, I never met this before. I think it is caused by torch.hub.load</p>
<p>Or do you have any idea about how to see the source code, and copy and paste the model structure, then upload the parameters I trained.</p>

username: Vasil_Popov
cooked: <p>Absolutely the same situation and problem, however I’m trying to test super - resolution model.</p>