Error in using cuda (ValueError: Expected a cuda device with a specified index or an integer, but got: )

username: Pengzhangzhi
cooked: <p>Error in using cuda.</p>
<p>Here is the Code:</p>
<pre><code class="lang-auto">import robust_loss_pytorch.general

adaptive = robust_loss_pytorch.adaptive.AdaptiveLossFunction(
    num_dims = 4, float_dtype=torch.cuda.FloatTensor, device="cuda")

</code></pre>
<p>And here is the document of robust_loss_pytorch.adaptive.AdaptiveLossFunction:</p>
<pre><code class="lang-auto">Args:
  num_dims: The number of dimensions of the input to come.
  float_dtype: The floating point precision of the inputs to come.
  device: The device to run on (cpu, cuda, etc).

</code></pre>
<p>Got the error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
&lt;ipython-input-414-22ef6dd72b45&gt; in &lt;module&gt;()
      2 
      3 adaptive = robust_loss_pytorch.adaptive.AdaptiveLossFunction(
----&gt; 4     num_dims = 4, float_dtype=torch.cuda.FloatTensor, device="cuda")

~/anaconda3/envs/torch36/lib/python3.6/site-packages/robust_loss_pytorch/adaptive.py in __init__(self, num_dims, float_dtype, device, alpha_lo, alpha_hi, alpha_init, scale_lo, scale_init)
    130        (isinstance(device, str) and 'cuda' in device) or\
    131        (isinstance(device, torch.device) and device.type == 'cuda'):
--&gt; 132         torch.cuda.set_device(self.device)
    133 
    134     self.distribution = distribution.Distribution()

~/anaconda3/envs/torch36/lib/python3.6/site-packages/torch/cuda/__init__.py in set_device(device)
    241             if this argument is negative.
    242     """
--&gt; 243     device = _get_device_index(device)
    244     if device &gt;= 0:
    245         torch._C._cuda_setDevice(device)

~/anaconda3/envs/torch36/lib/python3.6/site-packages/torch/cuda/_utils.py in _get_device_index(device, optional)
     32         else:
     33             raise ValueError('Expected a cuda device with a specified index '
---&gt; 34                              'or an integer, but got: '.format(device))
     35     return device_idx

ValueError: Expected a cuda device with a specified index or an integer, but got: 
</code></pre>
<p>When I try code  <code>adaptive = robust_loss_pytorch.adaptive.AdaptiveLossFunction(num_dims = 4, float_dtype=torch.cuda.FloatTensor, device="cuda:0")  </code></p>
<p>Got error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
&lt;ipython-input-415-c7998f8405f9&gt; in &lt;module&gt;()
      2 
      3 adaptive = robust_loss_pytorch.adaptive.AdaptiveLossFunction(
----&gt; 4     num_dims = 4, float_dtype=torch.cuda.FloatTensor, device="cuda:0")

~/anaconda3/envs/torch36/lib/python3.6/site-packages/robust_loss_pytorch/adaptive.py in __init__(self, num_dims, float_dtype, device, alpha_lo, alpha_hi, alpha_init, scale_lo, scale_init)
    154               latent_alpha_init.clone().detach().to(
    155                   dtype=self.float_dtype,
--&gt; 156                   device=self.device)[np.newaxis, np.newaxis].repeat(
    157                       1, self.num_dims),
    158               requires_grad=True))

TypeError: to() received an invalid combination of arguments - got (device=str, dtype=torch.tensortype, ), but expected one of:
 * (torch.device device, torch.dtype dtype, bool non_blocking, bool copy, *, torch.memory_format memory_format)
 * (torch.dtype dtype, bool non_blocking, bool copy, *, torch.memory_format memory_format)
 * (Tensor tensor, bool non_blocking, bool copy, *, torch.memory_format memory_format)
</code></pre>
<p>Or trying :</p>
<pre><code class="lang-auto">adaptive = robust_loss_pytorch.adaptive.AdaptiveLossFunction(
    num_dims = 4, float_dtype=torch.cuda.FloatTensor, device=torch.device("cuda"))
</code></pre>
<p>Got the same error as :</p>
<pre><code class="lang-auto">ValueError: Expected a cuda device with a specified index or an integer, but got:
</code></pre>
<p>Would you please provide me the correct usage of cuda?</p>
<p>If you need detailed information,please let me know!</p>

username: KarthikR
cooked: <p>Print this: torch.cuda.current_device(). The value you get from the print, can you pass it to the ‘device’ argument? Example if the value is 0, can you try as below?</p>
<p>robust_loss_pytorch.adaptive.AdaptiveLossFunction( num_dims = 4, float_dtype=torch.cuda.FloatTensor, <strong>device=0</strong>)</p>

username: Pengzhangzhi
cooked: <p>The value is 0, then I try this code,still get an error.</p>
<pre><code class="lang-auto">TypeError                                 Traceback (most recent call last)
&lt;ipython-input-418-2de402a52488&gt; in &lt;module&gt;()
----&gt; 1 robust_loss_pytorch.adaptive.AdaptiveLossFunction( num_dims = 4, float_dtype=torch.cuda.FloatTensor, device=0)

~/anaconda3/envs/torch36/lib/python3.6/site-packages/robust_loss_pytorch/adaptive.py in __init__(self, num_dims, float_dtype, device, alpha_lo, alpha_hi, alpha_init, scale_lo, scale_init)
    154               latent_alpha_init.clone().detach().to(
    155                   dtype=self.float_dtype,
--&gt; 156                   device=self.device)[np.newaxis, np.newaxis].repeat(
    157                       1, self.num_dims),
    158               requires_grad=True))

TypeError: to() received an invalid combination of arguments - got (device=int, dtype=torch.tensortype, ), but expected one of:
 * (torch.device device, torch.dtype dtype, bool non_blocking, bool copy, *, torch.memory_format memory_format)
 * (torch.dtype dtype, bool non_blocking, bool copy, *, torch.memory_format memory_format)
 * (Tensor tensor, bool non_blocking, bool copy, *, torch.memory_format memory_format)

</code></pre>

username: KarthikR
cooked: <p>So, to() received an invalid combination of arguments. This happened if we pass a string (“cuda:0”) and an int (0). One of the right approaches to to() is shown as in the message:</p>
<p>(torch.device device, torch.dtype dtype, bool non_blocking, bool copy, *, torch.memory_format memory_format)</p>
<p>Can you try this as here, we try to create a torch.device first?</p>
<p>cuda0 = torch.device(‘cuda:0’)<br>
adaptive = robust_loss_pytorch.adaptive.AdaptiveLossFunction(num_dims = 4, float_dtype=torch.cuda.FloatTensor, device=cuda0 )</p>

username: Pengzhangzhi
cooked: <aside class="quote no-group" data-username="KarthikR" data-post="4" data-topic="99027">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/eada6e/48.png" class="avatar"> KarthikR:</div>
<blockquote>
<p>cuda0 = torch.device(‘cuda:0’)<br>
adaptive = robust_loss_pytorch.adaptive.AdaptiveLossFunction(num_dims = 4, float_dtype=torch.cuda.FloatTensor, device=cuda0 )</p>
</blockquote>
</aside>
<p>Still not working. I am about crying <img src="https://discuss.pytorch.org/images/emoji/apple/pensive.png?v=12" title=":pensive:" class="emoji" alt=":pensive:" loading="lazy" width="20" height="20"> <img src="https://discuss.pytorch.org/images/emoji/apple/innocent.png?v=12" title=":innocent:" class="emoji" alt=":innocent:" loading="lazy" width="20" height="20"><br>
trying code:</p>
<pre><code class="lang-auto">cuda0 = torch.device("cuda:0")
adaptive a robust_loss_pytorch.adaptive.AdaptiveLossFunction(num_dims = 4, float_dtype=torch.cuda.FloatTensor, device=cuda0 )

</code></pre>
<p>got error:</p>
<pre><code class="lang-auto">

TypeError                                 Traceback (most recent call last)
&lt;ipython-input-420-7ab0f8c31272&gt; in &lt;module&gt;()
      1 cuda0 = torch.device("cuda:0")
----&gt; 2 adaptive = robust_loss_pytorch.adaptive.AdaptiveLossFunction(num_dims = 4, float_dtype=torch.cuda.FloatTensor, device=cuda0 )

~/anaconda3/envs/torch36/lib/python3.6/site-packages/robust_loss_pytorch/adaptive.py in __init__(self, num_dims, float_dtype, device, alpha_lo, alpha_hi, alpha_init, scale_lo, scale_init)
    154               latent_alpha_init.clone().detach().to(
    155                   dtype=self.float_dtype,
--&gt; 156                   device=self.device)[np.newaxis, np.newaxis].repeat(
    157                       1, self.num_dims),
    158               requires_grad=True))

TypeError: to() received an invalid combination of arguments - got (device=torch.device, dtype=torch.tensortype, ), but expected one of:
 * (torch.device device, torch.dtype dtype, bool non_blocking, bool copy, *, torch.memory_format memory_format)
 * (torch.dtype dtype, bool non_blocking, bool copy, *, torch.memory_format memory_format)
 * (Tensor tensor, bool non_blocking, bool copy, *, torch.memory_format memory_format)

</code></pre>

username: KarthikR
cooked: <p>When I exceute the below line in Google Colab, it runs without issues:</p>
<p>adaptive = robust_loss_pytorch.adaptive.AdaptiveLossFunction(num_dims = 4, float_dtype=<strong>torch.float32</strong>, device=“cuda:0”)</p>
<p>Notice the change in the float_dtype parameter. Can you run with the above and see if it helps?</p>

username: Pengzhangzhi
cooked: <aside class="quote no-group" data-username="KarthikR" data-post="6" data-topic="99027">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/eada6e/48.png" class="avatar"> KarthikR:</div>
<blockquote>
<p>adaptive = robust_loss_pytorch.adaptive.AdaptiveLossFunction(num_dims = 4, float_dtype= <strong>torch.float32</strong> , device=“cuda:0”)</p>
</blockquote>
</aside>
<p>Thanks sir! It does work.However,what I am wondering is that all of my data and model is in cuda, if I use torch.float32, isn’t  it a mismatch of data type?<br>
Again,Thanks for such an caring responses : )<br>
You have a nice day, sir!</p>

username: KarthikR
cooked: <p>In the code, there is the use of .to() and to this function, we pass the cuda value you give during initialization. So, I believe, your variables and model are moving to cuda.</p>
<p>You can print the model parameters’s device value, eg:</p>
<p>model = torchvision.models.resnet18()<br>
for name in next(model.parameters()):<br>
print(name.device)</p>

username: Pengzhangzhi
cooked: <p>Thanks sir.I really appreciate your  answer. Wish you all the best. : )</p>