Linear layer using GPU

zahra: <p>Hi,<br>
My network has two layers; the first one is a cnn layer and the second is a linear layer. Also, I try to use gpu for running it. The linear layer is as following:</p>
<pre><code class="lang-auto">self.fc1 = nn.Linear(rows_num_after_convolution, 1).to(torch.device("cuda:0"), dtype=torch.half, non_blocking=True)
</code></pre>
<p>but I receive this error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "MyCNNCustomized_v6_based_on_MyCNN13.py", line 576, in &lt;module&gt;
    outputs = net(current_image)
  File "/home/zahra/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "MyCNNCustomized_v6_based_on_MyCNN13.py", line 281, in forward
    x = torch.atan(self.fc1(self.convZ.output_after_pooling.squeeze(0).squeeze(1)))
  File "/home/zahra/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/zahra/.local/lib/python2.7/site-packages/torch/nn/modules/linear.py", line 92, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/zahra/.local/lib/python2.7/site-packages/torch/nn/functional.py", line 1408, in linear
    output = input.matmul(weight.t())
RuntimeError: Expected object of backend CPU but got backend CUDA for argument #2 'mat2'
</code></pre>
<p>I would appreciate it if anyone could guide me.</p>
<p>Many thanks before all</p>

sukuya: <aside class="quote no-group quote-modified" data-username="zahra" data-post="1" data-topic="50715">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/aeb1de/48.png" class="avatar"> zahra:</div>
<blockquote>
<p>self.fc1 = nn.Linear(rows_num_after_convolution, 1).to(torch.device(“cuda:0”), dtype=torch.half, non_blocking=True)</p>
</blockquote>
</aside>
<p><a class="mention" href="/u/zahra">@zahra</a> Can you share how you are using the input.</p>
<pre><code class="lang-auto">model = CNNModel()
input = torch.ones((1,4), device='cuda', dtype=torch.half)
output = model(input)
print(output)
</code></pre>
<p>This works perfectly fine for me with your Model.</p>

zahra: <p>Thanks, you are right, I forgot to set the device for input as ‘cuda’.<br>
Now, I have another problem that I appreciate it if you guide me about it, too:<br>
The backward function of my customized cnn is sth like this:</p>
<pre><code class="lang-auto">def backward(ctx, grad_output):
        grad_output = grad_output.detach()
        input, filter, bias = ctx.saved_tensors     
        grad_bias = torch.sum(grad_output)
        ....
</code></pre>
<p>and in main function is like this:</p>
<pre><code class="lang-auto">loss.backward()
</code></pre>
<p>I receive this error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "MyCNNCustomized_v6_based_on_MyCNN13.py", line 585, in &lt;module&gt;

&gt;     loss.backward(None)

File "/home/zahra/.local/lib/python2.7/site-packages/torch/tensor.py", line 107, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/zahra/.local/lib/python2.7/site-packages/torch/autograd/__init__.py", line 93, in backward
    allow_unreachable=True)  # allow_unreachable flag

&gt; RuntimeError: expected Variable or None (got numpy.ndarray)

</code></pre>
<p>I do not know what the problem is. Do you have any idea?<br>
It is interesting that the code was executable on cpu while I receive this error in gpu!</p>
<p>Many thanks before all</p>

sukuya: <aside class="quote no-group" data-username="zahra" data-post="3" data-topic="50715">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/aeb1de/48.png" class="avatar"> zahra:</div>
<blockquote>
<p>loss.backward()</p>
</blockquote>
</aside>
<p>Can you post the entire Class? You can follow <a href="https://github.com/jcjohnson/pytorch-examples" rel="noopener nofollow ugc">https://github.com/jcjohnson/pytorch-examples</a></p>

zahra: <p>Thanks, my problem solved.<br>
I used ScipyConv2dFunction class on this page:</p>
<p><a href="https://pytorch.org/tutorials/advanced/numpy_extensions_tutorial.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/numpy_extensions_tutorial.html</a></p>
<p>I customized correlate2d and convovle2d by myself. After that, each run for each sample of data lasts about 15 seconds on CPU. For decreasing the time, I switched to GPU and I changed numpy to tensor. Unfortunately, the time increased to 3 mins.<br>
Do you have any idea about how to improve the time?</p>
<p>Many thanks before all</p>

sukuya: <p><a class="mention" href="/u/zahra">@zahra</a> Not sure how you have customised the correlate2d and convolve2d, but this is most likely due to copying back and forth, the tensors between device and  host which is a slow process. Difficult to say more without looking at the code.</p>