RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

Jami_Anwesh_Kumar: <pre><code class="lang-auto">1. def forward(self, x):
2.         alpha = torch.randn(1,requires_grad= True).to(device = 'cuda')
3.         beta = torch.randn(1,requires_grad= True).to(device = 'cuda')
4.         out = F.relu(self.bn1(self.conv1(x)))
5.         out = self.bn2(self.conv2(out))
6.         num = torch.atan(torch.div((alpha*x),torch.sqrt((beta**2)+1)))
7.         num = num.to(device = 'cuda')
8.         den = alpha * torch.sqrt((beta**2)+1)
9.         den = den.to(device = 'cuda')
10.       x_tan = torch.div(num ,den).to(device = 'cuda')
11.       out = F.relu(out)
12.         return out

</code></pre>
<p>I have written above code while implementing my model which is subclass of torch.nn.module the problem i got at num (line 6) and mostly i will get them at den an x_tan. How to fix this??I am actually implementing BReGNext for facial expression recognition in Pytorch.Kindly help me with issue.Thanks in advance for the help.</p>

Usama_Hasan: <p>Hello <a class="mention" href="/u/jami_anwesh_kumar">@Jami_Anwesh_Kumar</a> it looks like that your input ‘x’ is on cpu while the rest of the tensors are on gpu, which is causing this error. Can you please cross check this<br>
Plus you can paste all your code within three quotes</p>

Jami_Anwesh_Kumar: <p>Thanks a lot <a class="mention" href="/u/usama_hasan">@Usama_Hasan</a> it means a lot with your reply but actually i figured it out just now that making alpha and beta part of nn.Parameter in <strong>init</strong> worked for me.See line 18 and 19 to understand what i meant and i commented some things in forward.</p>
<pre><code class="lang-auto">1. class BasicBlock(nn.Module):

2.     expansion = 1

3.     def __init__(self, in_planes, planes, stride=1):

4.         super(BasicBlock, self).__init__()

5.         self.conv1 = nn.Conv2d(

6.             in_planes, planes, kernel_size=3, stride=stride, padding=1, bias=False)

7.         self.bn1 = nn.BatchNorm2d(planes)

8.         self.conv2 = nn.Conv2d(planes, planes, kernel_size=3,

9.                                stride=1, padding=1, bias=False)

10.         self.bn2 = nn.BatchNorm2d(planes)

11.         self.shortcut = nn.Sequential()

12.         if stride != 1 or in_planes != self.expansion*planes:

13.             self.shortcut = nn.Sequential(

14.                 nn.Conv2d(in_planes, self.expansion*planes,

15.                           kernel_size=1, stride=stride, bias=False),

16.                 nn.BatchNorm2d(self.expansion*planes)

17.             )

18.         self.alpha = nn.Parameter(torch.rand(1))

19.         self.beta = nn.Parameter(torch.rand(1))  

20.     def forward(self, x):

21.         #alpha = torch.randn(1,requires_grad= True).to(device = 'cuda')

22.         #beta = torch.randn(1,requires_grad= True).to(device = 'cuda')

23.         

24.         out = F.relu(self.bn1(self.conv1(x)))

25.         out = self.bn2(self.conv2(out))

26.         # num = torch.atan(torch.div((alpha*x),torch.sqrt((beta**2)+1)))

27.         # num = num.to(device = 'cuda')

28.         # den = alpha * torch.sqrt((beta**2)+1)

29.         # den = den.to(device = 'cuda')

30.         x_tan = torch.div(torch.atan(torch.div((self.alpha*x),torch.sqrt((self.beta**2)+1))) ,self.alpha * torch.sqrt((self.beta**2)+1))

31.         out += self.shortcut(x_tan)

32.         out = F.relu(out)

33.         return out
</code></pre>

mhiyer: <p>Could this be elaborated? Why did this technique work? Thanks</p>