Using GPU ERROR

Sang_woon_Jeong: <p>Hello,</p>
<p>i using tansformer network.</p>
<p>This code bring in “<a href="https://github.com/LiamMaclean216/Pytorch-Transfomer" class="inline-onebox" rel="noopener nofollow ugc">GitHub - LiamMaclean216/Pytorch-Transfomer: My implementation of the transformer architecture from the Attention is All you need paper applied to time series.</a>”</p>
<p>I verify CPU version, so using GPU version.</p>
<p>Add to cuda .to(device)</p>
<p>BUt this ERROR</p>
<p>train_predict_outputs = model(train_data)</p>
<p>Traceback (most recent call last):<br>
File “/home/woon/anaconda3/envs/pytorch/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3417, in run_code<br>
exec(code_obj, self.user_global_ns, self.user_ns)<br>
File “”, line 1, in <br>
train_predict_outputs = model(train_data)<br>
File “/home/woon/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 550, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “/home/woon/06_pytorch/Reference_code_/Pytorch-Transfomer-master/Network.py”, line 74, in forward<br>
e = self.encs<a>0</a><br>
File “/home/woon/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 550, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “/home/woon/06_pytorch/Reference_code_/Pytorch-Transfomer-master/Network.py”, line 19, in forward<br>
a = self.attn(x)<br>
File “/home/woon/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 550, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “/home/woon/06_pytorch/Reference_code_/Pytorch-Transfomer-master/utils.py”, line 50, in forward<br>
a.append(h(x, kv = kv))<br>
File “/home/woon/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 550, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “/home/woon/06_pytorch/Reference_code_/Pytorch-Transfomer-master/utils.py”, line 30, in forward<br>
return attention(self.query(x), self.key(x), self.value(x))<br>
File “/home/woon/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 550, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “/home/woon/06_pytorch/Reference_code_/Pytorch-Transfomer-master/utils.py”, line 97, in forward<br>
x = self.fc1(x)<br>
File “/home/woon/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 550, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “/home/woon/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/linear.py”, line 87, in forward<br>
return F.linear(input, self.weight, self.bias)<br>
File “/home/woon/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/functional.py”, line 1612, in linear<br>
output = input.matmul(weight.t())<br>
RuntimeError: Expected object of device type cuda but got device type cpu for argument <span class="hashtag">#2</span> ‘mat2’ in call to _th_mm</p>
<p>i think  this class error</p>
<p>class AttentionBlock(torch.nn.Module):<br>
def <strong>init</strong>(self, dim_val, dim_attn):<br>
super(AttentionBlock, self).<strong>init</strong>()<br>
self.value = Value(dim_val, dim_val)<br>
self.key = Key(dim_val, dim_attn)<br>
self.query = Query(dim_val, dim_attn)</p>
<pre><code>def forward(self, x, kv = None):
    if(kv is None):
        #Attention with x connected to Q,K and V (For encoder)
        return attention(self.query(x), self.key(x), self.value(x))
    
    #Attention with x as Q, external vector kv as K an V (For decoder)
    return attention(self.query(x), self.key(kv), self.value(kv))
</code></pre>
<p>class MultiHeadAttentionBlock(torch.nn.Module):<br>
def <strong>init</strong>(self, dim_val, dim_attn, n_heads):<br>
super(MultiHeadAttentionBlock, self).<strong>init</strong>()<br>
self.heads = []<br>
for i in range(n_heads):<br>
self.heads.append(AttentionBlock(dim_val, dim_attn))</p>
<pre><code>    self.heads = nn.ModuleList(self.heads)
    
    self.fc = nn.Linear(n_heads * dim_val, dim_val, bias = False)
                  
    
def forward(self, x, kv = None):
    a = []
    for h in self.heads:
        a.append(h(x, kv = kv))
        
    a = torch.stack(a, dim = -1) #combine heads
    a = a.flatten(start_dim = 2) #flatten all head outputs
    
    x = self.fc(a)
    
    return x
</code></pre>
<p>but I dont know what to change.</p>