RuntimeError: Expected object of device type cuda but got device type cpu for argument #2 'mat1' in call to _th_addmm

username: chaslie
cooked: <p>Hi,</p>
<p>I have just recieved this error when trying to feed a sample into a CVAE. I’m not sure what it means.<br>
The section which generates the error is below, with <strong>test_out=model.sample(test_input)</strong> seems to be line of interest.</p>
<pre><code class="lang-auto">class model(nn.Module):
    def __init__(self,lat_dim=50 ):
        super(model, self).__init__()
        self.Encoder=Encoder(lat_dim)
        self.Decoder=Decoder(lat_dim)
        self.cuda()
        self.lat_dim=lat_dim
    
    def sample(self, epsilon=None):
        if epsilon is None:
            epsilon = torch.randn(100, self.lat_dim)
            print("epsilon=", epsilon)
        return self.decode(epsilon, apply_sigmoid=True)

def gen_example(model,epoch,test_input):
    x_axis=np.arange(1,1024,1)
    test_out=model.cuda()
    test_out=model.sample(test_input) ## this is the bit it is failing on. 
    fig=plt.figure(figsize(5,5))
    for i in range(test_out.shape[0]):
        plt.subplot(5, 5, i+1)
        plt.plot(test_out[i, :, 0], x_axis)
    plt.show()
</code></pre>
<p>The testing loop is initiated here:</p>
<pre><code class="lang-auto">    if epoch % 1 == 0:
        for i , (test_x, target) in enumerate( test_loader2):
            test_x=test_x.unsqueeze(1)
            test_x=test_x.cuda()
            loss,out_t,logqz_x,logpz,logpx_z,z2=loss_fn(model,test_x)
            loss=torch.mean(loss)
        elbo=-loss
        print("epoch=",epoch,"elbo=",elbo)
        gen_example(model,epoch,random_vector_for_gen)
</code></pre>
<p>regards,</p>
<p>Chaslie</p>

username: ptrblck
cooked: <p>In your <code>sample</code> function you are creating a new tensor on the CPU:</p>
<pre><code class="lang-python">epsilon = torch.randn(100, self.lat_dim)
</code></pre>
<p>while the submodules seem to be on the GPU.<br>
You could push this tensor to the appropriate device by checking the <code>.device</code> attribute of a known parameter of e.g. <code>self.Decoder</code>:</p>
<pre><code class="lang-python">epsilon = torch.randn(100, self.lat_dim, device=self.Decoder.layer.param.device)
</code></pre>
<p>Let me know, if that helps or if another line of code is raising this issue.</p>

username: chaslie
cooked: <p>hi Ptrblck,</p>
<p>after much head scratching, i think the following seems to have solved it. However i am intrigued on the difference between this and your solution, which would be more stable?</p>
<blockquote>
<p>test_input=test_input.cuda()</p>
</blockquote>
<p>Chaslie</p>

username: ptrblck
cooked: <p>Both approaches don’t seem to target the same issue and my approach was just a guess.<br>
If your code works, then you’ve found and got rid of the error. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: chaslie
cooked: <p>Hi Ptrblk,</p>
<p>thanks for the help, I will need to buy you a beer one day <img src="https://discuss.pytorch.org/images/emoji/apple/beer.png?v=9" title=":beer:" class="emoji" alt=":beer:"> <img src="https://discuss.pytorch.org/images/emoji/apple/beers.png?v=9" title=":beers:" class="emoji" alt=":beers:"></p>
<p>I’ll let you know when i get the next problem debugged and it runs through <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>chaslie</p>

username: ptrblck
cooked: <p>Haha, I didn’t do anything and you figured it out, but we can surely have a beer someday. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: mathematics
cooked: <p>sometimes problem got same known error. But i’ve implemented all right<br>
I was implementing siamese network and here is my model</p>
<pre><code class="lang-auto">class Siamese(nn.Module):
    def __init__(self, vocab_size, embed_dim, hidden_dim):
        super().__init__()
        self.vocab_size = vocab_size
        self.embed_dim = embed_dim
        self.hidden_dim = hidden_dim
        
        self.fc = nn.Sequential(
            nn.Linear(self.hidden_dim, self.hidden_dim)
        )
        
    def forward(self, x1, x2):
        #n1
        lstm_out_1, _ = self.build_net(x1)
        lstm_out_1 = lstm_out_1.contiguous().view(-1, self.hidden_dim)
        l1 = self.fc(lstm_out_1)
        #n2
        lstm_out_2, _ = self.build_net(x2)
        lstm_out_2 = lstm_out_2.contiguous().view(-1, self.hidden_dim)
        l2 = self.fc(lstm_out_2)
        
        return f.normalize(l1), f.normalize(l2)
    
    def build_net(self, x):
        net = self.build_model(self.vocab_size,self.embed_dim, self.hidden_dim)(x)
        return net
        
    def build_model(self, vocab_size, embed_dim, hidden_dim):
        layers = []
        layers.append(nn.Embedding(vocab_size, embed_dim))
        layers.append(nn.LSTM(embed_dim, hidden_dim))
        return  nn.Sequential(*layers)
</code></pre>
<p>and i was experimenting if it will work for random values and did like this<br>
after taking all to cuda <code>device = torch.device('cuda')</code></p>
<p>build a random tensor as<code>input = torch.LongTensor([[2,4,12,13]]).to(device)</code> and passed to model as<br>
<code>Siamese(200,128, 32).to(device).forward(input,input)</code></p>
<p>got same</p>
<p><code>RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_index_select</code></p>
<p>but when removing all cuda things it works like perfectly</p>
<pre><code class="lang-auto">input = torch.LongTensor([[2,4,12,13]])
Siamese(200,128, 128).forward(input,input)

</code></pre>
<p><strike>also please dont mine but my yesterday <code>normalize</code> function, couldnt be passed giving error <code>LongTensor not supported for torch.div</code>  so i sticked  to standard <code>f.normalize</code> to do operations.  that <code>Embedding</code> dim not support for FloatTensor if passed.</strike></p>

username: mathematics
cooked: <p>and suddenly solved , Although it looks no mistake but <code>build_model</code> function was caused,<br>
I had manually defined that function in <code>__init__</code>   and this is solved</p>
<pre><code class="lang-auto">class Siamese(nn.Module):
    def __init__(self, vocab_size, embed_dim, hidden_dim):
        super().__init__()
        self.vocab_size = vocab_size
        self.embed_dim = embed_dim
        self.hidden_dim = hidden_dim
        self.fc = nn.Sequential(
            nn.Linear(self.hidden_dim, self.hidden_dim)
        )
        
        self.em = nn.Sequential(
        nn.Embedding(self.vocab_size, self.embed_dim),
        nn.LSTM(self.embed_dim, self.hidden_dim)
        )
        
    def forward(self, x1, x2):
        #n1
        lstm_out_1, _ = self.build_net(x1)
        print(lstm_out_1)
        lstm_out_1 = lstm_out_1.contiguous().view(-1, self.hidden_dim)
        l1 = self.fc(lstm_out_1)
        #n2
        lstm_out_2, _ = self.build_net(x2)
        lstm_out_2 = lstm_out_2.contiguous().view(-1, self.hidden_dim)
        l2 = self.fc(lstm_out_2)
        
        return f.normalize(l1), f.normalize(l2)
    
    def build_net(self, x):
        net = self.em(x)
        return net
</code></pre>
<p>but Why it was not working on previous code??</p>

username: ptrblck
cooked: <p>If you are creating modules in the <code>forward</code> method, they were not properly registered in the model creation and thus never pushed to the device via <code>model.cuda()</code> or <code>model.to()</code>.<br>
Besides that, the parameters of these modules are also most likely not passed to the optimizer and recreated using random values in each iteration.</p>
<p>In your last code snippet you are creating the additional layers in the <code>__init__</code>, which is the right approach to register them properly.</p>

username: mathematics
cooked: <p>Yes thanks, and previous question was also solved, need not to pass LongTensor because output of model is FloatTensor, which worked in custom normalize function, silly i was doing <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>