I cannot get my model's weights to match the input type

GustavoVargasHakim: <p>Hi everyone,</p>
<p>I am working on a Neuroevolution project. I am building a model which is composed by several DenseBlocks, however, the skip connections of each of the denseblocks may vary. I tell you all of this to put into context, that the forward pass is built dynamically.</p>
<p>My code is the following:</p>
<pre><code class="lang-auto">'''Networks class'''
class CNN(nn.Module):
  def __init__(self, e, denseBlocks, links, classifier, init_weights = True):
    super(CNN, self).__init__()
    extraction = []
    for block in denseBlocks:
      for layer in block:
        extraction += layer
    self.extraction = nn.Sequential(*extraction)
    self.classifier = nn.Sequential(*classifier)
    self.denseBlocks = denseBlocks
    self.links = links
    self.connections = e.second_level
    self.first_level = e.first_level
    self.nblocks = e.n_block
    
    
  def forward(self, x):
    '''Feature extraction'''
    for i in range(self.nblocks):
        block = self.denseBlocks[i]
        connections = self.connections[i]
        link = self.links[i]
        prev = -1
        pos = 0
        outputs = []
        for j in range(self.first_level[i]['nconv']):
          if j == 0 or j == 1:
            x = nn.Sequential(*block[j])(x)
            outputs.append(x)
          else:
            conn = connections[pos:pos+prev]
            for c in range(len(conn)):
              if conn[c] == 1:
                x2 = outputs[c]
                x = torch.cat((x, x2), axis = 1)
            x = nn.Sequential(*block[j])(x)
            outputs.append(x)
            pos += prev
          prev += 1
        x = nn.Sequential(*link)(x)

    x = torch.flatten(x,1)
    '''Classification'''
    x = self.classifier(x)
    return nn.functional.log_softmax(x, dim=1)
</code></pre>
<p>Inside the <code>__init__</code>, I add the convolutional part (i.e. all the denseblocks and transition operations) as <code>self.extraction</code> and the classification part as <code>self.classification</code>. In the forward pass, I use the information inside <code>self.denseBlock</code> to dynamically pass the input <code>x</code> through the convolutional layers inside each block, until the end of the CNN.</p>
<p>The problem is that I want to pass a random tensor just to see if the network computes correctly. I have:</p>
<pre><code class="lang-auto">#Create network
net = CNN(e, network[0], network[1], network[2])
net.to(device, dtype = torch.float32)

#Create random tensor in cuda
image = torch.rand([1, 1, 256, 256], device = device)
net(image)
</code></pre>
<p>I what I get is:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-83-0987cf7a2c44&gt; in &lt;module&gt;()
----&gt; 1 net(image)

6 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight)
    418                             _pair(0), self.dilation, self.groups)
    419         return F.conv2d(input, weight, self.bias, self.stride,
--&gt; 420                         self.padding, self.dilation, self.groups)
    421 
    422     def forward(self, input: Tensor) -&gt; Tensor:

RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
</code></pre>
<p>I have tried many combinations, like the <code>.cuda()</code>, not using the <code>dtype = torch.float32</code> but I have no idea how to solve it. I have read that possibly not all the parts of the network are on the GPU. But I have defined two sequentials already, and the modules I use in the forward pass are the same that are inside the sequentials.</p>
<p>Thanks in advance for your help!</p>

ptrblck: <p>Could you check, if all passed arguments are <code>nn.Module</code> objects, i.e. <code>denseBlocks</code>, <code>links</code>, and <code>classifier</code>?<br>
It’s unclear from the error message which module is raising this issue and the code looks generally alright.<br>
Wrapping the modules in <code>nn.Sequential</code> inside the <code>forward</code> is unusual, but shouldn’t change the device.</p>
<p>Feel free to add the missing module definitions in case you get stuck, so that we can help debugging. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

GustavoVargasHakim: <p>Thank you very much! You were right! The elements inside the links list were not added correctly. I used the nn.ModuleList instead of a normal list and the problem finally solved! <img src="https://discuss.pytorch.org/images/emoji/apple/raised_hands.png?v=9" title=":raised_hands:" class="emoji" alt=":raised_hands:"></p>