Why is this not being send to GPU?

username: Aknw_Fen
cooked: <p>I’m sending the net to GPU, still complains about not being on GPU.</p>
<p>This is a piece of it:</p>
<pre data-code-wrap="python"><code class="lang-python">class DynamicEncoder(nn.Module):
    """Encoder.

    Args:
        increments: list of input_channel-output_channel tuples.
        config: parameters for layers configuration.

    """

    def __init__(self, increments: list[tuple[int, int]], config: DAEConfig) -&gt; None:
        super().__init__()
        self.config = config
        self.pool = (
            nn.MaxPool2d(
                kernel_size=config.get("p_kernel"),
                stride=config.get("p_stride"),
                return_indices=True,
            )
            if self.config.get("use_pool")
            else None
        )
        c, b = create_layers(
            channels_list=increments,
            is_transpose=False,
            config=config,
        )
        self.convs = c
        self.batch_norms = b
        self.drop = nn.Dropout(0.3)
        self.dense = None

    def forward(
        self,
        x: Tensor,
    ) -&gt; tuple[Tensor, list[Tensor], list[tuple[str, torch.Size]]]:
        """Forward pass for Encoding.

        Returns:
            x: Tensor
            indices: From the convolution
            shapes: before each pooling, and before flattening.

        """
        pool_indices = []
        shapes = []
        for c, b in zip(self.convs, self.batch_norms, strict=False):
            shapes.append(("conv", x.size()))
            x = self.drop(F.leaky_relu(b(c(x))))
            if self.pool is not None:
                shapes.append(("pool", x.size()))
                x, index = self.pool(x)
                pool_indices.append(index)
        shapes.append(("none", x.size()))  # unflattened size.
        x = x.view(x.size(0), -1)
        dense_i, dense_o = sum(x.shape[1:]), 128
        if self.dense is None:
            self.dense = nn.Linear(dense_i, dense_o)
        x = self.drop(nn.functional.leaky_relu(self.dense(x)))
        shapes.append(("dense", (dense_i, dense_o)))

        return x, pool_indices, shapes
</code></pre>
<p>All I get is:</p>
<p>—&gt; 81         x = self.drop(nn.functional.leaky_relu(self.dense(x)))</p>
<p>i don’t know what to do</p>

username: JuanFMontesinos
cooked: <p>How do you send the model to gpu?<br>
All this code does is defining the model, but not the device.</p>

username: Aknw_Fen
cooked: <p>i use this:</p>
<pre><code class="lang-auto">def to_device_available(net: nn.Module) -&gt; str:
    """Send to device (GPU/MPS) if available."""
    device = "cpu"
    if torch.cuda.is_available():
        device = "cuda"
        if torch.cuda.device_count() &gt; 1:
            net = nn.DataParallel(net)
            return device
    elif torch.mps.is_available():
        device = "mps"
    net.to(device)
    return net, device
</code></pre>
<p>i also print params and says “cuda” <img src="https://discuss.pytorch.org/images/emoji/apple/sob.png?v=12" title=":sob:" class="emoji" alt=":sob:" loading="lazy" width="20" height="20"></p>

username: JuanFMontesinos
cooked: <p>net.to(device) is not in-place.<br>
You should do net=net.to(device)</p>

username: Aknw_Fen
cooked: <p>Ive tried that as well; apparently layers initialised within <code>forward</code> aren’t created in cuda (the Decoder part in this case).</p>
<p>Running a dummy input beforehand seems to fix it? (then sending the whole net to cuda.)</p>

username: JuanFMontesinos
cooked: <p>Oh I didn’t read the forward. forward is not the right place to initiallize the modules.<br>
Pytorch’s nn is a tree structure and pretty much all the functions that call model.paremeters() and iterating over this tree.<br>
Defining layers within forward is preventing all the methods before being aware that these layers exist and will create tons of silent issues. I would encourage you not to do this.</p>
<p>The most obvious is:<br>
if you defined the optimizer before these layers are intialized by doing<br>
<code>optimizer=some_opt(model.paremeters())</code><br>
The optimizer will not upgrade the layers with the gradients for those intialized in forward, which will cause your network not to perform well and will be hard to catch.</p>

username: Aknw_Fen
cooked: <p>Hi there, thanks for the comment. Let me assert how I understand this, then you can correct me.</p>
<p>I’ve an autoencoder:</p>
<pre data-code-wrap="python"><code class="lang-python">class AE:
     self.encoder = Encoder()
     self.decoder = None
</code></pre>
<p>decoder is None because it takes results  <code>self.encoder(..)</code> for  instantiation.</p>
<p>…<img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=12" title=":thinking:" class="emoji" alt=":thinking:" loading="lazy" width="20" height="20">  …</p>
<p>PS: as I was writing this I realised it could be modified to:</p>
<pre><code class="lang-auto">```python
class AE:
     self.encoder = Encoder(...)
     self.decoder = Decoder(...)
</code></pre>
<p>But,</p>
<ul>
<li>Aren’t there some cases where what I did would make sense, given a decoder that needs parameters returned from the encoder forward method to instantiate it <a class="mention" href="/u/juanfmontesinos">@JuanFMontesinos</a> ?</li>
</ul>
<p>This seems a valid case to me.</p>

username: JuanFMontesinos
cooked: <p>I don’t recall a clear case where you cannot define the architecture without encoder parameters.<br>
Could you provide an example?<br>
In addition, there are many “lazy” layers where you don’t know number of channels of the input to a module and it is initialized at runtime:<br>
<a href="https://pytorch.org/docs/stable/generated/torch.nn.LazyConv2d.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.nn.LazyConv2d.html</a><br>
Most operations in the latests pytorch have an analogous lazy layer (conv2d, conv1d, batchnorm, etcetera…)</p>

username: nickums
cooked: <p>I just assign my model and the batches .to  ‘cuda’ without problem.<br>
It can also be helpful to import accelerate   which helps assign dataloaders, losses, etc to the GPU</p>

username: Aknw_Fen
cooked: <p>you may be right there, i’ve got no example in my mind either</p>

username: Aknw_Fen
cooked: <p>fantastic package, ty! ig ill need to do some more searches; yet it’s ok to implement things once</p>
<p>taking a look here now as well… <a href="https://pytorch.org/ecosystem/" class="inline-onebox" rel="noopener nofollow ugc">Ecosystem | PyTorch</a></p>