Error in Autoencoder

username: randinoo
cooked: <p>``class Autoencoder(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(Autoencoder, self).<strong>init</strong>()<br>
self.encoder = nn.Sequential(<br>
nn.Conv2d(1, 16, 3, stride=2, padding=1),<br>
nn.ReLU(),<br>
nn.Conv2d(16, 32, 3, stride=2, padding=1),<br>
nn.ReLU(),<br>
nn.Conv2d(32, 64, 3)<br>
)<br>
self.decoder = nn.Sequential(<br>
nn.ConvTranspose2d(64, 32, 3),<br>
nn.ReLU(),<br>
nn.ConvTranspose2d(32, 16, 3, stride=2, padding=1, output_padding=1),<br>
nn.ReLU(),<br>
nn.ConvTranspose2d(16, 1, 3, stride=2, padding=1, output_padding=1),<br>
nn.Sigmoid()<br>
)</p>
<pre><code>def forward(self, x):
    x = self.encoder(x)
    x = self.decoder(x)
    return x`
</code></pre>
<pre><code class="lang-auto">
the error in loss_train = criterion(output_train, y_train.long())

RuntimeError: only batches of spatial targets supported (3D tensors) but got targets of dimension: 1</code></pre>

username: ptrblck
cooked: <p>Since you are passing the targets as <code>LongTensor</code>s, I assume you are using <code>nn.CrossentropyLoss</code>.<br>
Also based on the posted architecture it seems you are working on a multi-class segmentation use case.<br>
If that’s the case, the model output is expected to contain logits (so remove the <code>sigmoid</code>) in the shape <code>[batch_size, nb_classes, height, width]</code>, while the target should be a <code>LongTensor</code> in the shape <code>[batch_size, height, width]</code> containing the class indices in the range <code>[0, nb_classes-1]</code>.</p>

username: randinoo
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> , it’s the same thing</p>
<pre><code class="lang-auto">class Autoencoder(nn.Module):
    def __init__(self):
        super(Autoencoder, self).__init__()
        self.encoder = nn.Sequential(
            nn.Conv2d(1, 16, 3, stride=2, padding=1),
            nn.ReLU(),
            nn.Conv2d(16, 32, 3, stride=2, padding=1),
            nn.ReLU(),
            nn.Conv2d(32, 64, 3)
        )
        self.decoder = nn.Sequential(
            nn.ConvTranspose2d(64, 32, 3),
            nn.ReLU(),
            nn.ConvTranspose2d(32, 16, 3, stride=2, padding=1, output_padding=1),
            nn.ReLU(),
            nn.ConvTranspose2d(16, 1, 3, stride=2, padding=1, output_padding=1),
            
        )

    def forward(self, x):
        x = self.encoder(x)
        x = self.decoder(x)
        return x

</code></pre>
<pre><code class="lang-auto">x_train, y_train = Variable(train_x), Variable(train_y)
   
    x_val, y_val = Variable(val_x), Variable(val_y)
  
    if torch.cuda.is_available():
      
        x_train = x_train.cuda()
        y_train = y_train.cuda()
        x_val = x_val.cuda()
        y_val = y_val.cuda()

  
    optimizer.zero_grad()

    
    output_train = model(x_train.float())
    output_val = model(x_val.float())
    

   
    loss_train = criterion(output_train, y_train.long())
</code></pre>
<p>help me plzz <img src="https://discuss.pytorch.org/images/emoji/apple/pensive.png?v=9" title=":pensive:" class="emoji" alt=":pensive:"></p>

username: pchandrasekaran
cooked: <p>Are x_train, y_train, x_val, y_val batched inputs or only a single sample? That may explain the shape/batch error.</p>
<p>Also, is this is an image autoencoder? If it is, then:</p>
<ol>
<li>Normalize your inputs</li>
<li>No need to pass targets as long.</li>
<li>Use MSELoss() as criterion</li>
</ol>

username: randinoo
cooked: <p>X_train in input, and y_train is output</p>
<p>hwo normalize my input?</p>

username: pchandrasekaran
cooked: <p>Here is a <a href="https://colab.research.google.com/drive/10xvESCwxFcALLCTpZAXfWT2SlgwO5u_r?usp=sharing" rel="noopener nofollow ugc">link</a> to a beginner friendly Image Autoencoder template that I have written. The same concept can be applied to any image feature extracting autoencoder. Please feel free to PM if you have any doubts as to what the code does, I’ll be happy to help.</p>
<p>Regards,</p>

username: randinoo
cooked: <p>RuntimeError: Calculated padded input size per channel: (1 x 1). Kernel size: (3 x 3). Kernel size can’t be greater than actual input size</p>

username: pchandrasekaran
cooked: <p>That would be because your inputs are too small. Since, that notebook is based on MNIST, the minimum input size needs to be 28x28.</p>

username: randinoo
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/4/549bb778c94aab7e5fc33ba34d4a88732d8579bb.png" data-download-href="https://discuss.pytorch.org/uploads/default/549bb778c94aab7e5fc33ba34d4a88732d8579bb" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/4/549bb778c94aab7e5fc33ba34d4a88732d8579bb.png" alt="image" data-base62-sha1="c4tJcctx9gziHlczkdkoAUiB1ft" width="690" height="245" data-dominant-color="F6F6F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">862×307 12.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: randinoo
cooked: <p>hi <a class="mention" href="/u/pchandrasekaran">@pchandrasekaran</a>  when I tested this code GAN,</p>
<pre><code class="lang-auto">class DiscriminatorNet(torch.nn.Module):
    """
    A three hidden-layer discriminative neural network
    """
    def __init__(self):
        super(DiscriminatorNet, self).__init__()
        n_features = 40
        n_out = 2
        
        self.hidden0 = nn.Sequential( 
            nn.Linear(n_features, 1024),
            nn.LeakyReLU(0.2),
            nn.Dropout(0.3)
        )
        self.hidden1 = nn.Sequential(
            nn.Linear(1024, 512),
            nn.LeakyReLU(0.2),
            nn.Dropout(0.3)
        )
        self.hidden2 = nn.Sequential(
            nn.Linear(512, 256),
            nn.LeakyReLU(0.2),
            nn.Dropout(0.3)
        )
        self.out = nn.Sequential(
            torch.nn.Linear(256, n_out),
            torch.nn.Sigmoid()
        )

    def forward(self, x):
        x = self.hidden0(x)
        x = self.hidden1(x)
        
        x = self.hidden2(x)
        x = self.out(x)
        return x

</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/6/76c128bbe6cf43d81c2e9598a9d05c7eaaf9fc63.png" data-download-href="https://discuss.pytorch.org/uploads/default/76c128bbe6cf43d81c2e9598a9d05c7eaaf9fc63" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/6/76c128bbe6cf43d81c2e9598a9d05c7eaaf9fc63.png" alt="image" data-base62-sha1="gWyblv5CA1VM5CXGEHC78CwA8nh" width="690" height="301" data-dominant-color="F6F5F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">852×372 17 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>help me plzz</p>

username: pchandrasekaran
cooked: <p>Now, I’m making an assumption here as I have limited information; I’m assuming you want a single output in the range (0, 1) and are tackling a binary classification problem.</p>
<ol>
<li>Change n_out to 1 and use nn.BCELoss(). <em>[If n_out is 2, use nn.NLLLoss(), and some extra changes are needed, so leave it for now]</em>
</li>
<li>If n_out=1, you’ll need to binarize the output from the network in order to use with sklearn’s accuracy_score since a sigmoided output is going to be a float in the interval [0, 1]. You can do that by:</li>
</ol>
<pre><code class="lang-auto">threshold = 0.5
network_output[network_output &gt; threshold] = 1
network_output[network_output &lt;= threshold] = 0
</code></pre>

username: randinoo
cooked: <p><a class="mention" href="/u/pchandrasekaran">@pchandrasekaran</a><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/2/5/255c137093a7e60855d64ee04ade3fe9a3328f51.png" alt="image" data-base62-sha1="5kuWap8ZVrEpKSifDtgcG8wCKOJ" width="682" height="135"></p>

username: randinoo
cooked: <p><a class="mention" href="/u/pchandrasekaran">@pchandrasekaran</a>  <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
RuntimeError: The size of tensor a (16) must match the size of tensor b (5177) at non-singleton dimension 3</p>
<p>with this code</p>
<pre><code class="lang-auto">class Autoencoder(nn.Module):
    def __init__(self):
        super(Autoencoder, self).__init__()
        self.encoder = nn.Sequential(
            nn.Conv2d(1, 16, 3, stride=2, padding=1),
            nn.ReLU(),
            nn.Conv2d(16, 32, 3, stride=2, padding=1),
            nn.ReLU(),
            nn.Conv2d(32, 64, 3)
        )
        self.decoder = nn.Sequential(
            nn.ConvTranspose2d(64, 32, 3),
            nn.ReLU(),
            nn.ConvTranspose2d(32, 16, 3, stride=2, padding=1, output_padding=1),
            nn.ReLU(),
            nn.ConvTranspose2d(16, 1, 3, stride=2, padding=1, output_padding=1),
            
        )

    def forward(self, x):
        x = self.encoder(x)
        x = self.decoder(x)
        return x
</code></pre>
<h1>
<a name="clearing-the-gradients-of-the-model-parameters-1" class="anchor" href="#clearing-the-gradients-of-the-model-parameters-1"></a>clearing the Gradients of the model parameters</h1>
<pre><code>optimizer.zero_grad()

# prediction for training and validation set

output_train = model(x_train.float())

output_val = model(x_val.float())


loss_train = criterion(output_train, y_train.long())

loss_val = criterion(output_val, y_val.long())</code></pre>

username: ptrblck
cooked: <p>Your model works correctly using random input shapes, so I guess the shape mismatch is caused in the loss calculation, in which case you would have to check the shapes of the model output and target tensor and make sure they have the expected shapes.<br>
I don’t know which <code>criterion</code> you are using, but the docs explain the expected shapes for them.</p>