Help :( Auto encoder predicts shapes 'ok' but lacks colours

username: Aknw_Fen
cooked: <p>A simple autoencoder is learning to decode images, currently this is the output:</p>
<p><strong>Original</strong></p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/0/501da039c27c2a1804a96da5f766dbf4ec662e9a.png" alt="image" data-base62-sha1="bqJFAwZ2CInkJOFlhykrbbMjq3M" width="274" height="70"></p>
<p><strong>Predicted</strong></p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/4/14b25c6b390db546ea6fa6e1387afe12ea722600.png" alt="image" data-base62-sha1="2X5GxpTYDOWGtxSsHMNbIheuG4g" width="274" height="70"></p>
<p>Which is not great but I’d assume it will improve.</p>
<p>I’m somewhat concerned that there aren’t colours.</p>
<p>The input to the network (CIFAR10, i.e 3x32x32 images) is transformed as: <code>transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),</code></p>
<p>Those are sent to the network.</p>
<p>Hence plotting the images I do: <code>X/2 + 0.5</code>.</p>
<p>The same for the network output.</p>
<p>The snippet looks like this:</p>
<pre data-code-wrap="python"><code class="lang-python">  with torch.no_grad():
    imgs, labels = next(iter(data))
    
    net.eval()
    r = net(imgs)

    imgs = imgs / 2 + 0.5
    r = r / 2 + 0.5
    img_grid = torchvision.utils.make_grid(imgs)
    net_img_grid = torchvision.utils.make_grid(r)
    # print(imgs[0], r[0])
    writer.add_image("images1", img_grid)
    writer.add_image("images2", net_img_grid)
  writer.close()
</code></pre>
<p>The transformation made to the coloured images is exactly the same one made to the outputs of the network.</p>
<p>What I see in the printed tensors is:</p>
<pre data-code-wrap="txt"><code class="lang-txt"># image (note it stores both signs.)
tensor([[[ 0.2392,  0.2471,  0.2941,  ...,  0.0745, -0.0118, -0.0902],
         [ 0.1922,  0.1843,  0.2471,  ...,  0.0667, -0.0196, -0.0667],
         [ 0.1843,  0.1843,  0.2392,  ...,  0.0902,  0.0196, -0.0588],
]) 

# prediction: all negative values.
tensor([[[-0.0537, -0.1358, -0.1682,  ..., -0.1811, -0.1710, -0.0690],
         [-0.1587, -0.2982, -0.3134,  ..., -0.2757, -0.2924, -0.1612],
         [-0.2067, -0.3414, -0.3217,  ..., -0.2506, -0.2766, -0.1937],
         ...,])
</code></pre>
<p>Where it’s pretty clear that the network is not learning the sign correctly.</p>
<p>I thought this could be due to MSELoss, but I doubt since not predicting the right sign gives a much larger error. ( <code>-.2 -.3 != -.2+.3</code>)</p>
<p>The only reason I can think of as being odd or unsure about is using Drop2d as opposed to just Drop, and I’m unsure which should be best, or whether both should be used.</p>
<p>I’ve read the docs, and <a href="https://discuss.pytorch.org/t/what-is-the-difference-between-nn-dropout2d-and-nn-dropout/108192">check some related posts</a>, but to no avail.</p>
<p>Any ideas to try out?</p>

username: mycul
cooked: <p>Hi, could you share the architecture for your model/encoder/decoder? I would think that pytorch would stop you if you tried to input a three-channel image into a one channel input, but perhaps that is your issue and it’s interpreting the image as grayscale.</p>

username: Aknw_Fen
cooked: <p>Finally getting to understand the cause. It’s neither the network, nor the normalisation of data. The key seems to be the combination of optimizer and learning rate.</p>
<p>Adam gets to coloured images with small learning rate, SGD doesn’t and instead it collapses to black and white channels.</p>
<p>I wonder if SGD would need a much smaller learning rate.</p>
<p>With just a few epochs, it looks like this, and now the improvement mostly depends on latent vector and number of layers:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/0/7019bc2c6fea1f7a62bbddcde90c5ebc45affa76.png" data-download-href="https://discuss.pytorch.org/uploads/default/7019bc2c6fea1f7a62bbddcde90c5ebc45affa76" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/0/7019bc2c6fea1f7a62bbddcde90c5ebc45affa76.png" alt="image" data-base62-sha1="fZGC3oFBcA2Emfye3beEwube0No" width="274" height="274"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">274×274 53.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>