RGB image issue at the output of model

jbs06: <p>Hi!</p>
<p>I have trained a convolutional autoencoder with rgb images for 20 epochs (I have taken the architecture already used for faces) and wanted to see the results at the output. However I have some color distorsion. I am using matplotlib imshow to show images.</p>
<p>The code is the following:</p>
<pre data-code-wrap="analysis_loader"><code class="lang-analysis_loader">x, _ = next(analysis_loader)
z_points = model.encoder(x.unsqueeze(0).to(device))
reconst_imgs = model.decoder(z_points)

plt.figure(figsize=(5,5))
x = x.permute(1,2,0)
plt.imshow(x)

plt.figure(figsize=(5,5))
img = reconst_imgs.squeeze()
img = img.detach().cpu().permute(1,2,0)
plt.imshow(img)
</code></pre>
<p>The result is the following:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/e/eeb2127d4897f92e11ac7f8f36fa430638298d07.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/eeb2127d4897f92e11ac7f8f36fa430638298d07" title="Capture"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/e/eeb2127d4897f92e11ac7f8f36fa430638298d07_2_256x500.jpeg" alt="Capture" data-base62-sha1="y3B9FsGyGnkQ6OvslshqpcBPs8L" width="256" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/e/eeb2127d4897f92e11ac7f8f36fa430638298d07_2_256x500.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/e/e/eeb2127d4897f92e11ac7f8f36fa430638298d07.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/e/e/eeb2127d4897f92e11ac7f8f36fa430638298d07.jpeg 2x" data-dominant-color="C2C3C0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture</span><span class="informations">328×639 26.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I have tried to change channel order with numpy.dstack but it only changes the tone into blue, green, red,…</p>
<p>I don’t know if it is related with tensor format. When plotting plt.imshow does not complaing about clipping however the range of the first image is [0,1] while the output image is [0.98,0.05]. Do I have to normalize the image? Should I train the model more?</p>
<p>Thanks in advance.</p>

ptrblck: <p>I’m not sure if the output is due to some color clipping in <code>matplotlib</code> or rather your model.<br>
I’ve got similar pictures when my GANs were not training.</p>

jbs06: <p>Thank you for the answer <a class="mention" href="/u/ptrblck">@ptrblck</a>. I have been checking matplotlib and other tools and it is still showing me the same image. It must be due to the model. I have also tried to change some hyperparameters but it is still not working. This is the convolutional autoencoder I am using:</p>
<pre><code class="lang-auto">class Autoencoder (nn.Module):

    def __init__(self, en_out1=32, en_out2=64, en_out3=64, en_out4=64, dec_out1=64, dec_out2=64, dec_out3=32):
        super(Autoencoder, self).__init__()
        #Encoder parameters
        self.en_conv1 = nn.Conv2d(in_channels=3, out_channels=en_out1, kernel_size=3, padding=1)
        self.bn1 = nn.BatchNorm2d(en_out1)
        self.en_conv2 = nn.Conv2d(in_channels=en_out1, out_channels=en_out2, kernel_size=3, stride=2, padding=1)
        self.bn2 = nn.BatchNorm2d(en_out2)
        self.en_conv3 = nn.Conv2d(in_channels=en_out2, out_channels=en_out3, kernel_size=3, stride=2, padding=1)
        self.bn3 = nn.BatchNorm2d(en_out3)
        self.en_conv4 = nn.Conv2d(in_channels=en_out3, out_channels=en_out4, kernel_size=3, padding=1)
        self.bn4 = nn.BatchNorm2d(en_out4)
        self.lin1 = nn.Linear(en_out4 * 25 * 25, 200)

        # Decoder parameters
        self.lin2 = nn.Linear(200, en_out4 * 25 * 25)
        self.dec_conv1 = nn.ConvTranspose2d(in_channels=en_out4, out_channels=dec_out1, kernel_size=3, padding=1)
        self.bn5 = nn.BatchNorm2d(dec_out1)
        self.dec_conv2 = nn.ConvTranspose2d(in_channels=dec_out1, out_channels=dec_out2, kernel_size=3, stride=2, padding=1, output_padding=1)
        self.bn6 = nn.BatchNorm2d(dec_out2)
        self.dec_conv3 = nn.ConvTranspose2d(in_channels=dec_out2, out_channels=dec_out3, kernel_size=3, stride=2, padding=1,
                                           output_padding=1)
        self.bn7 = nn.BatchNorm2d(dec_out3)
        self.dec_conv4 = nn.ConvTranspose2d(in_channels=dec_out3, out_channels=3, kernel_size=3, padding=1)
        self.dropout = nn.Dropout(0.25)
        self.leakyrelu = nn.LeakyReLU(0.01)
    
    def encoder(self, x):
        x = self.dropout(self.bn1(self.leakyrelu(self.en_conv1(x))))
        x = self.dropout(self.bn2(self.leakyrelu(self.en_conv2(x))))
        x = self.dropout(self.bn3(self.leakyrelu(self.en_conv3(x))))
        x = self.dropout(self.bn4(self.leakyrelu(self.en_conv4(x))))
        x = x.view(x.size(0), -1)
        latent_space = self.lin1(x)
        
        return latent_space
    
    def decoder(self, latent_space):
        x = self.lin2(latent_space)
        x = x.view(x.size(0), 64, 25, 25)
        x = self.dropout(self.bn5(self.leakyrelu(self.dec_conv1(x))))
        x = self.dropout(self.bn6(self.leakyrelu(self.dec_conv2(x))))
        x = self.dropout(self.bn7(self.leakyrelu(self.dec_conv3(x))))
        x = self.dec_conv4(x)
        x = torch.sigmoid(x)
        
        return x

    def forward(self, x):
        latent_space = self.encoder(x)
        x_ = self.decoder(latent_space) 

        return x_
</code></pre>
<p>Model intialization, criterion and optimizer:</p>
<pre><code class="lang-auto"># ----------------------------------------------- DEFINE MODEL --------------------------------------------------------#
torch.manual_seed(42)
device = torch.device('cuda')
model = Autoencoder(en_out1=32, en_out2=64, en_out3=64, en_out4=64, dec_out1=64, dec_out2=64, dec_out3=32)
model.cuda(device)


# ------------------------------------------- CRITERION &amp; OPTIMIZER ---------------------------------------------------#
criterion = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=5e-4, betas=(.9, .99), weight_decay=1e-2)
</code></pre>
<p>And training:</p>
<pre><code class="lang-auto"># ------------------------------------------------ TRAINING ----------------------------------------------------------#
torch.set_grad_enabled(True)

n_epochs = 20
cost_list = []
COST = 0

for epoch in tqdm.tqdm(range(n_epochs)):
    COST = 0
    if epoch == 10:
        optimizer = torch.optim.Adam(model.parameters(), lr=2e-4, betas=(.9, .99), weight_decay=1e-2)
    for i, data in enumerate(train_loader):
        img, _ = data
        img = img.to(device)
        optimizer.zero_grad()
        z = model.forward(img)
        loss = criterion(z,img)
        loss.backward()
        optimizer.step()
        COST += loss.data
        if i % 50 == 0:
            print(loss)

    cost_list.append(COST)
    
    torch.save(model.state_dict(), 'model.pth')
</code></pre>
<p>Am I missing something?</p>
<p>Thank you in advance</p>