Please help me edit generator and discriminator for new size

ljeonjko: <p>Hi all!</p>
<p>I’m trying to edit a generator to input differently shaped images. At the moment, it is working for <code>8 x 8</code> images. I wish to transform them to accept “images” <code>8x1</code>. I’ve already tried changing the kernel size, stride, padding etc, but I keep getting</p>
<pre><code class="lang-auto">RuntimeError: Input type (torch.cuda.DoubleTensor) and weight type (torch.cuda.FloatTensor) should be the same

</code></pre>
<p>Here is the generator:</p>
<pre><code class="lang-auto">#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Wed Jan 12 12:46:54 2022

@author: leon
"""

import torch
import torch.nn as nn
import torch.optim as optim


""
class Downscale(nn.Module):
    def __init__(self, in_size, out_size, normalize = True, dropout = 0.0):
        super(Downscale, self).__init__()
        
        model = [nn.Conv2d(
            in_size,
            out_size,
            kernel_size = 4,
            stride = 2,
            padding = 1,
            bias = False
            )]
        
        if normalize:
            model.append(
                nn.BatchNorm2d(out_size, 0.8)
                )
            
        model.append(
            nn.LeakyReLU(0.2)
            )
        
        if dropout:
            
            model.append(
                nn.Dropout(dropout)
                )
        
        self.model = nn.Sequential(*model)
        
    def forward(self, x):
        return self.model(x)
##############################################################################       
# #############################################################################   
class Upscale(nn.Module):
    def __init__(self, in_size, out_size, dropout = 0.0):
        super(Upscale, self).__init__()
        
        model =[
            nn.ConvTranspose2d(
                in_size,
                out_size,
                kernel_size = 4,
                stride = 2,
                padding = 1,
                bias = False
                ),
            nn.BatchNorm2d(out_size, 0.8),
            nn.ReLU(inplace = True)
            ]
        
        if dropout:
            model.append(
                nn.Dropout(dropout)
                )
            
        self.model = nn.Sequential(*model)
        
    def forward(self, x, skip_input):
        x = self.model(x)
        out = torch.cat((x, skip_input), dim = 1)
        return out
##############################################################################       
# #############################################################################   

class Generator(nn.Module):
    def __init__(self, features_g, num_channels):
        super(Generator, self).__init__()
        self.features_g = features_g
        self.num_channels = num_channels
        self.build()
        
        
    def build(self):
        # input: 1 x 8 x 8   
        self.down4 = Downscale(
            in_size = self.num_channels,
            out_size = self.features_g,
            dropout = 0.5
            )
        
        # input: 8 x 4 x 4
        self.down5 = Downscale(
            in_size = self.features_g,
            out_size = self.features_g * 2,
            dropout = 0.5
            )
        
        # input: 18 x 2 x 2
        self.down6 = Downscale(
            in_size = (self.features_g * 2  + self.num_channels),
            out_size = self.features_g * 4,
            dropout = 0.5
            )
        ## state: 32 x 1 x 1  ##
        
        
        
        
        
        # input: 32 x 1 x 1
        self.up1 = Upscale(
            in_size = self.features_g * 4,
            out_size = self.features_g * 2,
            dropout = 0.5
            )

        # input: 16 x 2 x 2
        self.up2 = Upscale(
            in_size = (self.features_g * 4 + self.num_channels),
            out_size = self.features_g * 1, 
            dropout = 0.5
            )
        

        ## state: 8 X 4 X 4 ##
        
        final = [
            nn.Upsample(scale_factor = 2),
            
            # input: 8 X 8 X 8
            
            nn.Conv2d(
                in_channels = self.features_g * 2, 
                out_channels = self.num_channels,
                kernel_size = 3,
                stride = 1,
                padding = 1
                ),
            
            # input: 1 X 8 X 8
            
            #nn.Tanh()
            nn.Sigmoid()
            ]
         
        self.final = nn.Sequential(*final)
            
    def forward(self, input, constraint_map):
        
       
        d4 = self.down4(input)
        d5 = self.down5(d4)
        d5 = torch.cat((d5, constraint_map), dim = 1)
        d6 = self.down6(d5)
        u1 = self.up1(d6, d5)
        u2 = self.up2(u1, d4)
        
        return self.final(u2)

    def define_optim(self, learning_rate, beta1):
        self.optimizer = optim.Adam(self.parameters(), lr = learning_rate, betas = (beta1, 0.999))
     
    @staticmethod    
    def init_weights(layers):
        classname = layers.__class__.__name__
        
        if classname.find('Conv') != -1:
            nn.init.normal_(layers.weight.data, 0.0, 0.02)
            
        elif classname.find('BatchNorm') != -1:
            nn.init.normal_(layers.weight.data, 1.0, 0.02)
            nn.init.constant_(layers.bias.data, 0)
         

    
</code></pre>
<p>and the discriminator:</p>
<pre><code class="lang-auto">class Discriminator(nn.Module):
    def __init__(self, latent_vector_size, features_d, num_channels):
        super(Discriminator, self).__init__()
        
        self.latent_vector_size = latent_vector_size
        self.features_d = features_d
        self.num_channels = 1 #num_channels 
        self.optimizer = None
        self.main = None
    
    @staticmethod    
    def discriminator_block(in_filters, out_filters, stride, normalize):
        
        layers = [
            nn.Conv2d(
                in_channels = in_filters,
                out_channels = out_filters,
                kernel_size = 3,
                stride = stride,
                padding = 1                
                )
            ]
        if normalize:
            layers.append(
                nn.InstanceNorm2d(out_filters)
                )
        
        layers.append(
            nn.LeakyReLU(0.2, inplace = True)
            )
        
        return layers
        
        
    def build(self):
        
        layers = []
        in_filters = self.num_channels
        
        for out_filters, stride, normalize in [(self.features_d, 2, False)]:#, (self.features_d * 2, 2, True),
                                               #(self.features_d * 4, 2, True), (self.features_d * 8, 1, True)]:
        
            layers.extend(self.discriminator_block(
                in_filters,
                out_filters,
                stride,
                normalize
                ))
            
            in_filters = out_filters
        
        layers.append(
            nn.Conv2d(
                in_channels = out_filters,
                out_channels = 1,
                kernel_size = 3,
                stride = 1,
                padding = 1
                )
            )
        self.main = nn.Sequential(*layers)
        
    def forward(self, input):
        return self.main(input)
    
    def define_optim(self, learning_rate, beta1):
        self.optimizer = optim.Adam(self.parameters(), lr = learning_rate, betas = (beta1, 0.999))
    
    @staticmethod   
    def init_weights(layers):
        classname = layers.__class__.__name__
        if classname.find('Conv') != -1:
            nn.init.normal_(layers.weight.data, 0.0, 0.02)
            
        elif classname.find('BatchNorm') != -1:
            nn.init.normal_(layers.weight.data, 1.0, 0.02)
            nn.init.constant_(layers.bias.data, 0)
</code></pre>
<p>I know it’s a lot to ask, but I’m utterly stuck and don’t know how to proceed.<br>
Any help is greately appreciated <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

ptrblck: <p>The error message points to a <code>dtype</code> mismatch to make sure the model and inputs use the same <code>dtype</code> by transforming one or the other.<br>
The common approach is to use the default <code>float32</code> <code>dtype</code>, so cast your inputs via:</p>
<pre><code class="lang-python">input = input.float()
</code></pre>
<p>which should fix the error.</p>

ljeonjko: <p>Hi. Thanks for the reply. That seems to fix the problem, but for some reason, my <code>[batch_size, 1, 8, 1]</code> images are being interpreted as having a dimension of <code>[batch_size, 1, 8, 20]</code>. I’ll guess I’ll have to rewrite the whole thing and switch to Conv1d. Thanks anyaways <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

ptrblck: <p>Could you explain the issue in a bit more detail and what “interpret as having a dimension of” means?</p>

ljeonjko: <p>The code I’ve pasted works for images of dimensions <code>8 x 8</code>, but I’ve since discovered that my training data is not sampled representatively (due to gaps in original data), so I had to reduce the images to <code>8 x 1</code> to match the original data as closely as possible. This now leaves me with the issue of network being inadequate, so I reckon I’m better off rewriting all of it to deal with, essentially, one dimensional data (e.g. <code>Conv1d</code> instead of <code>Conv2d</code> etc.), than to modify the existing network piece by piece. The object-oriented programming of the network is more confusing to me than anything else really.</p>

ljeonjko: <p>On that note, mathematically, is there any difference between applying <code>Conv2d</code> on images <code>HxW = 8x1</code> nad applying <code>Conv1d</code> on data <code>size = 8</code>, if we keep the kernel size, padding and stride to result with the same output dimensions?</p>

ptrblck: <p>No, there is no difference in both approaches.</p>
<aside class="quote no-group" data-username="ljeonjko" data-post="5" data-topic="176389">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/b5ac83/48.png" class="avatar"> ljeonjko:</div>
<blockquote>
<p>The object-oriented programming of the network is more confusing to me than anything else really.</p>
</blockquote>
</aside>
<p>If using <code>nn.Module</code> objects is confusing, you could also use the pure functional API (e.g. via <code>F.conv2d</code>, <code>F.linear</code> etc.) and would need to store the parameters and buffers yourself.</p>

ljeonjko: <p>Thanks for the help! <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20"></p>