RuntimeError: size mismatch, m1: [1 x 6], m2: [1 x 3]

rim_zayani: <p>Hi, everyone. I am new to Pytorch. I’m trying to use a variational Autoencoder with simple data (a scalar as input). I’m using a batch size=6.<br>
I’m reading the data from a csv file, taken from Kaggle. I’m using only one column as input.<br>
Here is some of my code:</p>
<pre><code class="lang-auto">eq=pd.read_csv('/home/zayanir/Documents/data/database.csv')
latitude=eq['Latitude']
train_dl=torch.utils.data.DataLoader(latitude, batch_size=args.batch_size)
</code></pre>
<pre><code class="lang-auto">class VAE(nn.Module):
    def __init__(self):
        super(VAE, self).__init__()
        self.fc1=nn.Linear(1,3)
        self.fc2=nn.Linear(3,1)

    def encoder(self, x):
        moy=F.relu(self.fc1(x))
        variance=F.relu(self.fc1(x))
        return moy, variance
    
    def decoder(self, z):
        x=self.fc2(z)
        x=torch.sigmoid(x)
        return x
        
    def reparameterise(self, mu, var):
         std = torch.exp(0.5*var)
         eps = torch.randn_like(std) 
         return mu + eps*std
     
    def forward(self, x):
        mu,var=self.encoder(x)
        z=self.reparameterise(mu, var)
        x=self.decoder(z)
        return x, mu, var
</code></pre>
<p>The problem is that I get this error:</p>
<pre><code class="lang-auto">RuntimeError: size mismatch, m1: [1 x 6], m2: [1 x 3] at /opt/conda/conda-bld/pytorch_1525909934016/work/aten/src/TH/generic/THTensorMath.c:2033
</code></pre>
<p>I don’t understand why the model considers the input as a vector of 6 scalars when i’m using just a scalar as input with a batch_size=6.</p>
<p>Can you please help me?<br>
Thank you very much.</p>

alex.veuthey: <p>Could you print a more detailed stacktrace? So that we know which line throws the error.</p>
<p>Then we can print shapes for different variables (once we know which are implied) and debug this <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

rim_zayani: <p>Here is more detail about the error:</p>
<pre><code class="lang-auto">runfile('/home/zayanir/Documents/code/tuto/vae_1D.py', wdir='/home/zayanir/Documents/code/tuto')
Traceback (most recent call last):

  File "&lt;ipython-input-49-b9687dd3ae78&gt;", line 1, in &lt;module&gt;
    runfile('/home/zayanir/Documents/code/tuto/vae_1D.py', wdir='/home/zayanir/Documents/code/tuto')

  File "/home/zayanir/anaconda3/envs/myenv/lib/python3.6/site-packages/spyder_kernels/customize/spydercustomize.py", line 786, in runfile
    execfile(filename, namespace)

  File "/home/zayanir/anaconda3/envs/myenv/lib/python3.6/site-packages/spyder_kernels/customize/spydercustomize.py", line 110, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "/home/zayanir/Documents/code/tuto/vae_1D.py", line 107, in &lt;module&gt;
    mu, var, loss, inp, out= train(train_dl, args.learning_rate, latitude.size)

  File "/home/zayanir/Documents/code/tuto/vae_1D.py", line 65, in train
    out, mu, var=model(data)

  File "/home/zayanir/anaconda3/envs/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/zayanir/Documents/code/tuto/vae_1D.py", line 43, in forward
    mu,var=self.encoder(x)

  File "/home/zayanir/Documents/code/tuto/vae_1D.py", line 19, in encoder
    moy=F.relu(self.fc1(x))

  File "/home/zayanir/anaconda3/envs/myenv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/zayanir/anaconda3/envs/myenv/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 55, in forward
    return F.linear(input, self.weight, self.bias)

  File "/home/zayanir/anaconda3/envs/myenv/lib/python3.6/site-packages/torch/nn/functional.py", line 994, in linear
    output = input.matmul(weight.t())

RuntimeError: size mismatch, m1: [1 x 6], m2: [1 x 3] at /opt/conda/conda-bld/pytorch_1525909934016/work/aten/src/TH/generic/THTensorMath.c:2033
</code></pre>
<p>The shapes for different variables:</p>
<pre><code class="lang-auto">eq shape= (23412, 21)
latitude shape= (23412,)
Input x of the encoder: torch.Size([6]) 
</code></pre>
<p>Here is all the code:</p>
<pre><code class="lang-auto">import torch
from torchvision.datasets import ImageFolder
from torchvision import transforms
import torch.nn as nn
from torch.autograd import Variable
import torch.nn.functional as F
import pandas as pd
import argparse

class VAE(nn.Module):
    def __init__(self):
        super(VAE, self).__init__()
        self.fc1=nn.Linear(1,3)
        self.fc2=nn.Linear(3,1)
        
    def encoder(self, x):
        print(x.shape)
        moy=F.relu(self.fc1(x))
        variance=F.relu(self.fc1(x))
        return moy, variance
    
    def decoder(self, z):
        x=self.fc2(z)
        x=torch.sigmoid(x)
        return x
                
    def reparameterise(self, mu, var):
         std = torch.exp(0.5*var)
         eps = torch.randn_like(std) 
         return mu + eps*std
     
    def forward(self, x):
        mu,var=self.encoder(x)
        z=self.reparameterise(mu, var)
        x=self.decoder(z)
        return x, mu, var


def loss_fct(x, x_rec, mu, var):
    reconstruction_loss=nn.MSELoss()
    ER=reconstruction_loss(x_rec, x)
    KL=-0.5*torch.sum(1 + var - mu.pow(2) - var.exp())  
    return ER + KL

def train(train_dl, lr1, bs):
    model=VAE()    
    model = model.double() 
    optimizer=torch.optim.Adam(model.parameters(), lr=lr1)
    train_loss=0.0
    for idx, data in enumerate(train_dl):
        data= Variable(data)
        ############ forward ##############
        out, mu, var=model(data)
        loss=loss_fct(data, out, mu, var)
        train_loss+=loss.data[0]
        ############ backward ##############
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    return mu, var, train_loss/bs, data, out 
  
if __name__ == '__main__':
    
    parser = argparse.ArgumentParser(description='Autoencoder Sentinel DB')
    parser.add_argument('-bs','--batch-size', type=int, default=6, metavar='',
                    help='input batch size for training (default: 6)')
    parser.add_argument('-ep','--epochs', type=int, default=100, metavar='',
                    help='number of epochs to train (default: 100)')
    parser.add_argument('-lr','--learning-rate', type=float, default=0.001, metavar='',
                    help='learning rate (default: 0.001)')
    parser.add_argument('-oimg','--output-img', type=str, default='outp_images/output12.png', metavar='',
                    help='output image (default: outp_images/output12.png)')
    parser.add_argument('-iimg','--input-img', type=str, default='outp_images/input12.png', metavar='',
                    help='input image (default: outp_images/input12.png)')
    parser.add_argument('-outp','--output', type=str, default='out3.txt', metavar='',
                    help='output file (default: out.txt)')
    args = parser.parse_args()

    eq=pd.read_csv('/home/zayanir/Documents/data/database.csv')
    latitude=eq['Latitude']
    train_dl=torch.utils.data.DataLoader(latitude, batch_size=args.batch_size)
    print("eq shape=", eq.shape)
    print("latitude shape=", latitude.shape)
    for epoch in range(1,args.epochs+1):
        mu, var, loss, inp, out= train(train_dl, args.learning_rate, latitude.size)
        print(f'loss {epoch} = {loss}')
</code></pre>

alex.veuthey: <p>Okay, then this comes from the shape of your input. As it is <code>torch.Size([6])</code> and not <code>torch.Size([6, 1])</code>, the model thinks it has an input shape of <code>[features]</code> instead of <code>[batch_size]</code>.</p>
<p>Add <code>x = x.unsqueeze(1)</code> in your <code>encoder()</code> method. This will add a new dimension to your data, and make it clear to the model that it has an input of shape <code>[batch_size, features]</code>.</p>