Size mismatch, m1: [46 x 20], m2: [12 x 12]

username: JessicaJane
cooked: <p>I am really new to the machine Learnining Community and I was doing a Pytorch/Captum Tutorial. To be specific this one: <a href="https://captum.ai/tutorials/Titanic_Basic_Interpret" rel="nofollow noopener">https://captum.ai/tutorials/Titanic_Basic_Interpret</a><br>
Now I was trying to do the same just with another Dataset:<br>
<a href="https://www.kaggle.com/hb20007/gender-classification" rel="nofollow noopener">https://www.kaggle.com/hb20007/gender-classification</a><br>
I have changed the gender values to 0 for male and 1 for female</p>
<p>My Code looks like this:</p>
<pre><code class="lang-auto">dataset_path = "assets/Transformed Data Set - Sheet1.csv"
gender_data = pd.read_csv(dataset_path)

gender_data = pd.concat([gender_data,
                          pd.get_dummies(gender_data['Favorite Color'], prefix="color"),
                          pd.get_dummies(gender_data['Favorite Music Genre'],prefix="music"),
                          pd.get_dummies(gender_data['Favorite Beverage'],prefix="beverage"),
                          pd.get_dummies(gender_data['Favorite Soft Drink'],prefix="drink")], axis=1)
gender_data = gender_data.drop(['Favorite Color','Favorite Music Genre','Favorite Beverage','Favorite Soft Drink'], axis=1)


# Set random seed for reproducibility.
np.random.seed(131254)

# Convert features and labels to numpy arrays.
labels = gender_data["Gender"].to_numpy()
gender_data = gender_data.drop(['Gender'], axis=1)
feature_names = list(gender_data.columns)
data = gender_data.to_numpy()

# Separate training and test sets using 
train_indices = np.random.choice(len(labels), int(0.7*len(labels)), replace=False)
test_indices = list(set(range(len(labels))) - set(train_indices))
train_features = data[train_indices]
train_labels = labels[train_indices]
test_features = data[test_indices]
test_labels = labels[test_indices]

import torch
import torch.nn as nn
torch.manual_seed(1)  # Set seed for reproducibility.
class GenderSimpleNNModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear1 = nn.Linear(12, 12)
        self.sigmoid1 = nn.Sigmoid()
        self.linear2 = nn.Linear(12, 8)
        self.sigmoid2 = nn.Sigmoid()
        self.linear3 = nn.Linear(8, 2)
        self.softmax = nn.Softmax(dim=1)

    def forward(self, x):
        lin1_out = self.linear1(x)
        sigmoid_out1 = self.sigmoid1(lin1_out)
        sigmoid_out2 = self.sigmoid2(self.linear2(sigmoid_out1))
        return self.softmax(self.linear3(sigmoid_out2))

net = GenderSimpleNNModel()
USE_PRETRAINED_MODEL = False

if USE_PRETRAINED_MODEL:
    net.load_state_dict(torch.load('models/gender_model.pt'))
    print("Model Loaded!")
else:
    criterion = nn.CrossEntropyLoss()
    num_epochs = 200

    optimizer = torch.optim.Adam(net.parameters(), lr=0.1)
    input_tensor = torch.from_numpy(train_features).type(torch.FloatTensor)
    label_tensor = torch.from_numpy(train_labels)
    for epoch in range(num_epochs):    
        output = net(input_tensor)
        loss = criterion(output, label_tensor)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        if epoch % 20 == 0:
            print ('Epoch {}/{} =&gt; Loss: {:.2f}'.format(epoch+1, num_epochs, loss.item()))

    torch.save(net.state_dict(), 'models/gender_model.pt')
</code></pre>
<p>Now I have done it like in the Tutorial, but I get an error, to be specific I get this error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-7-c565b5b7e2f2&gt; in &lt;module&gt;
     13     label_tensor = torch.from_numpy(train_labels)
     14     for epoch in range(num_epochs):
---&gt; 15         output = net(input_tensor)
     16         loss = criterion(output, label_tensor)
     17         optimizer.zero_grad()

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--&gt; 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

&lt;ipython-input-6-dbf1be8585cd&gt; in forward(self, x)
     13 
     14     def forward(self, x):
---&gt; 15         lin1_out = self.linear1(x)
     16         sigmoid_out1 = self.sigmoid1(lin1_out)
     17         sigmoid_out2 = self.sigmoid2(self.linear2(sigmoid_out1))

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--&gt; 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/linear.py in forward(self, input)
     85 
     86     def forward(self, input):
---&gt; 87         return F.linear(input, self.weight, self.bias)
     88 
     89     def extra_repr(self):

~/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py in linear(input, weight, bias)
   1368     if input.dim() == 2 and bias is not None:
   1369         # fused op is marginally faster
-&gt; 1370         ret = torch.addmm(bias, input, weight.t())
   1371     else:
   1372         output = input.matmul(weight.t())

RuntimeError: size mismatch, m1: [46 x 20], m2: [12 x 12] at /opt/conda/conda-bld/pytorch_1570710822989/work/aten/src/TH/generic/THTensorMath.cpp:197
</code></pre>
<p>I am desperate. I don’t know where to put what, so that the sizes will match and I won’t get that error anymore. Please Help me  :S</p>

username: mailcorahul
cooked: <p>In the first line of <code>forward</code> method under your network <code>GenderSimpleNNModel</code>, you do <code>lin1_out = self.linear1(x)</code> which expects the input dims as 12.<br>
But from the exception <code>size mismatch, m1: [46 x 20], m2: [12 x 12] </code>, it seems you are feeding an input of dimension 20. That’s why the size mismatch. Make sure the 2nd dimension is of size 12.</p>

username: mailcorahul
cooked: <p>In addition to it, I made a few observations.</p>
<ol>
<li>You have a linear transformation <code>self.linear1 = nn.Linear(12, 12)</code> which simply takes a vector of dimension 12 and transforms it to the same dimension(i.e 12). You can avoid it and simply start from <code>self.linear2 = nn.Linear(12, 8)</code>.</li>
<li>Since you are using <code>nn.CrossEntropyLoss</code>, you should not use <code>softmax</code> at the end of your network’s forward pass. <code>nn.CrossEntropyLoss</code> combines <code>log_softmax</code> and <code>NLLLoss</code>.</li>
</ol>

username: JessicaJane
cooked: <p>Hi mailcorahul,</p>
<p>thanks for your help and notes! I appreciate it a lot!<br>
How can I change its size to 12 ?</p>
<p>To your addition:<br>
1 So I just simply change the call in <code>lin1_out = self.linear1(x)</code> to <code>lin1_out = self.linear2(x)</code> ?<br>
2. What else should I use ?</p>
<p>Thanks a lot !!!</p>

username: mailcorahul
cooked: <p>Can you do <code>print(input_tensor.size())</code> to see what is the feature size?<br>
Based on the feature size, you can change<br>
<code>self.linear1 = nn.Linear(12, 12)</code> to <code>self.linear1 = nn.Linear(feature_size, 12)</code>.</p>
<p>Regarding (2), you can simply do <code>return self.linear3(sigmoid_out2)</code></p>

username: JessicaJane
cooked: <aside class="quote no-group quote-modified" data-username="mailcorahul" data-post="5" data-topic="62751">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mailcorahul/48/15919_2.png" class="avatar"> mailcorahul:</div>
<blockquote>
<p>print(input_tensor)</p>
</blockquote>
</aside>
<pre><code class="lang-auto">tensor([[1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1.,
         0., 0.],
        [1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 1.,
         0., 0.],
        [1., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1.,
         0., 0.],
        [1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0.,
         0., 1.],
        [1., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0.,
         1., 0.],
        [0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 1., 0.,
         0., 0.],
        [0., 0., 1., 0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1.,
         0., 0.],
        [1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0.,
         0., 1.],
        [1., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0.,
         1., 0.],
        [0., 1., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 1.,
         0., 0.],
        [1., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1.,
         0., 0.],
        [0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0.,
         0., 0.],
        [0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 1.,
         0., 0.],
        [0., 0., 1., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 1., 0., 0., 1.,
         0., 0.],
        [1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.,
         0., 1.],
        [1., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 1., 0., 0., 0., 0., 1.,
         0., 0.],
        [0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.,
         1., 0.],
        [1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1.,
         0., 0.],
        [1., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 1.,
         0., 0.],
        [1., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 1., 1., 0.,
         0., 0.],
        [0., 1., 0., 0., 0., 1., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0.,
         1., 0.],
        [1., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 1., 0., 0., 0., 0., 0., 0.,
         0., 1.],
        [0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0.,
         1., 0.],
        [1., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0.,
         1., 0.],
        [0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 1.,
         0., 0.],
        [0., 1., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 1.,
         0., 0.],
        [0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0.,
         1., 0.],
        [1., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 1., 0., 0., 0., 0., 0., 1.,
         0., 0.],
        [1., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 1.,
         0., 0.],
        [0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 1., 0., 0., 0., 1., 0.,
         0., 0.],
        [0., 0., 1., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.,
         1., 0.],
        [1., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 1., 0., 1., 0.,
         0., 0.],
        [0., 0., 1., 0., 0., 0., 0., 0., 1., 0., 0., 1., 0., 0., 0., 0., 0., 1.,
         0., 0.],
        [0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 1., 0., 1.,
         0., 0.],
        [0., 1., 0., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 0., 1., 0.,
         0., 0.],
        [0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 1.,
         0., 0.],
        [0., 1., 0., 0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 1., 0.,
         0., 0.],
        [1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0.,
         0., 0.],
        [1., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 1., 0., 0., 0., 0., 0., 1.,
         0., 0.],
        [0., 1., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 1., 0., 1.,
         0., 0.],
        [0., 0., 1., 0., 1., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0.,
         1., 0.],
        [0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0.,
         0., 0.],
        [1., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 1.,
         0., 0.],
        [0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0.,
         1., 0.],
        [1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.,
         1., 0.],
        [0., 0., 1., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 0.,
         0., 1.]])
</code></pre>
<p>Thats the output when I put a the print command right unter the</p>
<pre><code class="lang-auto"> input_tensor = torch.from_numpy(train_features).type(torch.FloatTensor)
</code></pre>

username: JessicaJane
cooked: <p>And when I enter print(input_tensor.shape) the result is<br>
torch.Size([46, 20])</p>

username: mailcorahul
cooked: <p>My bad. Wanted to check size of input_tensor.<br>
In that case, you can change<br>
<code>self.linear1 = nn.Linear(12, 12)</code>  to  <code>self.linear1 = nn.Linear(20, 12)</code></p>

username: JessicaJane
cooked: <p>YOU my friend, are a GENIUS! Thanks a lot !! It worked!! I am so happy right now !!</p>

username: mailcorahul
cooked: <p>Lol, not at all <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
Glad it helped! Make sure you remove <code>softmax</code> at the end.<br>
Thanks!</p>

username: JessicaJane
cooked: <p>Yes I will !<br>
Thanks !!</p>