Binary classification Different input sizes error

username: Tornike
cooked: <p>Hello, doing binary classification with BCELoss()</p>
<p>this is my dataset gettiem()</p>
<blockquote>
<p>def <strong>getitem</strong>(self,index):<br>
x=torch.tensor(self.df.iloc[index,:-1])<br>
y_label=torch.tensor(self.df.iloc[index,-1])<br>
return (x.float(),y_label.float())</p>
</blockquote>
<p>this is my Model</p>
<pre><code class="lang-auto">
class Credit_Module(nn.Module):
    def __init__(self):
        super().__init__()
#if i have 20 columns should i give first layer 20 input features?
        self.fc1=nn.Linear(20,50)
        self.fc2=nn.Linear(50,80)
        self.fc3=nn.Linear(80,20)
        self.fc4=nn.Linear(20,2)
        
    
    def forward(self,x):
        x=F.relu(self.fc1(x))
        x=F.relu(self.fc2(x))
        x=F.relu(self.fc3(x))
        x=self.fc4(x)
        print(x.shape)
        return x
</code></pre>
<p>This is The Error</p>
<pre><code class="lang-auto">

                    y_pred=model(X_train)
     11             print(y_train.shape)
---&gt; 12             loss=criterion(y_pred,y_train)
ValueError: Using a target size (torch.Size([10])) that is different to the input size (torch.Size([10, 2])) is deprecated. Please ensure they have the same size.


</code></pre>
<p>it gets different sizes. how can i make them same size?<br>
Should i change a Loss function ? or something in my model?<br>
Train loader batch size is 10</p>

username: RaLo4
cooked: <p>I don’t really know what you are trying to do here using BCELoss, so I will guess according to the info you have given.<br>
Since you said you want to do binary classification and your target is of size <code>torch.Size([10])</code>, I’m guessing it is filled with ones and zeros and you want your network to predict a number between <code>0.0</code> and <code>1.0</code> aka a probability.<br>
For this to happen you need to change the following:</p>
<aside class="quote no-group" data-username="Tornike" data-post="1" data-topic="119583">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tornike/48/36047_2.png" class="avatar"> Tornike:</div>
<blockquote>
<p><code>self.fc4=nn.Linear(20,2)</code></p>
</blockquote>
</aside>
<p>to <code>self.fc4=nn.Linear(20,1)</code><br>
and</p>
<aside class="quote no-group" data-username="Tornike" data-post="1" data-topic="119583">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tornike/48/36047_2.png" class="avatar"> Tornike:</div>
<blockquote>
<p><code>return x</code></p>
</blockquote>
</aside>
<p>to <code>return x.view(-1, 1).squeeze(1)</code></p>
<p>This will give you logits as output, but they can be in a greater range than just 0 - 1.<br>
So if you want the probability (0-1) instead of the logits you need to either add a <code>nn.Sigmoid()</code> layer to the end of your network or use <code>nn.BCEWithLogitsLoss</code>  instead of <code>nn.BCELoss</code>.<br>
Which is just BCELoss with a builtin sigmoid.<br>
Using <code>nn.BCEWithLogitsLoss</code> is the recommended option. You can read about it <a href="https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html?highlight=bce#torch.nn.BCEWithLogitsLoss" rel="noopener nofollow ugc">here</a>.</p>

username: Tornike
cooked: <p>Thanks for your feedback.<br>
I did what you advised and it worked.<br>
But when i evaluate the model it always returns -1.somethings so the predicted value never gets 0.</p>
<p><code>self.fc4=nn.Linear(20,2)</code></p>
<p>i had that 2 because i wanted to return tensor of 2 probabilities of 2 classes and to return higher probability of these 2 as a prediction.<br>
again, what you advised, it worked but it is always predicting that ‘1’ class</p>

username: RaLo4
cooked: <aside class="quote no-group" data-username="Tornike" data-post="4" data-topic="119583">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tornike/48/36047_2.png" class="avatar"> Tornike:</div>
<blockquote>
<p>i had that 2 because i wanted to return tensor of 2 probabilities of 2 classes and to return higher probability of these 2 as a prediction.<br>
again, what you advised, it worked but it is always predicting that ‘1’ class</p>
</blockquote>
</aside>
<p>Like I wrote in the above answer, I just assumed you had 1 class because your target tensor is of size <code>torch.Size([10])</code>.<br>
If you want to have two classes than this is were your problem lies and not with the model.<br>
If you have 2 classes, the target tensor would have to be of size <code>torch.Size([10, 2])</code></p>
<p>Just make sure you really want 2 classes. If you want a binary classification were the final output is either True or False, than a one dimensional output with a probability between 0.0 and 1.0 (which you get by putting the modles output into a sigmoid function) is what most people would use.</p>
<p>If you still want to have 2 classes, than your first model was totally fine as it was.<br>
You just need your target to have 2 classes (be 2 dimensional) as well</p>

username: Tornike
cooked: <p>Thanks a lot.<br>
I’m new in deep learning so it is still a little bit confusing for me.<br>
Just to make sure, classification should return either good ‘0’ and bad ‘1’ credit risk. I thought it was working like i said, but your answer makes more sense.</p>
<p>So i did like you said and it worked, but i have a very low accuracy. because it always returns one class. Maybe the problem is in dataset, i don’t know.</p>
<p>Anyways, Thanks for your time.</p>

username: RaLo4
cooked: <aside class="quote no-group" data-username="Tornike" data-post="6" data-topic="119583">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tornike/48/36047_2.png" class="avatar"> Tornike:</div>
<blockquote>
<p>Thanks a lot.</p>
</blockquote>
</aside>
<p>No problem.</p>
<p>I just want to quickly summarize everything for you, in order to not confuse you.</p>
<blockquote>
<p>Your network outputs, so called, logits, that are <strong>not</strong> in range 0-1.</p>
</blockquote>
<blockquote>
<p>These logits can be put into a 0-1 range by either passing them trough a <code>nn.Sigmoid()</code> layer at the end of your net, or by passing the logits through <code>torch.sigmoid()</code>.</p>
</blockquote>
<blockquote>
<p>If you use <code>nn.BCELoss</code> as your loss, you need the <code>nn.Sigmoid()</code> layer at the end of your net. But it is recommended to use <code>nn.BCEWithLogitsLoss</code> as your loss and <strong>not</strong> have a <code>nn.Sigmoid()</code> layer at the end of your net.</p>
</blockquote>
<blockquote>
<p>When using the recommended <code>nn.BCEWithLogitsLoss</code>, <code>torch.sigmoid()</code> is used during testing/interference. Since <code>nn.BCEWithLogitsLoss</code> takes in logits you build your net to output logits, but during testing/interference we want the 0-1 probability so we pass the nets logits through <code>torch.sigmoid()</code>.</p>
</blockquote>
<p>Hope this helps</p>

username: Tornike
cooked: <p>I did like you recommended, with <code>nn.BCEWithLogitsLoss</code> and <code>torch.sigmoid</code> at testing and it is working like it is supposed.</p>
<p>Once again, Thank you very much for Great, deep explanation.</p>

username: bbasaran
cooked: <p>Hello <a class="mention" href="/u/ralo4">@RaLo4</a> , I have the same problem but could not solve with this. Would you help me, too? I summarized my network below basically:</p>
<pre><code class="lang-auto">class Predictor(pl.LightningModule):
  def __init__(self, n_features):
    super().__init__()
    self.model = LSTM(n_features)
    self.criterion = nn.BCELoss()

  def forward(self, x, labels=None): 
    output = self.model(x)
    loss = 0
    if labels is not None:
      loss = self.criterion(output, labels)
    return loss, output

class LSTM(nn.Module): 
  def __init__(self, n_features, n_hidden=256, n_layers=3):        
    super().__init__()
    self.lstm = nn.LSTM( ... )
    self.classifier = nn.Sigmoid()            
  
  def forward(self, x):
    self.lstm.flatten_parameters()
    _, (hidden, _) = self.lstm(x)
    output = hidden[-1]
    return self.classifier(output)
</code></pre>
<p>This architecture causes the same error (batch size = 64):</p>
<blockquote>
<p>Using a target size (torch.Size([64])) that is different to the input size (torch.Size([64, 256])) is deprecated. Please ensure they have the same size.</p>
</blockquote>

username: RaLo4
cooked: <p>This is extreme difficult to answer since I do not know what your input looks like nor do I know what your desired output or labels look like.<br>
But you can easily debug this yourself. All the error is telling you, is that your networks output should be the same size as your labels, but it currently is not.<br>
Your labels are a one dimensional tensor of size <code>[64]</code> and your output is a two dimensional tensor of size <code>[64, 256]</code>.<br>
So you need to make sure that this output</p>
<aside class="quote no-group" data-username="bbasaran" data-post="9" data-topic="119583">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bbasaran/48/37602_2.png" class="avatar"> bbasaran:</div>
<blockquote>
<p><code>output = self.model(x)</code></p>
</blockquote>
</aside>
<p>Becomes the shape you want it to be (which is <code>torch.Size([64])</code> guessing by looking at the labels shape).<br>
You are using only the <code>h_n</code> output of your pytorchs lstm.</p>
<aside class="quote no-group" data-username="bbasaran" data-post="9" data-topic="119583">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bbasaran/48/37602_2.png" class="avatar"> bbasaran:</div>
<blockquote>
<p><code>_, (hidden, _) = self.lstm(x)</code></p>
</blockquote>
</aside>
<p>So I checked the <a href="https://pytorch.org/docs/stable/generated/torch.nn.LSTM.html#torch.nn.LSTM" rel="noopener nofollow ugc">documentation</a> and for the shape of <code>h_n</code> is states:</p>
<blockquote>
<ul>
<li><strong>h_n</strong> of shape (num_layers * num_directions, batch, hidden_size): tensor containing the hidden state for t = seq_len. If <code>proj_size &gt; 0</code> was specified, <code>h_n</code> shape will be (num_layers * num_directions, batch, proj_size).Like <em>output</em>, the layers can be separated using <code>h_n.view(num_layers, num_directions, batch, hidden_size)</code> and similarly for <em>c_n</em>.</li>
</ul>
</blockquote>
<p>You need to either adjust the network or change how you are handling the output (it depends on what you are trying to do) for it to work.<br>
Debugging shapes is not do difficult. You can use something like <code>print(output.shape)</code> or <code>print(output.size())</code> to print the shapes at different points in your code</p>

username: bbasaran
cooked: <p>Thank you for the answer <a class="mention" href="/u/ralo4">@RaLo4</a> ! As you have said, the problem is the inconsistency between the shapes of “hidden” and “labels”. Actually, I have just explained my problem here better: <a href="https://discuss.pytorch.org/t/problem-with-converting-my-lstm-multi-label-classification-model-to-a-binary-classification-model/119995" class="inline-onebox">Problem with converting my LSTM multi-label classification model to a binary classification model</a><br>
I have been trying to migrate my multi-class classification model to a binary classification model. In the former case, I was using “CrossEntropyLoss” and “Linear” Activation and there was no problem. As you can see now, problem arises when I alter them to “BCELoss” and “Sigmoid” Activation. Maybe instead of reshaping the output, I have to change the forward method of LSTM class from scratch, since it was written for a multi-class classification model?</p>

username: RaLo4
cooked: <p>I have looked at the other post you linked.<br>
Having read what you posted there, I think you were already on the right track.</p>
<p>I am not sure that what I am going to say now is “the final answer”, but concerning the shape issue, it would work if you combine your previous classifier</p>
<aside class="quote no-group" data-username="bbasaran" data-post="3" data-topic="119995">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bbasaran/48/37602_2.png" class="avatar"><a href="https://discuss.pytorch.org/t/problem-with-converting-my-lstm-multi-class-classification-model-to-a-binary-classification-model/119995/3">Problem with converting my LSTM multi-class classification model to a binary classification model</a></div>
<blockquote>
<p>self.classifier = nn.Linear(n_hidden, n_classes) # where n_classes = 2</p>
</blockquote>
</aside>
<p>with your current one</p>
<aside class="quote no-group" data-username="bbasaran" data-post="9" data-topic="119583">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bbasaran/48/37602_2.png" class="avatar"> bbasaran:</div>
<blockquote>
<p><code>self.classifier = nn.Sigmoid() </code></p>
</blockquote>
</aside>
<p>So it would be something like:</p>
<pre><code class="lang-auto">self.classifier = nn.Sequential(nn.Linear(n_hidden, n_classes),
                                nn.Sigmoid())
</code></pre>
<p><strong>With one more change!!!</strong><br>
Change the n_classes from 2 to 1. This might sound weird because you have two classes 0 and 1.<br>
But as I explained in Tornikes Problem above, if you have a binary classification with labels being 0 and 1, you normally want your output to be a probability in range 0-1.</p>
<p>If this works you can further optimize it by taking out the <code>nn.Sigmoid()</code> and switch to <code>nn.BCEWithLogitsLoss()</code> as a loss function, as recommended by pytorch.<br>
You can read about it above.</p>

username: bbasaran
cooked: <p>Thanks a lot for your detailed answer <a class="mention" href="/u/ralo4">@RaLo4</a></p>
<p>In the last couple of days, I decided to leave my model as a multi-class classifier, because besides using the same dataset with classes only 0-1, I can diversify my dataset like classes 0-1-2… in the following days. As far as I understood, even if I build the model as a multi-class classifier, I can feed it with a dataset containing only 0-1 labels. I am planning to use again CrossEntropyLoss, but my only problem right now is changing Linear Activation to Softmax.</p>