How to use pytorch to output the probability of binary classfication?

MadFrog: <p>The data in my <code>train.csv</code>  likes this:</p>
<pre><code class="lang-auto">[
[0, 0,……, 0],
[0, 1,……, 0],
[0, 2,……, 0],
[0, 3,……, 0],
[1, 0,……, 0],
[2, 0,……, 0],
[3, 0,……, 0],
[1, 1,……, 1],
[2, 1,……, 1],
[1, 2,……, 1],
[3, 1,……, 1],
]
</code></pre>
<p>The training set has 800,000 rows data and the test set has 20,000 rows data.<br>
The value of <code>y_train</code>  is only 0 and 1, and there are more than 150 columns of data in csv.</p>
<p>I want to use pytorch to predict the probability of 0 and 1 with <code>x_test</code>, how to do it?</p>
<p>Here is my code:</p>
<pre><code class="lang-auto">import pandas as pd

train = pd.read_csv(r'train.csv')
x_test = pd.read_csv(r'test.csv')

x_train = train[:, :-1]
y_train = train[:, -1]
</code></pre>
<p>I want 2 columns of probability, the first column is the probability of <code>0</code> , and the second column is the probability of <code>1</code> , such as this:</p>
<pre><code class="lang-auto">[[0.23334,0.76267]
 ……
 [0.84984,0.15685]
 [0.16663,0.83291]]
</code></pre>
<p>When I use <code>keras</code>,I can do it like this:</p>
<pre><code class="lang-auto">import numpy as np
from keras.models import Sequential
from keras.layers import Dense
from sklearn.preprocessing import LabelEncoder
from keras.utils import np_utils

data_train = np.array([
[0, 0, 0],
[0, 1, 0],
[0, 2, 0],
[0, 3, 0],
[1, 0, 0],
[2, 0, 0],
[3, 0, 0],
[1, 1, 1],
[2, 1, 1],
[1, 2, 1],
[3, 1, 1],
])

data_test = np.array([
[1, 3],
[0, 4],
[5, 0]
])

x_train = data_train[:, :-1]
y_train = data_train[:, -1]
x_test = data_test
encoder = LabelEncoder()
encoder.fit(y_train)
encoded_y = encoder.transform(y_train)
y_train = np_utils.to_categorical(encoded_y)

model = Sequential()
model.add(Dense(512, activation='relu', input_dim=2))
model.add(Dense(200, activation='relu'))
model.add(Dense(200, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(2, activation='softmax'))
model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['binary_accuracy'])

model.fit(x_train, y_train, epochs=5, batch_size=1, verbose=1)
predict = model.predict_proba(x_test, batch_size=1)
print(predict)
</code></pre>
<p>result:</p>
<pre><code class="lang-auto">[[0.46373594 0.53626406]
 [0.99079037 0.00920963]
 [0.72976    0.27024   ]]
</code></pre>
<p>But how to do it with pytorch?</p>

ptrblck: <p>You could create a model with two output neurons (e.g. via <code>nn.Linear</code>) and setup a multi-label classification use case using <code>nn.BCEWithLogitsLoss</code>.<br>
Since the model output would be logits, you could apply <code>torch.sigmoid</code> on them to get the probabilities for each class.<br>
Here is a very simple example:</p>
<pre><code class="lang-python">class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc1 = nn.Linear(128, 64)
        self.fc2 = nn.Linear(64, 40)
        
    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        return x

model = MyModel()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.BCEWithLogitsLoss()

data = torch.randn(64, 128)
target = torch.randint(0, 2, (64, 40)).float()

for epoch in range(100):
    optimizer.zero_grad()
    output = model(data)
    loss = criterion(output, target)
    loss.backward()
    optimizer.step()
    print('epoch {}, loss {}'.format(epoch, loss.item()))

# check predictions
output = model(data)
probs = torch.sigmoid(output)
print(probs)
print(target)
</code></pre>

MadFrog: <p><a class="mention" href="/u/ptrblck">@ptrblck</a><br>
Hi,thanks!<br>
I use this code:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.functional as F


train = pd.read_csv(r'train.csv')
x_test = pd.read_csv(r'test.csv')

x_train = train[:, :-1]
y_train = train[:, -1]

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc1 = nn.Linear(800000, 153)
        self.fc2 = nn.Linear(153, 2)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        return x

model = MyModel()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.BCEWithLogitsLoss()
x_train = torch.Tensor(x_train.values)
y_train = torch.Tensor(y_train.values)
x_test = torch.Tensor(x_test.values)
# data = torch.randn(64, 128)
# target = torch.randint(0, 2, (64, 40)).float()

for epoch in range(5):
    optimizer.zero_grad()
    output = model(x_train)
    loss = criterion(output, y_train)
    loss.backward()
    optimizer.step()
    print('epoch {}, loss {}'.format(epoch, loss.item()))

# check predictions
output = model(x_test)
probs = torch.sigmoid(output)
print(probs)
</code></pre>
<p>But the error messgae is:</p>
<blockquote>
<p>RuntimeError:<br>
size mismatch, m1: [800000 x 153], m2: [800000 x 153] at …\aten\src\TH/generic/THTensorMath.cpp:41</p>
</blockquote>
<p>How to set these 2 lines?</p>
<pre><code class="lang-auto">self.fc1 = nn.Linear(800000, 153)
self.fc2 = nn.Linear(153, 2)
</code></pre>

ptrblck: <p>It seems the <code>x_train</code> tensor has the shape <code>[153, 800000]</code> and should be permuted before passing it to the model via <code>x_train = x_train.permute(1, 0)</code>.</p>

MadFrog: <p><a class="mention" href="/u/ptrblck">@ptrblck</a><br>
Hi,when I do it as you said:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.functional as F


train = pd.read_csv(r'train.csv')
x_test = pd.read_csv(r'test.csv')

x_train = train[:, :-1]
y_train = train[:, -1]

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc1 = nn.Linear(800000, 153)
        self.fc2 = nn.Linear(153, 2)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        return x

model = MyModel()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.BCEWithLogitsLoss()
x_train = torch.Tensor(x_train.values)
y_train = torch.Tensor(y_train.values)
x_train = x_train.permute(1, 0)
x_test = torch.Tensor(x_test.values)
# data = torch.randn(64, 128)
# target = torch.randint(0, 2, (64, 40)).float()

for epoch in range(5):
    optimizer.zero_grad()
    output = model(x_train)
    loss = criterion(output, y_train)
    loss.backward()
    optimizer.step()
    print('epoch {}, loss {}'.format(epoch, loss.item()))

# check predictions
output = model(x_test)
probs = torch.sigmoid(output)
print(probs)
</code></pre>
<p>The error message is:</p>
<blockquote>
<p>ValueError:<br>
Target size (torch.Size([800000])) must be the same as input size (torch.Size([153, 2]))<br>
So how should I set this 2 lines?</p>
</blockquote>
<pre><code class="lang-auto">self.fc1 = nn.Linear(800000, 153)
self.fc2 = nn.Linear(153, 2)
</code></pre>

ptrblck: <p>Ok, I might be wrong with the <code>permute</code> suggestion.<br>
Could you remove the <code>permute</code> operation again and post the shape of <code>x_train</code> before passing it to the model?</p>

MadFrog: <p>Hi,my training set is 800,000 rows and 153 columns. The data I want to predict is 200,000 rows and 153 columns. How should I write these 2 lines?</p>
<pre><code class="lang-auto">self.fc1 = nn.Linear(800000, 153)
self.fc2 = nn.Linear(153, 2)
</code></pre>
<p>If I use this code,the error message is:</p>
<blockquote>
<p>RuntimeError:<br>
size mismatch, m1: [800000 x 153], m2: [800000 x 153] at …\aten\src\TH/generic/THTensorMath.cpp:41</p>
</blockquote>

ptrblck: <aside class="quote no-group" data-username="MadFrog" data-post="7" data-topic="101043">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/madfrog/48/30432_2.png" class="avatar"> MadFrog:</div>
<blockquote>
<p>my training set is 800,000 rows and 153 columns.</p>
</blockquote>
</aside>
<p>800,000 would then be the batch size (or the total number of samples) while 153 would be the feature dimension.<br>
In that case use</p>
<pre data-code-wrap="python"><code class="lang-python">self.fc1 = nn.Linear(153, any_value)
self.fc2 = nn.Linear(any_value, 2)
</code></pre>
<p>The batch size is not part of the layer definition, as all PyTorch layers accept a variable batch size.<br>
Your target should have the same batch size so you can calculate the loss using the model output.</p>