Multi-Class Classifier

username: ostaeria
cooked: <p>I’m facing immense confusion in the construction of my neural network.<br>
My input is of  (140000 x 200) dimension. My output is of (140000) dimension.</p>
<p>My input is basically a bunch of 200 dimensional floats. Its denoted by <strong>x_train_regressors</strong><br>
My output consists of the following classes represented by numbers: 1, 2, 3, 4, 5, its denoted by <strong>y_train_exact_target</strong> .</p>
<p>I am trying to build and train a network to properly classify my output, but I am not sure how to classify to handle/format the output values. My code is below:</p>
<pre><code class="lang-auto">class NeuralNet(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super(NeuralNet, self).__init__()
        self.layer1 = nn.Linear(input_size, hidden_size)
        self.relu = nn.ReLU()
        self.layer2 = nn.Linear(hidden_size, num_classes)
        
    def forward(self, x):
        out = self.layer1(x)
        out = self.relu(out)
        out = self.layer2(out)
        return out
</code></pre>
<pre><code class="lang-auto">exact_net = NeuralNet(200 , 1000, 5)
loss_function = nn.BCELoss()
exact_net_opt = torch.optim.SGD(exact_net.parameters(), lr = 0.02)

train_set = Variable(torch.from_numpy(np.array(x_train_regressors))).float()
test_set = Variable(torch.LongTensor(y_train_exact_target)).long()


epochs = 50
for epochs in range(epochs):
    exact_net_opt.zero_grad()
    out = exact_net(instance)
    loss = loss_function(out, label)
    loss.backward()
    exact_net_opt.step()
        
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
&lt;ipython-input-187-5e98211ac56e&gt; in &lt;module&gt;
      4     exact_net_opt.zero_grad()
      5     out = exact_net(instance)
----&gt; 6     loss = loss_function(out, label)
      7     loss.backward()
      8     exact_net_opt.step()

~\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    475             result = self._slow_forward(*input, **kwargs)
    476         else:
--&gt; 477             result = self.forward(*input, **kwargs)
    478         for hook in self._forward_hooks.values():
    479             hook_result = hook(self, input, result)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\loss.py in forward(self, input, target)
    484 
    485     def forward(self, input, target):
--&gt; 486         return F.binary_cross_entropy(input, target, weight=self.weight, reduction=self.reduction)
    487 
    488 

~\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\functional.py in binary_cross_entropy(input, target, weight, size_average, reduce, reduction)
   1595     if input.nelement() != target.nelement():
   1596         raise ValueError("Target and input must have the same number of elements. target nelement ({}) "
-&gt; 1597                          "!= input nelement ({})".format(target.nelement(), input.nelement()))
   1598 
   1599     if weight is not None:

ValueError: Target and input must have the same number of elements. target nelement (1) != input nelement (5)
</code></pre>

username: ptrblck
cooked: <p>If you want to classify each sample to one of your 5 classes, try to use <code>nn.CrossEntropyLoss</code>.<br>
Also, shift the target values to <code>[0, nb_classes-1]</code>, so <code>[0, 4]</code> in your case, as otherwise you’ll get an out of bounds error.</p>

username: ostaeria
cooked: <p>In this case, wouldn’t one hot encoding be necessary?</p>

username: ptrblck
cooked: <p>No, for multi-class classification (one target class for each sample), the targets should hold the class indices.<br>
Other frameworks often use one-hot encoded target vectors, which is not necessary in PyTorch.<br>
Have a look at the <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss">docs</a> for more information.</p>

username: Rojin
cooked: <p>Hey! shouldn’t we add a softmax activation at the end when we want to do a multi-class classification?</p>

username: ptrblck
cooked: <p>If you are using <code>nn.CrossentropyLoss</code>, you should pass the logits without any non-linearity to the criterion, as internally <code>F.log_softmax</code> and <code>nn.NLLLoss</code> will be used.<br>
Alternatively, you could apply <code>F.log_softmax</code> manually and pass it directly to <code>nn.NLLLoss</code>.</p>
<p>So usually you don’t use a softmax activation, unless e.g. you are implementing some custom criterion of course.</p>