When do I turn prediction numbers into 1 and 0 for binary classification?

julliet: <p>I have a binary classification problem. Right now I’m using several linear layers with ReLU activation.<br>
I’m using BCEWithLogitsLoss() for Loss, so I’m not implementing any Softmax on the layers.<br>
Predictions of the model look something like this:<br>
<code>-0.2443,   6.6122,  25.0909,  ..., -62.7383,   0.3066,  61.6255</code><br>
So I can’t really compare it to zeroes and ones of my label. I don’t understand in which moment and how I should convert my predictions into 1s and 0s?</p>
<p>If it’s important right now my model looks like this:</p>
<pre><code class="lang-auto">
    def __init__(self, num_cols, cat_cols, embedding_size_dict, n_classes,
                 embedding_dim_dict=None, learning_rate=0.01):
        super().__init__()

        self.cat_cols = cat_cols
        self.num_cols = num_cols
        self.embeddings, total_embedding_dim = self._create_embedding_layers(
            cat_cols, embedding_size_dict, embedding_dim_dict)
        
        in_features = len(num_cols) + total_embedding_dim
        self.layers = nn.Sequential(
            nn.Linear(in_features, 128),
            nn.ReLU(),
            nn.Linear(128, 256),
            nn.ReLU(),
            nn.Linear(256, n_classes)
        )

    @staticmethod
    def _create_embedding_layers(cat_cols, embedding_size_dict, embedding_dim_dict):
        """construct the embedding layer, 1 per each categorical variable"""
        total_embedding_dim = 0
        embeddings = {}
        for col in cat_cols:
            embedding_size = embedding_size_dict[col]
            embedding_dim = embedding_dim_dict[col]
            total_embedding_dim += embedding_dim
            embeddings[col] = nn.Embedding(embedding_size, embedding_dim)

        return nn.ModuleDict(embeddings), total_embedding_dim

    def forward(self, num_tensor, cat_tensor):
        cat_outputs = []
        for i, col in enumerate(self.cat_cols):
            embedding = self.embeddings[col]
            cat_output = embedding(cat_tensor[:, i])
            cat_outputs.append(cat_output)
        
        cat_outputs = torch.cat(cat_outputs, dim=1)
        all_outputs = torch.cat((num_tensor, cat_outputs), dim=1)
        
        final_outputs = self.layers(all_outputs).squeeze(dim=1)
        return final_outputs```</code></pre>

zetyquickly: <p>Hello <a class="mention" href="/u/julliet">@julliet</a>,</p>
<p>Basically when you decided not to go with Sigmoid or Softmax you lose ability to convert logits to probability like values. If you need probabilities try to use any continuous function maps <code>(-inf, +inf)</code> to <code>[0,1]</code> and pass this <code>f(logits)</code> to the call of criterion.</p>
<p>Examples of such mappings: <code>f(logits) = 0.5 * (1 + tanh(logits/2))</code> or <code>Sigmoid</code></p>

KFrank: <p>Hi Julia!</p>
<aside class="quote no-group quote-modified" data-username="julliet" data-post="1" data-topic="130075" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/3be4f8/48.png" class="avatar"> julliet:</div>
<blockquote>
<p>I’m using BCEWithLogitsLoss() for Loss, so I’m not implementing any Softmax on the layers.<br>
Predictions of the model look something like this:<br>
<code>-0.2443,   6.6122,  25.0909,  ..., -62.7383,   0.3066,  61.6255</code><br>
So I can’t really compare it to zeroes and ones of my label.<br>
…</p>
<pre><code class="lang-auto">            nn.Linear(256, n_classes)
</code></pre>
</blockquote>
</aside>
<p>In terms of comparing your predictions to the “zeroes and ones” of<br>
your label, <code>BCEWithLogitsLoss</code> does precisely this (without converting<br>
your predictions into 1s and 0s).</p>
<p><code>BCEWithLogitsLoss</code> takes predictions that are raw-score <em>logits</em><br>
(such as those produced by your final <code>Linear</code> layer and that run<br>
from <code>-inf</code> to <code>inf</code>) and compares them with ground-truth labels that<br>
are zeros and ones (or more generally, with ground-truth labels that<br>
are probabilities between zero and one).  That is, not converting your<br>
predictions (that are logits) into zeros and ones before passing them<br>
to <code>BCEWithLogitsLoss</code> is the correct thing to do.  (In this situation,<br>
<code>BCEWithLogitsLoss</code> will likely be your loss function used for training.)</p>
<aside class="quote no-group">
<blockquote>
<p>how I should convert my predictions into 1s and 0s?</p>
</blockquote>
</aside>
<p>On the other hand, if you wish to compute the <em>accuracy</em> of your<br>
predictions (an evaluation metric that you would most likely not use<br>
for training), that is, the percentage of your yes-no predictions are<br>
correct, you do want to convert your predictions to zeros and ones,<br>
and then simply count how many are equal to your zero-and-one<br>
ground-truth labels.</p>
<p>A logit of <code>0.0</code> corresponds to a probability (of being in the “1”-class)<br>
of <code>0.5</code>, so one would typically threshold the logit against <code>0.0</code>:</p>
<pre><code class="lang-auto">accuracy = ((predictions &gt; 0.0) == labels).float().mean()
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

julliet: <p>Hello K. Frank!<br>
Thank you for your thorough answer. That’s exactly what I needed. I trained the model, but I still needed to compute accuracy what I needed zeros and ones for.</p>

julliet: <p>I did use Sigmoid and BCE Loss function on first iteration, but my Loss stayed the same value during the training. Thus, I decided to remove it and change to BCEWithLogitsLoss</p>