How to threshold a tensor into binary values？

chaoyan1073: <p>I want to threshold a tensor used in self-defined loss function into binary values. Previously, I used torch.round(prob) to do it. Since my prob tensor value range in [0 1]. This is equivalent to threshold the tensor prob using a threshold value 0.5.</p>
<p>For example,<br>
prob = [0.1, 0.3, 0.7, 0.9], torch.round(prob) = [0, 0, 1, 1]</p>
<p>Now, I would like to use a changeable threshold value, how to do it?</p>

richard: <p>Try <a href="http://pytorch.org/docs/master/torch.html?highlight=where#torch.where">torch.where</a></p>

chaoyan1073: <p>It seems that only the lastest pytorch 0.4 has torch.where.</p>

richard: <p>Something else you can do is something like</p>
<pre><code class="lang-auto">prob = ...   # Tensor([0.1, 0.3, 0.7, 0.9])
t = Variable(torch.Tensor([0.5]))  # threshold
out = (prob &gt; t).float() * 1
</code></pre>

chaoyan1073: <p>Hi Richard, thanks for your kind help. I tried and just got an error.</p>
<p>RuntimeError: gt() received an invalid combination of arguments - got (numpy.float32), but expected one of:</p>
<ul>
<li>(float other)<br>
didn’t match because some of the arguments have invalid types: (numpy.float32)</li>
<li>(Variable other)<br>
didn’t match because some of the arguments have invalid types: (numpy.float32)</li>
</ul>

jpeg729: <p>Both of these work on pytorch 0.3</p>
<pre><code class="lang-auto">t = 0.5
</code></pre>
<pre><code class="lang-auto">t = torch.Tensor([0.5])</code></pre>

richard: <p>You have to make sure that in <code>(prob &gt; t)</code>, both elements are tensors.</p>

jpeg729: <p>It also works if both prob and t are Variables.</p>

Deeply: <p>I am curious to know what role does multiplying by 1, <code>*1</code>, play in <code>out = (prob &gt; t).float() * 1</code>  ?</p>

himat: <p>I would also like to know <a class="mention" href="/u/richard">@richard</a>.</p>

richard: <p>The <code>*1</code> is definitely not necessary</p>

hs271095: <p>Hey this gives me bool values and not 0 1</p>

bsridatta: <p>.float() should convert booleans to 0., 1.</p>

Aamir_M1: <pre><code class="lang-auto">import torch
from torch import nn
from torch.nn import functional as F
import math
import numpy as np


class SparseActivations(nn.Module):
    """ Custom Linear layer but mimics a standard linear layer """

    def __init__(self, size_in, size_out, threshold):
        super().__init__()
        self.size_in, self.size_out = size_in, size_out
        self.threshold = threshold
        weights = torch.Tensor(size_out, size_in)
        self.weights = nn.Parameter(weights)  # nn.Parameter is a Tensor that's a module parameter.
        bias = torch.Tensor(size_out)
        self.bias = nn.Parameter(bias)
        self.sigmoid = nn.Sigmoid()
        # initialize weights and biases
        self.init_weights()

    def init_weights(self):
        stdv = 1. / math.sqrt(self.weights.size(1))
        self.weights.data.uniform_(-stdv, stdv)
        if self.bias is not None:
            self.bias.data.uniform_(-stdv, stdv)

    def forward(self, x):
        w_times_x = torch.mm(x, self.weights.t())
        outs = self.sigmoid(torch.add(w_times_x, self.bias))  # w times x + b
        for i in range(0, self.size_out):
            if outs[:, i] &gt; self.threshold:
                outs[:, i] = 1
            else:
                outs[:, i] = 0
        return outs


class ForwardNetwork(nn.Module):

    # Using custom layer
    def __init__(self):
        super().__init__()
        self.l1 = nn.Linear(100, 10000)
        self.custom1 = SparseActivations(10000, 10000, threshold=0.56)
        self.l2 = nn.Linear(10000, 100)
        self.l3 = nn.Linear(100, 10)

    def forward(self, x):
        x = F.relu(self.l1(x))
        x = self.custom1(x)
        return x


def main():
    fr = ForwardNetwork()
    n = torch.tensor(np.random.uniform(0, 1, 100).reshape(1, 100).astype(np.float32))
    results = fr.forward(n).detach().numpy().flatten().astype(np.uint8)
    counter = 0
    on_bit_list = []
    for idx, r in enumerate(results):
        if r == 1:
            counter += 1
            on_bit_list.append(idx)
        else:
            pass
    print(counter)
    print(on_bit_list)


if __name__ == '__main__':
    main()

</code></pre>