Binary mask output by network

username: alwynmathew
cooked: <p>Below code will create a mask for <code>x</code></p>
<pre><code class="lang-auto">class get_mask(nn.Module):
  def __int__(self, in_c):
    self.conv1 = nn.Conv2d(in_c, 1 , 3, 1)
    self.norm = nn.BatchNorm2d(2)

  def forward(self, x):
    p = 1
    p2d =(p, p, p, p)
    x = self.conv1(F.pad(x,p2d))
    x = self.normalize(x)
    return F.sigmoid(x)
</code></pre>
<p>How modify the network to output a binary mask(zeros and ones) instead of <code>F.sigmoid(x)</code>?</p>

username: ptrblck
cooked: <p>If you don’t need to backpropagate through it, you could just apply a threshold on the sigmoid output of e.g. <code>0.5</code>.<br>
Do you just need the binary outputs for some accuracy calculation or visualization?</p>

username: alwynmathew
cooked: <p>No, I need to backprop. I want to use to this <code>mask</code> to mask out invalid pixels in <code>x</code> while calculating loss.</p>

username: ptrblck
cooked: <p>Ah OK, then I misunderstood your question, sorry.<br>
In that case you could create your mask vector (with zeros and ones) and multiply it with the loss.</p>
<p>Something like this should work:</p>
<pre><code class="lang-python">model = nn.Conv2d(3, 1, 3, 1, 1)
x = torch.randn(1, 3, 5, 5)
mask = torch.empty(1, 1, 5, 5).random_(2)
target = torch.empty(1, 1, 5, 5).random_(2)
criterion = nn.BCELoss(reduction='none')

output = torch.sigmoid(model(x))
loss = criterion(output, target)
loss = loss * mask
loss.mean().backward()
</code></pre>

username: alwynmathew
cooked: <p>Sorry, maybe I confused you.</p>
<p>I need something link this:</p>
<pre><code class="lang-auto">model = nn.Conv2d(3, 1, 3, 1, 1)
mask = torch.sigmoid(model(x)) # but I need mask to be binary instead of values between 0 and 1
masked_img = mask * x # point-wise
loss = cal_loss(masked_img, x)
loss.mean().backward()
</code></pre>

username: longbowking
cooked: <p>I think <a href="https://discuss.pytorch.org/t/binary-activation-function-with-pytorch/56674/4">link</a> could give us a hint, in your case,</p>
<pre><code class="lang-auto">mask = torch.relu(torch.sign(torch.sigmoid(model(x))-0.5))
</code></pre>
<p>should return mask with elements ∈ {0,1}.</p>
<p>Besides, as discussed in <a href="https://discuss.pytorch.org/t/circumventing-an-indifferentiablility-problem/41718/3">link</a>, the derivative of sign(.) is always 0,</p>
<p>suppose y = M(x1) * H(x2), where</p>
<ul>
<li>M(): mask layer</li>
<li>H(): some hidden layer</li>
</ul>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/2/520836ba294a5e2b59411134526d785eade891e5.png" alt="image" data-base62-sha1="bHGKwYNeSivxX1cu4C4FFXTFsot" width="499" height="60"></p>
<p>as</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/0/7/07f4b2bdeac17f56de85a3ddebb706b96c56d6fa.png" alt="image" data-base62-sha1="18nByEYbFEwUJLKDjfxI611J8IG" width="325" height="62"></p>
<p>Note that, since M(x1)  ∈ {0,1}, thus only the positive mask layer outputs take part in back-propagation.</p>
<p>b.t.w. how to insert equations in pytorch forum?</p>

username: Nicholas_Roth
cooked: <p>Thanks for this, exactly what I was looking for!</p>

username: zippeurfou
cooked: <p>I think there might be a slight mistake in the example. Not important because directionally correct but I believe that instead of being:<br>
<code>loss.mean().backward()</code><br>
you should write:</p>
<pre><code class="lang-auto">loss = loss / torch.sum(mask)
loss.backward()
</code></pre>