Discrete Output with Positive Integers

fermoren: <p>Hi,</p>
<p>I am trying to implement a neural network that produces a discrete output in the form of integers between zero and infinity. My initial thought was to use the torch.round() function, however, I have found that this approach is not suitable as the round function does not support backpropagation. I have also attempted the approach outlined in the link <a href="https://discuss.pytorch.org/t/torch-round-gradient/28628" class="inline-onebox">Torch.round() gradient</a> with no success.</p>
<p>What is the simplest method for achieving a discrete output of any positive integer in a neural network?</p>
<p>Thank you.</p>

KFrank: <p>Hi Fernando!</p>
<aside class="quote no-group quote-modified" data-username="fermoren" data-post="1" data-topic="170702" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/f/dbc845/48.png" class="avatar"> fermoren:</div>
<blockquote>
<p>I have also attempted the approach … with no success.</p>
</blockquote>
</aside>
<p>What did you actually try?  Did it fail with error messages?  No gradients?<br>
Zero gradients?</p>
<aside class="quote no-group">
<blockquote>
<p>What is the simplest method for achieving a discrete output of any positive integer in a neural network?</p>
</blockquote>
</aside>
<p>The core problem is that if you want backpropagation, discrete values are<br>
not usefully differentiable, in that their gradients are zero (almost) everywhere.</p>
<p>You have two choices:</p>
<p>You can produce continuous values that tend to be clustered around your<br>
desired discrete values and then figure out how to compute a loss function<br>
from these “approximately-discrete” values (that is usefully differentiable).</p>
<p>You can produce discrete values, but approximate their zero gradients with<br>
non-zero gradients that capture the relationship of your loss function to the<br>
parameters that produce your discrete values in a way that leads to useful<br>
backpropagation and training.</p>
<p>Best.</p>
<p>K. Frank</p>

fermoren: <p>Thanks for your answer K.Frank!</p>
<p>I tried to implement a “personalised” version of the round function which is ignored during the backpropagation:</p>
<pre><code class="lang-auto">class my_round_func(InplaceFunction):
    @staticmethod
    def forward(ctx, input):
        ctx.input = input
        return torch.round(input)

    @staticmethod
    def backward(ctx, grad_output):
        grad_input = grad_output.clone()
        return grad_input
</code></pre>
<p>Anyhow, the model seems to not be learning with that approach. Could you detail a little bit more about any of the solutions you mentioned and how to implement them?</p>
<p>Thanks so much!</p>

KFrank: <p>Hi Fernando!</p>
<aside class="quote no-group" data-username="fermoren" data-post="3" data-topic="170702" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/f/dbc845/48.png" class="avatar"> fermoren:</div>
<blockquote>
<p>Could you detail a little bit more about any of the solutions you mentioned</p>
</blockquote>
</aside>
<p>Whether a given approach might work will depend very much on your<br>
specific use case.</p>
<p>Here’s one hypothetical illustration:</p>
<p>Let’s say that your loss function is given by a look-up table with ten entries<br>
and your model predicts a discrete integer that runs from <code>0</code> to <code>9</code>.   So<br>
<code>loss = look_up_table[prediction]</code>.</p>
<p>You won’t be able to backpropagate through the discrete-integer prediction.</p>
<p>But you could modify your model to predict a continuous value that ranges<br>
from <code>0.0</code> to <code>9.0</code> and use your look-up table as an interpolation table so that<br>
your loss is now a continuous function of your (continuous) prediction.  You<br>
will now have valid, <em>useful</em> gradients, backpropagation will work, and your<br>
network might (or might not) train.</p>
<p>Best.</p>
<p>K. Frank</p>