Backward through Bernoulli sample

username: Ruggero_Ragonesi
cooked: <p>I am trying to backpropagate through a Bernoulli sample. A simplified version of my code is:</p>
<pre><code class="lang-auto">...
parameter = torch.nn.Parameter(torch.ones(8), requires_grad=True) # parameters to learn
prob = torch.sigmoid(parameter) #sigmoid to make a probability [0,1]
bern_sample = torch.distributions.bernoulli.Bernoulli(prob).sample() # draw a sample 

loss = torch.sum(bern_sample)
loss.backward()
</code></pre>
<p>Intuitively the variable <code>parameter</code> should be updated by gradient descent in order to make all probabilities close to 0, i.e. <code>bern_sample</code> should contain all zeros.</p>
<p>However when I check gradients wrt to parameter, i.e. <code>parameter.grad</code>, they are None.</p>
<p>What am I doing wrong?</p>
<p>Thanks in advance</p>

username: KFrank
cooked: <p>Hello Ruggero!</p>
<aside class="quote no-group quote-modified" data-username="Ruggero_Ragonesi" data-post="1" data-topic="98539" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ruggero_ragonesi/48/27458_2.png" class="avatar"> Ruggero_Ragonesi:</div>
<blockquote>
<p>I am trying to backpropagate through a Bernoulli sample.<br>
…<br>
However when I check gradients wrt to parameter, i.e. <code>parameter.grad</code>, they are None.</p>
</blockquote>
</aside>
<p>Sampling from a discrete distribution isn’t differentiable.  So pytorch’s<br>
autograd won’t (and can’t) backpropagate any gradients from<br>
subsequent functions back through the Bernoulli-sampling step.</p>
<p>The value returned from sampling a Bernoulli distribution (or any<br>
discrete distribution) is a discrete value – <code>0.0</code> or <code>1.0</code>.  So you can’t<br>
differentiate it.  That is, you can’t make your sample value a little<br>
smaller, say <code>0.99</code> instead of <code>1.0</code>, by  making your parameter <code>prob</code><br>
a little smaller.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Anurag_Pandey
cooked: <p>Hello K Frank,<br>
I’m just wondering if sampling from other distribution is a differentiable operation or not?</p>

username: KFrank
cooked: <p>Hi Anurag!</p>
<aside class="quote no-group" data-username="Anurag_Pandey" data-post="3" data-topic="98539" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/anurag_pandey/48/18967_2.png" class="avatar"> Anurag_Pandey:</div>
<blockquote>
<p>I’m just wondering if sampling from other distribution is a differentiable operation or not?</p>
</blockquote>
</aside>
<p>I do not believe that pytorch supports differentiating a sample from a<br>
(continuous) distribution with respect to parameters describing that<br>
distribution.  (But I haven’t checked.)</p>
<p>However, doing so can make mathematical sense, and you can<br>
sometimes structure your calculations so that it works.</p>
<p>Consider a gaussian distribution with mean <code>mu</code>, and standard<br>
deviation <code>sigma</code>.  You can sample from such a distribution by<br>
first sampling a value <code>z</code> from a normal distribution (mean = 0,<br>
standard deviation = 1), and then transforming that value:</p>
<pre data-code-wrap="python"><code class="lang-python">x = sigma * z + mu
</code></pre>
<p>If <code>mu</code> and <code>sigma</code> are packaged as <code>Tensor</code>s that have<br>
<code>requires_grad = True</code>, then calling <code>x.backward()</code> will<br>
compute gradients for <code>sigma</code> and <code>mu</code>.</p>
<p>(Whether you will get the result you expect depends on what you<br>
were expecting, but this scheme is conceptually self-consistent.)</p>
<p>Best.</p>
<p>K. Frank</p>

username: Ruggero_Ragonesi
cooked: <p>I solved the problem by using the reparametrization trick with Gumbel Softmax. I took inspiration from this: <a href="https://gist.github.com/yzh119/fd2146d2aeb329d067568a493b20172f" rel="noopener nofollow ugc">https://gist.github.com/yzh119/fd2146d2aeb329d067568a493b20172f</a></p>
<p>Thanks for your help anyway</p>

username: Ezgi
cooked: <p>Hi Ruggero,<br>
Could you please share your solution?<br>
Thank you,<br>
Ezgi</p>