Is sample operation differentiable？

zhping5678: <p>Is sample operation differentiable？<br>
When sampling from a torch distribution and trying to get parameter.grad, it says “element 0 of tensors does not require grad and does not have a grad_fn”, which I think sampling is not differentiable. Code is below.</p>
<pre><code class="lang-auto">loc = torch.tensor([0.0,2.0, 2.0, 3.0]).requires_grad_(True)
scale = torch.tensor([1.0, 1.0, 1.0, 1.0]).requires_grad_(True)
a = torch.distributions.normal.Normal(loc, scale).sample()
a.sum().backward()
</code></pre>
<p>However, when sampling by pyro.sample, I can get parameters.grad.</p>
<pre><code class="lang-auto">loc = torch.tensor([0.0,2.0, 2.0, 3.0]).requires_grad_(True)
scale = torch.tensor([1.0, 1.0, 1.0, 1.0]).requires_grad_(True)
a = pyro.sample("my_sample", pyro.distributions.Normal(loc, scale))
a.sum().backward()
</code></pre>
<p>As I know, pyro.distributions is wrapper of torch.distribution. Did pyro do something else to support grad of sample?</p>

eqy: <p>Are you trying to backward <em>through</em> the distribution somehow (which I don’t think is possible) or simply get the gradient of ·<code>a</code> (which I think is possible)?</p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; loc = torch.tensor([0.0,2.0, 2.0, 3.0]).requires_grad_(True)
&gt;&gt;&gt; scale = torch.tensor([1.0, 1.0, 1.0, 1.0]).requires_grad_(True)
&gt;&gt;&gt; a = torch.distributions.normal.Normal(loc, scale).sample()
&gt;&gt;&gt; a.sum().backward()
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "/home/eqy/.local/lib/python3.8/site-packages/torch/tensor.py", line 245, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/eqy/.local/lib/python3.8/site-packages/torch/autograd/__init__.py", line 145, in backward
    Variable._execution_engine.run_backward(
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
&gt;&gt;&gt; a.requires_grad = True
&gt;&gt;&gt; a.sum().backward()
&gt;&gt;&gt; a.grad
tensor([1., 1., 1., 1.])
&gt;&gt;&gt;
</code></pre>

KFrank: <p>Hi Zhping!</p>
<aside class="quote no-group" data-username="zhping5678" data-post="1" data-topic="172898" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zhping5678/48/57492_2.png" class="avatar"> zhping5678:</div>
<blockquote>
<p>Is sample operation differentiable？</p>
</blockquote>
</aside>
<p>As eqy notes, you cannot use autograd to get the gradient of a sample from<br>
a pytorch distribution with respect to that distribution’s parameters.</p>
<p>It is not unreasonable, however, to sample from a standard normal distribution<br>
and then use your parameters to transform that normal sample into your desired<br>
gaussian sample.  With this approach you can use autograd to differentiate the<br>
gaussian sample with respect to the parameters.</p>
<p>Consider:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; print (torch.__version__)
1.13.1
&gt;&gt;&gt;
&gt;&gt;&gt; _ = torch.manual_seed (2023)
&gt;&gt;&gt;
&gt;&gt;&gt; loc = torch.tensor ([0.0, 2.0, 2.0, 3.0]).requires_grad_ (True)
&gt;&gt;&gt; scale = torch.tensor ([1.0, 1.0, 1.0, 1.0]).requires_grad_ (True)
&gt;&gt;&gt;
&gt;&gt;&gt; # you could also use  a = torch.randn (4)
&gt;&gt;&gt; a = torch.distributions.normal.Normal (torch.zeros (4), torch.ones (4)).sample()
&gt;&gt;&gt; print (a)
tensor([-1.2075,  0.5493, -0.3856,  0.6910])
&gt;&gt;&gt;
&gt;&gt;&gt; # transform standard normal samples into gaussians with your loc and scale
&gt;&gt;&gt; a = scale * a + loc
&gt;&gt;&gt; print (a)
tensor([-1.2075,  2.5493,  1.6144,  3.6910], grad_fn=&lt;AddBackward0&gt;)
&gt;&gt;&gt;
&gt;&gt;&gt; a.sum().backward()
&gt;&gt;&gt;
&gt;&gt;&gt; print (loc.grad)
tensor([1., 1., 1., 1.])
&gt;&gt;&gt; print (scale.grad)
tensor([-1.2075,  0.5493, -0.3856,  0.6910])
</code></pre>
<aside class="quote no-group">
<blockquote>
<p>Did pyro do something else to support grad of sample?</p>
</blockquote>
</aside>
<p>I have no idea how pyro handles this.  If you cared, you could try to reverse<br>
engineer it by looking at the gradients you get from the pyro samples and see<br>
if they look anything like the gradients we get in the above example.</p>
<p>Best.</p>
<p>K. Frank</p>

zhping5678: <p>Thanks, eqy! I’m not trying to get the gradient of the sample result. I’m trying to figure out if sampling is differentiable, because I can get gradient of parameters like loc and scale by pyro. And I don’t know whether gradient of sample makes sense because sampling is stochastic.</p>

zhping5678: <p>Thank you, K. Frank! Why cannot we sample from Normal(loc, scale) directly, and choose to sample from a = Normal(0,1) and a * scale + loc ? Does the gradient make sense？<br>
Can I understand that sampling from a distribution is meaningless bacause it is stochastic?</p>

zhping5678: <p>After reading your reply, I tried to</p>
<pre><code class="lang-auto">loc = torch.zeros(4)
scale = torch.ones(4)
a = pyro.sample("my_sample", pyro.distributions.Normal(loc, scale))
a.sum().backward()
print(f"after backward, loc.grad = {loc.grad} and scale.grad = {scale.grad}")
</code></pre>
<p>and it reports errors “element 0 of tensors does not require grad and does not have a grad_fn”.<br>
I think that I know what pyro does, thank you!</p>