Expand vs repeat used within backward raise Error

jmaronas: <p>Hi.</p>
<p>I am implementing a Gaussian observation model for a multioutput task. The class I have implemented has the following init method. Basically the observation noise can be shared by the different outputs.</p>
<pre><code class="lang-auto">class GaussianLinearMean(nn.Module):

    def __init__(self,out_dim : int, noise_init: float, noise_is_shared : bool):
        super(GaussianLinearMean,self).__init__()

        self.out_dim = out_dim
        self.noise_is_shared = noise_is_shared

        if noise_is_shared: # if noise is shared create one parameter and expand to out_dim shape
            log_var_noise = nn.Parameter(torch.ones(1,1,dtype = cg.dtype)*torch.log(torch.tensor(noise_init,dtype = cg.dtype)))
            log_var_noise = log_var_noise.expand(out_dim,1)

        else: # creates a vector of noise variance parameters.
            log_var_noise = nn.Parameter(torch.ones(out_dim,1,dtype = cg.dtype)*torch.log(torch.tensor(noise_init,dtype = cg.dtype)))

        self.log_var_noise = log_var_noise
</code></pre>
<p>For the case in which the observation noise is shared, I just create one parameter that is expanded to match the number of outputs. If I use the expand method in <code>log_var_noise = log_var_noise.expand(out_dim,1)</code> everything works fine. However if I use repeat, i.e <code>log_var_noise = log_var_noise.repeat(out_dim,1)</code> I get the following error when backward the loss:</p>
<pre><code class="lang-auto">RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.
</code></pre>
<p>in the second iteration of the algorithm. As far as I understood the difference between repeat or expand is whether the memory is copied or not. I guess that the computed gradient is equivalent for both methods. I am just wondering where might be the error. This problem is solved if the <code>repeat</code> method is called at each call to the forward, rather than done in the <code>__init__</code> method. However, I prefer to do it only once for speed.</p>

albanD: <p>Hi,</p>
<p>The difference is that the current formula for repeat needs to access the original input (and so saves it) while expand does not.<br>
So, if you share these ops across multiple backward, the expand version does not have any issue (since nothing was saved, nothing was cleared by the first backward) but the repeat version does not have the saved Tensors anymore and lead to this error.</p>
<p>If you want to use repeat, you will have to do it at each forward I’m afraid.</p>
<p>The difference of behavior though could be removed (and repeat made a bit faster), opened an issue to track that: <a href="https://github.com/pytorch/pytorch/issues/40701">https://github.com/pytorch/pytorch/issues/40701</a></p>

jmaronas: <p>Thanks for your reply. I have dig a bit into and It seems that both methods have to be called withing the forward. If not, the gradient has a value of <code>None</code> for the expand case.</p>

albanD: <p>Right this is a different question:<br>
The short answer is that the <code>.grad</code> field is only populated for leaf Tensors (Tensors with no history). And the result of the repeat/expand is not a leaf. So what you see is expected.<br>
You will need to have 2 Tensors, one that is the original Parameter with a single element that is a leaf. And the expanded/repeated one that is used in the forward (that won’t get gradient).</p>
<p>Or you can move the repeat/expand to the forward (the safest bet as it does not rely on internal implementation details of expand/repeat).</p>