How does one obtain gradients as data efficiently?

username: pinocchio
cooked: <p>I am trying to extract gradients and later use them as data (so have them non-trainable). For that I am having to call <code>.backward()</code> twice. Doing that seems to mess up with my code as noted by mother question ( <a href="https://discuss.pytorch.org/t/what-does-doing-backward-twice-do/58057" class="inline-onebox">What does doing .backward twice do?</a>). However, for the code to work how I intend it (without the issue of calling backward twice) I am having to do clone the model parameters:</p>
<pre><code class="lang-auto">            ww = torch.randn(w.size(), requires_grad=True)
            ww.data = w.clone()
            loss = (ww*x-y)**2
            loss.backward()
            grad = ww.grad
            assert( (2*(w*x-y)*x).item() == grad.item() )
</code></pre>
<p>in this silly toy example its not an issue but I am worried that this might become an issue later. Anyone know how to get the gradients without having to clone the data and create extra vectors?</p>
<hr>
<p>all the code:</p>
<pre><code class="lang-auto">    if experiment_type == 'grad_constant':
        ## computes backard pass on J (i.e. dJ_dw) assuming g is constant
        # compute g
        if g_hard_coded:
            grad = 2*(w*x-y)*x # assumption, loss = (w*x - y)**2
        else:
            ww = torch.randn(w.size(), requires_grad=True)
            ww.data = w.clone()
            loss = (ww*x-y)**2
            loss.backward()
            grad = ww.grad
            assert( (2*(w*x-y)*x).item() == grad.item() )
        g = grad.item()
        # compute w_new
        w_new = w - (g+w**2) * g
        # compute final loss J
        J = (w_new + x2 + y2)**2
        # computes derivative of J
        J.backward()
        #dw_new_dw = w_new.grad.item()
        dJ_dw = w.grad.item()
</code></pre>
<hr>
<p>Cross posted:</p>
<ul>
<li><a href="https://stackoverflow.com/questions/58359410/how-does-one-obtain-gradients-as-data-efficiently-in-pytorch" rel="nofollow noopener">https://stackoverflow.com/questions/58359410/how-does-one-obtain-gradients-as-data-efficiently-in-pytorch</a></li>
<li><a href="https://discuss.pytorch.org/t/how-does-one-obtain-gradients-as-data-efficiently/58059" class="inline-onebox">How does one obtain gradients as data efficiently?</a></li>
<li><a href="https://www.quora.com/unanswered/How-does-one-obtain-gradients-as-data-efficiently-in-PyTorch" rel="nofollow noopener">https://www.quora.com/unanswered/How-does-one-obtain-gradients-as-data-efficiently-in-PyTorch</a></li>
</ul>

username: albanD
cooked: <p>Hi,</p>
<p>I’m not sure what the question is here.<br>
As a note, you should never have to use <code>.data</code>. It should either be replaced by <code>.detach()</code> or <code>with torch.no_grad():</code> (there are other posts in this forum that discuss this).</p>

username: pinocchio
cooked: <p>Hi AlbanD. What I am trying to do is make sure that the gradient information <code>w.grad</code> is treated as a number if the computation graph and not as a variable that one can differentiate. i.e. treated as a constant. Does that make sense?</p>
<p>Would <code>.data</code> still make things go wrong? In what way? I tried some code and printed values and the differentiation worked as I would have expected so I am not sure what you are suggesting or how it fixes my scenario.</p>

username: pinocchio
cooked: <p>the whole code just for reference:</p>
<pre><code class="lang-auto">'''
This script is written to test:
 1)what happens if we differentiate through pytorch's backward pass and to see
    if its consistent with the doing a second derivative
'''
import torch
import torch.nn as nn

from grad_test_sympy import symbolic_test

import sys

from pdb import set_trace as st

def test_does_backward_backward_match_second_derivative(experiment_type,grad_computation_type='dl_dw_zero_grad'):
    '''
     1)what happens if we differentiate through pytorch's backward pass and to see
        if its consistent with the doing a second derivative

    :param str experiment_type:
        - grad_constant: this one is to make sure the sympy code and pytorch matches.
            when g is constant there is no second call of backwards so its just
            a sanity test to make sure things are working.
        - grad_constant_but_use_backward_on_loss: this one calls
            backward twice (one on the loss, second on J that depends on loss)
            and the expected result for me it's to match the sympy code.
            It doesn't match
        -
    '''
    ## variable declaration
    w = torch.tensor([2.0], requires_grad=True)

    x = torch.tensor([3.0], requires_grad=False)
    y = torch.tensor([4.0], requires_grad=False)

    x2 = torch.tensor([5.0], requires_grad=False)
    y2 = torch.tensor([6.0], requires_grad=False)

    if experiment_type == 'grad_constant':
        ## computes backard pass on J (i.e. dJ_dw) assuming g is constant
        # compute g
        if grad_computation_type == 'g_hard_coded':
            grad = 2*(w*x-y)*x # assumption, loss = (w*x - y)**2
            g = grad.item()
        elif grad_computation_type == 'dl_dw_zero_grad':
            # compute the backward pass dl_dw
            loss = (w*x-y)**2
            loss.backward()
            g = w.grad.item()
            # zero out the gradients on the orginal tensor
            w.grad.zero_() # note this operation does NOT make g zero.
        else:
            ww = torch.randn(w.size(), requires_grad=True)
            ww.data = w.clone()
            loss = (ww*x-y)**2
            loss.backward()
            grad = ww.grad
            g = grad.item()
        assert( (2*(w*x-y)*x).item() == grad.item() )
        # compute w_new
        w_new = w - (g+w**2) * g
        # compute final loss J
        J = (w_new + x2 + y2)**2
        # computes derivative of J
        J.backward()
        #dw_new_dw = w_new.grad.item()
        dJ_dw = w.grad.item()
    elif experiment_type == 'grad_constant_but_use_backward_on_loss':
        ## computes backard pass on J (i.e. dJ_dw) but g is was backwarded passed already
        # compute g
        loss = (w*x-y)**2
        loss.backward()
        #g = w.grad.item() # dl_dw
        g = w.grad # dl_dw
        g.requires_grad = True
        print(g)
        # compute w_new
        w_new = w - (g+w**2) * g
        # compute final loss J
        J = (w_new + x2 + y2)**2
        # computes derivative of J
        J.backward()
        #dw_new_dw = w_new.grad.item()
        dJ_dw = w.grad.item()
    elif experiment_type == 'grad_analytic_only_backward_on_J':
        ## computes backard pass on J (i.e. dJ_dw) assuming g is constant
        # compute g
        grad = 2*(w*x-y)*x # assumption, loss = (w*x - y)**2
        g = grad
        # compute w_new
        w_new = w - (g+w**2) * g
        # compute final loss J
        J = (w_new + x2 + y2)**2
        # computes derivative of J
        J.backward()
        #dw_new_dw = w_new.grad.item()
        dJ_dw = w.grad.item()
    else:
        raise ValueError(f'This: experiment_type={experiment_type}, is not a valid test')
    ##
    print('---- test results ----')
    print(f'experiment_type = {experiment_type}')
    print('-- Pytorch results')
    print(f'g = {g}')
    #print(f'dw_new_dw = {dw_new_dw}')
    print(f'dJ_dw = {dJ_dw}')
    print('-- Sympy results')
    g, dw_new_dw, dJ_dw = symbolic_test(experiment_type)
    print(f'g_SYMPY = {g}')
    #print(f'dw_new_dw_SYMPY = {dw_new_dw}')
    print(f'dJ_dw_SYMPY = {dJ_dw}')

if __name__ == '__main__':
    test_does_backward_backward_match_second_derivative(experiment_type='grad_constant')
    #test_does_backward_backward_match_second_derivative(experiment_type='grad_constant_but_use_backward_on_loss')
    #test_does_backward_backward_match_second_derivative(experiment_type='grad_analytic_only_backward_on_J')

</code></pre>

username: pinocchio
cooked: <p>seems I need to read this: <a href="https://discuss.pytorch.org/t/clone-and-detach-in-v0-4-0/16861/2" class="inline-onebox">Clone and detach in v0.4.0</a></p>

username: albanD
cooked: <p>For " the gradient information  <code>w.grad</code>  is treated as a number if the computation graph and not as a variable that one can differentiate" you want to use <code>w.grad.detach()</code> for further computations to achieve that.</p>
<p><code>.data</code> will work but can be misleading as it does not allow the autograd to perform all the correctness checks. Which means that you can introduce subtle bugs that will make the gradients wrong. You cannot introduce such bugs with <code>.detach()</code> and <code>torch.no_grad()</code>.</p>

username: pinocchio
cooked: <aside class="quote no-group" data-username="albanD" data-post="6" data-topic="58059">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"> albanD:</div>
<blockquote>
<p>For " the gradient information <code>w.grad</code> is treated as a number if the computation graph and not as a variable that one can differentiate" you want to use <code>w.grad.detach()</code> for further computations to achieve that.</p>
</blockquote>
</aside>
<p>After some thought I think I am confused about your suggestion because I was thinking how gradients work mathematically and they are based on partial derivatives. Even if <code>w.grad.detach()</code> is called, when we do a partial derivative with respect to an independent variable (i.e. a leaf node in the computation graph) all other variables are constants. Thus, wether the gradient variable (i.e. <code>w.grad</code>) collects gradients or not I would have expected is irrelevant.</p>
<p>But your comment seems to suggest otherwise. Why is that the case? What part of my reasoning is wrong?</p>