Autograd: `require_grad=True`

username: andrew_su
cooked: <p>Hello</p>
<p>I am very very confused about the gradients when we do back-propagation. According to the Pytorch’s Autograd webpage, it says that if we want a tensor/variable to contain gradient information, we need to set <code>require_grad=True</code>. I have gone through many examples on the web, for instance: <a href="https://github.com/pytorch/examples/blob/master/reinforcement_learning/reinforce.py" rel="nofollow noopener">reinforce algorithm example </a>, or <a href="https://github.com/Finspire13/pytorch-policy-gradient-example/blob/master/pg.py" rel="nofollow noopener">policy gradient example</a>, they didn’t set <code>require_grad=True</code>.</p>
<p>I am wondering, in that case, when they do <code>loss.backward()</code>, and they didn’t even specify a single instance that <code>require_grad=True</code>, what exactly is happening? Do they event do back-propagation?</p>

username: Naruto-Sasuke
cooked: <p>Hi，we do not need the gradient of input(In most cases, they are useless, unless some special works like neural style transfer, where we only iteratively change the input to optimize the total loss).  Usually, we only want to get the model trained. Parameters in each layer are default to be <code>requires_grad=True</code>.<br>
So there is no worry about it.</p>

username: andrew_su
cooked: <p>Thanks Naruto for your input. Are you saying that, if we are building network in the following fashion, we don’t need to worry about the setting the any tensor to have <code>require_grad = True</code>?<br>
I am very confused because I was reading the tutorial from here: <a href="https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html" rel="nofollow noopener">autograd tutorial</a>, and it was emphasizing about the <code>require_grad= True</code>, where as in the example below, we don’t care about this.</p>
<p><strong>So in short, my question is, we only use <code>require_grad=True</code> when we are building network from scratch (like the pytorch tutorial link). If we are building network in the fashion given by below’s code, we don’t need to worry about this?</strong></p>
<pre><code class="lang-auto">class Policy(nn.Module):
    def __init__(self):
        super(Policy, self).__init__()
        self.affine1 = nn.Linear(4, 128)
        self.affine2 = nn.Linear(128, 2)

        self.saved_log_probs = []
        self.rewards = []

    def forward(self, x):
        x = F.relu(self.affine1(x))
        action_scores = self.affine2(x)
        return F.softmax(action_scores, dim=1)


policy = Policy()
optimizer = optim.Adam(policy.parameters(), lr=1e-2)
eps = np.finfo(np.float32).eps.item()
</code></pre>

username: Naruto-Sasuke
cooked: <p>This is a snippet of <code>Linear</code> layer,  as you can see.  The learnable weights are registered as <code>Parameter</code> which is default as <code>requires_grad=True</code>, see <a href="https://pytorch.org/docs/stable/nn.html#parameters" rel="nofollow noopener">here</a>.  Input of the networks needs no gradient(They are useless in most cases).   So everything is fine.</p>
<pre><code class="lang-python">    def __init__(self, in_features, out_features, bias=True):
        super(Linear, self).__init__()
        self.in_features = in_features
        self.out_features = out_features
        self.weight = Parameter(torch.Tensor(out_features, in_features))
        if bias:
            self.bias = Parameter(torch.Tensor(out_features))
        else:
            self.register_parameter('bias', None)
        self.reset_parameters()
</code></pre>

username: andrew_su
cooked: <p>Thanks Naruto. That clarifies things a lot for me.</p>
<p>One more question, I see many examples just <code>Autograd.Variable</code>, but now, <code>Variable is being deprecated</code>, do you see the need of using <code>variable</code>?</p>

username: Naruto-Sasuke
cooked: <p>I cannot think up one case, it is beyond me…</p>

username: erm
cooked: <p>wow, i had the same questions after following tutorials, thanks for clarifying. I was scratching my head thinking what was my NN network doing with no requires_grad= True anywhere.</p>