From one iteration to another Tensor is converting to a NoneType

username: yona
cooked: <p>Hello all !<br>
New in pytorch trying to code PGD .<br>
I have this error for the second iteration :<br>
sign(): argument ‘input’ (position 1) must be Tensor, not NoneType</p>
<p>I don’t understand why.<br>
Any help/ advice would be appreciated .</p>
<p>Find bellow the code .</p>
<p>Thanks</p>
<pre><code class="lang-auto">def compute(self, x, y):
    #  x ( 100,1,28, 28) and  y = [100]
    x_adv = x.requires_grad_(True)
    for i in range(0,self.num_iter):
      print("--------iteration"+str(i)+"-----------------")
      print("x_adv", x_adv)
      criterion = nn.NLLLoss()
      prediction = self.model(x_adv) 
      loss =  criterion(prediction, y) 
      print("loss", loss)  
      loss.backward() 
      
      print(type(x_adv), x_adv.shape)
      x_adv.requires_grad_() 
      gradients = x_adv.grad
      print("gradients",gradients)
      gradients = self.alpha*torch.sign(gradients)
      print("gradients with sign and alpha ", gradients)
      x_adv = x_adv + gradients  
      print("x_adv + gradients ", x_adv )
      print(type(x_adv))
      #x_adv = torch.max(torch.min(x_adv, x + self.eps), x - self.eps) # Project back into l_norm ball and correct range
      
</code></pre>

username: ptrblck
cooked: <p>You are overwriting <code>x_adv</code> in this line of code:</p>
<pre><code class="lang-python">x_adv = x_adv + gradients
</code></pre>
<p>which will create a non-leaf tensor and you should get a warning when trying to access it’s <code>.grad</code> attribute:</p>
<pre><code class="lang-python">UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the gradient for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations.
</code></pre>
<p>You could either use the suggested <code>.retain_grad()</code> operation or create a new leaf tensor via <code>x_adv = x_adv.detach().requires_grad_(True)</code> at the end of the iteration.</p>

username: Hiqmat_Nisa
cooked: <p>Hi, I am also getting the above mention error after first iteration. Below is the two layers  Residual LSTM. The input size is different for each layer but the hidden size is the same (256). For the first layer the input size is 1088 for second it is 256.  I think there is error in <code>self.weight_ir</code> but I am not sure. Can you guide me?</p>
<pre><code class="lang-auto">class RLSTMCell(jit.ScriptModule):
    def __init__(self, input_size, hidden_size, dropout=0.):
        super(RLSTMCell, self).__init__()
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.weight_ih = Parameter(torch.zeros(4 * hidden_size, input_size))
        torch.nn.init.xavier_uniform_(self.weight_ih)
        self.weight_hh = Parameter(torch.zeros(4 * hidden_size, hidden_size))
        torch.nn.init.xavier_uniform_(self.weight_hh)
        self.bias_ih = Parameter(torch.zeros(4 * hidden_size))
        self.bias_hh = Parameter(torch.zeros(4 * hidden_size))
        self.weight_ir = Parameter(torch.zeros(hidden_size, input_size))
        torch.nn.init.xavier_uniform_(self.weight_ir)


    @jit.script_method
    def forward(self, input: Tensor, state: Tuple[Tensor, Tensor]) -&gt; Tuple[Tensor, Tuple[Tensor, Tensor]]:
        hx, cx = state
   
        gates = (torch.mm(input, self.weight_ih.t()) + self.bias_ih +
                 torch.mm(hx, self.weight_hh.t()) + self.bias_hh)
        ingate, forgetgate, cellgate, outgate = gates.chunk(4, 1)

        ingate = torch.sigmoid(ingate)
        forgetgate = torch.sigmoid(forgetgate)
        cellgate = torch.tanh(cellgate)
        outgate = torch.sigmoid(outgate)

        cy = (forgetgate * cx) + (ingate * cellgate)
        
        ry = torch.tanh(cy) #eqution 12 in the paper https://arxiv.org/pdf/1701.03360v3.pdf
        #hy = outgate * torch.tanh(cy)
        if self.input_size == self.hidden_size:
            hy = outgate * (ry + input) #eqution 15 in the paper
        else:
            hy = outgate * (ry + torch.mm(input, self.weight_ir.t()))
        return hy, (hy, cy)
</code></pre>
<pre><code class="lang-auto">class LSTMLayer(jit.ScriptModule):
    def __init__(self, input_size, hidden_size):
        super(LSTMLayer, self).__init__()
        self.layer1 = RLSTMCell(input_size, hidden_size)
        self.layer2 = RLSTMCell(hidden_size, hidden_size)

    @jit.script_method
    def forward(self, input: Tensor, state: Tuple[Tensor, Tensor]) -&gt; Tuple[Tensor, Tuple[Tensor, Tensor]]:
        inputs = input.unbind(0)
        outputs = torch.jit.annotate(List[Tensor], [])
        for i in range(len(inputs)):
            out, state = self.layer1(inputs[i], state)
            out, state= self.layer2(state[0], state)
            outputs += [out]
        return torch.stack(outputs), state
</code></pre>

username: Hiqmat_Nisa
cooked: <p>When I made the input size of these RLSTM same, like</p>
<pre><code class="lang-auto">        self.layer1 = RLSTMCell(input_size, hidden_size)
        self.layer2 = RLSTMCell(input_size, hidden_size)
</code></pre>
<p>then it works fine when I access param.grad. But I guess in Multilayer lstms, 2nd layer lstm takes input from the output of 1st layer lstm. In above situation I am sending the original input to both layers. Is this approach is also okay?</p>