Loss function that uses the output to calculate index which is used to get value from an array and then loss is calculated

username: Aditya_Prakash
cooked: <p>In my neural network (RNN), I am defining the loss function such that the output of the neural network is used to find the index (binary) and then the index is used to extract the required element from an array which in turn will be used to calculate MSELoss.</p>
<p>However, the program gives <code>parameter().grad = None</code> error which is mostly because the graph is breaking somewhere. What is the problem with the error function defined.</p>
<p>Framework: Pytorch</p>
<p>The codes are as follow:<br>
Neural Network:</p>
<pre><code class="lang-auto">class RNN(nn.Module):
  def __init__(self):
    super(RNN, self).__init__()
    self.hidden_size = 8
    # self.input_size = 2
    self.h2o = nn.Linear(self.hidden_size, 1)
    self.h2h = nn.Linear(self.hidden_size, self.hidden_size)
    self.sigmoid = nn.Sigmoid()
  def forward(self,hidden):
    output = self.h2o(hidden)
    output = self.sigmoid(output)
    hidden = self.h2h(hidden)
    return output, hidden
  def init_hidden(self):
    return torch.zeros(1, self.hidden_size)
</code></pre>
<p>Loss Function, train step and training</p>
<pre><code class="lang-auto">rnn = RNN()
criterion = nn.MSELoss()

def loss_function(previous, output, index):
  code = 2*(output &gt; 0.5).long()
  current = Q_m2[code:code+2, i]
  return criterion(current, previous), current

def train_step():
  hidden = rnn.init_hidden()
  rnn.zero_grad()
  # Q_m2.requires_grad = True
  # Q_m2.create_graph = True 
  loss = 0
  previous = Q_m[0:2, 0]
  for i in range(1, samples):
    output, hidden = rnn(hidden)
    l, previous = loss_function(previous, output, i)
    loss+=l
  loss.backward()
  # Q_m2.retain_grad()
  for p in rnn.parameters():
    p.data.add_(p.grad.data, alpha=-0.05)
  return output, loss.item()/(samples - 1)

def training(epochs):
  running_loss = 0
  for i in range(epochs):
    output, loss = train_step()
    print(f'Epoch Number: {i+1}, Loss: {loss}')
    running_loss +=loss
</code></pre>
<p>Q_m2</p>
<pre><code class="lang-auto">Q_m = np.zeros((4, samples))
for i in range(samples):
  Q_m[:,i] = q_x(U_m[:,i])
Q_m = torch.FloatTensor(Q_m)
Q_m2 = Q_m
Q_m2.requires_grad = True
Q_m2.create_graph = True
</code></pre>
<p>Error:</p>
<pre><code class="lang-auto">&lt;ipython-input-36-feefd257c97a&gt; in train_step()
     21   # Q_m2.retain_grad()
     22   for p in rnn.parameters():
---&gt; 23     p.data.add_(p.grad.data, alpha=-0.05)
     24   return output, loss.item()/(samples - 1)
     25 

AttributeError: 'NoneType' object has no attribute 'data'
</code></pre>

username: KFrank
cooked: <p>Hi Aditya!</p>
<aside class="quote no-group" data-username="Aditya_Prakash" data-post="1" data-topic="132360" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aditya_prakash/48/42337_2.png" class="avatar"> Aditya_Prakash:</div>
<blockquote>
<p>However, the program gives <code>parameter().grad = None</code> error which is mostly because the graph is breaking somewhere.</p>
</blockquote>
</aside>
<p>I haven’t looked at your code or error messages in any detail.</p>
<p>But, yes, your “graph is breaking.”</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">  code = 2*(output &gt; 0.5).long()
</code></pre>
</blockquote>
</aside>
<p><code>code</code>, as a function of <code>output</code>, is not (usefully) differentiable, so,<br>
regardless of what you subsequently do with <code>code</code>, you won’t be<br>
able to back propagate through it.</p>
<p>I have no idea whether this would make sense for your use case,<br>
but one <em>possibility</em> would run as follows:</p>
<p>As I read it, <code>code</code> is calculated to be either <code>0</code> or <code>2</code>.  You could<br>
instead interpret <code>output</code> (processed appropriately, as necessary)<br>
to be the probability that <code>code</code> should be <code>0</code> vs. <code>2</code>, and then use<br>
that probability to form a weighted average of the <code>0</code> and <code>2</code> entries<br>
in your <code>Q_m2</code> array.</p>
<p>This will be differentiable and you will be able to backpropagate (but<br>
I’m not saying that it would make sense …).</p>
<p>Best.</p>
<p>K. Frank</p>