TypeError: 'NoneType' object is not iterable

username: abhinav_choudhury
cooked: <p>I am trying to implement a Deep Q network (DQN) using a graph convolution network (GCN) using the dynamic graph library (DGL). The base code is taken from this repository (<a href="https://github.com/louisv123/COLGE/blob/master/agent.py" rel="nofollow noopener">https://github.com/louisv123/COLGE/blob/master/agent.py</a>). However after I calculate the loss between the policy network and target network and run loss.backward() I get  <code>TypeError: 'NoneType' object is not iterable</code> . I have printed the loss value and it is not None.</p>
<p>I ran the original code from the repository and it is running perfectly. I have also implemented the GCN code in DGL and it seems to run. I have also visualized the graph using the torchviz. But I am unable to find why it is giving an error.</p>
<p>The code snippet is given below:</p>
<pre><code class="lang-auto">current_q_values= self.model(last_observation_tens, self.G)
next_q_values=current_q_values.clone()
current_q_values[range(self.minibatch_length),action_tens,:] = target
L=self.criterion(current_q_values,next_q_values)
print('loss:',L.item())
self.optimizer.zero_grad()
L.backward(retain_graph=True)
self.optimizer.step()
</code></pre>
<p>loss: 1461729.125</p>
<pre><code class="lang-auto">  ---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
&lt;ipython-input-17-cd5e862dd609&gt; in &lt;module&gt;()
     62 
     63 if __name__ == "__main__":
---&gt; 64     main()

7 frames
&lt;ipython-input-17-cd5e862dd609&gt; in main()
     55         print("Running a single instance simulation...")
     56         my_runner = Runner(env_class, agent_class, args.verbose)
---&gt; 57         final_reward = my_runner.loop(graph_dic,args.ngames,args.epoch, args.niter)
     58         print("Obtained a final reward of {}".format(final_reward))
     59         agent_class.save_model()

&lt;ipython-input-14-45cfc883a37b&gt; in loop(self, graphs, games, nbr_epoch, max_iter)
     45                         # if self.verbose:
     46                         #   print("Simulation step {}:".format(i))
---&gt; 47                         (obs, act, rew, done) = self.step()
     48                         action_list.append(act)
     49 

&lt;ipython-input-14-45cfc883a37b&gt; in step(self)
     16         #reward = torch.tensor([reward], device=device)
     17 
---&gt; 18         self.agent.reward(observation, action, reward,done)
     19 
     20         return (observation, action, reward, done)

&lt;ipython-input-16-76d612e8663c&gt; in reward(self, observation, action, reward, done)
    129               print('loss:',L.item())
    130               self.optimizer.zero_grad()
--&gt; 131               L.backward(retain_graph=True)
    132               self.optimizer.step()
    133 

/usr/local/lib/python3.6/dist-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)
    148                 products. Defaults to ``False``.
    149        
--&gt; 150         torch.autograd.backward(self, gradient, retain_graph, create_graph)
    151 
    152     def register_hook(self, hook):

/usr/local/lib/python3.6/dist-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
     97     Variable._execution_engine.run_backward(
     98         tensors, grad_tensors, retain_graph, create_graph,
---&gt; 99         allow_unreachable=True)  # allow_unreachable flag
    100 
    101 

/usr/local/lib/python3.6/dist-packages/torch/autograd/function.py in apply(self, *args)
     75 
     76     def apply(self, *args):
---&gt; 77         return self._forward_cls.backward(self, *args)
     78 
     79 

/usr/local/lib/python3.6/dist-packages/dgl/backend/pytorch/tensor.py in backward(ctx, grad_out)
    394     def backward(ctx, grad_out):
    395         reducer, graph, target, in_map, out_map, in_data_nd, out_data_nd, degs \
--&gt; 396             = ctx.backward_cache
    397         ctx.backward_cache = None
    398         grad_in = None

TypeError: 'NoneType' object is not iterable```

Kindly help.</code></pre>

username: albanD
cooked: <p>Hi,</p>
<p>Do you get any more information from the stack trace?<br>
Can you run with anomaly mode enabled <a href="https://pytorch.org/docs/stable/autograd.html#anomaly-detection" rel="nofollow noopener">doc</a> to see if you get a better stack trace?</p>

username: abhinav_choudhury
cooked: <p>Thank you for the reply.<br>
I ran the code with anomaly mode enabled</p>
<pre><code class="lang-auto">with autograd.detect_anomaly():
              (last_observation_tens, action_tens, reward_tens, observation_tens)=self.get_sample()
              target = reward_tens + self.gamma *torch.max(self.model_target(observation_tens, self.G) + observation_tens * (-1e5), dim=1)[0]
              current_q_values= self.model(last_observation_tens, self.G)
              next_q_values=current_q_values.clone()
              current_q_values[range(self.minibatch_length),action_tens,:] = target
              L=self.criterion(current_q_values,next_q_values)
              print('loss:',L.item())
              self.optimizer.zero_grad()
              L.backward(retain_graph=True)
              self.optimizer.step()'''
</code></pre>
<p>and got the following output<br>
<code>/pytorch/torch/csrc/autograd/python_anomaly_mode.cpp:40: UserWarning: No forward pass information available. Enable detect anomaly during forward pass for more information.</code><br>
How do I enable detect anomaly during forward pass?<br>
I also updated the question with the full stack trace.</p>

username: albanD
cooked: <p>You seem to have it already available during the forward…<br>
Can you provide a small code sample (30 lines) that reproduces this issue?</p>

username: reekjohns
cooked: <p>The error is self-explanatory. You are trying to subscript an object which you think is a list or dict, but actually is None. This means that you tried to do:</p>
<p><code>None[something]</code></p>
<p>This  error means that you attempted to index an object that doesn’t have that functionality. You might have noticed that the method sort() that only modify the list have no return value printed – they return the default None. ‘NoneType’ object is not <a href="http://net-informations.com/python/err/nonetype.htm" rel="nofollow noopener">subscriptable</a> is the one thrown by python when you use the square bracket notation object[key] where an object doesn’t define the <strong>getitem</strong> method . This is a design principle for all mutable data structures in Python.</p>

username: Digi_TMG
cooked: <p><a href="https://360digitmg.com/python-typeerror-nonetype-object-is-not-subsriptable" rel="nofollow noopener">TypeError: ‘NoneType’ object is not subscriptable</a><br>
The error is self-explanatory. You are trying to subscript an object which is a None actually…</p>
<p>Example 1</p>
<p>list1=[5,1,2,6]        # Create a simple list<br>
order=list1.sort()  # sort the elements in created list and store into another variable.<br>
order[0]               # Trying to access the first element after sorting</p>
<p>TypeError     Traceback (most recent call last)</p>
<p>in ()<br>
list1=[5,1,2,6]<br>
order=list1.sort()<br>
----&gt; order[0]</p>