RuntimeError: invalid argument 4: Index tensor must have same dimensions as input tensor at

username: Chris_Freiburg
cooked: <p>Hi,</p>
<p>Iam new to this forum and to pytorch.<br>
Could someone help me with that error<br>
RuntimeError: invalid argument 4: Index tensor must have same dimensions as input tensor at</p>
<p>def learn(self, batch, gamma):<br>
“”"Update value parameters using given batch of experience tuples.</p>
<pre><code>    Params
    ======
        experiences (Tuple[torch.Variable]): tuple of (s, a, r, s', done) tuples 
        gamma (float): discount factor
    """
    
    #states, actions, rewards, next_states, dones = experiences
    states = np.array([each[0][0] for each in batch], ndmin=3)
    actions = np.array([each[0][1] for each in batch])
    rewards = np.array([each[0][2] for each in batch]) 
    next_states = np.array([each[0][3] for each in batch], ndmin=3)

    dones = np.array([each[0][4] for each in batch])
    #dones  = np.array([map(lambda x: 1 if x else 0, dones)],dtype=np.float16)
    dones = dones.astype(np.int16)
    states = torch.from_numpy(states).float().cuda()
    next_states = torch.from_numpy(next_states).float().cuda()
    rewards = torch.from_numpy(rewards).float().cuda()
    actions = torch.from_numpy(actions).long().cuda()
    dones = torch.from_numpy(dones).float().cuda()
    #next_states = torch.from_numpy(next_states).float().unsqueeze(0).cuda()
    #print(states.shape)
    states = states.view(32,8)
    #print(states.shape)
    next_states = next_states.view(32,8)
    ## TODO: compute and minimize the loss
    "*** YOUR CODE HERE ***"
    Q_targets_next = self.qnetwork_target(next_states).detach().max(1)[0].unsqueeze(1)
    print(rewards.shape)
    print(Q_targets_next)
    Q_targets =  rewards + (gamma * Q_targets_next * (1-dones))
    
    Q_expected = self.qnetwork_local(states).gather(1, actions)
    
    loss = F.mse_loss(Q_expected, Q_targets)
    self.optimizer.zero_grad()
    loss.backward()
    self.optimizer.step()

    # ------------------- update target network ------------------- #
    self.soft_update(self.qnetwork_local, self.qnetwork_target, self.tau) 
</code></pre>
<p>I don’t know if this gather is correct here found the code online and it worked<br>
now I try to modify to my problem</p>
<p>Thank you very much for your help<br>
If you need more information let me know<br>
Best Regards<br>
Chris</p>

username: ptrblck
cooked: <p>Which line throws this error?<br>
Could you post the shapes and if necessary the values of all used tensors in this operation?</p>

username: Chris_Freiburg
cooked: <p>oh yes I forgot to post the trace-back</p>
<h2>/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:67: RuntimeWarning: divide by zero encountered in double_scalars</h2>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in ()<br>
----&gt; 1 scores = dqn()<br>
2<br>
3 # plot the scores<br>
4 fig = plt.figure()<br>
5 ax = fig.add_subplot(111)</p>
<p> in dqn(n_episodes, max_t, eps_start, eps_end, eps_decay)<br>
23             score += reward<br>
24             <span class="hashtag">#reward</span> = np.tanh(reward)<br>
—&gt; 25             agent.step(state, action, reward, next_state, done)<br>
26             state = next_state<br>
27             if done:</p>
<p> in step(self, state, action, reward, next_state, done)<br>
57             if np.count_nonzero(self.memory.tree.tree) &gt; self.batch_size:<br>
58                 tree_idx, batch, ISWeights_mb = self.memory.sample(BATCH_SIZE)<br>
—&gt; 59                 self.learn(batch, GAMMA)<br>
60<br>
61     def act(self, state, eps=0.):</p>
<p> in learn(self, batch, gamma)<br>
114         Q_targets =  rewards + (gamma * Q_targets_next * (1-dones))<br>
115<br>
–&gt; 116         Q_expected = self.qnetwork_local(states).gather(1, actions)<br>
117<br>
118         loss = F.mse_loss(Q_expected, Q_targets)</p>
<p>RuntimeError: invalid argument 4: Index tensor must have same dimensions as input tensor at /pytorch/aten/src/THC/generic/THCTensorScatterGather.cu:16</p>
<p>Thank you for your help</p>

username: ptrblck
cooked: <p>Thanks for the stack trace. Could you print out the shapes of <code>self.qnetwork_local(states)</code> and <code>actions</code>? If might be <code>actions</code> has to be unsqueezed, but I would need to know your shapes.</p>

username: Chris_Freiburg
cooked: <p>for actions : torch.Size([32])<br>
for the network: torch.Size([32, 4])</p>

username: Chris_Freiburg
cooked: <hr>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in ()<br>
----&gt; 1 scores = dqn()<br>
2<br>
3 # plot the scores<br>
4 fig = plt.figure()<br>
5 ax = fig.add_subplot(111)</p>
<p> in dqn(n_episodes, max_t, eps_start, eps_end, eps_decay)<br>
23             score += reward<br>
24             <span class="hashtag">#reward</span> = np.tanh(reward)<br>
—&gt; 25             agent.step(state, action, reward, next_state, done)<br>
26             state = next_state<br>
27             if done:</p>
<p> in step(self, state, action, reward, next_state, done)<br>
57             if np.count_nonzero(self.memory.tree.tree) &gt; self.batch_size:<br>
58                 tree_idx, batch, ISWeights_mb = self.memory.sample(BATCH_SIZE)<br>
—&gt; 59                 self.learn(batch, GAMMA)<br>
60<br>
61     def act(self, state, eps=0.):</p>
<p> in learn(self, batch, gamma)<br>
120         Q_expected = self.qnetwork_local(states).gather(1, actions.unsqueeze(1))<br>
121<br>
–&gt; 122         loss = F.mse_loss(Q_expected, Q_targets)<br>
123         self.optimizer.zero_grad()<br>
124         loss.backward()</p>
<p>/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py in mse_loss(input, target, size_average, reduce)<br>
1567     “”"<br>
1568     return _pointwise_loss(lambda a, b: (a - b) ** 2, torch._C._nn.mse_loss,<br>
-&gt; 1569                            input, target, size_average, reduce)<br>
1570<br>
1571</p>
<p>/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py in _pointwise_loss(lambd, lambd_optimized, input, target, size_average, reduce)<br>
1535         return torch.mean(d) if size_average else torch.sum(d)<br>
1536     else:<br>
-&gt; 1537         return lambd_optimized(input, target, size_average, reduce)<br>
1538<br>
1539</p>
<p>RuntimeError: input and target shapes do not match: input [32 x 1], target [32 x 32] at /pytorch/aten/src/THCUNN/generic/MSECriterion.cu:15</p>

username: Chris_Freiburg
cooked: <p>I tried and it worked but now a have a new problem<br>
Is there any tutorial where they explain this stuff with unsqueeze etc , because I have no idea what I’m doing there OK I see they need to have the right shape but still</p>
<p>Thanks for your help<br>
Maybe you also have an idea for this one</p>

username: Chris_Freiburg
cooked: <p>I finally managed, after a lot of reshaping the tensors it worked thanks for your help</p>

username: Bushra178
cooked: <p>I am facing the same error. can u plz help me.</p>