Issues with concatenating tensors for policy history

maxmatical: <p>I’m following an older tutorial on policy gradient RL for cartpole <a href="https://medium.com/@ts1829/policy-gradient-reinforcement-learning-in-pytorch-df1383ea0baf" rel="nofollow noopener">seen here</a>, and I’m getting the following error for the select action step</p>
<pre><code class="lang-auto">class Policy(nn.Module):
    def __init__(self):
        super(Policy, self).__init__()
        self.state_space = env.observation_space.shape[0]
        self.action_space = env.action_space.n
        
        self.l1 = nn.Linear(self.state_space, 128, bias=False)
        self.l2 = nn.Linear(128, self.action_space, bias=False)
        
        self.gamma = gamma
        
        # Episode policy and reward history 
        self.policy_history = Variable(torch.Tensor()) 
        self.reward_episode = []
        # Overall reward and loss history
        self.reward_history = []
        self.loss_history = []
def forward(self, x):    
        model = torch.nn.Sequential(
            self.l1,
            nn.Dropout(p=0.6),
            nn.ReLU(),
            self.l2,
            nn.Softmax(dim=-1)
        )
        return model(x)
policy = Policy()
optimizer = optim.Adam(policy.parameters(), lr=learning_rate)
</code></pre>
<pre><code class="lang-auto">def select_action(state):
    #Select an action (0 or 1) by running policy model and choosing based on the probabilities in state
    state = torch.from_numpy(state).type(torch.FloatTensor)
    state = policy(Variable(state))
    c = Categorical(state) # turns the 
    action = c.sample()
    
    # Add log probability of our chosen action to our history    
    if policy.policy_history.dim() != 0:
        policy.policy_history = torch.cat([policy.policy_history, c.log_prob(action)])
    else:
        policy.policy_history = (c.log_prob(action))
    return action
</code></pre>
<p>I get the following error when I begin to train the network</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-21-c1f1eef9c02f&gt; in &lt;module&gt;()
      1 episodes = 1000
----&gt; 2 main(episodes)

&lt;ipython-input-20-2b725c0b3414&gt; in main(episodes)
      7 
      8         for time in range(1000):
----&gt; 9             action = select_action(state)
     10             # Step through environment using chosen action
     11             state, reward, done, _ = env.step(action.data[0])

&lt;ipython-input-18-f02a0e7d07a5&gt; in select_action(state)
      8     # Add log probability of our chosen action to our history
      9     if policy.policy_history.dim() != 0:
---&gt; 10         policy.policy_history = torch.cat([policy.policy_history, c.log_prob(action)])
     11     else:
     12         policy.policy_history = (c.log_prob(action))

RuntimeError: zero-dimensional tensor (at position 1) cannot be concatenated
</code></pre>
<p>How can I fix this error?</p>

Yara: <p>Hey there. I am facing the same issue. Were you able to solve it? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>