Unable to use torch.cat

LastJedi: <p>I am receiving an error whilst looping through the batch The error: expected Tensor as element 0 in argument 0, but got builtin_function_or_method</p>
<p>I followed a tutorial on Udemy, but I am not sure what the issue is.</p>
<p>The Code:</p>
<pre><code class="lang-auto">def sample(self, batch_size):
        
        assert self.can_sample(batch_size)
        
        batch = random.sample(self.memory, batch_size)
        
    
        batch = zip(*batch)

        
        return [torch.cat(items) for items in batch]


    def can_sample(self, batch_size):
        
        return len(self.memory) &gt;= batch_size * 10
</code></pre>
<p>The error:</p>
<pre><code class="lang-auto">TypeError                                 Traceback (most recent call last)
Cell In[52], line 1
----&gt; 1 stats = deep_sarsa(q_network, policy, 2000, epsilon = 0.01)

Cell In[51], line 24, in deep_sarsa(q_network, policy, episodes, alpha, batch_size, gamma, epsilon)
     21 memory.insert([state, action, rewards, done, next_state])
     23 if (memory.can_sample(batch_size)):
---&gt; 24     state_b, action_b, reward_b, done_b, next__state_b = memory.sample(batch_size)
     26     qsa_b = q_network(state_b).gather(1, action_b)
     27     next_action_b = policy(next_state_b, epsilon)

Cell In[50], line 46, in ReplayMemory.sample(self, batch_size)
     38 batch = zip(*batch)
     41 #batch = torch.tensor(batch, dtype=torch.int8)
     42 #torch.cat() -&gt; concatinate the elements in a single tensor.
---&gt; 46 return [torch.cat(items) for items in batch]

Cell In[50], line 46, in &lt;listcomp&gt;(.0)
     38 batch = zip(*batch)
     41 #batch = torch.tensor(batch, dtype=torch.int8)
     42 #torch.cat() -&gt; concatinate the elements in a single tensor.
---&gt; 46 return [torch.cat(items) for items in batch]

TypeError: expected Tensor as element 0 in argument 0, but got builtin_function_or_method
</code></pre>

J_Johnson: <p>Seems to be used improperly. Not at a computer, but this should work:</p>
<pre><code class="lang-auto">import torch

# build list of tensors
x=[]
for i in range(10):
    x.append(torch.rand(4,3,5))

# test cat function
y = torch.cat([_ for _ in x])
</code></pre>

LastJedi: <p>I tried your method, does not work.<br>
I get the following:<br>
expected Tensor as element 0 in argument 0, but got tuple</p>

J_Johnson: <aside class="quote no-group" data-username="J_Johnson" data-post="2" data-topic="172982">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/j_johnson/48/55494_2.png" class="avatar"> J_Johnson:</div>
<blockquote>
<pre><code class="lang-auto">import torch

# build list of tensors
x=[]
for i in range(10):
    x.append(torch.rand(4,3,5))

# test cat function
y = torch.cat([_ for _ in x])
</code></pre>
</blockquote>
</aside>
<p><a class="mention" href="/u/lastjedi">@LastJedi</a> the above code worked for me. But it seems you do not have a list of tensors, but a list of tuples. Without knowing what your <code>memory</code> is, exactly, or how you want to use it, a specific solution is not possible.</p>
<p>You could try adding an index to it, i.e. <code>torch.cat([_[0] for _ in x])</code>. But that assumes the first index is what you need, that it is a tensor, and that you want to concatenate it on dim=0.</p>

LastJedi: <p>Here is the class for the replay memory</p>
<pre><code class="lang-auto">class ReplayMemory:
    
    
    def __init__(self, capacity = 1000000):
        
        self.capacity = capacity
        self.memory = []
        self.position = int(0)
        
    
    # insert(). method which will allow us to insert a state transition in memory\
    
    def insert(self, transition):
        
        if (len(self.memory) &lt; self.capacity):
            
            self.memory.append(None)
        
        self.memory[int(self.position)] = transition
        
        self.position = (self.position + 1) % self.capacity
    
    
    
    #Sample(). Sample batches of exp in memory
    
    def sample(self, batch_size):
        
        assert self.can_sample(batch_size)
        
        batch = random.sample(self.memory, batch_size)
        
        #list of transitions, has the shape [[s, a, r, s`], [s, a, r, s`]] ( 2 transitions present)
        # we want the following -&gt; [[s,s], [a,a], [r,r], [s`,s`]]
        
        # * -&gt; unpack all the transitions from the list
        # zip() -&gt; pick the first element in each one of the transitions an group them
        batch = zip(*batch)
        
       
       
        return [torch.cat(items) for items in batch]
        
        
    
    def can_sample(self, batch_size):
        
        return len(self.memory) &gt;= batch_size * 10
    
    
    def __len__(self):
        return len(self.memory)
</code></pre>
<p>Deep SARSA algorithm that was followed from a Udemy course</p>
<pre><code class="lang-auto">def deep_sarsa(q_network, policy, episodes, alpha = 0.001, batch_size = 32, gamma = 0.99, epsilon = 0.05):
    
    
    optim = AdamW(q_network.parameters(), lr = alpha)
    
    memory = ReplayMemory(capacity = 1e6)
    
    stats =  {'MSE Loss': [], 'Returns': []}
    
    
    for episodes in tqdm(range(1, episodes + 1)):
        
        state = env.reset()
        ep_return = 0.
        done = False
        
        while not done:
            
            action = policy(state, epsilon)
            next_state, rewards, done, _ = env.step(action)
            memory.insert([state, action, rewards, done, next_state])
            
            if (memory.can_sample(batch_size)):
                state_b, action_b, reward_b, done_b, next__state_b = memory.sample(batch_size)
                
                qsa_b = q_network(state_b).gather(1, action_b)
                next_action_b = policy(next_state_b, epsilon)
                
                next_qsa_b = target_q_network(next_state_b).gather(1, next_action_b)
            
                target_b = reward_b + -done_b * gamma * next_qsa_b
            
                loss = F.mse_loss(qsa_b, target_b)
                
                q_network.zero_grad() 

                loss.backward()
                
                optim.step()
                
                stats['MSE Loss'].append(loss.item())
                
            state = next_state

            ep_return += rewards().item()
            
        stats['Returns'].append(ep_return)
        
        if(episodes % 10 == 0):
            
            target_q_network.load_state_dict(q_network.state.dict())
    
    return stats
    
</code></pre>

LastJedi: <p>I solved the issue.</p>
<p>There was a typo in my code. Instead of using .float(), I used .float <img src="https://discuss.pytorch.org/images/emoji/apple/man_facepalming.png?v=12" title=":man_facepalming:" class="emoji" alt=":man_facepalming:" loading="lazy" width="20" height="20"></p>