Stack expects each equal size

username: Tim5
cooked: <p><code>stack expects each tensor to be equal size but got [7, 768] at entry 0 and [8, 768] at entry 1</code></p>
<pre><code>    ca_embedding = torch.stack(tns_embedding) 
    embeddings.append(ca_embedding)
</code></pre>
<p>the shape for the first is <code>ca_embedding shape is  torch.Size([7, 768])</code><br>
the shape for the second is <code>ca_embedding shape is  torch.Size([8, 768])</code></p>

username: anantguptadbl
cooked: <p><a class="mention" href="/u/tim5">@Tim5</a></p>
<p>Recreating the problem</p>
<pre><code class="lang-auto">a=torch.rand(7, 768)
b=torch.rand(8, 768)
torch.stack([a,b])
</code></pre>
<pre><code class="lang-auto">RuntimeError: stack expects each tensor to be equal size, but got [7, 768] at entry 0 and [8, 768] at entry 1
</code></pre>
<p>Solution</p>
<pre><code class="lang-auto">torch.concat([a,b])
</code></pre>

username: Tim5
cooked: <p>Thanks a lot , i will try but excuse me does concat work like stack ?</p>

username: Tim5
cooked: <p>i got <code>AttributeError: module 'torch' has no attribute 'concat'</code></p>

username: anantguptadbl
cooked: <p>Looks like you might be using a slightly older version of pytorch<br>
You can use <code>torch.cat</code> in its stead</p>

username: Tim5
cooked: <p>it works but excuse me does cat works as stack ? as the shape for both is different … i mean the shape of cat is one but for stack is two</p>

username: anantguptadbl
cooked: <p><a class="mention" href="/u/tim5">@Tim5</a> there ware different ways of solving a problem</p>
<pre><code class="lang-auto">a=[torch.rand(1,10), torch.rand(1,10), torch.rand(1,10)]
print(torch.cat(a, axis=0).size())
print(torch.cat(a, axis=1).size())
print(torch.stack(a).size())
</code></pre>
<pre><code class="lang-auto">torch.Size([3, 10])
torch.Size([1, 30])
torch.Size([3, 1, 10])

</code></pre>
<p>Generally if you use stack in a loop then it can give problems</p>

username: Tim5
cooked: <aside class="quote no-group" data-username="anantguptadbl" data-post="7" data-topic="143012">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/anantguptadbl/48/17784_2.png" class="avatar"> anantguptadbl:</div>
<blockquote>
<p><code>ch.Size([3, 10])</code></p>
</blockquote>
</aside>
<p>if my tensor gave me like that<br>
<code>_embedding shape is  torch.Size([7, 768])</code><br>
<code>_embedding shape is  torch.Size([8, 768])</code> … and so on<br>
and i need to make it to sum all different number like<code> 7 , 8 , ....</code> so i need to use this line<br>
<code>torch.cat(_embedding, axis=0)</code> ,right ? as i got the summation only without <code>768</code></p>

username: AlphaBetaGamma96
cooked: <p><code>torch.stack</code> and <code>torch.cat</code> both ‘concatenate’ Tensors but the <strong>key</strong> different is that <code>torch.stack</code> applies this concatenation along a new dimenion, thereby <em>stacking</em> the Tensors whereas <code>torch.cat</code> does a direct concatenation along already existing dimensions</p>
<p>For example,</p>
<pre><code class="lang-auto">a = torch.randn(7,768)
b = torch.randn(8,768)

torch.cat((a,b)).shape #returns torch.Size([15, 768])
</code></pre>
<p>Trying <code>torch.stack</code> results in the error,</p>
<pre><code class="lang-auto">RuntimeError: stack expects each tensor to be equal size, but got [7, 768] at entry 0 and [8, 768] at entry 1
</code></pre>
<p>because <code>torch.stack</code> takes Tensors of the <strong>same</strong> shape and <em>stacks</em> them into a larger Tensor. You can think of having a <code>B</code> number of matrices of size <code>[N, N]</code> and <em>stacking</em> them together to get a Tensor <code>[B,N,N]</code>.</p>
<p>I’ll repeat your original problem with two new tensors that should explain the difference between <code>torch.stack</code> and <code>torch.cat</code>.</p>
<p>For example,</p>
<pre><code class="lang-auto">aa = torch.randn(4,768)
bb = torch.randn(4,768)
 
torch.stack((aa,bb)).shape #returns torch.Size([2, 4, 768]) #stacked along new dimension
torch.cat((aa,bb)).shape #returns torch.Size([8, 768]) #concatenated along existing dimension
</code></pre>
<p>This example should clearly show the difference between <code>torch.stack</code> and <code>torch.cat</code>. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: Tim5
cooked: <p>really thanks for your help and the great illustration of the difference but i got the summation only … why 768 didn’t appear?  it returns with me <code>torch.Size([6144])</code></p>

username: AlphaBetaGamma96
cooked: <p>Which command did you use that returned <code>torch.Size([6144])</code>? (Also, what version of pytorch are you using?)</p>

username: Tim5
cooked: <p>the version is <code>1.8.1+cu111</code><br>
the command is<br>
<code> _embedding = torch.cat(tokens_embedding)</code> as tokens_embedding hold different values every time from another loop</p>

username: AlphaBetaGamma96
cooked: <p>what size is <code>tokens_embedding</code>?</p>
<p>If all your token embeddings are of size <code>torch.Size([768])</code> and say you have four of them. You’ll want to use <code>torch.stack</code> in this example,</p>
<pre><code class="lang-auto">tokens_embedding = [torch.randn(768) for _ in range(4)] #list of 4 Tensors of shape [768]

_embedding = torch.cat(tokens_embedding)
_embedding.shape #returns torch.Size([3072])
 
_embedding = torch.stack(tokens_embedding)
_embedding.shape #returns torch.Size([4, 768])
</code></pre>

username: Tim5
cooked: <p>i already used stack before but got different shape for each time and that made a problem when i tried to implement this line after that</p>
<pre><code class="lang-auto">    embeddings.append(_embedding)
</code></pre>

username: AlphaBetaGamma96
cooked: <p>Can you post a minimal reproducible example?  Because it’s hard to visualize what your error is without the broader context.</p>
<p>If it’s purely the original error, it’s because your Tensors are of different sizes and <code>torch.stack</code> won’t work, and you might need to use <code>torch.cat</code> instead.</p>

username: Tim5
cooked: <p>the code is very huge to post as for loops and the file that i have also … i will describe my problem in the following lines</p>
<p>i implemented this line  <code> _embedding = torch.stack(tokens_embedding)</code> inside loop every time it read a sentences and got from it  the following results</p>
<pre><code class="lang-auto">size is  torch.Size([8, 768])
size is  torch.Size([9, 768])
size is  torch.Size([10, 768])
size is  torch.Size([13, 768])
size is  torch.Size([7, 768])
and so on
</code></pre>
<p>after it finished, i should use this line<br>
<code> embeddings.append(_embedding)</code><br>
but got</p>
<pre><code class="lang-auto">RuntimeError: stack expects each tensor to be equal size, but got [7, 768] at entry 0 and [8, 768] at entry 1
</code></pre>

username: AlphaBetaGamma96
cooked: <aside class="quote no-group" data-username="Tim5" data-post="16" data-topic="143012">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/e9c0ed/48.png" class="avatar"> Tim5:</div>
<blockquote>
<pre><code class="lang-auto">size is  torch.Size([8, 768])
size is  torch.Size([9, 768])
size is  torch.Size([10, 768])
size is  torch.Size([13, 768])
size is  torch.Size([7, 768])
and so on
</code></pre>
</blockquote>
</aside>
<p>And you want to stack these Tensors here? That won’t work because of the original problem, you could use <code>torch.cat</code> instead. Although, you’ll loose the information about which sets of rows correspond to what original Tensors.</p>