How should we concatenate multiple tensors vertically into one tensor in torch?

username: Jared_shad
cooked: <p>I am wondering how should we concatenate tensor with different shapes in the torch. I tried <code>torch.cat</code> for that, but I got type erorr as follow:</p>
<pre><code class="lang-auto">print(tensor_1.shape)
print(tensor_2.shape)

new_tensor = torch.cat((tensor_1, tensor_2),1)
</code></pre>
<p>but I got the following error:</p>
<pre><code class="lang-auto">shape of tensor_1 (?, 30, 30, 128)
shape of tensor_2 (?, 26, 26, 128)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
&lt;ipython-input-20-492132d221f7&gt; in &lt;module&gt;()
      6 print("shape of tensor_1", tensor_1.shape)
      7 print("shape of tensor_2", tensor_2.shape)
----&gt; 8 torch.cat((tensor_1, tensor_2),1)

TypeError: expected Tensor as element 0 in argument 0, but got Tensor
</code></pre>
<p>I think in Conv1D, concatenation is simple, in Conv2D, it is not known to me how to make concatenation. Does anyone know how to do this in torch? any idea?</p>
<p>I was thinking getting concatenated tensor shape as <code>(56,56,128)</code> is not a good idea or wrong in math. Is there any workaround to do concatenation on those? any thoughts?</p>
<p><strong>desired output</strong></p>
<p>I am not sure how multiple tensor can be concatenated, so I am okay with output as tensor with shape of  <code>(W, H, C)</code>. How should we do this in torch? any idea?</p>

username: Bjorn_Lindqvist
cooked: <p>What dimensions do you want the concatenated tensor to have?</p>

username: Jared_shad
cooked: <p>I understand concatenation in 1D, for instance, we can concatenate <code>(12,20)</code>, <code>(6,20)</code> as <code>(18, 20)</code>, but not sure what would be the correct dimension of the concatenated output tensor in 2D. I was thinking output tensor shape could be <code>(56,56, 128)</code>  based on the above example, but not sure that’s the right intuition to do so. I am open to community suggestions at the best.  I think I need a new tensor that could have the shape of <code>(Width, Height, Channel)</code>. Do you have any possible thoughts? Thanks!</p>

username: Bjorn_Lindqvist
cooked: <p>That works because 12*20 + 6*20 = 18 * 20. But 30*30*128 + 26*26*128 != 56*56*128 so it won’t work.</p>

username: Jared_shad
cooked: <p>right, that’s why getting concatenated tensor shape as <code>(56,56,128)</code> is not good idea or wrong. What can we do about concatenating multiple tensors? Do we need to reshape it back to <code>(30,30, 128), (26,26,128) </code> into <code>(W, H)</code> shape then concatenate then reshape it back to <code>(W, H, C)</code>? Do you have any possible thoughts? Thanks!</p>

username: Bjorn_Lindqvist
cooked: <p>Suppose you have a 5x5 rectangle and a 10x10 rectangle. You want to concatenate these two rectangles so that the result is a new rectangle. How do you do it?</p>

username: Jared_shad
cooked: <p>I am not sure but I think I might do flatten the array and do padding then do concatenation. I really don’t know how should I concatenate 2dim tensor vertically. Do you have any idea or coding attempt that might do some trick on that?</p>