How to do convolution on tube tensors (3D conv)()

isalirezag: <p>I am trying to do convolution on frames videos (like tube of videos).<br>
So im reading video frams and make them to have the shape of <code>NxCinxDxHxW</code>, where Cin = 3 (channel size), and W,H= dimension (lets say they are equal) and D is 1. and  N is batch size, lets say 1 for simipilicity.</p>
<p>then i concatenate them, so my final output is having the size of <code>NxCxDxHxW</code>, where D is the number of frames.</p>
<p>now i want to do 3d convolution in a way that i do convolution along the frames, like  i have the input of <code>NxCxDxHxW</code> and kernel <code>CxDxKxK</code>.</p>
<p>This is an example,</p>
<pre><code class="lang-auto">m = nn.Conv3d(3, 30, (6,3,3), stride=1,padding=(0, 1, 1))
input = torch.randn(1,3 , 6, 10, 10)
output = m(input)
output.size()
torch.Size([1, 30, 3, 10, 10])
</code></pre>
<p>I dont get the concept of padding along D, how does it happen?</p>
<p>Can you please tell me how should i do it?</p>

aplassard: <p>Check out the Conv3D documentation - <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.Conv3d" rel="nofollow noopener">https://pytorch.org/docs/stable/nn.html#torch.nn.Conv3d</a></p>
<p>According to that link the proper input size is (N, C, D, H, W)</p>

isalirezag: <p>what does it mean to pad along D dimension?</p>

aplassard: <p>Where do you see that?</p>

isalirezag: <p>in the conv3d documentation, we can pad for 3 dimensions like (1,1,1), the first one is pad for D, the last two are for H and W.<br>
I cannot get what does it mean to pad for D</p>

aplassard: <p>That means to add additional frames to the sequence</p>

SungmanHong: <aside class="quote no-group" data-username="aplassard" data-post="2" data-topic="21446">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aplassard/48/5743_2.png" class="avatar"> aplassard:</div>
<blockquote>
<p>(N, C, D, H, W)</p>
</blockquote>
</aside>
<p>In N,C,D,H,W… What is D?<br>
and why the channels is in the second dimension, not the last dimension?</p>

ptrblck: <aside class="quote no-group" data-username="SungmanHong" data-post="7" data-topic="21446">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sungmanhong/48/36192_2.png" class="avatar"> SungmanHong:</div>
<blockquote>
<p>In N,C,D,H,W… What is D?</p>
</blockquote>
</aside>
<p><code>D</code> corresponds to the “depth” of the volume, which is the additional dimension spanning the volume besides the height and width.</p>
<aside class="quote no-group" data-username="SungmanHong" data-post="7" data-topic="21446">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sungmanhong/48/36192_2.png" class="avatar"> SungmanHong:</div>
<blockquote>
<p>and why the channels is in the second dimension, not the last dimension?</p>
</blockquote>
</aside>
<p>PyTorch uses the (user-facing) channels-first memory layout by default, so the channel dimension is placed in <code>dim1</code> (also for 2D layers).</p>

SungmanHong: <aside class="quote no-group" data-username="ptrblck" data-post="8" data-topic="21446">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>D</code> corresponds to the “depth” of the volume, which is the additional dimension spanning the volume besides the height and width.</p>
</blockquote>
</aside>
<p>Thank you, In my specific case, the D(depth) dimension can be regarded as time dimension. My input data is video.</p>