Tensor dimension issue when having BatchNorm2d under a Sequential

wongkenchiang97: <p>Hello, I’ve encounter Tensor dimension issue when calling <em>forward()</em> on line 80.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/5/e585bcc2553979bc73711a5a03d44d1e4cab8949.png" data-download-href="https://discuss.pytorch.org/uploads/default/e585bcc2553979bc73711a5a03d44d1e4cab8949" title="螢幕擷取畫面 2024-03-01 215257"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/5/e585bcc2553979bc73711a5a03d44d1e4cab8949_2_690x393.png" alt="螢幕擷取畫面 2024-03-01 215257" data-base62-sha1="wKrS2FGhNXX06hDdHmIjNNlR7kd" width="690" height="393" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/5/e585bcc2553979bc73711a5a03d44d1e4cab8949_2_690x393.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/5/e585bcc2553979bc73711a5a03d44d1e4cab8949_2_1035x589.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/5/e585bcc2553979bc73711a5a03d44d1e4cab8949_2_1380x786.png 2x" data-dominant-color="252727"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">螢幕擷取畫面 2024-03-01 215257</span><span class="informations">1524×870 101 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
line 78 perform conv2d once, It succeed when I manually <em>unsqueeze()</em> the tensor before <strong>BatchNorm2d</strong>.</p>
<p>This is the part describe how <strong>Sequential</strong> will initialize under <strong>DoubleConv</strong>’s constructor.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/6/069d1ff0da96c9d047a9b9ea848d56a58c2aff8e.png" data-download-href="https://discuss.pytorch.org/uploads/default/069d1ff0da96c9d047a9b9ea848d56a58c2aff8e" title="螢幕擷取畫面 2024-03-01 221112"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/6/069d1ff0da96c9d047a9b9ea848d56a58c2aff8e_2_690x260.png" alt="螢幕擷取畫面 2024-03-01 221112" data-base62-sha1="WvvbRPCoKKV69Vtvmig9HsmdXw" width="690" height="260" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/6/069d1ff0da96c9d047a9b9ea848d56a58c2aff8e_2_690x260.png, https://discuss.pytorch.org/uploads/default/optimized/3X/0/6/069d1ff0da96c9d047a9b9ea848d56a58c2aff8e_2_1035x390.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/0/6/069d1ff0da96c9d047a9b9ea848d56a58c2aff8e_2_1380x520.png 2x" data-dominant-color="242626"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">螢幕擷取畫面 2024-03-01 221112</span><span class="informations">1713×646 88.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Any idea to implement line 78(the <em>unsqueeze</em>) under a <strong>Sequential</strong>’s initialization?</p>
<p>cc: <a class="mention" href="/u/ptrblck">@ptrblck</a> <a class="mention" href="/u/fitanium">@Fitanium</a></p>

ptrblck: <p>Why do you <code>squeeze()</code> the activation tensor after the <code>relu</code> layer?<br>
By default PyTorch expects 4D inputs for 2D layers as <code>[batch_size, channels, height, width]</code>. Newer PyTorch releases accept also 3D inputs and assuming a batch size of 1. However, I don’t know if the C++ frontend supports if too for all layers.</p>

wongkenchiang97: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thanks for your quick reply!!</p>
<p>I’ve change the dimension of <code>data_tensor</code> into 4D(single batch) on <strong>line 75</strong>,and follow a <code>forward()</code> on <strong>line 89</strong>.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/5/f596d9093fca9e75b27efc540657adef7511cf6d.png" data-download-href="https://discuss.pytorch.org/uploads/default/f596d9093fca9e75b27efc540657adef7511cf6d" title="螢幕擷取畫面 2024-03-02 142913"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/5/f596d9093fca9e75b27efc540657adef7511cf6d_2_690x249.png" alt="螢幕擷取畫面 2024-03-02 142913" data-base62-sha1="z2Aay9yt9TxfO5kS56btiDqWf13" width="690" height="249" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/5/f596d9093fca9e75b27efc540657adef7511cf6d_2_690x249.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/5/f596d9093fca9e75b27efc540657adef7511cf6d_2_1035x373.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/5/f596d9093fca9e75b27efc540657adef7511cf6d_2_1380x498.png 2x" data-dominant-color="2A2B2B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">螢幕擷取畫面 2024-03-02 142913</span><span class="informations">1609×582 101 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I got <code>output_conv_double</code> with expected dimension.<br>
Appreciate to have your valuable advise!!</p>