Can I create a CNN sentence classifier without standardizing (padding) a sentence length?

vprokopev: <p>I am thinking about doing a 1d convolution over token embeddings matrix (over time dimension) and then just max pool it. After max pooling, sequences of any length will become a vector of fixed length. So I want my learning and inference steps both to be able to take sequences o any length.</p>
<p>Is it possible to do in Pytorch? I mean I know it is possible, but I am trying to figure out the best way to do it. Have somebody tried it yet?</p>
<p>Thank you!</p>

ptrblck: <p>I’m unsure if you are asking about the pooling operation creating the outputs in a fixed size, but it so you could use <a href="https://pytorch.org/docs/stable/generated/torch.nn.AdaptiveMaxPool2d.html#torch.nn.AdaptiveMaxPool2d"><code>nn.AdaptiveMaxPool2d</code></a> and define the desired output size.</p>

vprokopev: <p>Thank you, this will definitely help! Should I just use Python build ins like lists/tuples to represent a batch of sentences of varied length? Will it slow the training down significantly?</p>

ptrblck: <p>Different sample sizes in a single batch won’t work out of the box and the common approach would be to either pad them to the longest sequence in the batch or to use the experimental <code>NestedTensor</code> support.<br>
Note that the latter is used in some internal transformer layers, but I don’t know how well other modules are supported. <a class="mention" href="/u/vdw">@vdw</a> also provided an approach <a href="https://discuss.pytorch.org/t/using-batches-for-seq2seq-models/55205">here</a> where samples with the same length are sampled into a single batch to avoid padding in case that’s interesting for your use case.</p>

vdw: <p>As <a class="mention" href="/u/ptrblck">@ptrblck</a> said, within a batch you need to ensure that all sequence have the same length, either through padding or through some smarter organization (cf. linked post).</p>
<aside class="quote no-group" data-username="vprokopev" data-post="1" data-topic="170888">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vprokopev/48/46588_2.png" class="avatar"> vprokopev:</div>
<blockquote>
<p>After max pooling, sequences of any length will become a vector of fixed length.</p>
</blockquote>
</aside>
<p>This is not correct for standard max pooling. As max pooling basically also just slides a window over you vector and aggregates the values within that window. The longer your sequence, the more sliding, the more output values. See this slide:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/4/3472948384708e193099141ebde85f85f71fe0a2.png" data-download-href="https://discuss.pytorch.org/uploads/default/3472948384708e193099141ebde85f85f71fe0a2" title="Screenshot from 2023-01-24 17-51-02"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/4/3472948384708e193099141ebde85f85f71fe0a2_2_690x390.png" alt="Screenshot from 2023-01-24 17-51-02" data-base62-sha1="7tYjxS88kwo1bf4I0vpUVrQuJIm" width="690" height="390" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/4/3472948384708e193099141ebde85f85f71fe0a2_2_690x390.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/4/3472948384708e193099141ebde85f85f71fe0a2_2_1035x585.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/4/3472948384708e193099141ebde85f85f71fe0a2.png 2x" data-dominant-color="F1F6F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2023-01-24 17-51-02</span><span class="informations">1169×662 57.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>You might be referring to 1-max pooling, where the window size is the sames as the sequence length:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/4/d4fc7ebe7476ce01b4a87002c617d6e23d851835.png" data-download-href="https://discuss.pytorch.org/uploads/default/d4fc7ebe7476ce01b4a87002c617d6e23d851835" title="Screenshot from 2023-01-24 17-52-26"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/4/d4fc7ebe7476ce01b4a87002c617d6e23d851835_2_690x390.png" alt="Screenshot from 2023-01-24 17-52-26" data-base62-sha1="uoabTXOvz8GuyWko7p6rC8nkmW1" width="690" height="390" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/4/d4fc7ebe7476ce01b4a87002c617d6e23d851835_2_690x390.png, https://discuss.pytorch.org/uploads/default/optimized/3X/d/4/d4fc7ebe7476ce01b4a87002c617d6e23d851835_2_1035x585.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/d/4/d4fc7ebe7476ce01b4a87002c617d6e23d851835.png 2x" data-dominant-color="F6F5F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2023-01-24 17-52-26</span><span class="informations">1169×662 59.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>In this case, yes, different batches wit different sequence length would result in the same outputs, and you wouldn’t need to worry the dimensions of subsequent layers. For example, when you search on Google for text classification with CNNs, you very likely stumble on this architecture:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/8/88f31f86472e62250d2ced5bffcacd7b8250eb45.png" data-download-href="https://discuss.pytorch.org/uploads/default/88f31f86472e62250d2ced5bffcacd7b8250eb45" title="Screenshot from 2023-01-24 17-54-50"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/8/88f31f86472e62250d2ced5bffcacd7b8250eb45_2_690x390.png" alt="Screenshot from 2023-01-24 17-54-50" data-base62-sha1="jxvP8XBPnBkrYlB3c2WL3eULKND" width="690" height="390" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/8/88f31f86472e62250d2ced5bffcacd7b8250eb45_2_690x390.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/8/88f31f86472e62250d2ced5bffcacd7b8250eb45_2_1035x585.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/8/8/88f31f86472e62250d2ced5bffcacd7b8250eb45.png 2x" data-dominant-color="EEEFED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2023-01-24 17-54-50</span><span class="informations">1186×671 208 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Note that this architecture uses indeed 1-max pooling, which is not the same as the standard max pooling.</p>
<p>I hope this helps.</p>

vprokopev: <p>Thank you my friend! Yes, I do 1-Max pooling, but without standardizing the input sequence length it’s still a bit problematic to implement. The easy choice is to standardize lengths using padding, so I decided to stick to it for now, instead of using something like NestedTensor</p>

J_Johnson: <p>Another approach you can try is subgroup your data by sequence size and pad accordingly. Then call batches from each subgroup. For example:</p>
<ul>
<li>sequence sizes 1 to 32 would be padded to 32 and grouped</li>
<li>sequence sizes 33 to 64 padded to 64 and grouped</li>
<li>etc.</li>
</ul>
<p>The above would allow you to still perform down-sampling or pooling operations before your final average pooling.</p>