Why lengths should be given in sorted order in pack_padded_sequence?

Sandeep42: <p>Hi there,</p>
<p>I am training a hierarchical model using Pytorch. This proved to be considerably difficult as the masking needs a sorted list of lengths. In a hierarchical model, I can sort the samples based on sentences and get away with padding sentences. But I will definitely have to mask the words, and Pytorch only supports them in a sorted order.</p>
<p>Is there a reason for enforcing the sorted order, some sort of optimization? Can I make it take inputs without the sorting?</p>
<p>Regards<br>
Sandeep</p>

smth: <p>this restriction is a cudnn restriction that we’re dealing with <img src="//discuss.pytorch.org/images/emoji/apple/frowning.png?v=5" title=":frowning:" class="emoji" alt=":frowning:"></p>

Sandeep42: <p>Yeah, this puts a lot of constraints on any complicated networks that I want to try.</p>
<p>I can use GRUCell, but it does not have support for bidirectional or masking!</p>

simono: <p>I have also been working with padded sequences and the need to order them. I am currently running into a problem where I have 2 different inputs and 2 RNNs. I can sort both inputs according to their length, but then the respective dimensions don’t match each other anymore, so I have to re-instantiate the original order after running through the RNN (basically as shown here: <a href="https://discuss.pytorch.org/t/rnns-sorting-operations-autograd-safe/1461">RNNs Sorting operations autograd safe?</a>). I’m just worried that this operation is not autograd-safe, i.e. that the gradients get lost or are associated with the wrong matrix entries after resorting.</p>
<p>Do you know about similar problems or can help?</p>

Diego999: <p>is it still the case ?</p>

KelleyYin: <p>Does anyone know how to do ?</p>

Brando_Miranda: <p>Just out of curiosity, perhaps after 2 years its been solved so wanted to bump this so to document if it was solved <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> <code>pack_padded_sequence</code></p>
<p>Thanks for the help!</p>

smth: <p>it has been solved now. you can give unsorted lists and it’s fine.</p>

Paulo_Mann: <p>Hey <a class="mention" href="/u/smth">@smth</a>, we can give an unsorted list to <code>pack_padded_sequence</code> and it is going to work fine?</p>

Brando_Miranda: <aside class="quote no-group" data-username="Brando_Miranda" data-post="7" data-topic="3540">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/brando_miranda/48/14355_2.png" class="avatar"> Brando_Miranda:</div>
<blockquote>
<p>pack_padded_sequence</p>
</blockquote>
</aside>
<p>The documentation still talk about sorting. Thus the issue seems unresolved, maybe the docs should be updated.</p>
<p><a href="https://pytorch.org/docs/stable/nn.html#pack-padded-sequence" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/nn.html#pack-padded-sequence</a></p>
<blockquote>
<p>For unsorted sequences, use enforce_sorted = False. If  <code>enforce_sorted</code>  is  <code>True</code> , the sequences should be sorted by length in a decreasing order, i.e.  <code>input[:,0]</code>  should be the longest sequence, and  <code>input[:,B-1]</code>  the shortest one. enforce_sorted = True is only necessary for ONNX export.</p>
</blockquote>
<hr>
<p>git issue: <a href="https://github.com/pytorch/pytorch/issues/23079" class="inline-onebox" rel="noopener nofollow ugc">update docs that sorting is not needed in · Issue #23079 · pytorch/pytorch · GitHub</a></p>