Error with Lengths in pack_padded_sequence

username: ParticularlyPythonic
cooked: <p>Hi,<br>
I am trying to set up an RNN capable of utilizing a GPU but packed_padded_sequence gives me a</p>
<pre><code class="lang-auto">RuntimeError: 'lengths' argument should be a 1D CPU int64 tensor
</code></pre>
<p>here is how I direct gpu computing</p>
<pre><code class="lang-auto">parser = argparse.ArgumentParser(description='Trainer')
parser.add_argument('--disable-cuda', action='store_true',
                    help='Disable CUDA')
args = parser.parse_args()
args.device = None
if not args.disable_cuda and torch.cuda.is_available():
    args.device = torch.device('cuda')
    torch.set_default_tensor_type(torch.cuda.FloatTensor)
else:
    args.device = torch.device('cpu')
</code></pre>
<p>here is the relevant part of the code.</p>
<pre><code class="lang-auto">def Tensor_length(track):
    """Finds the length of the non zero tensor"""
    return int(torch.nonzero(track).shape[0] / track.shape[1])
.
.
.
def forward(self, tracks, leptons):
        self.rnn.flatten_parameters()
        # list of event lengths
        n_tracks = torch.tensor([Tensor_length(tracks[i])
                                 for i in range(len(tracks))])
        sorted_n, indices = torch.sort(n_tracks, descending=True)
        sorted_tracks = tracks[indices].to(args.device)
        sorted_leptons = leptons[indices].to(args.device)
        # import pdb; pdb.set_trace()
        output, hidden = self.rnn(pack_padded_sequence(sorted_tracks,
                                                       lengths=sorted_n.cpu().numpy(),
                                                       batch_first=True)) # this gives the error

        combined_out = torch.cat((sorted_leptons, hidden[-1]), dim=1)
        out = self.fc(combined_out)  # add lepton data to the matrix
        out = self.softmax(out)
        return out, indices  # passing indices for reorganizing truth
</code></pre>
<p>I have tried everything from casting sorted_n to a long tensor to having it be a list, but i aways seem to get the same error.<br>
I have not worked with pytorch on gpu before and any advice will be greatly appreciated.</p>
<p>Thanks!</p>

username: ptrblck
cooked: <p>It’s working with a 1D CPU <code>LongTensor</code> for me:</p>
<pre><code class="lang-python">T, B, C = 100, 10, 4  # longest sequence, batch_size, feature_size
input = torch.randn(T, B, C, device='cuda')
lengths, _ = torch.sort(torch.randint(1, 100, (10,)), descending=True)
print(input.type())
&gt; torch.cuda.FloatTensor
print(lengths.type())
&gt; torch.LongTensor
pack_padded_sequence(input, lengths)
</code></pre>
<p>Could you print the <code>type</code> and shape of <code>sorted_n</code>?</p>

username: ParticularlyPythonic
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/4/489ef8775c7fc2490f363cf015e54934c0a34173.png" data-download-href="https://discuss.pytorch.org/uploads/default/489ef8775c7fc2490f363cf015e54934c0a34173" title="test"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/4/489ef8775c7fc2490f363cf015e54934c0a34173_2_690x275.png" alt="test" data-base62-sha1="amqYbz2vKACwnPj5gMtqP8zJ6fN" width="690" height="275" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/4/489ef8775c7fc2490f363cf015e54934c0a34173_2_690x275.png, https://discuss.pytorch.org/uploads/default/optimized/2X/4/489ef8775c7fc2490f363cf015e54934c0a34173_2_1035x412.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/4/489ef8775c7fc2490f363cf015e54934c0a34173_2_1380x550.png 2x" data-dominant-color="2C2C2D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">test</span><span class="informations">1724×688 130 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>That’s weird, since <code>lengths=sorted_n.cpu()</code> should create a 1D cpu <code>LongTensor</code>.<br>
Could you just for the sake of debugging pass <code>lengths=torch.randint(1, 15, (100,))</code> and see if it’s working?</p>

username: ParticularlyPythonic
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/e/e935845253207b4fd8256b5f1a77acd3e44cf651.png" data-download-href="https://discuss.pytorch.org/uploads/default/e935845253207b4fd8256b5f1a77acd3e44cf651" title="Selection_106"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/e/e935845253207b4fd8256b5f1a77acd3e44cf651_2_690x257.png" alt="Selection_106" data-base62-sha1="xh3Uq3n5obu4h2Y4yafOPVuYEp3" width="690" height="257" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/e/e935845253207b4fd8256b5f1a77acd3e44cf651_2_690x257.png, https://discuss.pytorch.org/uploads/default/optimized/2X/e/e935845253207b4fd8256b5f1a77acd3e44cf651_2_1035x385.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/e/e935845253207b4fd8256b5f1a77acd3e44cf651_2_1380x514.png 2x" data-dominant-color="2B2B2B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Selection_106</span><span class="informations">1753×655 131 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>Thanks for the info!<br>
I just debugged it and it looks like <a href="https://github.com/pytorch/pytorch/blob/21193bf12314f9d1be5ea36cbce3162c54b35225/torch/nn/utils/rnn.py#L195" rel="nofollow noopener">this line of code</a> is problematic, if you’ve set a CUDA tensor as the default tensor type.<br>
Here is a small snippet to see that the <code>cpu()</code> call will be ignored, since the data will be rewrapped:</p>
<pre><code class="lang-python">lengths = torch.as_tensor(lengths.cpu(), dtype=torch.int64)
lengths.type()
&gt; 'torch.cuda.LongTensor'
</code></pre>
<p>Could you create a <a href="https://github.com/pytorch/pytorch/issues" rel="nofollow noopener">GitHub issue</a> and CC me there?<br>
Let me know, if you don’t want to do it and I’ll create the issue. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=6" title=":wink:" class="emoji" alt=":wink:"></p>
<p>The current workaround would be to avoid setting the default device or use this small hotfix:</p>
<pre><code class="lang-python">import torch
torch.set_default_tensor_type(torch.cuda.FloatTensor)
from torch.nn.utils.rnn import pack_padded_sequence, PackedSequence


def hotfix_pack_padded_sequence(input, lengths, batch_first=False, enforce_sorted=True):
    lengths = torch.as_tensor(lengths, dtype=torch.int64)
    lengths = lengths.cpu()
    if enforce_sorted:
        sorted_indices = None
    else:
        lengths, sorted_indices = torch.sort(lengths, descending=True)
        sorted_indices = sorted_indices.to(input.device)
        batch_dim = 0 if batch_first else 1
        input = input.index_select(batch_dim, sorted_indices)

    data, batch_sizes = \
        torch._C._VariableFunctions._pack_padded_sequence(input, lengths, batch_first)
    return PackedSequence(data, batch_sizes, sorted_indices)


T, B, C = 100, 10, 4  # longest sequence, batch_size, feature_size
input = torch.randn(T, B, C, device='cuda')
lengths, _ = torch.sort(torch.randint(1, 100, (10,)), descending=True)
print(input.type())
print(lengths.type())
pack_padded_sequence(input, lengths.cpu())  # Error due to default type set to CUDA
hotfix_pack_padded_sequence(input, lengths)
</code></pre>

username: ParticularlyPythonic
cooked: <p>Thanks! I will do so asap.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/e/e72af0748fd2c42ec7c82a0faebe66f7a19db487.png" data-download-href="https://discuss.pytorch.org/uploads/default/e72af0748fd2c42ec7c82a0faebe66f7a19db487" title="Selection_106"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/e/e72af0748fd2c42ec7c82a0faebe66f7a19db487_2_690x112.png" alt="Selection_106" data-base62-sha1="wZ0i8XBjl0XMW52YnrhdESlFTN5" width="690" height="112" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/e/e72af0748fd2c42ec7c82a0faebe66f7a19db487_2_690x112.png, https://discuss.pytorch.org/uploads/default/optimized/2X/e/e72af0748fd2c42ec7c82a0faebe66f7a19db487_2_1035x168.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/e/e72af0748fd2c42ec7c82a0faebe66f7a19db487_2_1380x224.png 2x" data-dominant-color="2D2D2D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Selection_106</span><span class="informations">1554×254 57.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
but the fix gives me this error</p>

username: ptrblck
cooked: <p>Which PyTorch version are you using?<br>
Mine might be a bit too old (<code>1.0.0.dev20190104</code>).</p>

username: ParticularlyPythonic
cooked: <p>i am using 1.0,0.<br>
removing sorted_indices fixed it.<br>
Thanks a ton!</p>

username: ParticularlyPythonic
cooked: <p><aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/16542#issue-404589478" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <a href="https://github.com/ParticularlyPythonicBS" rel="nofollow noopener">
<img src="https://avatars1.githubusercontent.com/u/36012889?v=2&amp;s=96" class="thumbnail onebox-avatar" width="60" height="60">
</a>

<h4><a href="https://github.com/pytorch/pytorch/issues/16542#issue-404589478" target="_blank" rel="nofollow noopener">Issue: Error with setting tensors to use cpu in packed_padded_sequence when CUDA tensor is set as default</a></h4>

<div class="date" style="margin-top:10px;">
	<div class="user" style="margin-top:10px;">
	opened by <a href="https://github.com/ParticularlyPythonicBS" target="_blank" rel="nofollow noopener">ParticularlyPythonicBS</a>
	on <a href="https://github.com/pytorch/pytorch/issues/16542#issue-404589478" target="_blank" rel="nofollow noopener">2019-01-30</a>
	</div>
	<div class="user">
	</div>
</div>

<pre class="content" style="white-space: pre-wrap;">🐛 Bug
@ptrblck helped me diagnose the issue
if Default tensors are set to gpu, setting length to cpu for pack_padded_sequence doesn't work
To...</pre>

<div class="labels">
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
<br>
here is the issue</p>

username: Mrxiexianzhao
cooked: <p>maybe i have the same problem,i try two ways,<br>
First:<br>
when i spcify the function code ‘pack_padded_sequence’:</p>
<pre><code class="lang-auto">def pack_padded_sequence(input, lengths, batch_first=False, enforce_sorted=True):
    if torch._C._get_tracing_state() and not isinstance(lengths, torch.Tensor):
        warnings.warn('pack_padded_sequence has been called with a Python list of '
                      'sequence lengths. The tracer cannot track the data flow of Python '
                      'values, and it will treat them as constants, likely rendering '
                      'the trace incorrect for any other combination of lengths.',
                      stacklevel=2)
    lengths = torch.as_tensor(lengths, dtype=torch.int64)
    lengths = lengths.cpu() # 此处给pytorch提的issue解决的
    if enforce_sorted:
        sorted_indices = None
    else:
        lengths, sorted_indices = torch.sort(lengths, descending=True)
        sorted_indices = sorted_indices.to(input.device)
        batch_dim = 0 if batch_first else 1
        input = input.index_select(batch_dim, sorted_indices)

    data, batch_sizes = \
        _VF._pack_padded_sequence(input, lengths, batch_first)
    return PackedSequence(data, batch_sizes, sorted_indices, None)
</code></pre>
<p>when i debug my code, the type of input and lengths is all ‘‘torch.LongTensor’’  and ‘‘torch.FloatTensor’’,<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/8/8821cb8b23aae768ae0b4e64feedfea9348869b4.png" data-download-href="https://discuss.pytorch.org/uploads/default/8821cb8b23aae768ae0b4e64feedfea9348869b4" title="企业微信截图_15903898892171"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/8/8821cb8b23aae768ae0b4e64feedfea9348869b4_2_611x500.png" alt="企业微信截图_15903898892171" data-base62-sha1="jqhlbekHtVKUrNWXe7C1wPgy6iM" width="611" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/8/8821cb8b23aae768ae0b4e64feedfea9348869b4_2_611x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/8/8821cb8b23aae768ae0b4e64feedfea9348869b4_2_916x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/8/8821cb8b23aae768ae0b4e64feedfea9348869b4_2_1222x1000.png 2x" data-dominant-color="3E4143"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">企业微信截图_15903898892171</span><span class="informations">1662×1360 150 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
but it raise the error:</p>
<blockquote>
<p>File “C:\Anaconda3\lib\site-packages\torch\nn\utils\rnn.py”, line 277, in pack_padded_sequence<br>
_VF._pack_padded_sequence(input, lengths, batch_first)<br>
RuntimeError: ‘lengths’ argument should be a 1D CPU int64 tensor</p>
</blockquote>
<p>Second  i try def the function ‘hotfix_pack_padded_sequence’:</p>
<pre><code class="lang-auto">def hotfix_pack_padded_sequence(input, lengths, batch_first=False, enforce_sorted=True):
    lengths = torch.as_tensor(lengths, dtype=torch.int64)
    lengths = lengths.cpu()
    if enforce_sorted:
        sorted_indices = None
    else:
        lengths, sorted_indices = torch.sort(lengths, descending=True)
        sorted_indices = sorted_indices.to(input.device)
        batch_dim = 0 if batch_first else 1
        input = input.index_select(batch_dim, sorted_indices)

    data, batch_sizes = \
        torch._C._VariableFunctions._pack_padded_sequence(input, lengths, batch_first)
    return PackedSequence(data, batch_sizes, sorted_indices)
</code></pre>
<p>when i debug my code, the type of input and lengths is all ‘‘torch.LongTensor’’  and ‘‘torch.FloatTensor’’,<br>
but it raise the same error:</p>
<blockquote>
<p>“D:\soft\NLP_Frame\algorithm\modules\encoder\lstm_encoder.py”, line 55, in hotfix_pack_padded_sequence<br>
torch._C._VariableFunctions._pack_padded_sequence(input, lengths, batch_first)<br>
RuntimeError: ‘lengths’ argument should be a 1D CPU int64 tensor</p>
</blockquote>
<p>I’m going crazy， <img src="https://discuss.pytorch.org/images/emoji/apple/sob.png?v=12" title=":sob:" class="emoji" alt=":sob:" loading="lazy" width="20" height="20"></p>

username: ParticularlyPythonic
cooked: <p>bounding the packing inside a different default tensor type is a hack that I have found works</p>
<pre><code class="lang-auto">torch.set_default_tensor_type(torch.FloatTensor)
&lt;call to pack padded sequences&gt;
if self.device == torch.device("cuda"):
        torch.set_default_tensor_type(torch.cuda.FloatTensor)
</code></pre>
<p>Hopefully this works for you</p>

username: Mrxiexianzhao
cooked: <p>i find the debug，because my input.shape is [6, 574, 574, 100], lengths.shape is[6, 574], so the pack_padded_sequence(input, lengths) can not run, the error is</p>
<blockquote>
<p>‘lengths’ argument should be a 1D CPU int64 tensor</p>
</blockquote>
<p>what can i specify it?</p>

username: bhashithe
cooked: <p>I think this is a device error rather than an error in the function. Looking at other discussion on the same exception, they are mentioning that you need to move the lengths into the CPU (which is exactly what the error says).</p>
<p><a href="https://github.com/pytorch/pytorch/issues/16542#issuecomment-458932691" rel="nofollow noopener">link to the issue in github</a></p>

username: Anamul_Hasan_Dhusor
cooked: <p>it works for me<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/f/9f029ccaaf71ebd31cdb5386b26396113fbb28d0.png" data-download-href="https://discuss.pytorch.org/uploads/default/9f029ccaaf71ebd31cdb5386b26396113fbb28d0" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/f/9f029ccaaf71ebd31cdb5386b26396113fbb28d0.png" alt="image" data-base62-sha1="mGFvVBZkDu8o80ocAnjx1LcLFFm" width="690" height="240" data-dominant-color="F1F2F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">869×303 7.54 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>