Expected tensor for argument #1 'indices' to have scalar type Long; but got CPUFloatTensor instead (while checking arguments for embedding)

se7en: <p>What does the embedding expect for input? I got this error code when trying to embed the sentences.</p>
<blockquote>
<blockquote>
<blockquote>
<p>x = torch.randint(0, 10, (2, 5))<br>
embedding = nn.Embedding(5, 3)<br>
seq_len = len(x)<br>
embedded = embedding(x)<br>
Traceback (most recent call last):<br>
File “”, line 1, in <br>
File “/usr/local/lib/python2.7/site-packages/torch/nn/modules/module.py”, line 477, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “/usr/local/lib/python2.7/site-packages/torch/nn/modules/sparse.py”, line 110, in forward<br>
self.norm_type, self.scale_grad_by_freq, self.sparse)<br>
File “/usr/local/lib/python2.7/site-packages/torch/nn/functional.py”, line 1110, in embedding<br>
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)<br>
RuntimeError: Expected tensor for argument <span class="hashtag-raw">#1</span> ‘indices’ to have scalar type Long; but got CPUFloatTensor instead (while checking arguments for embedding)</p>
</blockquote>
</blockquote>
</blockquote>

ptrblck: <p>The first argument to <code>nn.Embedding</code> should be the <code>num_embeddings</code>, i.e. the size of your dictionary.<br>
In your code sample it looks like you are using a dictionary of 10 words, so try to create your embedding as:</p>
<pre><code class="lang-python">embedding = nn.Embedding(10, 3)
</code></pre>
<p>and run your code again.</p>
<p>The error message seems to be a bit strange, as <code>x</code> should be a <code>LongTensor</code>.<br>
Which PyTorch version are you using?</p>

se7en: <aside class="quote no-group" data-username="se7en" data-post="1" data-topic="32441">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/2bfe46/48.png" class="avatar"> se7en:</div>
<blockquote>
<p>embedding = nn.Embedding(5, 3)</p>
</blockquote>
</aside>
<p>No, it’s not correct. I figured it out in this way:</p>
<blockquote>
<blockquote>
<blockquote>
<p>x = torch.tensor([3])<br>
x<br>
tensor([3])<br>
x.size()<br>
(1,)<br>
embembedding = nn.Embedding(5, 3)<br>
embedded = embedding(x)<br>
embedded<br>
tensor([[-0.3584, -0.4176,  0.1584]], grad_fn=)</p>
</blockquote>
</blockquote>
</blockquote>

se7en: <blockquote>
<blockquote>
<blockquote>
<p>x = torch.tensor([[1, 2, 3, 4, 5], [6, 7, 8, 9, 0]])<br>
x<br>
tensor([[1, 2, 3, 4, 5],<br>
[6, 7, 8, 9, 0]])<br>
embedded = embedding(x)<br>
embedded<br>
tensor([[[ 0.2623, -0.6216,  1.8112],<br>
[ 0.5538, -1.0141, -1.6522],<br>
[-0.3584, -0.4176,  0.1584],<br>
[ 1.3007, -1.4878,  0.7318],<br>
[ 1.0062,  0.8589, -1.6106]],</p>
</blockquote>
</blockquote>
</blockquote>
<pre><code>    [[-0.4756,  1.3294,  0.3395],
     [ 0.4187,  0.9915, -0.5702],
     [ 0.3695,  0.8041, -0.6507],
     [-0.0904,  0.4995, -0.6367],
     [ 0.4574, -1.1187, -0.2644]]], grad_fn=&lt;EmbeddingBackward&gt;)
</code></pre>

se7en: <pre><code class="lang-auto">&gt;&gt;&gt; x = torch.LongTensor(2, 5).random_(0, 10)
&gt;&gt;&gt; x
tensor([[5, 3, 9, 3, 8],
        [8, 1, 8, 6, 9]])
&gt;&gt;&gt; embedded = embedding(x)
&gt;&gt;&gt; embedded
tensor([[[ 1.0062,  0.8589, -1.6106],
         [-0.3584, -0.4176,  0.1584],
         [-0.0904,  0.4995, -0.6367],
         [-0.3584, -0.4176,  0.1584],
         [ 0.3695,  0.8041, -0.6507]],

        [[ 0.3695,  0.8041, -0.6507],
         [ 0.2623, -0.6216,  1.8112],
         [ 0.3695,  0.8041, -0.6507],
         [-0.4756,  1.3294,  0.3395],
         [-0.0904,  0.4995, -0.6367]]], grad_fn=&lt;EmbeddingBackward&gt;)
</code></pre>

Richard_Kahn: <p>I’m getting this error. What exactly did you learn?</p>

ecdrid: <p>make sure your embeddings are having integer and not float like values in them; that’s what the error means actually (like label encoding the values(if going for entity embeddings))</p>

Ashima_Garg: <p>Loading glove vector embeddings have vector of float values.</p>

ecdrid: <p>That’s not what i meant, embedding layer is just a lookup table, that table index (dtype for your entity embeddings) needs to be int type or long etc…</p>

Ashima_Garg: <p>Yes thanks. I understood that.</p>

saba: <p>Hi Ptrblck,</p>
<p>I am trying to apply conditional DCGAN. The deined discriminator and generator are as follow, But I got an error when using Discriminator. ( RuntimeError: index out of range: Tried to access index 14 out of table with 9 rows. at /pytorch/aten/src/TH/generic/THTensorEvenMoreMath.cpp:418</p>
<p>)</p>
<pre><code class="lang-auto">class Discriminator(nn.Module):
    def __init__(self, ngpu):
        super(Discriminator, self).__init__()
        self.ngpu = ngpu
        self.label_embedding = nn.Embedding(10, 10)
            # input is (nc) x 64 x 64
        self.l1= nn.Sequential(nn.Conv2d(1, ndf, 4, 2, 1, bias=False),nn.LeakyReLU(0.2, inplace=True))
             # state size. (ndf) x 32 x 32
        self.l2=nn.Sequential(nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False),nn.BatchNorm2d(ndf * 2),nn.LeakyReLU(0.2, inplace=True))
        self.drop_out2 = nn.Dropout(0.5)

            # state size. (ndf*2) x 16 x 16
        self.l3= nn.Sequential(nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False), nn.BatchNorm2d(ndf * 4), nn.LeakyReLU(0.2, inplace=True))
        self.drop_out3 = nn.Dropout(0.5)

            # state size. (ndf*4) x 8 x 8
        self.l4= nn.Sequential(nn.Conv2d(ndf * 4, 1, 4, 2, 1, bias=False),nn.Sigmoid()

    def forward(self, x,Real_volume):
        
         c = self.label_embedding(Real_volume)
         x = torch.cat([x, c], 1)
         out = self.l1(x)
#         print("outsize1",out.shape)
         out=self.l2(out)
         out=self.drop_out2(out)
#         print("outsize2",out.shape)
         out=self.l3(out)
         out=self.drop_out3(out)

#         print("outsize3",out.shape)
         out=self.l4(out)
#         print("outsize4",out.shape)
      
         return out

class Generator(nn.Module):
    def __init__(self):
        super(Generator, self).__init__()
        self.label_embedding = nn.Embedding(10, 10)
        
        self.l1= nn.Sequential(nn.ConvTranspose2d( nz+10, ngf * 8, 3, 1, 0, bias=False),
        nn.BatchNorm2d(ngf * 8),
        nn.ReLU(True))
        # state size. (ngf*8) x 4 x 4
        self.l2=nn.Sequential(nn.ConvTranspose2d(ngf * 8, ngf * 4, 3, 1, 0, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(True))
        # state size. (ngf*4) x 8 x 8
        self.l3=nn.Sequential(nn.ConvTranspose2d( ngf * 4, ngf * 2, 3, 1, 0, bias=False),
        nn.BatchNorm2d(ngf * 2),
        nn.ReLU(True))
        # state size. (ngf*2) x 16 x 16
        self.l4=nn.Sequential(nn.ConvTranspose2d( ngf * 2, ngf, 3, 2, 1, bias=False),
        nn.BatchNorm2d(ngf),
        nn.ReLU(True))
        # state size. (ngf) x 32 x 32
        self.l5=nn.Sequential(nn.ConvTranspose2d( ngf, 1, 3, 2, 3, bias=False),nn.Sigmoid())
#            nn.Tanh()
            # state size. (nc) x 64 x 64


    def forward(self, x,Real_volume):
        c = self.label_embedding(Real_volume)
        x = torch.cat([x,c], 1)
        output = self.l1(x)
        output = self.l2(output)
        output = self.l3(output)
        output = self.l4(output)
        output = self.l5(output)
        return output

I used this command:
        output = netD(real_cpu,Real_volume).view(-1)
Do you think is there any thing wrong with my code and discriminator and generator implementation? I think sth is wrong with structure. What do you think? the z dimension is 100, and the training dataset are 21*21 patches with 64 batch size.</code></pre>

ptrblck: <p>Which line of code is raising this error?<br>
Could you print the input shape as well as the min and max values of the input tensor to this particular layer once you’ve isolated it?</p>

saba: <p>The discriminator and generator are as follow:</p>
<pre><code class="lang-auto">### Discriminator-----
class Discriminator(nn.Module):
    def __init__(self, ngpu):
        super(Discriminator, self).__init__()
        self.ngpu = ngpu
        self.label_embedding = nn.Embedding(10,10)
            # input is (nc) x 64 x 64
        self.l1= nn.Sequential(nn.Conv2d(1, ndf, 4, 2, 1, bias=False),nn.LeakyReLU(0.2, inplace=True))
             # state size. (ndf) x 32 x 32
        self.l2=nn.Sequential(nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False),nn.BatchNorm2d(ndf * 2),nn.LeakyReLU(0.2, inplace=True))
        self.drop_out2 = nn.Dropout(0.5)

            # state size. (ndf*2) x 16 x 16
        self.l3= nn.Sequential(nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False), nn.BatchNorm2d(ndf * 4), nn.LeakyReLU(0.2, inplace=True))
        self.drop_out3 = nn.Dropout(0.5)

            # state size. (ndf*4) x 8 x 8
        self.l4= nn.Sequential(nn.Conv2d(ndf * 4, 1, 4, 2, 1, bias=False),nn.Sigmoid())

#        self.l4= nn.Sequential(nn.Conv2d(ndf * 4, ndf * 8, 4, 2, 1, bias=False), nn.BatchNorm2d(ndf * 8),nn.LeakyReLU(0.2, inplace=True),nn.Sigmoid())
#            # state size. (ndf*8) x 4 x 4
#        self.l5= nn.Sequential(nn.Conv2d(ndf * 8, 1, 4, 1, 0, bias=False),nn.Sigmoid())
     

    def forward(self, x,Real_volume):
        
         c = self.label_embedding(Real_volume)
         print(c.shape)
         print(x.shape)
         x = torch.cat([x, c], 1)
         out = self.l1(x)
#         print("outsize1",out.shape)
         out=self.l2(out)
         out=self.drop_out2(out)
#         print("outsize2",out.shape)
         out=self.l3(out)
         out=self.drop_out3(out)

         out=self.l4(out)
      
         return out

## -----------Generator ------------
class Generator(nn.Module):
    def __init__(self):
        super(Generator, self).__init__()
        self.label_embedding = nn.Embedding(10,10)
        
        self.l1= nn.Sequential(nn.ConvTranspose2d( nz+10, ngf * 8, 3, 1, 0, bias=False),
        nn.BatchNorm2d(ngf * 8),
        nn.ReLU(True))
        # state size. (ngf*8) x 4 x 4
        self.l2=nn.Sequential(nn.ConvTranspose2d(ngf * 8, ngf * 4, 3, 1, 0, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(True))
        # state size. (ngf*4) x 8 x 8
        self.l3=nn.Sequential(nn.ConvTranspose2d( ngf * 4, ngf * 2, 3, 1, 0, bias=False),
        nn.BatchNorm2d(ngf * 2),
        nn.ReLU(True))
        # state size. (ngf*2) x 16 x 16
        self.l4=nn.Sequential(nn.ConvTranspose2d( ngf * 2, ngf, 3, 2, 1, bias=False),
        nn.BatchNorm2d(ngf),
        nn.ReLU(True))
        # state size. (ngf) x 32 x 32
        self.l5=nn.Sequential(nn.ConvTranspose2d( ngf, 1, 3, 2, 3, bias=False),nn.Sigmoid())
#            nn.Tanh()
            # state size. (nc) x 64 x 64


    def forward(self, x,Real_volume):
        c = self.label_embedding(Real_volume)
        print(c.shape)
        x = torch.cat([x,c], 1)
        output = self.l1(x)
        output = self.l2(output)
        output = self.l3(output)
        output = self.l4(output)
        output = self.l5(output)
        return output
        
  The error is when I use discriminator ( output = netD(real_cpu,Real_volume.squeeze(1)).view(-1)) and the error referred to the (c = self.label_embedding(Real_volume))

The real_cpu  is the training set (64x1x21x21) and the Readl_volume are the 64x1 volume (float numbers) from 2 to 89, the nz=100.
As an suggestion I set it to the
        self.label_embedding = nn.Embedding(100,10)
I set to this it give not any error but I am not sure if it works correctly or not!?</code></pre>

saba: <p>would you please help me that with defining the layer for conditional DCGAN? I am really confused with them.</p>
<p>I appreciate your help.</p>

ptrblck: <p><code>Readl_volume</code> should contain <code>long</code> values, in the range <code>[0, embedding_dim-1]</code>, while it seems you are trying to pass <code>float</code> values and are not using the full range.<br>
If you don’t need to provide <code>100</code> different indices, you wouldn’t need to use <code>nz=100</code>.</p>
<p>However, after this layer the <code>torch.cat([x, c], 1)</code> would most likely fail, as <code>x</code> and <code>c</code> would have a different number of dimensions.</p>
<p>I’m not sure what your current use case is, but you won’t be able to concatenate two tensors with different shapes in more then a single dimension.</p>

saba: <p>Yes, you are right. the size is different . I am trying to apply conditional GAN. The volumes are the condition and 100 is the size of the noise for generator. I need to cat the condition and noise in generator . and cat the condition with the training datasets in the discriminator. The size of noise for cat is 100 and the size of training dataset is 21x21. I don’t know how to embed to have these size for the conditions (volume) and cat with the noise for generator and training data (21x21) for discriminator.</p>

ptrblck: <p>The concatenation won’t work out of the box, if you want to keep the dimensions.<br>
You could of course flatten the condition tensor to <code>[batch_size, -1]</code> and concatenate in <code>dim1</code>, however you would have to check, if it fits your use case or not.</p>

mathematics: <p>when building vocabulary  , changing to <code>torch.FloatTensor</code> to <code>torch.LongTensor</code>  solved model not accepting data in my case. <code>nn.Embedding</code> expects inputs <code>LongTensor</code> only</p>

Navya: <p>Input data is looking this:<br>
tensor([[-1.0000, -1.0000, -1.0000,  …, -0.1132, -1.0000, -1.0000],<br>
[-1.0000, -1.0000, -1.0000,  …, -0.1253, -1.0000, -1.0000],<br>
[-1.0000, -1.0000, -1.0000,  …, -0.1316, -1.0000, -1.0000],<br>
…,<br>
[-1.0000, -1.0000, -1.0000,  …, -0.1269, -1.0000, -1.0000],<br>
[-1.0000, -1.0000, -1.0000,  …, -0.1322, -1.0000, -1.0000],<br>
[-1.0000, -1.0000, -1.0000,  …, -0.1047, -1.0000, -1.0000]],<br>
device=‘cuda:0’)</p>
<p>data = self.embeddingd(data)<br>
File “/home/ubuntu/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 1051, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “/home/ubuntu/anaconda3/lib/python3.7/site-packages/torch/nn/modules/sparse.py”, line 160, in forward<br>
self.norm_type, self.scale_grad_by_freq, self.sparse)<br>
File “/home/ubuntu/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py”, line 2043, in embedding<br>
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)<br>
RuntimeError: Expected tensor for argument <span class="hashtag">#1</span> ‘indices’ to have one of the following scalar types: Long, Int; but got torch.cuda.FloatTensor instead (while checking arguments for embedding)</p>
<p>I am having this error too. Please help me on this.</p>

ptrblck: <p>As the error message explains, <code>nn.Embedding</code> expects indices as its input (as <code>LongTensor</code> or <code>IntTensor</code>), while you are passing <code>FloatTensor</code>s to it.<br>
Note that an embedding is working as a “lookup table” and outputs dense feature tensors.<br>
Based on your input you also won’t be able to simply transform the input to a <code>LongTensor</code>, since it contains negative indices (besides the obvious rounding, which I don’t know if it would fit your use case).</p>