Embedding Error Index out of Range in self

username: csblacknet
cooked: <p>I tried the stackoverflow and other threads in forum but still my issues wasn’t resolved. I am a starter please help me understand what went wrong.</p>
<pre><code class="lang-auto">id_2_token = dict(enumerate(set(n for name in names for n in name),1))
token_2_id = {value:key for key,value in id_2_token.items()}
print(len(id_2_token))
print(len(token_2_id))
</code></pre>
<p>Output :</p>
<pre><code class="lang-auto">56
56
</code></pre>
<pre><code class="lang-auto">feature_id,target_id = batch_maker(names) #batching function
print(feature_id.shape) #Shape - [124,64,17]
</code></pre>
<p><span class="hashtag">#RNN</span> MODEL</p>
<pre><code class="lang-auto">class CharMaker(nn.Module):
    def __init__(self, input_size, hidden_size, output_size,n_layers=1):
        super(CharMaker,self).__init__()
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.output_size = output_size
        
        self.encoder = nn.Embedding(self.input_size, self.hidden_size)
        
        self.rnn = nn.RNN(self.hidden_size,self.hidden_size, num_layers=1,batch_first=True)
        self.linear = nn.Linear(self.hidden_size, self.output_size)
        
        self.softmax = torch.nn.Softmax(dim=output_size)
        
    def forward(self, inputs, hidden):
        batch_size = inputs.size(0)
        
        if hidden == None:
            hidden = torch.zeros(1,inputs.size(1),self.hidden_size)
        print(inputs.shape)
        encoded = self.encoder(inputs)
        output, hidden = self.rnn(encoded, hidden)
        outout = self.linear(hidden,self.output_size)
        
        output = self.softmax(output)
        
        return output,hidden
</code></pre>
<p>Initializing my model</p>
<pre><code class="lang-auto">cm = CharMaker(input_size=len(token_2_id),hidden_size=20,output_size=len(token_2_id))
</code></pre>
<p>Reshaping and Texting The Data</p>
<pre><code class="lang-auto">hidden = None
names_id_tensor = torch.from_numpy(features_id[0])
names_id_tensor = names_id_tensor.reshape(names_id_tensor.shape[0],names_id_tensor.shape[1],1)
</code></pre>
<p>Shapes</p>
<pre><code class="lang-auto">print(names_id_tensor.shape) #torch.Size([64, 17, 1])
</code></pre>
<pre><code class="lang-auto">output,hidden = cm(names_id_tensor,hidden)
</code></pre>
<p>Error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
&lt;ipython-input-139-d0d9f66f3192&gt; in &lt;module&gt;
----&gt; 1 output,hidden = cm(names_id_tensor,hidden)

~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--&gt; 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

&lt;ipython-input-129-f8a6cdd31a7a&gt; in forward(self, inputs, hidden)
     19             hidden = torch.zeros(1,inputs.size(1),self.hidden_size)
     20         print(inputs.shape)
---&gt; 21         encoded = self.encoder(inputs)
     22         output, hidden = self.rnn(encoded, hidden)
     23         outout = self.linear(hidden,self.output_size)

~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--&gt; 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~/.local/lib/python3.6/site-packages/torch/nn/modules/sparse.py in forward(self, input)
    112         return F.embedding(
    113             input, self.weight, self.padding_idx, self.max_norm,
--&gt; 114             self.norm_type, self.scale_grad_by_freq, self.sparse)
    115 
    116     def extra_repr(self):

~/.local/lib/python3.6/site-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1722         # remove once script supports set_grad_enabled
   1723         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-&gt; 1724     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1725 
   1726 

IndexError: index out of range in self
</code></pre>

username: ptrblck
cooked: <p>Could you print the min and max values of <code>names_id_tensor</code>?<br>
The <code>embedding_dim</code> is currently set as <code>self.input_size</code>, which is the length of <code>token_2_id</code>.<br>
Note that the embedding dimension refers to the max. input index you would like to provide.</p>

username: csblacknet
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>, I understand what are you trying to say but I am having an issue with that.</p>
<ol>
<li>This was just a batch for testing the model if it works or not. The highest value in that batch was 53 while my vocab(token_2_id) size is 56. What if another batch comes up with the highest value other than 53, what will happen then? How will I resolve that problem?</li>
</ol>
<p>Can you please guide me through this?</p>
<pre><code class="lang-auto">print(torch.max(names_id_tensor))
print(torch.min(names_id_tensor))
</code></pre>
<p>Output</p>
<pre><code class="lang-auto">tensor(53)
tensor(1)
</code></pre>

username: ptrblck
cooked: <aside class="quote no-group" data-username="csblacknet" data-post="3" data-topic="81550">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/csblacknet/48/24046_2.png" class="avatar"> csblacknet:</div>
<blockquote>
<p>The highest value in that batch was 53 while my vocab(token_2_id) size is 56. What if another batch comes up with the highest value other than 53, what will happen then? How will I resolve that problem?</p>
</blockquote>
</aside>
<p>You cannot pass indices higher than <code>embedding_dim-1</code>, since the embedding layer is working as a lookup table. The input is used to index the corresponding embedding vector, so you should set <code>embedding_dim</code> as the highest value you would expect in your use case.</p>

username: csblacknet
cooked: <p>Got it. I had to just fix the embedding layer. <a class="mention" href="/u/ptrblck">@ptrblck</a> Thanks it’s solved now.</p>
<p><strong>Last time my vocab was create by enumerating from 1. So if I just enumerate from 0 I can keep the same embedding otherwise if I had insisted on keeping enumeration from 1. then all I had to do was:</strong></p>
<pre><code class="lang-auto">self.encoder = nn.Embedding(self.input_size+1, self.hidden_size) #[57,20] but still 56 dimensions as 0th index is still empty.
</code></pre>
<p>Solution :</p>
<pre><code class="lang-auto">class CharMaker(nn.Module):
    def __init__(self, input_size, hidden_size, output_size,n_layers=1):
        super(CharMaker,self).__init__()
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.output_size = output_size
        print("Input &amp; Output Size", input_size,output_size)
        print("Hidden Size ", hidden_size )
        
        self.encoder = nn.Embedding(self.input_size, self.hidden_size) #[56,20]
        
        self.rnn = nn.RNN(self.hidden_size,self.hidden_size, num_layers=1,batch_first=True) #[20,20]
        self.linear = nn.Linear(self.hidden_size, self.output_size) #[20,56]
        
        
    def forward(self, inputs, hidden):
        batch_size = inputs.size(0)
        
        if hidden == None:
            hidden = torch.zeros(1,batch_size,self.hidden_size)
        #print("Original Input : ",inputs.shape)
        encoded = self.encoder(inputs)
        #print("Encoded Input : ",encoded.shape)
        output, hidden = self.rnn(encoded, hidden)
        output = self.linear(output)
        
        return output,hidden
</code></pre>

username: mathematics
cooked: <p>When using random data, I noticed, <code>nn.Embedding</code> only requires posetive numbers to be passed. This error gone if used <code>.abs()</code> or using any natural number.</p>

username: Jaideep_Valani
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> suppose the distinct values for my seq is 0,1,2 .<br>
What should size of embeddings here<br>
3 or 2…</p>

username: ptrblck
cooked: <p>The <code>num_embeddings</code> should be 3 so that all indices can be used to index the weight matrix.</p>

username: TheOraware
cooked: <p>even my indices is lower than embedding_dim-1 within a batch , i am still getting IndexError: index out of range in self , here is my data and code. In each batch for Age category size is 20 and input embedding size is 70 , dont know why indexing error is throwing</p>
<pre><code class="lang-auto">data = pd.read_csv('Churn_Modelling.csv')
print("Shape:", data.shape)
data.head()

X_train = data[['Age','Balance']]
y_train = pd.DataFrame(data['Exited'])
X_train

Shape: (10000, 14)
</code></pre>
<pre><code class="lang-auto">	Age  	Balance
	----	-------
0	 42	        0.00
1	41	    83807.86
2	42	   159660.80
3	39	        0.00
4	43	   125510.82

10000 rows × 2 columns

y_train

	Exited
	-------
0	1
1	0
2	1
3	0
4	0

10000 rows × 1 columns
</code></pre>
<pre><code class="lang-auto">features  = ['Age']
for col in features:
    X_train.loc[:,col] = X_train.loc[:,col].astype('category')
X_train.dtypes
</code></pre>
<pre><code class="lang-auto">Age        category
Balance     float64
dtype: object
</code></pre>
<pre><code class="lang-auto">embedded_cols = {n: len(col.cat.categories) for n,col in X_train[features].items()}
embedded_cols
</code></pre>
<p><code>{'Age': 70}</code></p>
<pre><code class="lang-auto">
 class ShelterOutcomeDataset(Dataset):
     def __init__(self, X, Y, embedded_col_names):
         X = X.copy()
         self.X1 = X.loc[:,embedded_col_names].copy().values.astype(np.int64) #categorical columns
         self.X2 = X.drop(columns=embedded_col_names).copy().values.astype(np.float32) #numerical columns
         self.y = Y.copy().values.astype(np.int64)
        
    def __len__(self):
        return len(self.y)
    
    def __getitem__(self, idx):
        return self.X1[idx], self.X2[idx], self.y[idx]

embedding_sizes = [(n_categories, min(50, (n_categories+1)//2)) for _,n_categories in embedded_cols.items()]
embedding_sizes

[(70, 35)]   

train_ds = ShelterOutcomeDataset(X_train,y_train , ['Age'])

class testNet(nn.Module):
    def __init__(self, emb_dims, n_cont):
        super().__init__()
        
        self.embeddings = nn.ModuleList([nn.Embedding(categories, size) for categories,size in emb_dims])
        no_of_embs = sum(e.embedding_dim for e in self.embeddings) #length of all embeddings combined
        self.n_emb, self.n_cont = no_of_embs, n_cont
        self.lin1 = nn.Linear(self.n_emb + self.n_cont,6)
        self.lin2 = nn.Linear(6, 4)
        self.lin3 = nn.Linear(4, 2)

        self.bn1 = nn.BatchNorm1d(self.n_cont)
        self.bn2 = nn.BatchNorm1d(6)
        self.bn3 = nn.BatchNorm1d(4)
        self.emb_drop = nn.Dropout(0.6)
        self.drops = nn.Dropout(0.3)

    def forward(self, x_cat, x_cont):

        x = [e(x_cat[:,i]) for i,e in enumerate(self.embeddings)]
        x = torch.cat(x, 1)
        
        x = self.emb_drop(x)
        # batch normalization over continous features
        x2 = self.bn1(x_cont)
        # concatenate both embedding and continous feature , here 1 means dim 
        # the dimension over which the tensors are concatenated we are concatenating columns
        x = torch.cat([x, x2], 1)
        x = F.relu(self.lin1(x))
        x = self.drops(x)
        x = self.bn2(x)
        x = F.relu(self.lin2(x))
        x = self.drops(x)
        x = self.bn3(x)
        x = self.lin3(x)

        return x
        
import torch.nn as nn

criterion = nn.CrossEntropyLoss()
def train_model(model, optim, train_dl):
    model.train()
    total = 0
    sum_loss = 0
    for cat, cont, y in train_dl:
        batch = y.shape[0]
        
        print(cat.size())  # &lt;--- size of features whihc has to be embeded

        y = y.to(torch.float32)
        
        output = model(cat, cont)
        _,pred = torch.max(output,1)
        
        loss = criterion(output, y.squeeze(1).long())
        optim.zero_grad()
        loss.backward()
        optim.step()
        total += batch
        sum_loss += batch*(loss.item())
    return sum_loss/total,pred



def train_loop(model, epochs, lr=0.01, wd=0.0):
    optim = get_optimizer(model, lr = lr, wd = wd)
    for epoch in range(epochs): 
        loss,pred = train_model(model, optim, train_dl)
        if (epoch+1) % 5 ==0:
            print(f'epoch : {epoch+1},training loss : {loss}, output : {output}')

batch_size = 20
train_dl = DataLoader(train_ds, batch_size=batch_size,shuffle=True)
#valid_dl = DataLoader(valid_ds, batch_size=batch_size,shuffle=True)    

train_dl = DeviceDataLoader(train_dl, device)
# valid_dl = DeviceDataLoader(valid_dl, device)

# model = ShelterOutcomeModel(embedding_sizes,0)

model = testNet(embedding_sizes,1)
print(model)

from collections import defaultdict
opt = torch.optim.Adam(model.parameters(), lr=1e-2)
# to_device(model, device)

train_loop(model, epochs=100, lr=0.01, wd=0.00001)

testNet(
  (embeddings): ModuleList(
    (0): Embedding(70, 35)
  )
  (lin1): Linear(in_features=36, out_features=6, bias=True)
  (lin2): Linear(in_features=6, out_features=4, bias=True)
  (lin3): Linear(in_features=4, out_features=2, bias=True)
  (bn1): BatchNorm1d(1, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (bn2): BatchNorm1d(6, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (bn3): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (emb_drop): Dropout(p=0.6, inplace=False)
  (drops): Dropout(p=0.3, inplace=False)
)
torch.Size([20, 1])

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
&lt;ipython-input-3281-888e52d4559c&gt; in &lt;module&gt;
     74 # to_device(model, device)
     75 
---&gt; 76 train_loop(model, epochs=100, lr=0.01, wd=0.00001)

&lt;ipython-input-3281-888e52d4559c&gt; in train_loop(model, epochs, lr, wd)
     46     optim = get_optimizer(model, lr = lr, wd = wd)
     47     for epoch in range(epochs):
---&gt; 48         loss,pred = train_model(model, optim, train_dl)
     49         if (epoch+1) % 5 ==0:
     50             print(f'epoch : {epoch+1},training loss : {loss}, output : {output}')

&lt;ipython-input-3281-888e52d4559c&gt; in train_model(model, optim, train_dl)
     15 
     16 
---&gt; 17         output = model(cat, cont)
     18         _,pred = torch.max(output,1)
     19 

~/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--&gt; 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

&lt;ipython-input-3280-681fc4d5712d&gt; in forward(self, x_cat, x_cont)
     30 
     31 
---&gt; 32         x = [e(x_cat[:,i]) for i,e in enumerate(self.embeddings)]
     33         x = torch.cat(x, 1)
     34 

&lt;ipython-input-3280-681fc4d5712d&gt; in &lt;listcomp&gt;(.0)
     30 
     31 
---&gt; 32         x = [e(x_cat[:,i]) for i,e in enumerate(self.embeddings)]
     33         x = torch.cat(x, 1)
     34 

~/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--&gt; 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

~/anaconda3/lib/python3.8/site-packages/torch/nn/modules/sparse.py in forward(self, input)
    122 
    123     def forward(self, input: Tensor) -&gt; Tensor:
--&gt; 124         return F.embedding(
    125             input, self.weight, self.padding_idx, self.max_norm,
    126             self.norm_type, self.scale_grad_by_freq, self.sparse)

~/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1812         # remove once script supports set_grad_enabled
   1813         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-&gt; 1814     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1815 
   1816 

IndexError: index out of range in self</code></pre>

username: ptrblck
cooked: <p>An index value of 70 for an embedding layer size of 70 won’t work, since the valid indices would be in the range <code>[0, 69]</code>, so you would either need to increase the <code>num_embeddings</code> value or clip the input.</p>

username: TheOraware
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> even i increase +1 into num_embeddings i am getting same error</p>
<p><code>self.embeddings = nn.ModuleList([nn.Embedding(categories+1, size) for categories,size in emb_dims])</code></p>
<p>Interesting thing is when i increase it by 23 it does not give error</p>
<pre><code class="lang-auto">class testNet(nn.Module):
    def __init__(self, emb_dims, n_cont):
        super().__init__()

        for categories,size in emb_dims:
            print(f'catagrorize is {categories}, size is {size}')
        
        self.embeddings = nn.ModuleList([nn.Embedding(categories+23, size) for categories,size in emb_dims])
        no_of_embs = sum(e.embedding_dim for e in self.embeddings) #length of all embeddings combined
        self.n_emb, self.n_cont = no_of_embs, n_cont
        self.lin1 = nn.Linear(self.n_emb + self.n_cont,6)
        self.lin2 = nn.Linear(6, 4)
        self.lin3 = nn.Linear(4, 2)

        self.bn1 = nn.BatchNorm1d(self.n_cont)
        self.bn2 = nn.BatchNorm1d(6)
        self.bn3 = nn.BatchNorm1d(4)
        self.emb_drop = nn.Dropout(0.6)
        self.drops = nn.Dropout(0.3)
        
    def forward(self, x_cat, x_cont):
        # take the embedding list and grab an embedding and pass in our single row of data.
        x = [e(x_cat[:,i]) for i,e in enumerate(self.embeddings)]
        x = torch.cat(x, 1)
        
        x = self.emb_drop(x)
        x2 = self.bn1(x_cont)
        x = torch.cat([x, x2], 1)
        x = F.relu(self.lin1(x))
        x = self.drops(x)
        x = self.bn2(x)
        x = F.relu(self.lin2(x))
        x = self.drops(x)
        x = self.bn3(x)
        x = self.lin3(x)

        return x
</code></pre>
<pre><code class="lang-auto">import torch.nn as nn

criterion = nn.CrossEntropyLoss()
def train_model(model, optim, train_dl):
    model.train()
    total = 0
    sum_loss = 0
    for cat, cont, y in train_dl:
        batch = y.shape[0]

        y = y.to(torch.float32)
        
        output = model(cat, cont)
        _,pred = torch.max(output,1)

        loss = criterion(output, y.squeeze(1).long())
        optim.zero_grad()
        loss.backward()
        optim.step()
        total += batch
        sum_loss += batch*(loss.item())
    return sum_loss/total,pred


def train_loop(model, epochs, lr=0.01, wd=0.0):
    optim = get_optimizer(model, lr = lr, wd = wd)
    for epoch in range(epochs): 
        loss,pred = train_model(model, optim, train_dl)
        if (epoch+1) % 5 ==0:
            print(f'epoch : {epoch+1},training loss : {loss}, output : {output}')

batch_size = 100
train_dl = DataLoader(train_ds, batch_size=batch_size,shuffle=True)

train_dl = DeviceDataLoader(train_dl, device)



print(f'embedding_sizes is {embedding_sizes}')
model = testNet(embedding_sizes,1)


from collections import defaultdict
opt = torch.optim.Adam(model.parameters(), lr=1e-2)


train_loop(model, epochs=100, lr=0.01, wd=0.00001)
</code></pre>
<pre><code class="lang-auto">embedding_sizes is [(70, 35)]
catagrorize is 70, size is 35
epoch : 5,training loss : 0.4648001512885094, output : 0.6111002564430237
epoch : 10,training loss : 0.4541498306393623, output : 0.6111002564430237
epoch : 15,training loss : 0.45384191155433656, output : 0.6111002564430237
epoch : 20,training loss : 0.45079687386751177, output : 0.6111002564430237
epoch : 25,training loss : 0.4511949673295021, output : 0.6111002564430237
epoch : 30,training loss : 0.45295464009046554, output : 0.6111002564430237
epoch : 35,training loss : 0.45299509912729263, output : 0.6111002564430237
epoch : 40,training loss : 0.45105998665094377, output : 0.6111002564430237
epoch : 45,training loss : 0.4528631994128227, output : 0.6111002564430237
epoch : 50,training loss : 0.4509485891461372, output : 0.6111002564430237
epoch : 55,training loss : 0.4534462735056877, output : 0.6111002564430237
epoch : 60,training loss : 0.4507604452967644, output : 0.6111002564430237
epoch : 65,training loss : 0.4527029529213905, output : 0.6111002564430237
epoch : 70,training loss : 0.4511090362071991, output : 0.6111002564430237
epoch : 75,training loss : 0.4510712164640427, output : 0.6111002564430237
epoch : 80,training loss : 0.4523083609342575, output : 0.6111002564430237
epoch : 85,training loss : 0.4539755055308342, output : 0.6111002564430237
epoch : 90,training loss : 0.4536020648479462, output : 0.6111002564430237
epoch : 95,training loss : 0.4528249257802963, output : 0.6111002564430237
epoch : 100,training loss : 0.45215955764055255, output : 0.6111002564430237
</code></pre>

username: ptrblck
cooked: <aside class="quote no-group" data-username="TheOraware" data-post="11" data-topic="81550">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/theoraware/48/30315_2.png" class="avatar"> TheOraware:</div>
<blockquote>
<p>Interesting thing is when i increase it by 23 it does not give error</p>
</blockquote>
</aside>
<p>This could mean that the max. index is larger than you expect, so you could add <code>assert</code> statements to your code to narrow down the largest values.</p>

username: TheOraware
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> thanks for pointing me out , could you please give me this statement as i am very new to pytorch , how and where i need to add assert statements?</p>

username: ptrblck
cooked: <p>In your training loop you could use something like this:</p>
<pre><code class="lang-python"># works
target = torch.tensor([69, 68])
assert (target&lt;70).all(), "target: {} invalid".format(target)

# fails
target = torch.tensor([69, 70])
assert (target&lt;70).all(), "target: {} invalid".format(target)
</code></pre>
<p>which would trigger the errors for target values larger than the used threshold.<br>
Once you are running into these errors, check why you expect them to be <code>&lt;70</code> and why they are apparently not in this range.</p>

username: TheOraware
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> thanks mate , i work on it and get back to you here if i need more assitance - you are great</p>

username: TheOraware
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> , can you please help me where to put your above suggested code , as you mentioned in training loop then i guess it could be like</p>
<pre><code class="lang-auto">def train_loop(model, epochs, lr=0.01, wd=0.0):
    optim = get_optimizer(model, lr = lr, wd = wd)
    for epoch in range(epochs): 
        
        target = torch.tensor([69, 68])&lt;---------------
        assert (target&lt;70).all(), "target: {} invalid".format(target)&lt;-------------
        
        loss,pred = train_model(model, optim, train_dl)
        if (epoch+1) % 5 ==0:
            print(f'epoch : {epoch+1},training loss : {loss}, output : {output}')
</code></pre>

username: TheOraware
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> i fixed issue , actually that was due to not converting Age column into Label Encoder , after transforming into label encoder , data needs to represented by its Index for embedding. Label Encoding is for index before your input to the embedding layer. It means some of Age which were not transformed into label encoder initially should have value greater than 70 or 69 was causing this problem. Is my understanding correct ? Do we really need to convert columns into label encoding before embedding?</p>

username: ptrblck
cooked: <p>You don’t <em>need</em> to use a label encoding, but it can be useful if you want to map your targets to <code>[0, nb_classes-1]</code>, which is expected by the embedding layer (and usually also loss functions).<br>
Alternatively, you could also use the max. value of the “unencoded” target and set it as the <code>num_embeddings</code> in case you are expecting to get the missing values in the future.</p>

username: TheOraware
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> can you please shed some light on it <a href="https://discuss.pytorch.org/t/embedding-layer/121969">https://discuss.pytorch.org/t/embedding-layer/121969</a></p>

username: M_M_Kamalraj
cooked: <p>Faced the IndexError: index out of range in self when experimenting with nn.Embedding.</p>
<p>I feel the below observations might provide the necessary intuition.</p>
<p>emb = nn.Embedding(13, 3)</p>
<p>input = torch.LongTensor([13, 8, 5, 0])</p>
<p>print(emb(input)) # will throw IndexError</p>
<p>Error Cause: 13 at Index 0 of input causes the error, as it can be max 12</p>
<p>input = torch.LongTensor([12, 8, 5, 0])</p>
<p>print(emb(input)) # will work</p>
<p>No Error: 12 at Index 0 of input can be embedded</p>