IndexError: Target is out of bounds

PresidentDoggo: <p>Hey guys, I have a neural network with an output of 23 classes</p>
<pre><code class="lang-auto">self.out = nn.Linear(in_features = 180, out_features = 23) 
</code></pre>
<p>I’m using cross entropy loss:</p>
<pre><code class="lang-auto">loss = F.cross_entropy(predictions, batch_y)
</code></pre>
<p>The inputs to the cross entropy function are of shape:</p>
<pre><code class="lang-auto">predictions.shape
&gt;&gt; torch.Size([128, 23])

batch_y.shape
&gt;&gt; torch.Size([128])
</code></pre>
<p>and I get an out of bounds error:</p>
<pre><code class="lang-auto">---&gt; 38         loss = F.cross_entropy(predictions, batch_y) 

IndexError: Target 23 is out of bounds.
</code></pre>
<p>However when I change the out_features in the output linear layer of the network from ‘23’ to ‘24’, it resolves the issue.<br>
My labels are not one-hot encoded.<br>
Because I now have an extra output class how will that effect the system?<br>
Why has increasing the number of output neurons resolved the issue in the first place?<br>
Have I missed something important?</p>
<p>Thanks for any advice, I can post the full code on request <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

chetan_patil: <p>Hi,<br>
Classes must span between 0 to N-1( 22 ). I think your classes span from 1 to 23.<br>
Change <code>batch_y</code> to span between 0 to 22, and keep your output neurons to 23.<br>
I suspect this is the issue.!</p>

PresidentDoggo: <p>My targets are categorical data that I have encoded numerically 1 - 23. That is most likely the problem. I will try to fix and see if that works,</p>
<p>Thanks!</p>

chetan_patil: <p>yes thats the issue, have a look at this<br>
<a href="https://pytorch.org/docs/master/generated/torch.nn.CrossEntropyLoss.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/docs/master/generated/torch.nn.CrossEntropyLoss.html</a></p>
<p>It needs classes to span between 0 to N-1</p>

PresidentDoggo: <p>I’ve marked your first post as solution, thanks for your help <a class="mention" href="/u/chetan_patil">@chetan_patil</a> !</p>

chetan_patil: <p>You’re welcome <a class="mention" href="/u/presidentdoggo">@PresidentDoggo</a> !</p>

Balazs_Gonczy: <p>We have the similar issue but doing this is not solving it, pls help! <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=12" title=":frowning:" class="emoji" alt=":frowning:" loading="lazy" width="20" height="20"></p>

Harshal_Dharpure: <pre><code class="lang-auto">
class Classifier(pl.LightningModule):

  def __init__(self):
    super().__init__()
    self.MFB = MFB(512,768,True,256,64,0.1)
    self.fin_y_shape = torch.nn.Linear(768,512)
    self.fin_old = torch.nn.Linear(64,2)
    self.fin = torch.nn.Linear(16 * 768, 64)
    self.fin_inten = torch.nn.Linear(64,6)
    self.fin_e1 = torch.nn.Linear(64,2)
    self.fin_e2 = torch.nn.Linear(64,2)
    self.fin_e3 = torch.nn.Linear(64,2)
    self.fin_e4 = torch.nn.Linear(64,2)
    self.fin_e5 = torch.nn.Linear(64,2)
    self.fin_e6 = torch.nn.Linear(64,2)
    self.fin_e7 = torch.nn.Linear(64,2)
    self.fin_e8 = torch.nn.Linear(64,2)
    self.fin_e9 = torch.nn.Linear(64,2)
    


    self.validation_step_outputs = []
    self.test_step_outputs = []

  def forward(self, x,y,rag):
      x_,y_,rag_ = x,y,rag
      print("x.shape", x.shape)
      z = self.MFB(torch.unsqueeze(y, axis=1), torch.unsqueeze(x, axis=1))
      #cross_attention= (rag and  x/y)
      z_new = torch.squeeze(z, dim=1)
      c_inten = self.fin_inten(z_new)
      c_e1 = self.fin_e1(z_new)
      c_e2 = self.fin_e2(z_new)
      c_e3 = self.fin_e3(z_new)
      c_e4 = self.fin_e4(z_new)
      c_e5 = self.fin_e5(z_new)
      c_e6 = self.fin_e6(z_new)
      c_e7 = self.fin_e7(z_new)
      c_e8 = self.fin_e8(z_new)
      c_e9 = self.fin_e9(z_new)
      c = self.fin_old(z_new)


      output = torch.log_softmax(c, dim=1)
      c_inten = torch.log_softmax(c_inten, dim=1)
      c_e1 = torch.log_softmax(c_e1, dim=1)
      c_e2 = torch.log_softmax(c_e2, dim=1)
      c_e3 = torch.log_softmax(c_e3, dim=1)
      c_e4 = torch.log_softmax(c_e4, dim=1)

      c_e5 = torch.log_softmax(c_e5, dim=1)
      c_e6 = torch.log_softmax(c_e6, dim=1)
      c_e7 = torch.log_softmax(c_e7, dim=1)

      c_e8 = torch.log_softmax(c_e8, dim=1)
      c_e9 = torch.log_softmax(c_e9, dim=1)


      return output,c_e1,c_e2,c_e3,c_e4,c_e5,c_e6,c_e7,c_e8,c_e9,c_inten

  def cross_entropy_loss(self, logits, labels):
    return F.nll_loss(logits, labels)

  def training_step(self, train_batch, batch_idx):
      #lab,txt,rag,img,name,per,iro,alli,ana,inv,meta,puns,sat,hyp= train_batch
      lab,txt,rag,img,name,e1,e2,e3,e4,e5,e6,e7,e8,e9,intensity = train_batch
      #logit_offen,a,b,c,d,e,f,g,h,i,logit_inten_target= self.forward(txt,img,rag)
      lab = train_batch[lab]
      #print(lab)
      name= train_batch[name]
      txt = train_batch[txt]
      rag = train_batch[rag]
      img = train_batch[img]
      e1 = train_batch[e1]
      e2 = train_batch[e2]
      e3 = train_batch[e3]
      e4 = train_batch[e4]
      e5 = train_batch[e5]
      e6 = train_batch[e6]
      e7 = train_batch[e7]
      e8 = train_batch[e8]
      e9 = train_batch[e9]
      intensity = train_batch[intensity]
      # per = train_batch[per]
      # iro= train_batch[iro]
      # alli = train_batch[alli]
      # ana = train_batch[ana]
      # inv = train_batch[inv]
      # meta = train_batch[meta]
      # puns = train_batch[puns]
      # sat = train_batch[sat]
      # hyp = train_batch[hyp]

      logit_offen,a,b,c,d,e,f,g,h,i,logit_inten_target= self.forward(txt,img,rag)

      loss1 = self.cross_entropy_loss(logit_offen, lab)
      loss4 = self.cross_entropy_loss(a, e1)
      loss5 = self.cross_entropy_loss(b, e2)
      loss6 = self.cross_entropy_loss(c, e3)
      loss7 = self.cross_entropy_loss(d, e4)
      loss8 = self.cross_entropy_loss(e, e5)
      loss9 = self.cross_entropy_loss(f, e6)
      loss10 = self.cross_entropy_loss(g, e7)
      loss11 = self.cross_entropy_loss(h, e8)
      loss12 = self.cross_entropy_loss(i, e9)
      loss17 = self.cross_entropy_loss(logit_inten_target, intensity)
      # loss2 = self.cross_entropy_loss(a,per)
      # loss3 = self.cross_entropy_loss(b,iro)
      # loss4 = self.cross_entropy_loss(c, alli)
      # loss5 = self.cross_entropy_loss(d,ana)
      # loss6 = self.cross_entropy_loss(e,inv)
      # loss7 = self.cross_entropy_loss(f,meta)
      # loss8 = self.cross_entropy_loss(g,puns)
      # loss9 = self.cross_entropy_loss(h,sat)
      # loss10 = self.cross_entropy_loss(i,hyp)

      loss = loss1 + loss4 + loss5 + loss6 + loss7 + loss8 +loss9 + loss10 +loss11 +loss12

      self.log('train_loss', loss)
      return loss


  def validation_step(self, val_batch, batch_idx):
      #lab,txt,rag,img,name,per,iro,alli,ana,inv,meta,puns,sat,hyp = val_batch
      lab,txt,rag,img,name,e1,e2,e3,e4,e5,e6,e7,e8,e9,intensity= val_batch
      lab = val_batch[lab]
      #print(lab)
      txt = val_batch[txt]
      rag = val_batch[rag]
      img = val_batch[img]
      name = val_batch[name]
      e1 = val_batch[e1]
      e2 = val_batch[e2]
      e3 = val_batch[e3]
      e4 = val_batch[e4]
      e5 = val_batch[e5]
      e6 = val_batch[e6]
      e7 = val_batch[e7]
      e8 = val_batch[e8]
      e9 = val_batch[e9]
      intensity = val_batch[intensity]
      # per = val_batch[per]
      # iro = val_batch[iro]
      # alli = val_batch[alli]
      # ana = val_batch[ana]
      # inv = val_batch[inv]
      # meta = val_batch[meta]
      # puns = val_batch[puns]
      # sat = val_batch[sat]
      # hyp = val_batch[hyp]


      logits,a,b,c,d,e,f,g,h,i,inten = self.forward(txt,img,rag)


      logits=logits.float()
    

      tmp = np.argmax(logits.detach().cpu().numpy(),axis=1)
      loss = self.cross_entropy_loss(logits, lab)
      lab = lab.detach().cpu().numpy()
      self.log('val_acc', accuracy_score(lab,tmp))
      self.log('val_roc_auc',roc_auc_score(lab,tmp))
      self.log('val_loss', loss)
      tqdm_dict = {'val_acc': accuracy_score(lab,tmp)}
      self.validation_step_outputs.append({'progress_bar': tqdm_dict,'val_f1 offensive': f1_score(lab,tmp,average='macro')})

      return {
                'progress_bar': tqdm_dict,
      'val_f1 offensive': f1_score(lab,tmp,average='macro')
      }

  def on_validation_epoch_end(self):
    outs = []
    outs14=[]
    for out in self.validation_step_outputs:
       outs.append(out['progress_bar']['val_acc'])
       outs14.append(out['val_f1 offensive'])
    self.log('val_acc_all_offn', sum(outs)/len(outs))
    self.log('val_f1 offensive', sum(outs14)/len(outs14))
    print(f'***val_acc_all_offn at epoch end {sum(outs)/len(outs)}****')
    print(f'***val_f1 offensive at epoch end {sum(outs14)/len(outs14)}****')
    self.validation_step_outputs.clear()

  def test_step(self, batch, batch_idx):
      lab,txt,rag,img,name,e1,e2,e3,e4,e5,e6,e7,e8,e9,intensity= batch
      #lab,txt,rag,img,name,per,iro,alli,ana,inv,meta,puns,sat,hyp= batch

      lab = batch[lab]
      #print(lab)
      rag = batch[rag]

      txt = batch[txt]
      img = batch[img]
      name = batch[name]
      e1 = batch[e1]
      e2 = batch[e2]
      e3 = batch[e3]
      e4 = batch[e4]
      e5 = batch[e5]
      e6 = batch[e6]
      e7 = batch[e7]
      e8 = batch[e8]
      e9 = batch[e9]
      intensity = batch[intensity]
      # per = batch[per]
      # iro = batch[iro]
      # alli = batch[alli]
      # ana = batch[ana]
      # inv = batch[inv]
      # meta = batch[meta]
      # puns = batch[puns]
      # sat = batch[sat]
      # hyp = batch[hyp]

      logits,a,b,c,d,e,f,g,h,i,inten= self.forward(txt,img,rag)
      logits = logits.float()
      tmp = np.argmax(logits.detach().cpu().numpy(force=True),axis=-1)
      loss = self.cross_entropy_loss(logits, lab)
      lab = lab.detach().cpu().numpy()
      self.log('test_acc', accuracy_score(lab,tmp))
      self.log('test_roc_auc',roc_auc_score(lab,tmp))
      self.log('test_loss', loss)
      tqdm_dict = {'test_acc': accuracy_score(lab,tmp)}
      self.test_step_outputs.append({'progress_bar': tqdm_dict,'test_acc': accuracy_score(lab,tmp), 'test_f1_score': f1_score(lab,tmp,average='macro')})
      return {
                'progress_bar': tqdm_dict,
                'test_acc': accuracy_score(lab,tmp),
                'test_f1_score': f1_score(lab,tmp,average='macro')
      }
  def on_test_epoch_end(self):
      # OPTIONAL
      outs = []
      outs1,outs2,outs3,outs4,outs5,outs6,outs7,outs8,outs9,outs10,outs11,outs12,outs13,outs14 = \
      [],[],[],[],[],[],[],[],[],[],[],[],[],[]
      for out in self.test_step_outputs:
        outs.append(out['test_acc'])
        outs2.append(out['test_f1_score'])
      self.log('test_acc', sum(outs)/len(outs))
      self.log('test_f1_score', sum(outs2)/len(outs2))
      self.test_step_outputs.clear()

  def configure_optimizers(self):
    # optimizer = torch.optim.Adam(self.parameters(), lr=3e-2)
    optimizer = torch.optim.Adam(self.parameters(), lr=1e-5)

    return optimizer


"""
Main Model:
Initialize
Forward Pass
Training Step
Validation Step
Testing Step

Pp
"""

class HmDataModule(pl.LightningDataModule):

  def setup(self, stage):
    self.hm_train = t_p
    self.hm_val = v_p
    # self.hm_test = test
    self.hm_test = te_p

  def train_dataloader(self):
    return DataLoader(self.hm_train, batch_size=10, drop_last=True)

  def val_dataloader(self):
    return DataLoader(self.hm_val, batch_size=10, drop_last=True)

  def test_dataloader(self):
    return DataLoader(self.hm_test, batch_size=10, drop_last=True)

data_module = HmDataModule()
checkpoint_callback = ModelCheckpoint(
     monitor='val_acc_all_offn',
     dirpath='mrinal/',
     filename='epoch{epoch:02d}-val_f1_all_offn{val_acc_all_offn:.2f}',
     auto_insert_metric_name=False,
     save_top_k=1,
    mode="max",
 )
all_callbacks = []
all_callbacks.append(checkpoint_callback)
# train
from pytorch_lightning import seed_everything
seed_everything(42, workers=True)
hm_model = Classifier()
gpus=1
#if torch.cuda.is_available():gpus=0
trainer = pl.Trainer(deterministic=True,max_epochs=2,precision=16,callbacks=all_callbacks)
trainer.fit(hm_model, data_module)






INFO:lightning_fabric.utilities.seed:Seed set to 42
/usr/local/lib/python3.10/dist-packages/lightning_fabric/connector.py:563: `precision=16` is supported for historical reasons but its usage is discouraged. Please set your precision to 16-mixed instead!
/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py:552: You passed `Trainer(accelerator='cpu', precision='16-mixed')` but AMP with fp16 is not supported on CPU. Using `precision='bf16-mixed'` instead.
INFO:pytorch_lightning.utilities.rank_zero:Using bfloat16 Automatic Mixed Precision (AMP)
INFO:pytorch_lightning.utilities.rank_zero:GPU available: False, used: False
INFO:pytorch_lightning.utilities.rank_zero:TPU available: False, using: 0 TPU cores
INFO:pytorch_lightning.utilities.rank_zero:IPU available: False, using: 0 IPUs
INFO:pytorch_lightning.utilities.rank_zero:HPU available: False, using: 0 HPUs
/usr/local/lib/python3.10/dist-packages/pytorch_lightning/callbacks/model_checkpoint.py:653: Checkpoint directory /content/LLaVA/mrinal exists and is not empty.
INFO:pytorch_lightning.callbacks.model_summary:
   | Name        | Type   | Params
----------------------------------------
0  | MFB         | MFB    | 21.0 M
1  | fin_y_shape | Linear | 393 K 
2  | fin_old     | Linear | 130   
3  | fin         | Linear | 786 K 
4  | fin_inten   | Linear | 390   
5  | fin_e1      | Linear | 130   
6  | fin_e2      | Linear | 130   
7  | fin_e3      | Linear | 130   
8  | fin_e4      | Linear | 130   
9  | fin_e5      | Linear | 130   
10 | fin_e6      | Linear | 130   
11 | fin_e7      | Linear | 130   
12 | fin_e8      | Linear | 130   
13 | fin_e9      | Linear | 130   
----------------------------------------
22.2 M    Trainable params
0         Non-trainable params
22.2 M    Total params
88.745    Total estimated model params size (MB)
x.shape torch.Size([10, 768])
x.shape torch.Size([10, 768])
***val_acc_all_offn at epoch end 0.3****
***val_f1 offensive at epoch end 0.29292929292929293****
/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/fit_loop.py:298: The number of training batches (34) is smaller than the logging interval Trainer(log_every_n_steps=50). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.
Epoch 0:   0%
 0/34 [00:00&lt;?, ?it/s]
x.shape torch.Size([10, 768])
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
&lt;ipython-input-90-b8b97f911905&gt; in &lt;cell line: 296&gt;()
    294 #if torch.cuda.is_available():gpus=0
    295 trainer = pl.Trainer(deterministic=True,max_epochs=2,precision=16,callbacks=all_callbacks)
--&gt; 296 trainer.fit(hm_model, data_module)

29 frames
/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
   2702     if size_average is not None or reduce is not None:
   2703         reduction = _Reduction.legacy_get_string(size_average, reduce)
-&gt; 2704     return torch._C._nn.nll_loss_nd(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
   2705 
   2706 

IndexError: Target 2 is out of bounds.
</code></pre>
<p>can you please help me in this problem</p>

ptrblck: <p>Double post from <a href="https://discuss.pytorch.org/t/the-error-is-indexerror-target-2-is-out-of-bounds-i-need-guidance-please/159629/13">here</a>.</p>