I got the error: RuntimeError: CUDA error: device-side assert triggered

Sanjay_Bhandari1: <p>Traceback (most recent call last):<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py”, line 44, in _call_and_handle_interrupt<br>
return trainer_fn(*args, **kwargs)<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py”, line 559, in _fit_impl<br>
self._run(model, ckpt_path=ckpt_path)<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py”, line 935, in _run<br>
results = self._run_stage()<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py”, line 976, in _run_stage<br>
self._run_sanity_check()<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py”, line 1005, in _run_sanity_check<br>
val_loop.run()<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/pytorch_lightning/loops/utilities.py”, line 174, in _decorator<br>
return loop_run(self, *args, **kwargs)<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/pytorch_lightning/loops/evaluation_loop.py”, line 115, in run<br>
self._evaluation_step(batch, batch_idx, dataloader_idx)<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/pytorch_lightning/loops/evaluation_loop.py”, line 375, in _evaluation_step<br>
output = call._call_strategy_hook(trainer, hook_name, *step_kwargs.values())<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py”, line 288, in _call_strategy_hook<br>
output = fn(*args, **kwargs)<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py”, line 378, in validation_step<br>
return self.model.validation_step(*args, **kwargs)<br>
File “/mnt/Enterprise/sanjay/image-captioning/Image-Captioning/models/Image_captioner.py”, line 102, in validation_step<br>
return self.step(batch, mode=“val”)<br>
File “/mnt/Enterprise/sanjay/image-captioning/Image-Captioning/models/Image_captioner.py”, line 87, in step<br>
preds = self.model(text_tokens,imgs,masks,pad_mask)<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1501, in _call_impl<br>
return forward_call(*args, **kwargs)<br>
File “/mnt/Enterprise/sanjay/image-captioning/Image-Captioning/models/Image_captioner.py”, line 45, in forward<br>
x = self.decoder(text,x,target_subsequent_mask=mask,target_pad_mask=pad_mask)<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1501, in _call_impl<br>
return forward_call(*args, **kwargs)<br>
File “/mnt/Enterprise/sanjay/image-captioning/Image-Captioning/models/decoder.py”, line 57, in forward<br>
pos_x = pos_x + self.pos_embedding(text_input)<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1501, in _call_impl<br>
return forward_call(*args, **kwargs)<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/torch/nn/modules/sparse.py”, line 162, in forward<br>
return F.embedding(<br>
File “/mnt/Enterprise/sanjay/new_env/lib/python3.10/site-packages/torch/nn/functional.py”, line 2210, in embedding<br>
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)<br>
RuntimeError: CUDA error: device-side assert triggered<br>
Compile with <code>TORCH_USE_CUDA_DSA</code> to enable device-side assertions.</p>
<p>My code is as ::</p>
<p>class Decoder(nn.Module):<br>
def <strong>init</strong>(self,vocab_size,max_seq_length,embed_dim,num_heads,num_layers,dropout=0.0) → None:<br>
super().<strong>init</strong>()</p>
<pre><code>      self.token_embedding = nn.Embedding(vocab_size,embed_dim)
      
      self.pos_embedding = nn.Embedding(max_seq_length,embed_dim)
      
      
      self.decoder_layer = nn.TransformerDecoderLayer(embed_dim, num_heads, embed_dim, dropout, batch_first=True)
      self.decoder = nn.TransformerDecoder(self.decoder_layer, num_layers)
      
      
      self.dropout = nn.Dropout(dropout)
      
      self.ln = nn.LayerNorm(embed_dim)
      
      self.linear = nn.Linear(embed_dim, vocab_size)
      
  def forward(self,text_input,encoder_memory,target_subsequent_mask=None,target_pad_mask=None):
      
      B,T = text_input.shape
      
      """
      TODO: layernorm still before embedding
      """
      pos_x = self.token_embedding(text_input)
      
      
      pos_x = pos_x + self.pos_embedding(text_input)
      
      pos_x = self.dropout(pos_x)
      
      # print(encoder_memory.shape)
      # print(pos_x.shape)
      
      x=  self.decoder(
          pos_x,
          encoder_memory,
          tgt_mask=target_subsequent_mask,
          tgt_key_padding_mask=target_pad_mask,
      )
      
      
      print(x.shape)
      
      x = self.linear(x)
      # x = self.linear(self.ln(x))
      
      return x
</code></pre>

ptrblck: <p>Based on the stacktrace:</p>
<pre><code class="lang-python">return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: CUDA error: device-side assert triggered
</code></pre>
<p>I guess you are passing invalid indices to your <code>nn.Embedding</code> layer which then fails with an indexing error. Make sure the input tensor contains values in the range <code>[0, num_embeddings-1]</code> and it should work.</p>