RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
Author: lcoandrade
Pull request: no
Labels: name: solved

### System Info

- `transformers` version: 4.30.2
- Platform: Linux-5.15.120+-x86_64-with-glibc2.31
- Python version: 3.10.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.0+cpu (False)
- Tensorflow version (GPU?): 2.12.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.7.0 (cpu)
- Jax version: 0.4.13
- JaxLib version: 0.4.13
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker and @younesbelkada

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I'm trying to make a Sarcasm detector with Lightning in this Kaggle [notebook](https://www.kaggle.com/code/luizclaudioandrade/nlp-with-pytorch).

When I start the training, I get this error:
`RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn`

This is my LightningModule:
```
class SarcasmTagger(pl.LightningModule):

    def __init__(
        self, 
        model_name: str, 
        n_classes: int, 
        n_training_steps=None, 
        n_warmup_steps=None
    ):
        super().__init__()
        self.bert = BertModel.from_pretrained(model_name, return_dict=True)
        #self.bert =  BertForSequenceClassification.from_pretrained(model_name, return_dict=True)
        self.classifier = nn.Linear(self.bert.config.hidden_size, n_classes)
        self.n_training_steps = n_training_steps
        self.n_warmup_steps = n_warmup_steps

    def forward(self, input_ids, attention_mask):
        outputs = self.bert(input_ids=input_ids, attention_mask=attention_mask)
        #print(outputs)
        logits = self.classifier(outputs.pooler_output)
        return logits
    
    def shared_step(self, batch, batch_idx):
        input_ids = batch["input_ids"]
        attention_mask = batch["attention_mask"]
        label = batch["label"].view(-1, 1)
        logits = self(input_ids=input_ids, attention_mask=attention_mask)
        loss = nn.functional.cross_entropy(logits, label)
        return logits, loss, label
        

    def training_step(self, batch, batch_idx):
        logits, loss, label = self.shared_step(batch, batch_idx)
        self.log("train_loss", loss, prog_bar=True, logger=True)
        return {"loss": loss, "predictions": logits, "label": label}

    def validation_step(self, batch, batch_idx):
        logits, loss, label = self.shared_step(batch, batch_idx)
        self.log("val_loss", loss, prog_bar=True, logger=True)
        return loss

    def test_step(self, batch, batch_idx):
        logits, loss, label = self.shared_step(batch, batch_idx)
        self.log("test_loss", loss, prog_bar=True, logger=True)
        return loss

    def configure_optimizers(self):
        optimizer = AdamW(self.parameters(), lr=2e-5)

        scheduler = get_linear_schedule_with_warmup(
          optimizer,
          num_warmup_steps=self.n_warmup_steps,
          num_training_steps=self.n_training_steps
        )

        return dict(
            optimizer=optimizer,
            lr_scheduler=dict(
                scheduler=scheduler,
                interval='step')
        )
```

What is the problem here? I'm lost.

Thanks!

### Expected behavior

Execute the training without errors.

Comments:
Even setting the model to train model with:
`self.bert.train()`

I get the same error.

Hey 👋🏻  
We need a minimal reproducing script in order to help: you are using an external library `pytorch lightning` and I have no idea what is happening inside. `transformers` also has a trainer class, we can help if a bug is related to it, but in this case I cannot ping anyone. 

Hi there!
Is it possible to check my Kaggle notebook ([https://www.kaggle.com/code/luizclaudioandrade/nlp-with-pytorch](https://www.kaggle.com/code/luizclaudioandrade/nlp-with-pytorch))?

The notebook has a [dataset](https://www.kaggle.com/datasets/rmisra/news-headlines-dataset-for-sarcasm-detection) with headlines and a tag if it is sarcastic or not (1 or 0). I'm creating a pandas dat frame and creating a dataset with this dataset encoded.

The dataset:
```
class SarcasticHeadlineDataset(Dataset):

    def __init__(
        self, 
        data: pd.DataFrame, 
        tokenizer: BertTokenizer, 
        max_token_len: int,
    ):
        self.tokenizer = tokenizer
        self.data = data
        self.max_token_len = max_token_len
    
    def __len__(self):
        return len(self.data)

    def __getitem__(self, index: int):
        data_row = self.data.iloc[index]

        headline = data_row.headline
        label = data_row.is_sarcastic

        encoding = self.tokenizer.encode_plus(
            headline,
            padding='max_length',
            max_length=self.max_token_len,
            return_tensors='pt',
        )

        return dict(
          headline=headline,
          input_ids=encoding["input_ids"].flatten(),
          attention_mask=encoding["attention_mask"].flatten(),
          label=torch.tensor(label, dtype=torch.float)
        )
```

The datamodule:
```
class SarcasticHeadlineDataModule(pl.LightningDataModule):

    def __init__(
        self, 
        train_df: pd.DataFrame, 
        test_df: pd.DataFrame, 
        tokenizer: BertTokenizer, 
        batch_size: int = 8, 
        max_token_len: int = 128, 
        num_workers: int = 4
    ):
        super().__init__()
        self.batch_size = batch_size
        self.train_df = train_df
        self.test_df = test_df
        self.tokenizer = tokenizer
        self.max_token_len = max_token_len
        self.num_workers = num_workers

    def setup(self, stage=None):
        self.train_dataset = SarcasticHeadlineDataset(
            self.train_df,
            self.tokenizer,
            self.max_token_len
        )

        self.test_dataset = SarcasticHeadlineDataset(
            self.test_df,
            self.tokenizer,
            self.max_token_len
        )

    def train_dataloader(self):
        return DataLoader(
            self.train_dataset,
            batch_size=self.batch_size,
            shuffle=True,
            num_workers=self.num_workers,
            pin_memory=True,
            persistent_workers=True,
        )

    def val_dataloader(self):
        return DataLoader(
            self.test_dataset,
            batch_size=self.batch_size,
            num_workers=self.num_workers,
            pin_memory=True,
            persistent_workers=True,
        )

    def test_dataloader(self):
        return DataLoader(
            self.test_dataset,
            batch_size=self.batch_size,
            num_workers=self.num_workers,
            pin_memory=True,
            persistent_workers=True,
        )
```

Some parameters for the trainer:
```
chekpoint_dir = os.path.join(OUTPUT_DIR, 'checkpoints')
checkpoint_callback = ModelCheckpoint(
  dirpath=chekpoint_dir,
  filename="best-checkpoint",
  save_top_k=1,
  verbose=True,
  monitor="val_loss",
  mode="min"
)

early_stopping_callback = EarlyStopping(monitor='val_loss', patience=2)

logger = CSVLogger(OUTPUT_DIR, name='lightning_logs')
```

The trainer:
```
trainer = pl.Trainer(
    accelerator='auto',
    devices='auto',
    strategy='auto',
    callbacks=[checkpoint_callback, early_stopping_callback],
    max_epochs=N_EPOCHS,
    logger=logger,
)
```

The LightningModule:
```
class SarcasmTagger(pl.LightningModule):

    def __init__(
        self, 
        model_name: str, 
        n_classes: int, 
        n_training_steps=None, 
        n_warmup_steps=None
    ):
        super().__init__()
        
        self.save_hyperparameters()
        
        self.bert = BertModel.from_pretrained(model_name, return_dict=True)
        self.bert.train()
        self.classifier = nn.Linear(self.bert.config.hidden_size, n_classes)
        self.n_training_steps = n_training_steps
        self.n_warmup_steps = n_warmup_steps

    def forward(self, input_ids, attention_mask):
        outputs = self.bert(input_ids=input_ids, attention_mask=attention_mask)
        #print(outputs)
        logits = self.classifier(outputs.pooler_output)
        return logits
    
    def shared_step(self, batch, batch_idx):
        input_ids = batch["input_ids"]
        attention_mask = batch["attention_mask"]
        label = batch["label"].view(-1, 1)
        logits = self(input_ids=input_ids, attention_mask=attention_mask)
        loss = nn.functional.cross_entropy(logits, label)
        return logits, loss, label
        

    def training_step(self, batch, batch_idx):
        logits, loss, label = self.shared_step(batch, batch_idx)
        self.log("train_loss", loss, prog_bar=True, logger=True)
        return {"loss": loss, "predictions": logits, "label": label}

    def validation_step(self, batch, batch_idx):
        logits, loss, label = self.shared_step(batch, batch_idx)
        self.log("val_loss", loss, prog_bar=True, logger=True)
        return loss

    def test_step(self, batch, batch_idx):
        logits, loss, label = self.shared_step(batch, batch_idx)
        self.log("test_loss", loss, prog_bar=True, logger=True)
        return loss

    def configure_optimizers(self):
        optimizer = AdamW(self.parameters(), lr=2e-5)

        scheduler = get_linear_schedule_with_warmup(
          optimizer,
          num_warmup_steps=self.n_warmup_steps,
          num_training_steps=self.n_training_steps
        )

        return dict(
            optimizer=optimizer,
            lr_scheduler=dict(
                scheduler=scheduler,
                interval='step')
        )
```

And the actual training part:
```
checkpoint_file = os.path.join(chekpoint_dir, 'best-checkpoint.ckpt') 

if os.path.isfile(checkpoint_file):
    print('Resuming training from previous checkpoint...')
    trainer.fit(
        sarcasm_tagger, 
        datamodule=data_module,
        ckpt_path=checkpoint_file
    )
else:
    print('Starting training from scratch...')
    trainer.fit(
        sarcasm_tagger, 
        datamodule=data_module
    )
```

The problem doesn't seem to be in the trainer, I think it is related to the model as this error is related to the backpropagation tensors don't having a gradient function set. But as I'm not calling detach anywhere in the code and I'm setting the model to training mode, I'm lost...

Thanks in advance!

Again, I am sorry but I won't have time to debug your kaggle notbook no. 
A minimal reproducing script is supposed to be ~10 lines of code where you show the issue with the model. 
Here is one, showing that the loss is properly back-propagating:
```python 
>>> from transformers import BertForSequenceClassification
>>> model = BertForSequenceClassification.from_pretrained("bert-base-uncased", return_dict=True)
>>> model(torch.ones(2,34).long(), labels = torch.ones(2,2)).loss
tensor(0.8236, grad_fn=<BinaryCrossEntropyWithLogitsBackward0>)
```
In this case, gradients properly flow and the output is the result of the `multi_label_classification` loss computation (BCE). 
I understand your frustration, but I have no idea what is happening behind the `class SarcasmTagger(pl.LightningModule):` inheritance, so I cannot really help you on this part.

If you are using a model + classifier:
```python 
>>> from transformers import BertModel
>>> model = BertModel.from_pretrained("bert-base-uncased", return_dict=True)
>>> pooler_out = model(torch.ones(2,34).long()).pooler_output
tensor([[-0.0807, -0.3833, -0.7920,  ..., -0.9157, -0.5491, -0.0386],
        [-0.0807, -0.3833, -0.7920,  ..., -0.9157, -0.5491, -0.0386]],
       grad_fn=<TanhBackward0>)
>>> classifier = torch.nn.Linear(model.bert.config.hidden_size, 2)
>>> classifier(pooler_out)
tensor([[-0.1094,  0.2125],
        [-0.1094,  0.2125]], grad_fn=<AddmmBackward0>)
```
In these simple snippets, grad function is kept. 

These kind of question should be asked on [the forum](https://discuss.huggingface.co/), as from what I see there is no bug in transformers.
 


Thanks for your help!

I have a similar issue.

With `pytorch-lightning==1.9.4` and `transformers==4.26.1` the code runs fine (and has done with previous versions of both libraries for months/years - yes there have been code changes in that time but the core has been rather stable).

(Also just tested with `transformers==4.29.2` and works fine)

However, when I change nothing in the code and change no other dependencies (so `pytorch-lightning==1.9.4` and all others the same) except to upgrade to `transformers==4.30.2` the code fails with the error message:
```
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
```

The problem is that my codebase is very large and it will take me a while to generate a minimal reproducing script. I will try to put this together, but in the time it takes me to do this, perhaps someone else will have a simpler solution (considering the information I am sharing) and/or a simpler minimal reproducing script.

Perhaps also @lcoandrade you could try your script with `transformers==4.26.1` or `transformers==4.29.2` and see if that works for you?

Thanks a lot for the additional information! If you can isolate which version of transformers made it fail for your, in that case we can look into this as a regression! Would be very helpful if @lcoandrade can do this! 


some more details:

These combinations work:
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.26.1`
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.27.4`
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.28.1`
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.29.2`

These combinations don't:
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.30.0`
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.30.2`
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.31.0`

So the regression must have been introduced in `transformers==4.30.0`?

I'll try to see if I can get a minimal reproducing script together.

Thanks for the help, @Alex-ley-scrub !
I changed my install packages part to:
```
!pip install torch==2.0.0+cu117
!pip install pytorch-lightning==1.9.4
!pip install accelerate==0.21.0
!pip install tokenizers==0.13.3
!pip install transformers==4.26.1
```

But the error was still popping up. So, I thought the error could be related to the optimizer used. My optimizer was this one:
```
def configure_optimizers(self):
        optimizer = AdamW(self.parameters(), lr=2e-5)

        scheduler = get_linear_schedule_with_warmup(
          optimizer,
          num_warmup_steps=self.n_warmup_steps,
          num_training_steps=self.n_training_steps
        )

        return dict(
            optimizer=optimizer,
            lr_scheduler=dict(
                scheduler=scheduler,
                interval='step')
        )
```

When I changed my method to use a simple Adam optimizer:
```
def configure_optimizers(self):
        optimizer = torch.optim.Adam(self.parameters(), lr=2e-5)
        return [optimizer]
```

It worked!

So, the problem is in the AdamW with a scheduler. Reversing the install packages to just:
```
!pip install -q transformers
```

Makes the training work.

As the AdamW is deprecated, I think it is a good idea change the code to use the torch.optim.Adam for instance.

Should this be considered a bug in AdamW?

What do you think, @ArthurZucker and @Alex-ley-scrub?

Thanks again!

Most probably yes! Thanks for investigating, I'm sure this will help others! 

some more details after I swapped this line of code:
```
from transformers import AdamW
```
with this line:
```
from torch.optim import AdamW
```

now all the versions of transformers I tested earlier work on my existing codebase:
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.26.1`
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.27.4`
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.28.1`
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.29.2`
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.30.0`
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.30.2`
- `torch==2.0.0+cu117`, `pytorch-lightning==1.9.4`, `accelerate==0.21.0`, `tokenizers==0.13.3`, `transformers==4.31.0`

therefore, there is pretty strong evidence that something in `transformers.AdamW` in `transformers==4.30.0` caused a regression?

thanks a lot @lcoandrade for that! 🙌  I can now upgrade our transformers dependency to the latest!

Pinging @muellerzr and @pacman100 as they are more familiar than me with the recent changes!

Not entirely sure this is worth looking into too much, given @stas00 point here: https://github.com/huggingface/transformers/pull/23417#issuecomment-1550506298

> This is a very old and deprecated implementation since it doesn't even follow the AdamW algorithm exactly. One should use torch.optim.AdamW instead, which also has a fused version since pt-2.0.0 which is almost as fast as apex's fused AdamW. So really you shouldn't be using this version anyway.

> The only reason it was kept is for BC for those who rely on exact results remaining exact after new transformers versions are released, otherwise we would have just replaced it with torch.optim.AdamW in the first place.

So yes, AdamW is slated for deprecation and you should use `torch.optim.AdamW`. @sgugger do we know when that is going to be? Or should we look into this more.

There wasn't anything explicit in the change to AdamW since v0.29.0, so it'll take some digging to find the exact commit certainly.

If our AdamW is not working properly, all the more reasons to switch the default to the PyTorch one. Users will still be able to switch back if they do not like the change.

Hi all, the default has been changed on main now and will populate on the next release. Install with `pip install git+https://github.com/huggingface/transformers` to use it OOTB!

Same problem here, as suggested, it was resolved with the switch of optimizers

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.