RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
Author: AbdulmohsenA
Pull request: no
Labels: name: trainer

name: Quantization

name: bug

### System Info

- `transformers` version: 4.45.0.dev0
- Platform: Linux-5.15.154+-x86_64-with-glibc2.35
- Python version: 3.10.14
- Huggingface_hub version: 0.25.0
- Safetensors version: 0.4.4
- Accelerate version: 0.34.2
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.4.1+cu121 (True)
- Tensorflow version (GPU?): 2.16.1 (True)
- Flax version (CPU?/GPU?/TPU?): 0.8.4 (gpu)
- Jax version: 0.4.26
- JaxLib version: 0.4.26.dev20240620
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: Tesla P100-PCIE-16GB

### Who can help?

@SunMarc 
@muellerz

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

``` python
model_name = "Abdulmohsena/Faseeh_LoRA"

## Quantization
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

model = AutoModelForSeq2SeqLM.from_pretrained(model_name, quantization_config=bnb_config)
tokenizer = AutoTokenizer.from_pretrained(model_name, src_lang="eng_Latn", tgt_lang="arb_Arab")
generation_config = GenerationConfig.from_pretrained(model_name)

model.gradient_checkpointing_enable()
model = prepare_model_for_kbit_training(model) # prepares the whole model for kbit training

for param in model.parameters():
    param.requires_grad = False  # freeze the model - train adapters later
    if param.ndim == 1:
        # cast the small parameters (e.g. layernorm) to fp32 for stability
        param.data = param.data.to(torch.float32)

## Low Rank Adaptation
lora_config = LoraConfig(
    task_type=TaskType.SEQ_2_SEQ_LM,
    inference_mode=False, 
    r=16, 
    lora_alpha=32, 
    lora_dropout=0.1,
    target_modules=["k_proj", "q_proj", "v_proj", "out_proj", "fc1", "fc2"],
    # modules_to_save=['decoder']
)


model.enable_input_require_grads()
model = get_peft_model(model, lora_config)
#trainable params: 8,650,752 || all params: 623,724,544 || trainable%: 1.3870

# Inference works, but training does not:
from transformers import Seq2SeqTrainingArguments, Seq2SeqTrainer
train_batch_size = 4
torch.cuda.empty_cache()

training_args = Seq2SeqTrainingArguments(
    output_dir=f"{model_name}",
    save_total_limit=3,
    load_best_model_at_end=True,

    per_device_train_batch_size=train_batch_size,
    per_device_eval_batch_size=train_batch_size,
    gradient_accumulation_steps=16 // train_batch_size,
    gradient_checkpointing=True,
#     torch_compile=False,
    
    logging_strategy="steps",
    logging_steps=500,
    
    save_strategy='steps',
    
    eval_strategy='steps',
    metric_for_best_model="bertscore-f1",
    greater_is_better=True,
    
    weight_decay=0.01,
    warmup_steps=1_000,
    learning_rate=3e-5,
    lr_scheduler_type="cosine",
    
    remove_unused_columns=True,  # required as the PeftModel forward doesn't have the signature of the wrapped model's forward
    label_names=["labels"],  # same reason as above
    
    num_train_epochs=2,
    
    predict_with_generate=True,
    fp16=True,
    
    push_to_hub=True,
    report_to='wandb'
)

trainer = Seq2SeqTrainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_dataset["train"],
    eval_dataset=tokenized_dataset["test"],
    tokenizer=tokenizer,
    data_collator=data_collator,
    compute_metrics=compute_metrics,
)

model.config.use_cache = False

trainer.train()
```

yields:
```
/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:92: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[17], line 3
      1 from datetime import datetime
      2 # wandb.init(project="Faseeh", name=f"Run @ {datetime.now()}")
----> 3 trainer.train()
      4 # wandb.finish()

File /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:2012, in Trainer.train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   2009 try:
   2010     # Disable progress bars when uploading models during checkpoints to avoid polluting stdout
   2011     hf_hub_utils.disable_progress_bars()
-> 2012     return inner_training_loop(
   2013         args=args,
   2014         resume_from_checkpoint=resume_from_checkpoint,
   2015         trial=trial,
   2016         ignore_keys_for_eval=ignore_keys_for_eval,
   2017     )
   2018 finally:
   2019     hf_hub_utils.enable_progress_bars()

File /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:2357, in Trainer._inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
   2354     self.control = self.callback_handler.on_step_begin(args, self.state, self.control)
   2356 with self.accelerator.accumulate(model):
-> 2357     tr_loss_step = self.training_step(model, inputs)
   2359 if (
   2360     args.logging_nan_inf_filter
   2361     and not is_torch_xla_available()
   2362     and (torch.isnan(tr_loss_step) or torch.isinf(tr_loss_step))
   2363 ):
   2364     # if loss is nan or inf simply add the average of previous logged losses
   2365     tr_loss += tr_loss / (1 + self.state.global_step - self._globalstep_last_logged)

File /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:3487, in Trainer.training_step(***failed resolving arguments***)
   3485         scaled_loss.backward()
   3486 else:
-> 3487     self.accelerator.backward(loss, **kwargs)
   3489 return loss.detach() / self.args.gradient_accumulation_steps

File /opt/conda/lib/python3.10/site-packages/accelerate/accelerator.py:2192, in Accelerator.backward(self, loss, **kwargs)
   2190     return
   2191 elif self.scaler is not None:
-> 2192     self.scaler.scale(loss).backward(**kwargs)
   2193 elif learning_rate is not None and self.has_lomo_optimizer:
   2194     self.lomo_backward(loss, learning_rate)

File /opt/conda/lib/python3.10/site-packages/torch/_tensor.py:521, in Tensor.backward(self, gradient, retain_graph, create_graph, inputs)
    511 if has_torch_function_unary(self):
    512     return handle_torch_function(
    513         Tensor.backward,
    514         (self,),
   (...)
    519         inputs=inputs,
    520     )
--> 521 torch.autograd.backward(
    522     self, gradient, retain_graph, create_graph, inputs=inputs
    523 )

File /opt/conda/lib/python3.10/site-packages/torch/autograd/__init__.py:289, in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)
    284     retain_graph = create_graph
    286 # The reason we repeat the same comment below is that
    287 # some Python versions print out the first line of a multi-line function
    288 # calls in the traceback and some print out the last line
--> 289 _engine_run_backward(
    290     tensors,
    291     grad_tensors_,
    292     retain_graph,
    293     create_graph,
    294     inputs,
    295     allow_unreachable=True,
    296     accumulate_grad=True,
    297 )

File /opt/conda/lib/python3.10/site-packages/torch/autograd/graph.py:769, in _engine_run_backward(t_outputs, *args, **kwargs)
    767     unregister_hooks = _register_logging_hooks_on_whole_graph(t_outputs)
    768 try:
--> 769     return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
    770         t_outputs, *args, **kwargs
    771     )  # Calls into the C++ engine to run the backward pass
    772 finally:
    773     if attach_logging_hooks:

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
```

### Expected behavior

The training to commence successfuly

Comments:
Thanks for the issue @AbdulmohsenA! cc @SunMarc 

@AbdulmohsenA Can you try setting `gradient_checkpointing_kwargs={"use_reentrant":False}` in training args and check if that fixes it ?!

@RUFFY-369 It works! Much appreciated.

Cheers!