RuntimeError: unscale_() has already been called on this optimizer since the last update().
Author: kafkasl
Pull request: no

### System Info

- `transformers` version: 4.30.0.dev0
- Platform: Linux-5.4.0-122-generic-x86_64-with-glibc2.31
- Python version: 3.9.16
- Huggingface_hub version: 0.14.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): 2.9.2 (True)
- Flax version (CPU?/GPU?/TPU?): 0.6.3 (gpu)
- Jax version: 0.4.1
- JaxLib version: 0.4.1
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: not sure, see colab below


### Who can help?

@younesbelkada @pacman100 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Run the colab: https://colab.research.google.com/drive/1ARmlaZZaKyAg6HTi57psFLPeh0hDRcPX?usp=sharing#scrollTo=Duak7T_B3VpJ

@younesbelkada @pacman100  I have reinstalled from source as suggested after the fix in (https://github.com/huggingface/transformers/pull/23914/files) but I still get the error. I'm in the latest commit `transformers @ git+https://github.com/huggingface/transformers.git@fabe17a726bbf6081cfbcc975d8ac451a81f3e2d` and you can tell from the stacktrace that the line numbers are different (due to the changes to fix the problem when using QLora). The script does not use QLora (afaik)


am I missing something? 

### Expected behavior

The call `trainer.train()` should work and instead produces the following exception:
```╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <cell line: 17>:17                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:1661 in train                    │
│                                                                                                  │
│   1658 │   │   inner_training_loop = find_executable_batch_size(                                 │
│   1659 │   │   │   self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size  │
│   1660 │   │   )                                                                                 │
│ ❱ 1661 │   │   return inner_training_loop(                                                       │
│   1662 │   │   │   args=args,                                                                    │
│   1663 │   │   │   resume_from_checkpoint=resume_from_checkpoint,                                │
│   1664 │   │   │   trial=trial,                                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:1995 in _inner_training_loop     │
│                                                                                                  │
│   1992 │   │   │   │   │   │   │   │   args.max_grad_norm,                                       │
│   1993 │   │   │   │   │   │   │   )                                                             │
│   1994 │   │   │   │   │   │   else:                                                             │
│ ❱ 1995 │   │   │   │   │   │   │   self.accelerator.clip_grad_norm_(                             │
│   1996 │   │   │   │   │   │   │   │   model.parameters(),                                       │
│   1997 │   │   │   │   │   │   │   │   args.max_grad_norm,                                       │
│   1998 │   │   │   │   │   │   │   )                                                             │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py:1817 in clip_grad_norm_        │
│                                                                                                  │
│   1814 │   │   │   # `accelerator.backward(loss)` is doing that automatically. Therefore, its i  │
│   1815 │   │   │   # We cannot return the gradient norm because DeepSpeed does it.               │
│   1816 │   │   │   return None                                                                   │
│ ❱ 1817 │   │   self.unscale_gradients()                                                          │
│   1818 │   │   return torch.nn.utils.clip_grad_norm_(parameters, max_norm, norm_type=norm_type)  │
│   1819 │                                                                                         │
│   1820 │   def clip_grad_value_(self, parameters, clip_value):                                   │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py:1780 in unscale_gradients      │
│                                                                                                  │
│   1777 │   │   │   for opt in optimizer:                                                         │
│   1778 │   │   │   │   while isinstance(opt, AcceleratedOptimizer):                              │
│   1779 │   │   │   │   │   opt = opt.optimizer                                                   │
│ ❱ 1780 │   │   │   │   self.scaler.unscale_(opt)                                                 │
│   1781 │                                                                                         │
│   1782 │   def clip_grad_norm_(self, parameters, max_norm, norm_type=2):                         │
│   1783 │   │   """                                                                               │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/torch/cuda/amp/grad_scaler.py:275 in unscale_            │
│                                                                                                  │
│   272 │   │   optimizer_state = self._per_optimizer_states[id(optimizer)]                        │
│   273 │   │                                                                                      │
│   274 │   │   if optimizer_state["stage"] is OptState.UNSCALED:                                  │
│ ❱ 275 │   │   │   raise RuntimeError("unscale_() has already been called on this optimizer sin   │
│   276 │   │   elif optimizer_state["stage"] is OptState.STEPPED:                                 │
│   277 │   │   │   raise RuntimeError("unscale_() is being called after step().")                 │
│   278                                                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: unscale_() has already been called on this optimizer since the last update().
```

Comments:
Hi @kafkasl 
I can confirm the training works if I install `accelerate` from source using your notebook.

![Screenshot 2023-06-01 at 14 47 20](https://github.com/huggingface/transformers/assets/49240599/55f8ecbd-5761-43b9-895f-a73849fd16a6)

Can you add:
```bash
!pip install -q git+https://github.com/huggingface/peft.git git+https://github.com/huggingface/transformers.git git+https://github.com/huggingface/accelerate.git
```
On the setup block and re-run it again?

@younesbelkada  I seem to still have  the issue, the accelerate version used:
```
!pip freeze | grep accelerate
accelerate @ git+https://github.com/huggingface/accelerate.git@4d583ad6a1f13d1d7617e6a37f791ec01a68413a
```

how did you make it work? I just have this setup but it still fails:
```
!pip install -q bitsandbytes datasets loralib
# !pip install -q git+https://github.com/huggingface/peft.git git+https://github.com/huggingface/transformers.git
!pip install git+https://github.com/huggingface/peft.git git+https://github.com/huggingface/transformers.git git+https://github.com/huggingface/accelerate.git
```

what am I missing? 

Had the same issue yesterday. Kept getting the error even though I updated the transformer library. I believe what helped was to `pip uninstall transformers`, restart the kernel, and then `pip install -U git+https://github.com/huggingface/transformers@de9255de27abfcae4a1f816b904915f0b1e23cd9`.

@kafkasl 
I just re-ran the notebook again in a fresh colab instance and it works on my end

![Screenshot 2023-06-01 at 16 07 15](https://github.com/huggingface/transformers/assets/49240599/808d4d3d-5aa7-4410-afb4-2b734c7d66f8)

Here is a copy of the notebook: https://colab.research.google.com/drive/1VmNo77ub8IVe-LGSZtjdjEzWY45aN0Tm?usp=sharing

I confirm it works although I have no idea why because the code looks the same, and I've used the same env twice. Anyway thanks for the help!

@younesbelkada I actually still receive this error when running with 4bit quantization. Using the same installations and running this notebook I run into the same error. Note that without qlora I can run just fine (ie. the notebook linked here does work but [this notebook](https://huggingface.co/dfurman/falcon-40b-chat-oasst1/blob/main/finetune_falcon40b_oasst1_with_bnb_peft.ipynb) does not).

Any idea why this could be?

Output of `pip freeze` is below for relevant libraries
```bash
accelerate @ git+https://github.com/huggingface/accelerate.git@eba6eb79dc2ab652cd8b44b37165a4852768a8ac
bitsandbytes==0.39.0
einops==0.6.1
loralib==0.1.1
peft @ git+https://github.com/huggingface/peft.git@7fb5f90a38cb39a31396de7e638ead9ecea692af
transformers @ git+https://github.com/huggingface/transformers.git@460b844360131c99d3dd4dbd9c08545ea2e6ac9e
```

Hi @sam-h-bean 
Hmm, looking at the logs it seems that something is wrong with the tensorboard logging:
```bash
─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <cell line: 23>:23                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:1696 in train                    │
│                                                                                                  │
│   1693 │   │   inner_training_loop = find_executable_batch_size(                                 │
│   1694 │   │   │   self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size  │
│   1695 │   │   )                                                                                 │
│ ❱ 1696 │   │   return inner_training_loop(                                                       │
│   1697 │   │   │   args=args,                                                                    │
│   1698 │   │   │   resume_from_checkpoint=resume_from_checkpoint,                                │
│   1699 │   │   │   trial=trial,                                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/accelerate/utils/memory.py:132 in decorator              │
│                                                                                                  │
│   129 │   │   │   if batch_size == 0:                                                            │
│   130 │   │   │   │   raise RuntimeError("No executable batch size found, reached zero.")        │
│   131 │   │   │   try:                                                                           │
│ ❱ 132 │   │   │   │   return function(batch_size, *args, **kwargs)                               │
│   133 │   │   │   except Exception as e:                                                         │
│   134 │   │   │   │   if should_reduce_batch_size(e):                                            │
│   135 │   │   │   │   │   gc.collect()                                                           │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:2052 in _inner_training_loop     │
│                                                                                                  │
│   2049 │   │   │   │   │   self.state.epoch = epoch + (step + 1 + steps_skipped) / steps_in_epo  │
│   2050 │   │   │   │   │   self.control = self.callback_handler.on_step_end(args, self.state, s  │
│   2051 │   │   │   │   │                                                                         │
│ ❱ 2052 │   │   │   │   │   self._maybe_log_save_evaluate(tr_loss, model, trial, epoch, ignore_k  │
│   2053 │   │   │   │   else:                                                                     │
│   2054 │   │   │   │   │   self.control = self.callback_handler.on_substep_end(args, self.state  │
│   2055                                                                                           │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:2338 in _maybe_log_save_evaluate │
│                                                                                                  │
│   2335 │   │   │   self._globalstep_last_logged = self.state.global_step                         │
│   2336 │   │   │   self.store_flos()                                                             │
│   2337 │   │   │                                                                                 │
│ ❱ 2338 │   │   │   self.log(logs)                                                                │
│   2339 │   │                                                                                     │
│   2340 │   │   metrics = None                                                                    │
│   2341 │   │   if self.control.should_evaluate:                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:2700 in log                      │
│                                                                                                  │
│   2697 │   │                                                                                     │
│   2698 │   │   output = {**logs, **{"step": self.state.global_step}}                             │
│   2699 │   │   self.state.log_history.append(output)                                             │
│ ❱ 2700 │   │   self.control = self.callback_handler.on_log(self.args, self.state, self.control,  │
│   2701 │                                                                                         │
│   2702 │   def _prepare_input(self, data: Union[torch.Tensor, Any]) -> Union[torch.Tensor, Any]  │
│   2703 │   │   """                                                                               │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer_callback.py:390 in on_log           │
│                                                                                                  │
│   387 │                                                                                          │
│   388 │   def on_log(self, args: TrainingArguments, state: TrainerState, control: TrainerContr   │
│   389 │   │   control.should_log = False                                                         │
│ ❱ 390 │   │   return self.call_event("on_log", args, state, control, logs=logs)                  │
│   391 │                                                                                          │
│   392 │   def on_prediction_step(self, args: TrainingArguments, state: TrainerState, control:    │
│   393 │   │   return self.call_event("on_prediction_step", args, state, control)                 │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer_callback.py:397 in call_event       │
│                                                                                                  │
│   394 │                                                                                          │
│   395 │   def call_event(self, event, args, state, control, **kwargs):                           │
│   396 │   │   for callback in self.callbacks:                                                    │
│ ❱ 397 │   │   │   result = getattr(callback, event)(                                             │
│   398 │   │   │   │   args,                                                                      │
│   399 │   │   │   │   state,                                                                     │
│   400 │   │   │   │   control,                                                                   │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/integrations.py:655 in on_log               │
│                                                                                                  │
│    652 │   │   │   │   │   │   "This invocation of Tensorboard's writer.add_scalar() "           │
│    653 │   │   │   │   │   │   "is incorrect so we dropped this attribute."                      │
│    654 │   │   │   │   │   )                                                                     │
│ ❱  655 │   │   │   self.tb_writer.flush()                                                        │
│    656 │                                                                                         │
│    657 │   def on_train_end(self, args, state, control, **kwargs):                               │
│    658 │   │   if self.tb_writer:                                                                │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/torch/utils/tensorboard/writer.py:1200 in flush          │
│                                                                                                  │
│   1197 │   │   if self.all_writers is None:                                                      │
│   1198 │   │   │   return                                                                        │
│   1199 │   │   for writer in self.all_writers.values():                                          │
│ ❱ 1200 │   │   │   writer.flush()                                                                │
│   1201 │                                                                                         │
│   1202 │   def close(self):                                                                      │
│   1203 │   │   if self.all_writers is None:                                                      │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/torch/utils/tensorboard/writer.py:150 in flush           │
│                                                                                                  │
│    147 │   │   Call this method to make sure that all pending events have been written to        │
│    148 │   │   disk.                                                                             │
│    149 │   │   """                                                                               │
│ ❱  150 │   │   self.event_writer.flush()                                                         │
│    151 │                                                                                         │
│    152 │   def close(self):                                                                      │
│    153 │   │   """Flushes the event file to disk and close the file.                             │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/tensorboard/summary/writer/event_file_writer.py:125 in   │
│ flush                                                                                            │
│                                                                                                  │
│   122 │   │   Call this method to make sure that all pending events have been                    │
│   123 │   │   written to disk.                                                                   │
│   124 │   │   """                                                                                │
│ ❱ 125 │   │   self._async_writer.flush()                                                         │
│   126 │                                                                                          │
│   127 │   def close(self):                                                                       │
│   128 │   │   """Performs a final flush of the event file to disk, stops the                     │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/tensorboard/summary/writer/event_file_writer.py:190 in   │
│ flush                                                                                            │
│                                                                                                  │
│   187 │   │   │   if self._closed:                                                               │
│   188 │   │   │   │   raise IOError("Writer is closed")                                          │
│   189 │   │   │   self._byte_queue.join()                                                        │
│ ❱ 190 │   │   │   self._writer.flush()                                                           │
│   191 │   │   │   # Check the status again in case the background worker thread has              │
│   192 │   │   │   # failed in the meantime to avoid waiting until the next call to               │
│   193 │   │   │   # surface the error.                                                           │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/tensorboard/summary/writer/record_writer.py:43 in flush  │
│                                                                                                  │
│   40 │   │   self._writer.write(header + header_crc + data + footer_crc)                         │
│   41 │                                                                                           │
│   42 │   def flush(self):                                                                        │
│ ❱ 43 │   │   self._writer.flush()                                                                │
│   44 │                                                                                           │
│   45 │   def close(self):                                                                        │
│   46 │   │   self._writer.close()                                                                │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/tensorflow/python/lib/io/file_io.py:221 in flush         │
│                                                                                                  │
│    218 │   data would survive an application crash but not necessarily an OS crash.              │
│    219 │   """                                                                                   │
│    220 │   if self._writable_file:                                                               │
│ ❱  221 │     self._writable_file.flush()                                                         │
│    222                                                                                           │
│    223   def close(self):                                                                        │
│    224 │   r"""Closes the file.                                                                  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
FailedPreconditionError: /content/drive/MyDrive/Colab 
Files/WM/falcon-chat-40b/runs/May30_05-06-25_3e61552a1fc1/events.out.tfevents.1685423187.3e61552a1fc1.231.0; 
Transport endpoint is not connected
```
I would probably try to use something else than tensorboard or change the log_steps in the trainign arguments, what do you think?

In my case I hit this due to dataset being too small. See how I can consistently reproduce when dataset has 2 entries but issue goes away once I have 5 entries: https://colab.research.google.com/drive/1jc1hab4pJBWHJNKeuScMDqMyPv0QqRro?usp=sharing

> `pip install -U git+https://github.com/huggingface/transformers@de9255de27abfcae4a1f816b904915f0b1e23cd9`.

Thanks. This worked for me.

I am still able to reproduce this double `unscale_()` issue with the original stack trace.

Using [Falcon-Guanaco.ipynb](https://colab.research.google.com/drive/1BiQiw31DT7-cDp1-0ySXvvhzqomTdI-o?usp=sharing#scrollTo=mNnkgBq7Q3EU) and making the following modifications:

1) Add `dataset = dataset.shard(num_shards=80, index=0)` before constructing `SFTTrainer`.
2) Change `max_seq_length = 512` to `max_seq_length = 1024`.

After these modifications the `trainer.train()` call reliably fails.

Debugging I see the following steps happen before the error:

1) The `step()` call in `accelerate/optimizer.py` returns immediately because of the [self.gradient_state.sync_gradients condition](https://github.com/huggingface/accelerate/blob/543c59af224e3ea273633732319916b0698234ab/src/accelerate/optimizer.py#L128). As a result, the `optimizer_state["stage"]` is never transitioned to `OptState.READY`.
2) `optimizer_was_run` in the call from `transformers/trainer.py` is (incorrectly?) set to `True` [here](https://github.com/huggingface/transformers/blob/70c79940957fb25b54bd1b106935c756b90345eb/src/transformers/trainer.py#L1881).
3) On the next iteration, the double `unscale_()` error is raised since we call `clip_grad_norm_` every time.

Edit: [Fix on my fork](https://github.com/huggingface/accelerate/commit/644038e3859f7ada492bb0053fd360d87c4b4d0a) is working.

@PhilDakin how to pip install your fork ?

Thanks

Edit : what works now is using 
```
!pip install git+https://github.com/huggingface/transformers@de9255de27abfcae4a1f816b904915f0b1e23cd9
```
(colab)

!pip uninstall transformers
restart kernel
!pip install git+https://github.com/huggingface/transformers@de9255de27abfcae4a1f816b904915f0b1e23cd9

Solves the error

> !pip uninstall transformers restart kernel !pip install git+https://github.com/huggingface/transformers@de9255de27abfcae4a1f816b904915f0b1e23cd9
> 
> Solves the error

do we need any other thing?

Hi everyone,
I just re-ran the notebook again and it seems to work fine (c.f. screenshot below)
![Screenshot 2023-06-13 at 15 17 59](https://github.com/huggingface/transformers/assets/49240599/4b231900-8a4d-4d8f-a19a-c1c46c393134)
![Screenshot 2023-06-13 at 15 32 32](https://github.com/huggingface/transformers/assets/49240599/1afeffdc-887a-41ce-8cf0-c8db7b6d11e3)
Make sure to use a fresh new environment when running the experiments 

@younesbelkada no need to use previous transformer build ?

Hi @x4080 
I just tried the notebook with the latest build of transformers, accelerate and trl and everything seems to work fine. I have updated the notebook accordingly: https://colab.research.google.com/drive/1BiQiw31DT7-cDp1-0ySXvvhzqomTdI-o?usp=sharing 
Make sure to use a fresh environment with these libraries installed

Hey I still have this issue when using huggingface trainer instead of trl. And its fixed by ```git+https://github.com/huggingface/transformers@de9255de27abfcae4a1f816b904915f0b1e23cd9``` however as a side effect there is no adapter_config.json saved.

https://drive.google.com/file/d/1jWKCRWwM1R0d00yFI8HUsakhQrbSeLFu/view?usp=share_link

Update:

fixed the adapter_config saving issue by

```
from transformers import TrainerCallback
class PeftSavingCallback(TrainerCallback):
    def on_save(self, args, state, control, **kwargs):
        checkpoint_path = os.path.join(args.output_dir, f"checkpoint-{state.global_step}")
        kwargs["model"].save_pretrained(checkpoint_path)

        if "pytorch_model.bin" in os.listdir(checkpoint_path):
            os.remove(os.path.join(checkpoint_path, "pytorch_model.bin"))
```

@younesbelkada I can confirm that using regular import works

Thanks for all

@younesbelkada while it is the case that the notebook **with no modifications** runs correctly, the issue I've [described above persists](https://github.com/huggingface/transformers/issues/23935#issuecomment-1588134562).

I'm maintaining a fix on my fork but don't have the contextual knowledge to confidently PR.

Would you please take another look?

(This is using latest of `accelerate` `trl` `transformers` installed via `git`)

I tried with different jsonl data, and the unscale_() error appears again -> its kinda related to data then, but I dont know what - anybody has similar experience ?

Edit : I tried to add 2 more data and it works, but I recalled I add 1 data for the working data, and it error, so its kinda fuzzy right now

Here's error because of adding 1 more row of training data, the data is in the same data format - so no problem with new data either :
```
/tmp/ipykernel_29/1990004115.py:27 in <module>                                                   │
│                                                                                                  │
│ [Errno 2] No such file or directory: '/tmp/ipykernel_29/1990004115.py'                           │
│                                                                                                  │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:1539 in train                    │
│                                                                                                  │
│   1536 │   │   inner_training_loop = find_executable_batch_size(                                 │
│   1537 │   │   │   self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size  │
│   1538 │   │   )                                                                                 │
│ ❱ 1539 │   │   return inner_training_loop(                                                       │
│   1540 │   │   │   args=args,                                                                    │
│   1541 │   │   │   resume_from_checkpoint=resume_from_checkpoint,                                │
│   1542 │   │   │   trial=trial,                                                                  │
│                                                                                                  │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:1850 in _inner_training_loop     │
│                                                                                                  │
│   1847 │   │   │   │   │   │   │   │   args.max_grad_norm,                                       │
│   1848 │   │   │   │   │   │   │   )                                                             │
│   1849 │   │   │   │   │   │   else:                                                             │
│ ❱ 1850 │   │   │   │   │   │   │   self.accelerator.clip_grad_norm_(                             │
│   1851 │   │   │   │   │   │   │   │   model.parameters(),                                       │
│   1852 │   │   │   │   │   │   │   │   args.max_grad_norm,                                       │
│   1853 │   │   │   │   │   │   │   )                                                             │
│                                                                                                  │
│ /opt/conda/lib/python3.10/site-packages/accelerate/accelerator.py:1913 in clip_grad_norm_        │
│                                                                                                  │
│   1910 │   │   │   # `accelerator.backward(loss)` is doing that automatically. Therefore, its i  │
│   1911 │   │   │   # We cannot return the gradient norm because DeepSpeed does it.               │
│   1912 │   │   │   return None                                                                   │
│ ❱ 1913 │   │   self.unscale_gradients()                                                          │
│   1914 │   │   return torch.nn.utils.clip_grad_norm_(parameters, max_norm, norm_type=norm_type)  │
│   1915 │                                                                                         │
│   1916 │   def clip_grad_value_(self, parameters, clip_value):                                   │
│                                                                                                  │
│ /opt/conda/lib/python3.10/site-packages/accelerate/accelerator.py:1876 in unscale_gradients      │
│                                                                                                  │
│   1873 │   │   │   for opt in optimizer:                                                         │
│   1874 │   │   │   │   while isinstance(opt, AcceleratedOptimizer):                              │
│   1875 │   │   │   │   │   opt = opt.optimizer                                                   │
│ ❱ 1876 │   │   │   │   self.scaler.unscale_(opt)                                                 │
│   1877 │                                                                                         │
│   1878 │   def clip_grad_norm_(self, parameters, max_norm, norm_type=2):                         │
│   1879 │   │   """                                                                               │
│                                                                                                  │
│ /opt/conda/lib/python3.10/site-packages/torch/cuda/amp/grad_scaler.py:275 in unscale_            │
│                                                                                                  │
│   272 │   │   optimizer_state = self._per_optimizer_states[id(optimizer)]                        │
│   273 │   │                                                                                      │
│   274 │   │   if optimizer_state["stage"] is OptState.UNSCALED:                                  │
│ ❱ 275 │   │   │   raise RuntimeError("unscale_() has already been called on this optimizer sin   │
│   276 │   │   elif optimizer_state["stage"] is OptState.STEPPED:                                 │
│   277 │   │   │   raise RuntimeError("unscale_() is being called after step().")    
```

Is there some kind of do and donts with the quantity of data ? should be even or odd ? But I dont think that makes a difference since its like all is random

Edit : As expected, I remove that data and training works again
Edit2 : I tried to copy the last data (that works) and training it again -> not working again, maybe there's some kind of how many data needed makes a difference ?

I'm also using jsonl and hitting this issue. I could consistently reproduce it when my dataset had 2 lines instead of 5 lines. I also could reproduce this issue with larger dataset when num_train_epochs was set to 3 vs 1.

@samos123 Thanks for confirmation, I thought I was crazy to watch that sometimes error and sometime its not 

😄 

Another thing I noticed is that if I use num_train_epochs=1 that it works fine but the steps is way less than my dataset size. Dataset size is 282 however steps is ~80

> 



> I am still able to reproduce this double `unscale_()` issue with the original stack trace.
> 
> Using [Falcon-Guanaco.ipynb](https://colab.research.google.com/drive/1BiQiw31DT7-cDp1-0ySXvvhzqomTdI-o?usp=sharing#scrollTo=mNnkgBq7Q3EU) and making the following modifications:
> 
> 1. Add `dataset = dataset.shard(num_shards=80, index=0)` before constructing `SFTTrainer`.
> 2. Change `max_seq_length = 512` to `max_seq_length = 1024`.
> 
> After these modifications the `trainer.train()` call reliably fails.
> 
> Debugging I see the following steps happen before the error:
> 
> 1. The `step()` call in `accelerate/optimizer.py` returns immediately because of the [self.gradient_state.sync_gradients condition](https://github.com/huggingface/accelerate/blob/543c59af224e3ea273633732319916b0698234ab/src/accelerate/optimizer.py#L128). As a result, the `optimizer_state["stage"]` is never transitioned to `OptState.READY`.
> 2. `optimizer_was_run` in the call from `transformers/trainer.py` is (incorrectly?) set to `True` [here](https://github.com/huggingface/transformers/blob/70c79940957fb25b54bd1b106935c756b90345eb/src/transformers/trainer.py#L1881).
> 3. On the next iteration, the double `unscale_()` error is raised since we call `clip_grad_norm_` every time.
> 
> Edit: [Fix on my fork](https://github.com/huggingface/accelerate/commit/644038e3859f7ada492bb0053fd360d87c4b4d0a) is working.


Hello, able to reproduce this. cc @muellerzr

Reason:
Gradient Accumulation in trainer is happening across the epochs because of `total_batched_samples `. However, Accelerate resets step at the end of an epoch leading to `sync_gradients` being `False` and optimizer not being run and when the next time `clip_grad_norm_ ` is called, it leads to `unscale_() has already been called on this optimizer since the last update().`
```
def _do_sync(self):
        "Sets the right `sync_gradients` context and either resets or increases `self.step`"
        if self.gradient_state.end_of_dataloader:
            self.step = 0
            self.gradient_state._set_sync_gradients(True)
        else:
            self.step += 1
            self.gradient_state._set_sync_gradients((self.step % self.gradient_state.num_steps) == 0)
```

Adapter size is 443 bytes, so it does not work.

But this error has disappeared when I installed recommended - https://colab.research.google.com/drive/1BiQiw31DT7-cDp1-0ySXvvhzqomTdI-o?usp=sharing#scrollTo=i-tTvEF1RT3y

Still, have no idea why adapter is not saved.





Hello everyone, the above PR #24415 should resolve the issues with grad_acc around epoch boundaries.

> Had the same issue yesterday. Kept getting the error even though I updated the transformer library. I believe what helped was to `pip uninstall transformers`, restart the kernel, and then `pip install -U git+https://github.com/huggingface/transformers@de9255de27abfcae4a1f816b904915f0b1e23cd9`.

@leoplusx thanks,  this solution worked for me

> Had the same issue yesterday. Kept getting the error even though I updated the transformer library. I believe what helped was to `pip uninstall transformers`, restart the kernel, and then `pip install -U git+https://github.com/huggingface/transformers@de9255de27abfcae4a1f816b904915f0b1e23cd9`.

worked for me, thanks!

I changed to 4.29.0 and it's all good. Previously using 4.31.0

```bash
pip install transformers==4.29.0
```

```bash
Name: transformers
Version: 4.29.0
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: /root/anaconda3/envs/hw/lib/python3.11/site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, tokenizers, tqdm
Required-by: peft, sentence-transformers, trl
```


This is a crazy problem. Huggingface libraries on Kaggle platform have constant issues. Also the default wandb enablement on Kaggle is also very inconsistent and is headache particularly in code submission competitions, given internet is off. Many issues pop up on some runs and won't pop up other times. While I definitely have learnt something new through these experiments, I feel I waste an enormous amount of time trying to find such technical glitches.  I rather would like to spend my time on real DS tasks than on troubleshooting and finding fixes. Extremely frustrating.

I have this error when trying to use gradient_accumulation_steps > 1 in Accelerator.

> I have this error when trying to use gradient_accumulation_steps > 1 in Accelerator.

I got this when applying clip_grad_norm_ with gradient_accumulation_steps > 1 in Accelerator. Seems like clip_grad_norm_ is called multiple times without updating the optimizer. I could resolve this with:
`if accelerator.sync_gradients:
                accelerator.clip_grad_norm_(model.parameters(), max_norm=1.0)`

> > I have this error when trying to use gradient_accumulation_steps > 1 in Accelerator.
> 
> I got this when applying clip_grad_norm_ with gradient_accumulation_steps > 1 in Accelerator. Seems like clip_grad_norm_ is called multiple times without updating the optimizer. I could resolve this with: `if accelerator.sync_gradients: accelerator.clip_grad_norm_(model.parameters(), max_norm=1.0)`

This works for me. Thanks @NisKarki