Allow resuming of logging to WANDB with the Trainer
Author: BramVanroy
Pull request: no

### Feature request

As far as I can tell, it is currently not possible to resume a training run and continue logging to the same run on WANDB when using the Trainer. The reason is that WANDB would [require](https://github.com/wandb/wandb/blob/a32638cb1c6ab775e9ed431d9a9b4b8a30685453/wandb/sdk/wandb_init.py#L1036-L1053) you to set `resume=True` and a run `id` in `wandb.init` (or env `WANDB_RUN_ID`) for this to work.

The Trainer currently does not allow for these options as far as I can see

https://github.com/huggingface/transformers/blob/c9a82be592ca305180a7ab6a36e884bca1d426b8/src/transformers/integrations.py#L726-L737
### 

### Motivation

Make it easy to resume logging to wandb without any code changes, directly through a config or CLI (TrainingArguments).

### Your contribution

I can work on this. I would update TrainingArguments to add two new arguments:

- `wandb_resume`
- `wandb_id` (which backs off to the environment variable `WANDB_RUN_ID`)

which would then be passed to the `wandb` integration Callback as part of `args`

https://github.com/huggingface/transformers/blob/c9a82be592ca305180a7ab6a36e884bca1d426b8/src/transformers/integrations.py#L684

which can then be used in `init`

https://github.com/huggingface/transformers/blob/c9a82be592ca305180a7ab6a36e884bca1d426b8/src/transformers/integrations.py#L725-L737


Comments:
cc @parambharat who has done a lot of work on the Wandb integration

Hi @BramVanroy , Thanks for bringing this up. If I understood the issue correctly, you want to resume a wandb run while resuming training. There are two solutions I can think of that don't require changing `TrainingArguments`. Can you try these instead ?


1. Set the env vars `WANDB_RESUME="must"` and `WANDB_RUN_ID=<run_id_you_want_to_resume>` before running your training script.  wandb should read these env vars upon initialization and should resume your training run. [W&B Env Vars Reference](https://docs.wandb.ai/guides/track/environment-variables)
2. Initialize a `run` with `wandb.init(resume=must, id=<run_id_you_want_to_resume>` before initializing the `Trainer`. The trainer will [not initialize a new run](https://github.com/huggingface/transformers/blob/8f1f0bf50f402881c0aa53b18f21736a151adf5b/src/transformers/integrations.py#L733C16-L733C38) if a run already exists. Here's a [colab](https://colab.research.google.com/github/wandb/examples/blob/master/colabs/huggingface/wandb_hf_example.ipynb#scrollTo=RK5HRy1JQ0yX) example of this from the wandb examples repo.

@parambharat  Oh, I was not aware of the resume environment variable! That would make life indeed much easier in combination with the wandb run ID!

A run's ID is the alphanumeric part of the URL, right? So in this example

> https://wandb.ai/username/projectname/runs/phup4zp1/

it would be `phup4zp1`?

If it's that easy then I am a happy man! 

@BramVanroy , Yes, that's the `run_id` 👍 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Ah, sorry forgot to come back to this. This works indeed, thanks a lot @parambharat!