Repository: huggingface/transformers

Issue:
wandb integration gags during hyperparameter search
## Environment info

- transformers version: 4.6.1
- Platform: Linux-4.19.0-16-cloud-amd64-x86_64-with-glibc2.10
- Python version: 3.8.10
- PyTorch version (GPU?): 1.8.1+cu111 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

wandb version is 0.10.26, but I don't think it matters. 

### Who can help
Maybe @sgugger since this is Trainer-related; I don't know who did the wandb integration specifically. 

## Information

Model I am using: custom Pytorch model. 

The problem arises when using:
* [ ] the official example scripts: (probably, haven't tried)
* [x] my own modified scripts: custom training script using the Trainer

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task:
* [x] my own task or dataset: custom MLM training

## To reproduce

Steps to reproduce the behavior:

1. Train a model using the Trainer with the wandb logging integration and run a hyperparameter search using Optuna (also maybe Ray, but I haven't tried with Ray)
2. After the first run, you'll get an exception like below when wandb tries to log. The issue is that the previous run has finished but a new one hasn't been started. 


```
..... (first trial runs fine; logs to wandb and finishes)

wandb: Synced /home/josh/runs/hps_test: https://wandb.ai/mindful/projectname/runs/2vojg06h                                                                                              
  5%|▌         | 1/19 [00:03<01:02,  3.47s/it][W 2021-05-30 07:41:43,979] Trial 1 failed because of the following error: Error('You must call wandb.init() before wandb.log()')               
Traceback (most recent call last):                                                                                                                                                            
  File "/home/josh/anaconda3/envs/project/lib/python3.8/site-packages/optuna/_optimize.py", line 217, in _run_trial                                                                             
    value_or_values = func(trial)                                                                                                                                                             
  File "/home/josh/anaconda3/envs/project/lib/python3.8/site-packages/transformers/integrations.py", line 138, in _objective                                                                    
    trainer.train(resume_from_checkpoint=checkpoint, trial=trial)                                                                                                                             
  File "/home/josh/anaconda3/envs/project/lib/python3.8/site-packages/transformers/trainer.py", line 1332, in train                                                                             
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch)                                                                                                                               
  File "/home/josh/anaconda3/envs/project/lib/python3.8/site-packages/transformers/trainer.py", line 1405, in _maybe_log_save_evaluate                                                          
    self.log(logs)                                                                                                                                                                            
  File "/home/josh/anaconda3/envs/project/lib/python3.8/site-packages/transformers/trainer.py", line 1692, in log                                                                               
    self.control = self.callback_handler.on_log(self.args, self.state, self.control, logs)                                                                                                    
  File "/home/josh/anaconda3/envs/project/lib/python3.8/site-packages/transformers/trainer_callback.py", line 371, in on_log                                                                    
    return self.call_event("on_log", args, state, control, logs=logs)                                                                                                                         
  File "/home/josh/anaconda3/envs/project/lib/python3.8/site-packages/transformers/trainer_callback.py", line 378, in call_event                                                                
    result = getattr(callback, event)(                                                                                                                                                        
  File "/home/josh/anaconda3/envs/project/lib/python3.8/site-packages/transformers/integrations.py", line 754, in on_log                                                                        
    self._wandb.log({**logs, "train/global_step": state.global_step})                                                                                                                         
  File "/home/josh/anaconda3/envs/project/lib/python3.8/site-packages/wandb/sdk/lib/preinit.py", line 38, in preinit_wrapper                                                                    
    raise wandb.Error("You must call wandb.init() before {}()".format(name))                                                                                                                  
wandb.errors.Error: You must call wandb.init() before wandb.log()                                                                                                                             
wandb: ERROR You must call wandb.init() before wandb.log() 
```

## Expected behavior

wandb should just reinitialize per training run so that each run is logged separately.
Note that as far as I can tell this is a one-line fix (set `_initialized` to `False` in `WandbCallback.on_train_begin` when running an hyperparameter search) so I'll open a PR with that. I just figured there should be an issue as well for clarity.




Discussion:

Resolving patch:
diff --git a/src/transformers/integrations.py b/src/transformers/integrations.py
--- a/src/transformers/integrations.py
+++ b/src/transformers/integrations.py
@@ -713,6 +713,7 @@ def on_train_begin(self, args, state, control, model=None, **kwargs):
         hp_search = state.is_hyper_param_search
         if hp_search:
             self._wandb.finish()
+            self._initialized = False
         if not self._initialized:
             self.setup(args, state, model, **kwargs)
 


Test patch: