Error when running a wandb sweeps on run_summarization.py
Author: KMFODA
Pull request: no

## Environment info
- `transformers` version: 4.16.0.dev0
- Platform: Linux-5.11.0-37-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.8.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes

### Who can help
Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj

## Information

Model I am using T5-Base or Pegasus

The problem arises when using:
* [x] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Replace the if __name__ == "__main__" function in the run_summarization.py example script with:

```
if __name__ == "__main__":
#     main()
    wandb.login()

    config_defaults = {
        'num_train_epochs': 3,
        'learning_rate': 0.00003,
        'weight_decay': 0.1
        
    }
    
    wandb.init(project="kaizan-sum", entity="kmfoda_kaizan", config=config_defaults)

    
    sweep_config = {
        "name": "lr-epoch-weight-decay-sweep-batch-",
        "method": "bayes",
        "metric": {"name": "bert_rogue", "goal": "maximize"},
        "parameters": {
            "weight_decay": {"min": 0.0, "max": 1.0},
            "num_train_epochs": {"min": 1, "max": 40},
            "learning_rate": {"min": 0.0, "max": 4e-4},
        },
        "early_terminate": {"type": "hyperband", "min_iter": 6,},
    }

    sweep_id = wandb.sweep(sweep_config)

    wandb.agent(sweep_id, function=main)
```

2. Run the following:

```
python3 transformers/examples/pytorch/summarization/run_summarization.py \
    --model_name_or_path t5-base \
    --per_device_train_batch_size 2 \
    --output_dir output_dir \
    --overwrite_output_dir \
    --fp16 \
    --do_train \
    --predict_with_generate \
    --report_to wandb \
    --load_best_model_at_end True \
    --greater_is_better True \
    --evaluation_strategy steps \
    --save_steps 1200 \
    --eval_steps 50 \
    --logging_steps 400 \
    --max_train_samples 100 \
    --max_eval_samples 10 \
    --dataset_name samsum
```

3. After the 1st run finished I get the following error:

```
wandb: ERROR Problem finishing run
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/wandb_run.py", line 1788, in _atexit_cleanup
    self._on_finish()
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/wandb_run.py", line 1936, in _on_finish
    self._console_stop()  # TODO: there's a race here with jupyter console logging
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/wandb_run.py", line 1828, in _console_stop
    self._restore()
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/wandb_run.py", line 1758, in _restore
    self._err_redir.uninstall()
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/lib/redirect.py", line 754, in uninstall
    _WSCH.remove_fd(self._pipe_read_fd)
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/lib/redirect.py", line 667, in remove_fd
    self._unregister()
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/lib/redirect.py", line 655, in _unregister
    signal.signal(signal.SIGWINCH, self._old_handler)
  File "/usr/lib/python3.6/signal.py", line 47, in signal
    handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
ValueError: signal only works in main thread
/usr/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 6 leaked semaphores to clean up at shutdown
  len(cache))]([url](url))
```

## Expected behavior

Wandb sweeps should save the run and kickstart a new run without this Value Error

Comments:
The stack trace only shows errors coming from WandB, not the example, so I'm not sure what you want us to fix.

Hi @sgugger, you're right it doesn't. The reason I raised it with HF rather than WandB is that I can run a sweep on a function with a trainer in it just fine but whenever I try and run a sweep on the main function of run_summarizer.py I get this error.

@KMFODA I'm from W&B, let me check if this way of calling Sweeps is supported, possibly not.

Have you tried just calling sweeps from the `wandb` command line? 
- use the unmodified run_summarization.py script
- Put the location of the script and the sweep config in a yaml file
- set up the sweep config via the wandb cli
- Kick off the training via the wandb cli 

The [sweeps quickstart here](https://docs.wandb.ai/guides/sweeps/quickstart) should get you going hopefully.

Amazing thanks @morganmcg1 calling sweeps from the `wandb` command line fixes this!