"Cannot re-initialize CUDA in forked subprocess." error when running "transformers/notebooks/05-benchmark.ipynb" notebook
Author: ClaartjeBarkhof
Pull request: no
Labels: name: wontfix

## Environment info 
I am getting this error on a server, but also on Collab, so giving the Collab specs:

- `transformers` version: 3.3.1
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (True)
- Tensorflow version (GPU?): 2.3.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No?

And on my server:
- `transformers` version: 3.3.1
- Platform: Linux-4.19.0-11-amd64-x86_64-with-debian-10.6
- Python version: 3.6.12
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No?


### Who can help
 @patrickvonplaten

## Information

Model I am using (Bert, XLNet ...): Any model.

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. [Run the colab on benchmarking provided on the transformers GitHub](https://github.com/huggingface/transformers/blob/master/notebooks/05-benchmark.ipynb).

```
2020-10-15 14:20:38.078717: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
1 / 5
Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
Traceback (most recent call last):
  File "run_benchmark.py", line 47, in <module>
    main()
  File "run_benchmark.py", line 43, in main
    benchmark.run()
  File "/usr/local/lib/python3.6/dist-packages/transformers/benchmark/benchmark_utils.py", line 674, in run
    memory, inference_summary = self.inference_memory(model_name, batch_size, sequence_length)
ValueError: too many values to unpack (expected 2)
```

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
Might be of interest to @patrickvonplaten 

Facing same issue here.

Try this.
You can check other parameters for PyTorchBenchmarkArguments at [here](https://github.com/huggingface/transformers/blob/acf56408d81fdc04a32af139a45ae7b76e0c5b0d/src/transformers/benchmark/benchmark_args_utils.py#L34-L123).

```python
# main.py
def main():
    from transformers import PyTorchBenchmark, PyTorchBenchmarkArguments

    args = PyTorchBenchmarkArguments(models=["bert-base-uncased"],
                                     batch_sizes=[8],
                                     sequence_lengths=[8, 32, 128, 512],
                                     multi_process=False)
    print(args.do_multi_processing)
    benchmark = PyTorchBenchmark(args)
    results = benchmark.run()
    print(results)


if __name__ == '__main__':
    main()
```
```
CUDA_VISIBLE_DEVICES=0 python main.py
```

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.