HyperParameter search in sagemaker
Author: Marc-Morcos
Pull request: no

- `transformers` version: 4.6.1 (higher is not supported on Sagemaker)
- Platform: Sagemaker

### Who can help
Models:
- gpt2: @patrickvonplaten, @LysandreJik

Library:

- ray/raytune: @richardliaw, @amogkam

## Information

Model I am using GPT2-Medium

The problem arises when using:
I was following this guide https://huggingface.co/docs/sagemaker/train#prepare-a-transformers-fine-tuning-script
But I also wanted to do hyperparameter search https://huggingface.co/blog/ray-tune

I got everything to work on google colab, but on amazon sagemaker, I run into this error when using raytune
```
/opt/conda/lib/python3.6/site-packages/ray/autoscaler/_private/cli_logger.py:61: FutureWarning: Not all Ray CLI dependencies were found. In Ray 1.4+, the Ray CLI, autoscaler, and dashboard will only be usable via `pip install 'ray[default]'`. Please update your install command.
  "update your install command.", FutureWarning)
2021-08-04 21:09:18,214#011INFO services.py:1247 -- View the Ray dashboard at #033[1m#033[32mhttp://127.0.0.1:8265#033[39m#033[22m
Traceback (most recent call last):
  File "train.py", line 148, in <module>
    best = hyperParamSearch_trainer.hyperparameter_search(direction="minimize", hp_space=my_hp_space_ray, n_trials =args.numTrials )
  File "/opt/conda/lib/python3.6/site-packages/transformers/trainer.py", line 1668, in hyperparameter_search
    best_run = run_hp_search(self, n_trials, direction, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/transformers/integrations.py", line 236, in run_hp_search_ray
    **kwargs,
  File "/opt/conda/lib/python3.6/site-packages/ray/tune/tune.py", line 297, in run
    _ray_auto_init()
  File "/opt/conda/lib/python3.6/site-packages/ray/tune/tune.py", line 670, in _ray_auto_init
    ray.init()
  File "/opt/conda/lib/python3.6/site-packages/ray/_private/client_mode_hook.py", line 82, in wrapper
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/ray/worker.py", line 940, in init
    hook()
  File "/opt/conda/lib/python3.6/site-packages/ray/tune/registry.py", line 197, in flush
    self.references[k] = ray.put(v)
  File "/opt/conda/lib/python3.6/site-packages/ray/_private/client_mode_hook.py", line 82, in wrapper
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/ray/worker.py", line 1597, in put
    object_ref = worker.put_object(value)
  File "/opt/conda/lib/python3.6/site-packages/ray/worker.py", line 287, in put_object
    serialized_value = self.get_serialization_context().serialize(value)
  File "/opt/conda/lib/python3.6/site-packages/ray/serialization.py", line 331, in serialize
    return self._serialize_to_msgpack(value)
  File "/opt/conda/lib/python3.6/site-packages/ray/serialization.py", line 311, in _serialize_to_msgpack
    self._serialize_to_pickle5(metadata, python_objects)
  File "/opt/conda/lib/python3.6/site-packages/ray/serialization.py", line 271, in _serialize_to_pickle5
    raise e
  File "/opt/conda/lib/python3.6/site-packages/ray/serialization.py", line 268, in _serialize_to_pickle5
    value, protocol=5, buffer_callback=writer.buffer_callback)
  File "/opt/conda/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 73, in dumps
    cp.dump(obj)
  File "/opt/conda/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 580, in dump
    return Pickler.dump(self, obj)
TypeError: can't pickle _thread.RLock objects

```
I also tried optuna but since there is no option for garabage cleaning, I always run into CUDA out of memory

Comments:
Hey @MarcM0 this has been fixed in the later transformer versions.

Since you can't upgrade the version, can you use this workaround instead: https://github.com/huggingface/transformers/issues/11249#issuecomment-860144744

now I get this error
```


1.4+, the Ray CLI, autoscaler, and dashboard will only be usable via `pip install 'ray[default]'`. Please update your install command.
  "update your install command.", FutureWarning)
2021-08-04 21:42:50,092#011INFO services.py:1247 -- View the Ray dashboard at #033[1m#033[32mhttp://127.0.0.1:8265#033[39m#033[22m
Traceback (most recent call last):
  File "train.py", line 149, in <module>
    best = hyperParamSearch_trainer.hyperparameter_search(direction="minimize", hp_space=my_hp_space_ray, n_trials =args.numTrials )
  File "/opt/conda/lib/python3.6/site-packages/transformers/trainer.py", line 1668, in hyperparameter_search
    best_run = run_hp_search(self, n_trials, direction, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/transformers/integrations.py", line 236, in run_hp_search_ray
    **kwargs,
  File "/opt/conda/lib/python3.6/site-packages/ray/tune/tune.py", line 297, in run
    _ray_auto_init()
  File "/opt/conda/lib/python3.6/site-packages/ray/tune/tune.py", line 670, in _ray_auto_init
    ray.init()
  File "/opt/conda/lib/python3.6/site-packages/ray/_private/client_mode_hook.py", line 82, in wrapper
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/ray/worker.py", line 940, in init
    hook()
  File "/opt/conda/lib/python3.6/site-packages/ray/tune/registry.py", line 197, in flush
    self.references[k] = ray.put(v)
  File "/opt/conda/lib/python3.6/site-packages/ray/_private/client_mode_hook.py", line 82, in wrapper
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/ray/worker.py", line 1597, in put
    object_ref = worker.put_object(value)
  File "/opt/conda/lib/python3.6/site-packages/ray/worker.py", line 287, in put_object
    serialized_value = self.get_serialization_context().serialize(value)
  File "/opt/conda/lib/python3.6/site-packages/ray/serialization.py", line 331, in serialize
    return self._serialize_to_msgpack(value)
  File "/opt/conda/lib/python3.6/site-packages/ray/serialization.py", line 311, in _serialize_to_msgpack
    self._serialize_to_pickle5(metadata, python_objects)
  File "/opt/conda/lib/python3.6/site-packages/ray/serialization.py", line 271, in _serialize_to_pickle5
    raise e
  File "/opt/conda/lib/python3.6/site-packages/ray/serialization.py", line 268, in _serialize_to_pickle5
    value, protocol=5, buffer_callback=writer.buffer_callback)
  File "/opt/conda/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 73, in dumps
    cp.dump(obj)
  File "/opt/conda/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 580, in dump
    return Pickler.dump(self, obj)
  File "pyarrow/io.pxi", line 1021, in pyarrow.lib.Buffer.__reduce_ex__
AttributeError: module 'pickle' has no attribute 'PickleBuffer'
```

what version of ray do you have? how was it installed?

Since I can't directly access the terminal of the computer where the train script is run, I put this line in my train script
`subprocess.check_call([sys.executable, "-m", "pip", "install", "ray[tune]==1.5.1"])`


This seems to be an option as well but I can't find any documentation for how to do it  
https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face
![image](https://user-images.githubusercontent.com/30278842/128376012-1d2ae3e2-25c5-4f45-bbe8-2b19d1573553.png)


@MarcM0 can you also do a `pip install pickle5`. I think that should do the trick.

That worked, thank you!