connection issue while downloading data
State: closed | Author: rabeehkarimimahabadi (NONE)

Hi
I am running my codes on google cloud, and I am getting this error resulting in the failure of the codes when trying to download the data, could you assist me to solve this? also as a temporary solution, could you tell me how I can increase the number of retries and timeout to at least let the models run for now. thanks 

```
Traceback (most recent call last):
  File "finetune_t5_trainer.py", line 361, in <module>
    main()
  File "finetune_t5_trainer.py", line 269, in main
    add_prefix=False if training_args.train_adapters else True)
  File "/workdir/seq2seq/data/tasks.py", line 70, in get_dataset
    dataset = self.load_dataset(split=split)
  File "/workdir/seq2seq/data/tasks.py", line 306, in load_dataset
    return datasets.load_dataset('glue', 'cola', split=split)
  File "/usr/local/lib/python3.6/dist-packages/datasets/load.py", line 589, in load_dataset
    path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
  File "/usr/local/lib/python3.6/dist-packages/datasets/load.py", line 263, in prepare_module
    head_hf_s3(path, filename=name, dataset=dataset)
  File "/usr/local/lib/python3.6/dist-packages/datasets/utils/file_utils.py", line 200, in head_hf_s3
    return http_head(hf_bucket_url(identifier=identifier, filename=filename, use_cdn=use_cdn, dataset=dataset))
  File "/usr/local/lib/python3.6/dist-packages/datasets/utils/file_utils.py", line 403, in http_head
    url, proxies=proxies, headers=headers, cookies=cookies, allow_redirects=allow_redirects, timeout=timeout
  File "/usr/local/lib/python3.6/dist-packages/requests/api.py", line 104, in head
    return request('head', url, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 504, in send
    raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /datasets.huggingface.co/datasets/datasets/glue/glue.py (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f47db511e80>, 'Connection to s3.amazonaws.com timed out. (connect timeout=10)'))
```

Comments: could you tell me how I can avoid download, by pre-downloading the data first, put them in a folder so the code does not try to redownload? could you tell me the path to put the downloaded data, and how to do it? thanks
@lhoestq 

Does your instance have an internet connection ?

If you don't have an internet connection you'll need to have the dataset on the instance disk.
To do so first download the dataset on another machine using `load_dataset` and then you can save it in a folder using `my_dataset.save_to_disk("path/to/folder")`. Once the folder is copied on your instance you can reload the dataset with `datasets.load_from_disk("path/to/folder")`