distributed_eval does not run
Author: rabeehk
Pull request: no
Labels: name: wontfix

Hi
I am trying to run distributed_eval with latest version of huggingface codes installed from the webpage, (internship) 
please find the command below and the errors, thank you for your help. 

Info on versions/machine: 
python = 3.7
8 gpus
transformers              4.0.0rc1                 pypi_0    pypi

```
rabeeh@gpu8:~/transformers/examples/seq2seq$ python -m torch.distributed.launch --nproc_per_node=8  run_distributed_eval.py     --model_name sshleifer/distilbart-large-xsum-12-3      --save_dir xsum_generations     --data_dir xsum     --fp16 
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
2020-11-22 16:51:27.894686: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-22 16:51:27.894685: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-22 16:51:27.894686: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-22 16:51:27.894688: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-22 16:51:27.894688: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-22 16:51:27.894685: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-22 16:51:27.894688: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-22 16:51:27.896156: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json
Traceback (most recent call last):
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 389, in get_config_dict
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 987, in cached_path
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 1108, in get_from_cache
    r.raise_for_status()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_distributed_eval.py", line 248, in <module>
    run_generate()
  File "run_distributed_eval.py", line 180, in run_generate
    **generate_kwargs,
  File "run_distributed_eval.py", line 57, in eval_data_dir
    model = AutoModelForSeq2SeqLM.from_pretrained(model_name).cuda()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/modeling_auto.py", line 1141, in from_pretrained
    pretrained_model_name_or_path, return_unused_kwargs=True, **kwargs
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 341, in from_pretrained
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 401, in get_config_dict
    raise EnvironmentError(msg)
OSError: Can't load config for 'sshleifer/distilbart-large-xsum-12-3'. Make sure that:

- 'sshleifer/distilbart-large-xsum-12-3' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'sshleifer/distilbart-large-xsum-12-3' is the correct path to a directory containing a config.json file


404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json
Traceback (most recent call last):
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 389, in get_config_dict
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 987, in cached_path
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 1108, in get_from_cache
    r.raise_for_status()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_distributed_eval.py", line 248, in <module>
    run_generate()
  File "run_distributed_eval.py", line 180, in run_generate
    **generate_kwargs,
  File "run_distributed_eval.py", line 57, in eval_data_dir
    model = AutoModelForSeq2SeqLM.from_pretrained(model_name).cuda()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/modeling_auto.py", line 1141, in from_pretrained
    pretrained_model_name_or_path, return_unused_kwargs=True, **kwargs
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 341, in from_pretrained
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 401, in get_config_dict
    raise EnvironmentError(msg)
OSError: Can't load config for 'sshleifer/distilbart-large-xsum-12-3'. Make sure that:

- 'sshleifer/distilbart-large-xsum-12-3' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'sshleifer/distilbart-large-xsum-12-3' is the correct path to a directory containing a config.json file


404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json
Traceback (most recent call last):
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 389, in get_config_dict
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 987, in cached_path
404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json
Traceback (most recent call last):
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 389, in get_config_dict
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 1108, in get_from_cache
    r.raise_for_status()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/requests/models.py", line 943, in raise_for_status
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 987, in cached_path
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_distributed_eval.py", line 248, in <module>
    run_generate()
  File "run_distributed_eval.py", line 180, in run_generate
    **generate_kwargs,
  File "run_distributed_eval.py", line 57, in eval_data_dir
    local_files_only=local_files_only,    
model = AutoModelForSeq2SeqLM.from_pretrained(model_name).cuda()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 1108, in get_from_cache
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/modeling_auto.py", line 1141, in from_pretrained
    pretrained_model_name_or_path, return_unused_kwargs=True, **kwargs
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 341, in from_pretrained
    r.raise_for_status()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/requests/models.py", line 943, in raise_for_status
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 401, in get_config_dict
    raise EnvironmentError(msg)
OSError: Can't load config for 'sshleifer/distilbart-large-xsum-12-3'. Make sure that:

- 'sshleifer/distilbart-large-xsum-12-3' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'sshleifer/distilbart-large-xsum-12-3' is the correct path to a directory containing a config.json file

    
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_distributed_eval.py", line 248, in <module>
    run_generate()
  File "run_distributed_eval.py", line 180, in run_generate
    **generate_kwargs,
  File "run_distributed_eval.py", line 57, in eval_data_dir
    model = AutoModelForSeq2SeqLM.from_pretrained(model_name).cuda()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/modeling_auto.py", line 1141, in from_pretrained
    pretrained_model_name_or_path, return_unused_kwargs=True, **kwargs
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 341, in from_pretrained
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 401, in get_config_dict
    raise EnvironmentError(msg)
OSError: Can't load config for 'sshleifer/distilbart-large-xsum-12-3'. Make sure that:

- 'sshleifer/distilbart-large-xsum-12-3' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'sshleifer/distilbart-large-xsum-12-3' is the correct path to a directory containing a config.json file


404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json
Traceback (most recent call last):
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 389, in get_config_dict
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 987, in cached_path
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 1108, in get_from_cache
    r.raise_for_status()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_distributed_eval.py", line 248, in <module>
    run_generate()
  File "run_distributed_eval.py", line 180, in run_generate
    **generate_kwargs,
  File "run_distributed_eval.py", line 57, in eval_data_dir
    model = AutoModelForSeq2SeqLM.from_pretrained(model_name).cuda()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/modeling_auto.py", line 1141, in from_pretrained
    pretrained_model_name_or_path, return_unused_kwargs=True, **kwargs
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 341, in from_pretrained
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 401, in get_config_dict
    404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json
raise EnvironmentError(msg)
OSError: Can't load config for 'sshleifer/distilbart-large-xsum-12-3'. Make sure that:

- 'sshleifer/distilbart-large-xsum-12-3' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'sshleifer/distilbart-large-xsum-12-3' is the correct path to a directory containing a config.json file


Traceback (most recent call last):
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 389, in get_config_dict
404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json
Traceback (most recent call last):
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 389, in get_config_dict
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 987, in cached_path
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 1108, in get_from_cache
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 987, in cached_path
    r.raise_for_status()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/requests/models.py", line 943, in raise_for_status
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 1108, in get_from_cache
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_distributed_eval.py", line 248, in <module>
    run_generate()
  File "run_distributed_eval.py", line 180, in run_generate
    r.raise_for_status()
      File "/opt/conda/envs/internship/lib/python3.7/site-packages/requests/models.py", line 943, in raise_for_status
**generate_kwargs,
  File "run_distributed_eval.py", line 57, in eval_data_dir
    model = AutoModelForSeq2SeqLM.from_pretrained(model_name).cuda()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/modeling_auto.py", line 1141, in from_pretrained
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_distributed_eval.py", line 248, in <module>
    pretrained_model_name_or_path, return_unused_kwargs=True, **kwargs
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 341, in from_pretrained
    run_generate()
  File "run_distributed_eval.py", line 180, in run_generate
    **generate_kwargs,
  File "run_distributed_eval.py", line 57, in eval_data_dir
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 401, in get_config_dict
    model = AutoModelForSeq2SeqLM.from_pretrained(model_name).cuda()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/modeling_auto.py", line 1141, in from_pretrained
    raise EnvironmentError(msg)
OSError: Can't load config for 'sshleifer/distilbart-large-xsum-12-3'. Make sure that:

- 'sshleifer/distilbart-large-xsum-12-3' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'sshleifer/distilbart-large-xsum-12-3' is the correct path to a directory containing a config.json file


    pretrained_model_name_or_path, return_unused_kwargs=True, **kwargs
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 341, in from_pretrained
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 401, in get_config_dict
    raise EnvironmentError(msg)
OSError: Can't load config for 'sshleifer/distilbart-large-xsum-12-3'. Make sure that:

- 'sshleifer/distilbart-large-xsum-12-3' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'sshleifer/distilbart-large-xsum-12-3' is the correct path to a directory containing a config.json file


404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json
Traceback (most recent call last):
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 389, in get_config_dict
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 987, in cached_path
    local_files_only=local_files_only,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/file_utils.py", line 1108, in get_from_cache
    r.raise_for_status()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/sshleifer/distilbart-large-xsum-12-3/resolve/main/config.json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_distributed_eval.py", line 248, in <module>
    run_generate()
  File "run_distributed_eval.py", line 180, in run_generate
    **generate_kwargs,
  File "run_distributed_eval.py", line 57, in eval_data_dir
    model = AutoModelForSeq2SeqLM.from_pretrained(model_name).cuda()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/modeling_auto.py", line 1141, in from_pretrained
    pretrained_model_name_or_path, return_unused_kwargs=True, **kwargs
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 341, in from_pretrained
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/configuration_utils.py", line 401, in get_config_dict
    raise EnvironmentError(msg)
OSError: Can't load config for 'sshleifer/distilbart-large-xsum-12-3'. Make sure that:

- 'sshleifer/distilbart-large-xsum-12-3' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'sshleifer/distilbart-large-xsum-12-3' is the correct path to a directory containing a config.json file


Traceback (most recent call last):
  File "/opt/conda/envs/internship/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/conda/envs/internship/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/torch/distributed/launch.py", line 260, in <module>
    main()
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/torch/distributed/launch.py", line 256, in main
    cmd=cmd)
subprocess.CalledProcessError: Command '['/opt/conda/envs/internship/bin/python', '-u', 'run_distributed_eval.py', '--local_rank=7', '--model_name', 'sshleifer/distilbart-large-xsum-12-3', '--save_dir', 'xsum_generations', '--data_dir', 'xsum', '--fp16']' returned non-zero exit status 1.
```


Comments:
with t5-base seems to run fine, so perhaps the specific model in the README is not available. Thank you. 

Hello! Indeed, it seems this model does not exist. Do you want to open a PR with a model that works? Thanks!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.