Repository: Unbabel/COMET
Issue: Get list of models
Root cause: Documentation Deficiency

## 🚀 Feature
COMET should output a list of available models if `-m` is used with an invalid model name.

### Motivation

It's a bit of a pain to figure out the available models from the CLI. I had to come to the Github page.

### Alternatives

<!-- A clear and concise description of any alternative solutions or features you've considered if any. -->

### Additional context

<!-- Add any other context or screenshots about the feature request here. -->


On a related question, how does this dictionary  map to the list of models in https://unbabel.github.io/COMET/html/models.html#faq

```
{'emnlp20-comet-rank': 'https://unbabel-experimental-models.s3.amazonaws.com/comet/wmt20/emnlp20-comet-rank.tar.gz',
 'wmt20-comet-da': 'https://unbabel-experimental-models.s3.amazonaws.com/comet/wmt20/wmt20-comet-da.tar.gz',
 'wmt20-comet-qe-da': 'https://unbabel-experimental-models.s3.amazonaws.com/comet/wmt20/wmt20-comet-qe-da.tar.gz',
 'wmt21-comet-mqm': 'https://unbabel-experimental-models.s3.amazonaws.com/comet/wmt21/wmt21-comet-mqm.tar.gz',
 'wmt21-cometinho-da': 'https://unbabel-experimental-models.s3.amazonaws.com/comet/wmt21/wmt21-cometinho-da.tar.gz',
 'wmt21-comet-qe-mqm': 'https://unbabel-experimental-models.s3.amazonaws.com/comet/wmt21/wmt21-comet-qe-mqm.tar.gz'
}
```





@alvations that's something I have to fix a long time ago already! Documentation is extremely outdated...

wmt-large-da-estimator-1719 | wmt20-comet-da
-- | --
↑wmt-base-da-estimator-1719 | not supported in v1.0
↑wmt-large-qe-estimator-1719 | wmt20-comet-qe-da
↓wmt-large-hter-estimator | not supported in v1.0
↓wmt-base-hter-estimator | not supported in v1.0
↑emnlp-base-da-ranker | emnlp20-comet-rank

All other models are from our 2021 paper and are not in that documentation!

Regarding @mjpost request, it does print the available metrics when you pass an invalid model name:

> comet-score -s src.de -t hyp1.en -r ref.en --gpus 0 --model test

```bash
usage: comet-score [-h] [-s SOURCES] [-t TRANSLATIONS [TRANSLATIONS ...]] [-r REFERENCES] [-d SACREBLEU_DATASET] [--batch_size BATCH_SIZE] [--gpus GPUS] [--accelerator {dp,ddp}] [--to_json TO_JSON]
                   [--model MODEL] [--model_storage_path MODEL_STORAGE_PATH] [--mc_dropout MC_DROPOUT] [--seed_everything SEED_EVERYTHING] [--num_workers NUM_WORKERS] [--disable_bar] [--disable_cache]
                   [--disable_length_batching] [--print_cache_info]
comet-score: error: test is not a valid checkpoint path or model choice. Choose from dict_keys(['emnlp20-comet-rank', 'wmt20-comet-da', 'wmt20-comet-qe-da', 'wmt21-comet-mqm', 'wmt21-cometinho-da', 'wmt21-comet-qe-mqm'])
```

When I pass --model wmt21-comet-qe-da 

Global seed set to 12
Downloading refless-wmt21-comet-da.tar.gz
refless-wmt21-comet-da.tar.gz: 0.00B [00:00, ?B/s]
Traceback (most recent call last):
  File "comet/cli/score.py", line 327, in <module>
    score_command()
  File "comet/cli/score.py", line 170, in score_command
    model_path = download_model(cfg.model, saving_directory=cfg.model_storage_path)
  File "/content/COMET/comet/download_utils.py", line 209, in download_model
    available_metrics[model], directory=saving_directory
  File "/content/COMET/comet/download_utils.py", line 165, in download_file_maybe_extract
    urllib.request.urlretrieve(url, filename=filepath, reporthook=_reporthook(t))
  File "/usr/lib/python3.7/urllib/request.py", line 247, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden


which means the model name (taken from doc) is not wrong according to your post, but not able to download.
I'll use wmt20-comet-qe-da instead


you are right @vince62s! The link is broken for that model. I'll create a new issue for this