Repository: huggingface/evaluate
Issue: Metric returned instead of the actual COMET scores
Root cause: External Dependency Breakage

It looks like that when using the `.compute()` method to obtain `wmt20-comet-da` scores it returns the string corresponding to the score names instead of the actual value (i.e. `system_score` and `scores` instead of the real values). Here's the snippet I'm running:

```
> from evaluate import load
> comet = load('comet')
> comet_results = comet.compute(predictions=predictions, references=references_list, sources=sources_list)
> comet_results
{'mean_score': 'system_score', 'scores': 'scores'}
```

After a bit of investigation, it seems the scores dictionary should return the values corresponding to these keys instead of the keys themselves. See [L168 here](https://github.com/huggingface/evaluate/blob/af3c30561d840b83e54fc5f7150ea58046d6af69/metrics/comet/comet.py#L168). Any idea why this is happening on version 0.4.0?

As of now, I patched it by collecting the `.values()` at that line, happy to open a PR if this looks fine for you.
```
- unbabel-comet==2.0.2
- torch==2.0.1
- torchmetrics==0.8.2
- pytorch-lightning==1.9.5
```

Same here! 
`import evaluate`
`mt_metrics = evaluate.combine( ["bleu", "chrf", "comet"])`
`sources = ["Dem Feuer konnte Einhalt geboten werden", "Schulen und Kindergärten wurden eröffnet."]`
`predictions = ["They were able to control the fire.", "Schools and kindergartens opened"]`
`references = ["They were able to control the fire.", "Schools and kindergartens opened"]`
`results = mt_metrics.compute(predictions=predictions, references=references, sources=sources)`
`print(results)`
Only COMET scores can't print out in the result as below:
`{'bleu': 1.0, 'precisions': [1.0, 1.0, 1.0, 1.0], 'brevity_penalty': 1.0, 'length_ratio': 1.0, 'translation_length': 12, 'reference_length': 12, 'score': 100.0, 'char_order': 6, 'word_order': 0, 'beta': 2, 'mean_score': 'system_score', 'scores': 'scores'}`


Can you investigate what the type is returned by the comet metric? Maybe there is an easy fix.

I found it odd and decided to make a new (mini) conda environment with the specified settings. Surprisingly, the problem disappeared. What's interesting is that I realised I had been running the code in the main branch instead of v0.4.0, which is now functioning well. I suspect I had an incorrect version of COMET installed. It appears that upcoming releases will behave differently depending on the COMET version, causing the evaluate library to process dictionary keys rather than their associated values.

Maybe @lvwerra you have better guesses? In any case, feel free to close this issue.

I observe the same behaviour that the score keys are returned `{'mean_score': 'system_score', 'scores': 'scores'}` - I checked my dependencies and I have evaluate `0.4.0` installed and comet `2.0.2`. The returned type is a dict with two entries:
![image](https://github.com/huggingface/evaluate/assets/56544999/fc8c0e98-1ab0-43ab-b935-c2e58bd5bab0)


I have the same issue with unbabel-comet 2.1.1 and evaluate 0.4.0

Has anyone been able to resolve this issue? Which package versions will not cause this error?

I confirm it has been fixed with the new release: `unbabel-comet==2.2.1` and `evaluate==0.4.1`.