Repository: huggingface/evaluate
Issue: ERROR: Using words with different lenght to compute CER and WER
Root cause: Validation Gap

I was trying to compute WER and CER metrics with a single input (without being in a list) with different lengths.
### INPUT: 
**wer_score = wer.compute(predictions="good night moon", references="have a great day")**

And I get the following error:

```
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_6873/1470097976.py in <module>
----> 5 wer_score = wer.compute(predictions="good night moon", references="have a great day")
      6 print(wer_score)

/anaconda/envs/CondaEnv/lib/python3.10/site-packages/evaluate/module.py in compute(self, predictions, references, **kwargs)
    423 
    424         if any(v is not None for v in inputs.values()):
--> 425             self.add_batch(**inputs)
    426         self._finalize()
    427 

/anaconda/envs/CondaEnv/lib/python3.10/site-packages/evaluate/module.py in add_batch(self, predictions, references, **kwargs)
    501                     f"Input references: {summarize_if_long_list(references)}"
    502                 )
--> 503             raise ValueError(error_msg) from None
    504 
    505     def add(self, *, prediction=None, reference=None, **kwargs):

ValueError: Mismatch in the number of predictions (15) and references (16)
```
However, if the input is a list it works well:

### INPUT: 
**predictions = ["hello world", "good night moon"]**
**references = ["hi everyone", "have a great day"]**
**wer_score = wer.compute(predictions="good night moon", references="have a great day")**

Hi @Coliverfelt, the inputs should be a list, even if you only have one element:

```py
wer_score = wer.compute(predictions=["good night moon"], references=["have a great day"])
```

Does that solve your issue?


Indeed, it should be and works @lvwerra. However when I put:

```
wer_score = wer.compute(predictions="helloworld", references="helloworld")
print(wer_score)

OUTPUT: 0.0
```

It works as well. But if I put:

```
`wer_score = wer.compute(predictions="hell oworld", references="helloworld")
print(wer_score)`

OUTPUT:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_10817/1257524053.py in <module>
      3 # references = ["50.0", "6.5"]
      4 # wer_score = wer.compute(predictions=predictions, references=references)
----> 5 wer_score = wer.compute(predictions="hell oworld", references="helloworld")
      6 print(wer_score)

/anaconda/envs/CondaEnv/lib/python3.10/site-packages/evaluate/module.py in compute(self, predictions, references, **kwargs)
    423 
    424         if any(v is not None for v in inputs.values()):
--> 425             self.add_batch(**inputs)
    426         self._finalize()
    427 

/anaconda/envs/CondaEnv/lib/python3.10/site-packages/evaluate/module.py in add_batch(self, predictions, references, **kwargs)
    501                     f"Input references: {summarize_if_long_list(references)}"
    502                 )
--> 503             raise ValueError(error_msg) from None
    504 
    505     def add(self, *, prediction=None, reference=None, **kwargs):

ValueError: Mismatch in the number of predictions (11) and references (10)
```

That's because predictions expects (and interprets) the inputs as a list of samples and as such expects as many predictions as references. When you pass just a string instead of a list the string is interpreted as a list of characters (`"hello" -> ["h", "e", "l", "l", "0"]`) and therefore the result you get is equivalent to the following:

```py
wer_score = wer.compute(predictions=["h", "e", "l", "l", "o", "w", "o", "r", "l", "d"], references=["h", "e", "l", "l", "o", "w", "o", "r", "l", "d"])
``` 

So this just works by accident and you should always pass the inputs as a list.

Thank you very much. It makes things clear for me.