Repository: Lightning-AI/torchmetrics

Issue:
The BLEU score computed by the SacreBLEUScore metric is not the same as the original implementation
Hi, I'm sorry in advance for my english, it isn't my first language. Anyway, I didn't see any issue or discussion about this small bug, so, let's get started.

## 🐛 Bug

This a very edge case that came out while I was evaluating some machine translation models on the WMT14 test set.
The BLEU score computed by the SacreBLEUScore metric from torchmetrics (with 13a as the tokenizer) is different from the [original implementation](https://github.com/mjpost/sacrebleu), this is due to the fact that the way the words are tokenized is different when a number is followed by a punctuation symbol at the end of a string.

### To Reproduce
 
I used the [HuggingFace version of SacreBLEU](https://huggingface.co/spaces/evaluate-metric/sacrebleu/) (which directly relies on the original one) as baseline.

```python
# Just a toy example, notice the space between 2023 and the full stop in the first string of the predictions
> predictions = ["We are in 2023 .", "Wait until I count to 10, then move."]
> references = [["We are in 2023."], ["Wait until I count to 10, then move."]]

# HugginFace implementation
> import evaluate
> hf_scb = evaluate.load("sacrebleu")
> hf_score = hf_scb.compute(predictions=predictions, references=references)
> hf_bleu_score = "{0:.2f}".format(hf_score["score"])
> print(f"HF implementation\nBLEU score: {hf_bleu_score}\npreds_len: {hf_score['sys_len']}\ntarget_len: {hf_score['ref_len']}")
... HF implementation
... BLEU score: 100.00
... preds_len: 15
... target_len: 15

# Torchmetrics implementation
> from torchmetrics import SacreBLEUScore
> tm_scb = SacreBLEUScore()
> tm_score = "{0:.5f}".format(tm_scb(predictions, references))
>print(f"Torchmetrics implementation\nBLEU score: {tm_score}\npreds_len: {tm_scb.preds_len}\ntarget_len: {tm_scb.target_len}")
... Torchmetrics implementation
... BLEU score: 0.82652
... preds_len: 15.0
... target_len: 14.0
```

### Expected behavior

The BLEU score computed by SacreBLEUScore should be the same as the one given by the original implementation, therefore, the predictions and references' lengths should be computed in the same way.

### Environment

- TorchMetrics version: `0.11.4` installed using `conda install -c conda-forge torchmetrics`
- Python version: `3.10.11`
- PyTorch version: `1.12.1` with `CUDA 11.5`
- OS: `Ubuntu 20.04 LTS`

### Additional context

The solution would be modifying the following line inside the `_tokenize_13a` method
https://github.com/Lightning-AI/torchmetrics/blob/3ea6062b26b65d0c3b75152b81472e8be861e96d/src/torchmetrics/functional/text/sacre_bleu.py#L196
with the following
```python
return cls._tokenize_regex(f" {line} ")
```
which is the same as the [original implementation](https://github.com/mjpost/sacrebleu/blob/4f4124642c4eb0b7120e50119c669f0570a326a7/sacrebleu/tokenizers/tokenizer_13a.py#L34).


Discussion:
Hi! thanks for your contribution!, great first issue!
@stancld makes sense to you?
@SkafteNicki Makes sense to fix this to be equivalent to the original implementation. @RistoAle97 Do you wanna send a PR?

Otherwise, I'm mostly AFK today and this weekend, but can have a look on Monday.
@stancld I will open a PR as soon as I get home today. At the end of the day I just need to change one line of code.

Resolving patch:
diff --git a/src/torchmetrics/functional/text/sacre_bleu.py b/src/torchmetrics/functional/text/sacre_bleu.py
--- a/src/torchmetrics/functional/text/sacre_bleu.py
+++ b/src/torchmetrics/functional/text/sacre_bleu.py
@@ -174,7 +174,7 @@ def _tokenize_base(cls, line: str) -> str:
 
     @classmethod
     def _tokenize_13a(cls, line: str) -> str:
-        """Tokenizes an line using a relatively minimal tokenization that is equivalent to mteval-v13a, used by WMT.
+        """Tokenizes a line using a relatively minimal tokenization that is equivalent to mteval-v13a, used by WMT.
 
         Args:
             line: input sentence
@@ -193,7 +193,7 @@ def _tokenize_13a(cls, line: str) -> str:
             line = line.replace("&lt;", "<")
             line = line.replace("&gt;", ">")
 
-        return cls._tokenize_regex(line)
+        return cls._tokenize_regex(f" {line} ")
 
     @classmethod
     def _tokenize_zh(cls, line: str) -> str:


Test patch: