Repository: nltk/nltk

Issue:
Is not the max of meteor score 100?  (nltk.translate.meteor_score)
I used [nltk.translate.meteor_score](https://github.com/nltk/nltk/blob/develop/nltk/translate/meteor_score.py) to calculate the meteor score of a example, strangely I got a score of 105.6. 
Is not the max of meteor score 100?

The example is following:

> **candidate:**   creates or updates a virtual machine scale set
**reference:**   create or update a vm scale set
**METEOR score:**   105.6

Is there anyone know that? Thank you very much!
Meteor unmatched-lists calculation
Hi, 

I think [unmatched-lists calculation by _enum_stem_match()](https://github.com/nltk/nltk/blob/e4444c9b15762e6d86f5f6c4f5faadb87632c72a/nltk/translate/meteor_score.py#L109) is wrong,
```
enum_unmat_hypo_list = (
        list(zip(*enum_unmat_hypo_list)) if len(enum_unmat_hypo_list) > 0 else []
    )
enum_hypothesis_list = list(
        filter(lambda x: x[0] not in enum_unmat_hypo_list, enum_hypothesis_list)
    )
```
because the upper zip(*) generates [(unmatched indices), (unmatched stems)]
but what we want is to filter by (unmatched indices).
Also, "not in" would filter **out** unmatched, not the opposite.

When I fix the calculation to:
```
enum_unmat_hypo_list = (
        list(zip(*enum_unmat_hypo_list))[0] if len(enum_unmat_hypo_list) > 0 else []
    )
enum_hypothesis_list = list(
       filter(lambda x: x[0] in enum_unmat_hypo_list, enum_hypothesis_list)
    )
```

the score below is 0.655 with matches = [(0, 0), (1, 2), (2, 3)]
(Before fixing, it was 0.809 with matches = [(0, 0), (1, 2), (1, 2), (2, 3)] which contains duplicated elements)
```
hypothesis = 'I likes pizza'
reference = 'I really like pizza'
single_meteor_score(reference, hypothesis)
```


I would like to know if this is wrong.
Thanks.


Discussion:

Resolving patch:
diff --git a/nltk/translate/meteor_score.py b/nltk/translate/meteor_score.py
--- a/nltk/translate/meteor_score.py
+++ b/nltk/translate/meteor_score.py
@@ -2,12 +2,12 @@
 #
 # Copyright (C) 2001-2021 NLTK Project
 # Author: Uday Krishna <udaykrishna5@gmail.com>
+# Contributor: Tom Aarsen
 # URL: <http://nltk.org/>
 # For license information, see LICENSE.TXT
 
 
 from itertools import chain, product
-
 from nltk.corpus import wordnet
 from nltk.stem.porter import PorterStemmer
 
@@ -102,28 +102,10 @@ def _enum_stem_match(
         (word_pair[0], stemmer.stem(word_pair[1])) for word_pair in enum_reference_list
     ]
 
-    word_match, enum_unmat_hypo_list, enum_unmat_ref_list = _match_enums(
+    return _match_enums(
         stemmed_enum_list1, stemmed_enum_list2
     )
 
-    enum_unmat_hypo_list = (
-        list(zip(*enum_unmat_hypo_list)) if len(enum_unmat_hypo_list) > 0 else []
-    )
-
-    enum_unmat_ref_list = (
-        list(zip(*enum_unmat_ref_list)) if len(enum_unmat_ref_list) > 0 else []
-    )
-
-    enum_hypothesis_list = list(
-        filter(lambda x: x[0] not in enum_unmat_hypo_list, enum_hypothesis_list)
-    )
-
-    enum_reference_list = list(
-        filter(lambda x: x[0] not in enum_unmat_ref_list, enum_reference_list)
-    )
-
-    return word_match, enum_hypothesis_list, enum_reference_list
-
 
 def stem_match(hypothesis, reference, stemmer=PorterStemmer()):
     """
@@ -199,7 +181,7 @@ def wordnetsyn_match(hypothesis, reference, wordnet=wordnet):
     )
 
 
-def _enum_allign_words(
+def _enum_align_words(
     enum_hypothesis_list, enum_reference_list, stemmer=PorterStemmer(), wordnet=wordnet
 ):
     """
@@ -240,7 +222,7 @@ def _enum_allign_words(
     )
 
 
-def allign_words(hypothesis, reference, stemmer=PorterStemmer(), wordnet=wordnet):
+def align_words(hypothesis, reference, stemmer=PorterStemmer(), wordnet=wordnet):
     """
     Aligns/matches words in the hypothesis to reference by sequentially
     applying exact match, stemmed match and wordnet based synonym match.
@@ -257,7 +239,7 @@ def allign_words(hypothesis, reference, stemmer=PorterStemmer(), wordnet=wordnet
     :rtype: list of tuples, list of tuples, list of tuples
     """
     enum_hypothesis_list, enum_reference_list = _generate_enums(hypothesis, reference)
-    return _enum_allign_words(
+    return _enum_align_words(
         enum_hypothesis_list, enum_reference_list, stemmer=stemmer, wordnet=wordnet
     )
 
@@ -268,7 +250,7 @@ def _count_chunks(matches):
     of each chunk are adjacent to each other. This is used to calculate the
     fragmentation part of the metric.
 
-    :param matches: list containing a mapping of matched words (output of allign_words)
+    :param matches: list containing a mapping of matched words (output of align_words)
     :return: Number of chunks a sentence is divided into post alignment
     :rtype: int
     """
@@ -318,8 +300,8 @@ def single_meteor_score(
     >>> round(meteor_score('this is a cat', 'non matching hypothesis'),4)
     0.0
 
-    :param references: reference sentences
-    :type references: list(str)
+    :param reference: reference sentence
+    :type reference: str
     :param hypothesis: a hypothesis sentence
     :type hypothesis: str
     :param preprocess: preprocessing function (default str.lower)
@@ -343,7 +325,7 @@ def single_meteor_score(
     )
     translation_length = len(enum_hypothesis)
     reference_length = len(enum_reference)
-    matches, _, _ = _enum_allign_words(enum_hypothesis, enum_reference, stemmer=stemmer)
+    matches, _, _ = _enum_align_words(enum_hypothesis, enum_reference, stemmer=stemmer, wordnet=wordnet)
     matches_count = len(matches)
     try:
         precision = float(matches_count) / translation_length


Test patch:
diff --git a/nltk/test/meteor.doctest b/nltk/test/meteor.doctest
--- a/nltk/test/meteor.doctest
+++ b/nltk/test/meteor.doctest
@@ -43,3 +43,11 @@ Testing on `wikipedia examples <https://en.wikipedia.org/wiki/METEOR#Examples>`_
     ...       'The cat was sat on the mat'
     ...       ),4)
     0.9654
+
+Test corresponding to issue #2751, where METEOR score > 1
+
+    >>> round(meteor(
+    ...       ['create or update a vm set'],
+    ...       'creates or updates a virtual machine scale set'
+    ...       ),4)
+    0.7806
\ No newline at end of file