Add rouge-2 in rouge_types for metric calculation
State: closed | Author: Shashi456 (NONE)

The description of the ROUGE metric says, 
```
_KWARGS_DESCRIPTION = """
Calculates average rouge scores for a list of hypotheses and references
Args:
    predictions: list of predictions to score. Each predictions
        should be a string with tokens separated by spaces.
    references: list of reference for each prediction. Each
        reference should be a string with tokens separated by spaces.
Returns:
    rouge1: rouge_1 f1,
    rouge2: rouge_2 f1,
    rougeL: rouge_l f1,
    rougeLsum: rouge_l precision
"""
```

but the `rouge_types` argument defaults to  `rouge_types = ["rouge1", "rougeL"]`, this PR updates and add `rouge2` to the list so as to reflect the description card.

Comments: Indeed there's currently a mismatch between the description and what it rouge actually returns.
Thanks for proposing this fix :) 

I think it's better to return rouge 1-2-L.
Was there a reason to only include rouge 1 and rouge L @thomwolf ? 

rougeLsum is also missing, could you add it ?

Adding `RougeLSum` would fix https://github.com/huggingface/datasets/issues/617

I am opening a PR with both of them right now actually :)

Also the format of the output isn't exactly ideal, It's usually only the F-1 score that is cared about. 

Formatting the output to reflect how `ROUGE-1-5-5` (the perl version thats usually used and pyrouge is a wrapper over it), would be better.



I'll close this since you seem to have already added it in another PR. Sorry for the delay in responding to you @lhoestq.

What do you mean by "Formatting the output to reflect how ROUGE-1-5-5" @Shashi456 ?

I like the idea of returning all the scores for two reason:
- Rouge's aggregator does sampling and therefore it returns "low" "mid" and "high" scores
- It is interesting to have the precision and recall to see how the F1 score was computed
But I understand your point that returning only the F1 score makes sense since it's the one that's always used 

@thomwolf the scores now returned look like this:
```
{'rouge1': AggregateScore(low=Score(precision=0.16620308156871524, recall=0.18219819615984395, fmeasure=0.16226017699359463), mid=Score(precision=0.17274338501705871, recall=0.1890957812369246, fmeasure=0.16823877588620403), high=Score(precision=0.17934569582981455, recall=0.1965626706042028, fmeasure=0.17491509794856058)), 
'rouge2': AggregateScore(low=Score(precision=0.12478835737689957, recall=0.1362113231755514, fmeasure=0.12055941950062395), mid=Score(precision=0.1303967602691664, recall=0.1423747229852964, fmeasure=0.1258363976151122), high=Score(precision=0.13654527560789362, recall=0.1488071465116122, fmeasure=0.13184989406704056)), 
'rougeL': AggregateScore(low=Score(precision=0.16568068818352072, recall=0.1811919016674486, fmeasure=0.1614784523482225), mid=Score(precision=0.17156684723552357, recall=0.1879777628247058, fmeasure=0.16720699286250762), high=Score(precision=0.17788847350584547, recall=0.1948899838530898, fmeasure=0.17316501523379826))}
```

While when computed through the perl rouge script, it looks like:
```
ROUGE-1 Average_R: 0.34775 (95%-conf.int. 0.34546 - 0.35025)
ROUGE-1 Average_P: 0.19381 (95%-conf.int. 0.19246 - 0.19538)
ROUGE-1 Average_F: 0.24070 (95%-conf.int. 0.23925 - 0.24230)
---------------------------------------------
ROUGE-2 Average_R: 0.07160 (95%-conf.int. 0.07010 - 0.07298)
ROUGE-2 Average_F: 0.04845 (95%-conf.int. 0.04741 - 0.04942)
---------------------------------------------
ROUGE-L Average_R: 0.26404 (95%-conf.int. 0.26215 - 0.26598)
ROUGE-L Average_P: 0.14696 (95%-conf.int. 0.14576 - 0.14815)
ROUGE-L Average_F: 0.18245 (95%-conf.int. 0.18120 - 0.18367)
```
while the wrapper returns the much more readable:
```
[2020-07-30 18:13:38,556 INFO] Rouges at step 13000                                       
>> ROUGE-F(1/2/3/l): 43.43/20.42/39.78              
ROUGE-R(1/2/3/l): 53.91/25.34/49.32
```

The formatting allows for easy reading, and although "low", "mid", "high" make sense, this is more concise and effective. 

One way of changing this might be to return a dictionary that returns values like `rouge_1_precision`, `rouge_1_F1`, `rouge_1_recall`, and maybe also having the ability to get the values you are interested in and keeping `recall` and `F1` as default.

cc: @lhoestq 

Ok I see.
I think it's also important to follow one of the existing output format (there are already too many different formats, let's try not to add another different one)
I'd still stick with the current format and not transform the output of the python implementation of rouge since it's already widely used.
What do you think ?

Maybe we could convert the dataclasses in dictionnaries, would that help @Shashi456 ?

@thomwolf yeah I think that would help. I initially didn't understand the high low mid categories. Dictionaries could help in this case I guess, and if we allow the user to choose what they want i.e F1 and precision or recall.