BLEU
BLEU (bilingual evaluation understudy) is an algorithm for evaluating the quality of text which has been machine-translated from one natural language to another. Quality is considered to be the correspondence between a machine's output and that of a human: "the closer a machine translation is to a professional human translation, the better it is" – this is the central idea behind BLEU. BLEU was one of the first metrics to claim a high correlation with human judgements of quality, and remains one of the most popular automated and inexpensive metrics. Scores are calculated for individual translated segments—generally  sentences—by comparing them with a set of good quality reference translations. Those scores are then averaged over the whole corpus to reach an estimate of the translation's overall quality. Intelligibility or grammatical correctness are not taken into account. BLEU's output is always a number between 0 and 1. This value indicates how similar the candidate text is to the reference texts, with values closer to 1 representing more similar texts. Few human translations will attain a score of 1, since this would indicate that the candidate is identical to one of the reference translations. For this reason , it is not necessary to attain a score of 1. Because there are more opportunities to match, adding additional reference translations will increase the BLEU score. Algorithm. This is illustrated in the following example from Papineni et al. (2002): Of the seven words in the candidate translation, all of them appear in the reference translations. Thus the candidate text is given a unigram precision of, where formula_2 is number of words from the candidate that are found in the reference, and formula_3 is the total number of words in the candidate. This is a perfect score, despite the fact that  the candidate translation above retains little of the content of either of the references. The modification that BLEU makes is fairly straightforward. For each word in the candidate translation, the algorithm takes its maximum total count, formula_4, in any of the reference translations. In the example above, the word "the" appears twice in reference 1, and once in reference 2. Thus formula_5. For the candidate translation, the count formula_6 of each word is clipped to a maximum of formula_7 for that word. In this case, "the" has formula_8 and formula_9, thus formula_10 is clipped to 2. These clipped countsformula_10 are  then summed over all distinct words in the candidate. This sum is then divided by the total number of unigrams in the candidate translation. In the above example, the modified unigram precision score would be: In practice, however, using individual words as the unit of comparison is not optimal. Instead, BLEU computes the same modified precision metric using n-grams. The length which has the "highest correlation with monolingual human judgements" was found to be four. The unigram scores are found to account for the adequacy of the translation, how much information is retained. The longer -gram scores account for  the fluency of the translation, or to what extent it reads like "good English". An example of a candidate translation for the same references as above might be: In this example, the modified unigram precision would be, as the word 'the' and the word 'cat' appear once each in the candidate, and the total number of words is two. The modified bigram precision would be formula_14 as the bigram, "the cat" appears once in the candidate. It has been pointed out that precision is usually twinned with recall to overcome this problem , as the unigram recall of this example would  be formula_15 or formula_16. The problem being that as there are multiple reference translations, a bad translation could easily have an inflated recall, such as a translation which consisted of all the words in each of the references. To produce a score for the whole corpus, the modified precision scores for the segments are combined using the geometric mean multiplied by a brevity penalty to prevent very short candidates from receiving too high a score. Let be the total length of the reference corpus, and the total length of the translation corpus. If formula_17, the brevity penalty applies, defined to  be formula_18. (In the case of multiple reference sentences, is taken to be the sum of the lengths of the sentences whose lengths are closest to the lengths of the candidate sentences. However, in the version of the metric used by NIST evaluations prior to 2009, the shortest reference sentence had been used instead.) iBLEU is an interactive version of BLEU that allows a user to visually examine the BLEU scores obtained by the candidate translations. It also allows comparing two different systems in a visual and interactive manner which is useful for system development. Performance. BLEU has frequently been  reported as correlating well with human judgement, and remains a benchmark for the assessment of any new evaluation metric. There are however a number of criticisms that have been voiced. It has been noted that, although in principle capable of evaluating translations of any language, BLEU cannot, in its present form, deal with languages lacking word boundaries. It has been argued that although BLEU has significant advantages, there is no guarantee that an increase in BLEU score is an indicator of improved translation quality.