Differential Evaluation: a Qualitative Analysis of Natural Language Processing System Behavior Based Upon Data Resistance to Processing (2021)

Abstract: Most of the time, when dealing with a particular Natural Language Processing task, systems are compared on the basis of global statistics such as recall, precision, F1-score, etc. While such scores provide a general idea of the behavior of these systems, they ignore a key piece of information that can be useful for assessing progress and discerning remaining challenges: the relative difficulty of test instances. To address this shortcoming, we introduce the notion of differential evaluation which effectively defines a pragmatic partition of instances into gradually more difficult bins by leveraging the predictions made by a set of systems. Comparing systems along these difficulty bins enables us to produce a finer-grained analysis of their relative merits, which we illustrate on two use-cases: a comparison of systems participating in a multi-label text classification task (CLEF eHealth 2018 ICD-10 coding), and a comparison of neural models trained for biomedical entity detection (BioCreative V chemical-disease relations dataset).

## introduction the analysis of nlp system results has mainly focused on evaluation scores meant to rank systems and feed leaderboards. in tasks such as information extraction, text classification, etc., evaluation generally relies on the comparison of a hypothesis (typically a system output) with a gold standard, generally produced through manual annotation. since the muc-6 conference @xcite , the metrics used were created for information retrieval @xcite : recall (true positive rate), precision (positive predictive value) and their harmonic (possibly weighted) mean, the f1score. evaluation scripts are widely available nowadays, for instance those of the conll shared tasks @xcite . these scripts rely on an annotation scheme based on the bio prefix used to specify whether a token is at the beginning, inside or outside of an annotation span, making it a de facto standard for ner evaluation @xcite . many other nlp tasks have developed or used their own metrics, such as accuracy for classification, bleu @xcite for machine translation, rouge for machine translation and text summarization @xcite , word error rate for automatic speech recognition, etc. while evaluation is the key step in shared tasks, developers also need to evaluate the performance of their systems for feature selection or architecture design choices, especially when several systems are combined @xcite . however, scores only are insufficient to capture the behavior of systems and to provide a finergrained analysis of their pros and cons. indeed, though widely used, scores are not free of imperfections, as demonstrated by @xcite who discuss the use of the average to aggregate evaluation scores. they show that very different system behaviors can yield similar scores when using the average and suggest an alternative aggregation mechanism. some researchers also call for going beyond performance scores: ethayarajh and jurafsky (2020) suggest that performance-based evaluation (as promoted by leaderboards) overlooks aspects such as utility, prediction cost, and robustness of models. they recommend considering the point of view of the user of models rather than just performance scores to estimate their relevance. trying to provide a finer understanding of the issues raised by the input text and of the limitations of the evaluated systems, we propose a new qualitative analysis method that takes into account the observed relative difficulty of predicting gold labels for each input. this difficulty is assessed pragmatically based upon the number of systems that predict a gold label (a true positive) for a given input. as a qualitative method, its aim is not to compute an evaluation measure nor to rank systems, but instead to obtain an overview of how different systems achieve the task, and thus understand where their strengths and weaknesses are. after explaining how the method works globally (section 2), we illustrate it with data from two shared tasks from the biomedical domain, one for multi-label classification and another for named entity recognition (section 3), then discuss a few points and directions for future investigation (section 4). ## differential evaluation: highlighting the 'difficulty' of examples our qualitative analysis method, which we call differential evaluation 1 , globally considers the various sets of correct instances ('true positives', or 'gold instances') that were discovered by a set of systems. since the aim of the method is not to produce a ranking, the considered systems can be different systems performing the same task, as in a shared task for example, or different versions of the same system also performing a given task, as in a development context. as input, the algorithm takes a matrix of instances and systems, as shown in figure 1 . for each instance, it then computes how many systems discovered it correctly (i.e., in figure 1 , '762_levodopa' has been discovered by 6 systems, '1034_cyp' has been discovered by 4 systems, etc.) this enables it to compute then how many instances have been detected by all systems, by all systems but one, by all systems but two, etc., and by no system at all. this yields a grouping of instances into bins depending on the number of systems that discovered them. there are as many bins as there are systems plus one for the set of instances that were discovered by none of the systems. bin-1 is the set of instances detected by exactly one system, bin-2 the set of instances detected by exactly two systems, etc.; and bin-0, the set of instances that no system was able to detect (see section 3 for illustrated examples). figure 2 shows the composition of bin-5 in a case where six systems are compared, and displays the percentage coverage of the bin for each system. figure 3 shows a schema of the global scenario of the method. instances in bin-n (where n is the number of considered systems), which holds the set of entities discovered by all systems, can be considered as the easiest to predict, while instances in bin-0, which holds the set of entities that no system was able to detect, can be seen as the most difficult. moretrue positives (tps) are displayed with absolute and relative values (percentage of the number of instances in the bin) in the output matrix, as in figure 4 presents one of the outputs of the method, a heatmap of percentages of system tps relative to the total number of instances in each bin, in this case for the clef ehealth 2018 icd-10 coding task for italian (we analyse this example in detail in section 3.1.1). the first column on the left is bin-0, holding only 0 values as we have said that bin-0 is the bin of instances missed by all systems (as shown by ## experiments in this section, we present insights that can be drawn from the use of differential evaluation on data related to two shared tasks addressing respectively multi-label text classification and named entity recognition, both in the biomedical domain. note that our algorithm processes the systems in the order in which they are presented and that it is not intended to create a new ranking of the systems, but rather to provide more fine-grained information to analyze how a given system has performed or achieved its ranking.