# From Scores to Predictions in Multi-Label Classification: Neural Thresholding Strategies
###### Abstract
From Scores to Predictions in Multi-Label Classification: Neural Thresholding Strategies  
[PERSON], [PERSON], J. From Scores to Predictions in Multi-Label Classification: Neural Thresholding Strategies Strategies  
[PERSON], D.  
relevant categories, a single image can contain many objects, and a sound can be a mix of acoustic waves from various sources. A typical classification system, while calculating its output (subset of labels) for a given input, internally operates in continuous latent space and provides a certain floating point value--a _score_--for all (or the most promising) labels. The higher the label score, the more appropriate this label is for a given input, according to the classifier. In order to return a set of labels, the classifier must make final decisions about the prediction of labels, basing them on these per-label scores. This stage of making ultimate predictions based on scores is called _thresholding_. All classifiers, implicitly (e.g., [10, 11]) or explicitly (e.g., [17, 18, 19, 20, 21]) have to use some form of thresholding of the calculated per-label scores to obtain the final bipartition of the set of all available labels into subsets of relevant and irrelevant ones. The thresholding phase is typically meant to be as simple as taking labels, where scores exceed a suitably chosen _threshold_ value (e.g., it can be 0.5 if the scores have a probabilistic interpretation, see, e.g., [20, 21]). Thresholding can also be more complex (see Section 3.1), which can lead to overall better label predictions. When there are thousands or even millions of available labels to choose from, the task is known as eXtreme Multi-Label Classification (XMLC) [22]. Such a large number of labels in XMLC makes it a suitable framework for analyzing and even solving other problems closely related to classification, such as recommendation, tagging, and ranking [23, 24, 25, 26]. These are the prevailing applications and use cases of extreme multi-label classifiers. In the ranking, recommendation, and most tagging applications, we care about the top \(k\) outputs of the system, such as the \(k\)-most appropriate adverts, \(k\)-most relevant search engine results, or \(k\)-top-scoring topics describing a given post or tweet (not necessarily the whole subset of relevant tags). In the literature on XMLC (e.g., [25, 26, 27, 28, 29, 30, 31]), the quality of extreme multi-label classifiers is evaluated based on label lists sorted in descending order, according to their real-valued scores calculated by the classifier. The XMLC repository ([[http://manikvarma.org/downloads/XC/XMLRepository.html](http://manikvarma.org/downloads/XC/XMLRepository.html)]([http://manikvarma.org/downloads/XC/XMLRepository.html](http://manikvarma.org/downloads/XC/XMLRepository.html)), accessed on 26 May 2023) curates benchmark results of many extreme multi-label classifiers on a collection of datasets using the following metrics: precision at \(k\) (P@\(k\)), propensity scored precision at \(k\) (PSP@\(k\)), discounted cumulative gain at \(k\) (DCG@\(k\)), normalized discounted cumulative gain at \(k\) (nDCG@\(k\)), etc. The \(k\) values for which performances are reported are as follows: 1, 3, and 5 (e.g., P@1, PSP@3, nDCG@5). The important thing to notice here is that all these metrics are _ranking_ quality metrics, not _classification_ quality metrics, e.g., F1 score [1, 32]. There are a couple of reasons for the choice of ranking-based metrics over classification-based metrics in XMLC [24]. In addition to the aforementioned typical applications of extreme classifiers (recommendation systems), there is also the inevitable problem of missing labels in ground truth predictions in the presence of millions of labels, which makes the ground truth in XMLC less trustworthy than in other tasks; the difference in relevance between true positive prediction (especially for rare labels) and true negative prediction (most of the labels should not be assigned to a given input); and the high imbalance in label sizes.