Calibrating Language Models for Neural Ranking under Noisy Supervision with Relaxed Labels (2025)

Abstract: In recent years, we have seen an increased usage of neural ranking models in the information retrieval domain. Although language model-based rankers have shown significant progress in performing ranking tasks, little to no work has addressed the issue of fine-tuning them in the presence of label noise in the training data. In a general learning setting, training models in the presence of noisy labeled data is studied extensively. To this end, confidence calibration approaches have shown significant promise; however, their usage in training neural ranking models is relatively less studied. In this work, we address this gap by adapting and analyzing regularization-based calibration approaches to reduce the effect of label noise in ranking tasks. Specifically, we study label relaxation in neural ranking models. We demonstrate the effectiveness of this approach by performing extensive evaluations comparing the label relaxation approach to standard loss functions. Additionally, we analyze the calibration error associated with the loss functions.After evaluating on five different noise levels, two different ranking models, and four diverse ranking datasets, the results suggest that label relaxation can improve the performance of the ranking models under noisy labels. Furthermore, we find that label relaxation reduces calibration error, although it suggests a better metric to be used for neural ranking models.

## introduction the advancements of language models have enabled their rapid usage in various application domains. one of such prominent application areas is neural ranking wherein the task is to estimate the relevance of several candidate documents or entities based on their relevance to the given query @xcite , which is typically a question presented in a natural language form. with the recent progress in nlp domains, models like bert @xcite have achieved significant progress in capturing the semantic contextual information for a given query. existing works focus on improving the ranking tasks considering several aspects of the learning framework @xcite @xcite @xcite . however, to the best of our knowledge, only a few works considered approaches to develop robust ranking models when noisy labels are prevalent in the training data. label noise in the training data for ranking tasks can be caused due to several reasons. for instance, in a question answering dataset, noise can stem from distant supervision, weakly supervised data generation, bad annotations, among other reasons. such noise can essentially lead to the generation of models with degraded generalization and unstable predictions @xcite @xcite . this issue becomes particularly critical in ranking tasks, where the quality of predictions directly impacts the rank order of documents or entities, thereby affecting the overall effectiveness of the system. furthermore, studying the risks associated with label noise in ranking models is also important, as improper handling of noise can lead to misleading rankings and reduced model reliability. one of the few works in the nlp domain by @xcite studied the robustness of the bert model and showed that in sentence classification tasks, weakly supervised noise can severely degrade the performance of the model. in classification and general learning settings, this problem has been tackled often by using several types of model calibration approaches by @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite . these approaches typically work by ensuring that the confidence of the underlying model in predicting an input instance should also reflect the true likelihood of the prediction. in other words, the model should not confidently predict wrong labels, and in contrast, when predicting the correct labels, it should exhibit sufficient confidence. the idea is to calibrate the overconfident models, which are vulnerable to memorizing incorrect labels @xcite . label smoothing @xcite is considered a standard approach, wherein the idea is to distribute a specific amount (decided based on a hyperparameter) of probability mass taken from the actual label to all the other labels. although label smoothing can be quite effective, it still relies on precise probabilistic labels, which might degrade the generalization performance @xcite . therefore, @xcite proposed label relaxation, which considered a set of candidate distributions, instead of a single smoothed distribution. label relaxation essentially replaces fixed (and possibly incorrect) label distributions with sets of plausible distributions, thereby allowing the learner to learn a bounded range of acceptable target labels. in this paper, we tackle label noise in order to develop robust ranking models in the fine-tuning step. we consider two different directions, considering model calibration techniques. firstly, we introduce label relaxation into the ranking paradigm as a principled approach to fine-tune models under noisy conditions. more specifically, considering the pairwise ranking loss, we integrate relaxation in several widely used neural ranking models. then we compare the performance of two different calibration approaches, i.e., smoothing with relaxation, to gain some initial insights into which approach performs better. secondly, by analyzing the calibration error, we aim to understand how well the models' confidence reflects their true performance under noisy conditions. thus, we assess the associated risks of poor confidence calibration, which can lead to suboptimal ranking decisions. we model the label noise in the ranking tasks by considering a proximity-aware approach. experimental results considering these two different calibration approaches, 5 different noise levels, 4 diverse datasets, and two ranking models suggest the potential of relaxation under label noise in fine-tuning ranking models. our contributions can be summarized as, ## related work ranking models as mentioned beforehand, with the advancement of language models, we have seen significant progress in the domain of neural ranking @xcite @xcite @xcite . one of the first works was sentence-bert @xcite , which adapted the bert architecture into a siamese network to produce sentencelevel embeddings. nogueira and cho (2019) extended this idea by further showing that bertbased models could be fine-tuned specifically for passage re-ranking. this has shown substantial improvements in retrieval performance. subsequent work has continued to explore more scalable and generalizable ranking solutions. for example, @xcite introduced a family of embedding models trained with contrastive learning on massive collections of text pairs. cross encoders @xcite are shown to outperform the previous approaches in re-ranking tasks at the cost of a high training time. calibrated loss calibration refers to the alignment between the model's predicted confidence and the actual likelihood of correctness. a perfectly calibrated model assigns a probability of 0.7 to a prediction if, on average, 70% of such predictions are correct. there exist two categories of approaches that perform model calibration, (i) post-hoc and (ii) regularization-based. in order to perform calibration, post hoc approaches adjust the output predictions @xcite @xcite . however, this requires additional validation on held-out datasets. furthermore, this approach assumes the training and test distributions to be the same, which often is not. regularization-based approaches do not require any extra data and perform calibration during the training step while computing the loss @xcite @xcite . label smoothing is often used as a standard technique to soften the hard target labels by redistributing the probability mass to non-target labels @xcite . however, typical smoothing distributes the probability mass uniformly. there exist approaches that essentially follow more advanced approaches, such as bootstrapping techniques @xcite , wherein a self-supervised approach is used to distribute the probability mass. self-distillation and model distillation approaches also follow a similar approach by replacing the hard labels with the soft ones from the teacher model @xcite . although in typical classification settings such approaches have been extensively studied, in the nlp domain, this is relatively less explored. @xcite introduced confidence-aware label smoothing for alignment tasks considering language models and have shown the potential of the calibration approaches. @xcite also showed the potential of several calibrated approaches in fine-tuning language models. note that we consider the idea of label relaxation introduced by @xcite wherein a single fixed target distribution is replaced with a set of candidate probability distributions. another work by @xcite proposed relaxed labels in metric learning, which relaxes binary pairwise relation labels by replacing them with continuous similarity weights from a source embedding space. alike our work, @xcite also study learning to rank from relevance judgment distributions. they use kl divergence to align model predictions with empirical distributions, thereby directly capturing inter-annotator disagreement. note that we assume that such distributions are not consistently available across ranking datasets. instead, we propose label relaxation, which defines a credal set of admissible label distributions. this approach allows us to model epistemic uncertainty and mitigate label noise without requiring multiple annotations per query. to the best of our knowledge, this is the first work that studies calibration in this context. ## calibration in ranking model in our work, we consider two different ranking approaches, both of which fall under the category of bi-encoder models. more specifically, we use the pre-trained bert @xcite and the e5 @xcite models. e5 model @xcite encodes both the query and candidate entities using a language model, producing dense vector embeddings. these token-level embeddings are then averaged via a pooling layer to obtain fixed-size vectors. finally, a scoring function computes a probability score ŷ ∈ [0, 1], reflecting the likelihood that the candidate entity is the correct match for the query. bert model follows a similar approach to e5 model in performing ranking tasks. however, the only difference is that bert is pre-trained to perform binary relevance classification tasks between a query and a document @xcite , whereas in contrast, e5, is additionally pre-trained on several ranking tasks. next, we formalize the ranking task and subsequently define the term label relaxation in this context. note that the document ranking step often consists of two different steps, namely retrieval and ranking. in this work, we only consider the ranking stage. in a typical supervised ranking setup, each training sample consists of a query q and a set of candidate documents defined as d @xmath0, d 2 , . . . , d k }, with only one document labeled as relevant @xcite @xcite . herein, we assume the set of candidate documents is already correctly retrieved by a retrieval model. typically, within a training step, a batch of queries and corresponding documents are presented, wherein the size of the batch is determined by the training configuration. for a batch consisting of n queries and k candidates per query, we define the label matrix @xmath1, 1} n ×k as follows. herein r q i ⊆ d q i is the set of relevant documents for query q i , typically of cardinality 1. let us assume the ranking model as f , then it produces a score f (q i , d j ) for candidate d j to query q i . in the ranking tasks, the goal is to ensure that relevant documents are scored higher than non-relevant ones. to achieve this, the pairwise ranking loss is often used herein. let (q i , d j + , d j -) denote a training triplet, where d j + is a document relevant to query q i , and d j -is a non-relevant (or less relevant) document, i.e., d j -∈ d q i \ r q i . since we adopt an in-batch negative sampling strategy, we have a training batch containing n queries, each associated with k candidate documents (one relevant and k -1 non-relevant). for every query q i , the model compares the relevant document d j + against all k -1 non-relevant candidates in the batch. the pairwise ranking loss scores a relevant document so that it exceeds that of a non-relevant one by γ > 0, a defined margin. this can be defined as, here, j + denotes the index of the relevant document among the k candidates for q i . this loss penalizes cases where a non-relevant document scores too closely or higher than the relevant one. label smoothing is a regularization technique that softens target labels to mitigate overconfidence @xcite . rather than encoding the correct document as a one-hot vector, label smoothing redistributes a small fraction of the probability mass across all other candidates. formally, the smoothed label distribution ỹ ∈ [0, 1] n ×k can be defined as follows. as mentioned previously, we consider in-batch pairwise training; therefore, we use the smoothed score ỹi,j in place of a hard label of 1 in the marginbased loss, resulting in the label-smoothed pairwise ranking loss as follows. label relaxation in pairwise ranking unlike label smoothing, which redistributes the probability mass of the target label uniformly, label relaxation replaces the target with a set of plausible distributions that reflect epistemic uncertainty @xcite . this can help to reduce the uncertainty regarding the correct label. label relaxation introduces a relaxed set of acceptable target distributions parameterized by α ∈ [0, 1]. we define q α as the set of all relevance probability distributions p satisfying p(+) ≥ 1α and p(-) ≤ α. while q α is a set, in our implementation, we instantiate it via a canonical representative distribution p r for loss computation as this set essentially defines that the relevant document should be preferred with high probability; this does not pertain to a specific numeric value; rather, we allow the model to match any distribution within q α . the model can then generate any label that falls inside this plausible region, without penalizing it for deviations that are within the acceptable uncertainty bounds. the relaxation parameter α ∈ [0, 1] controls the degree of permissible deviation from the one-hot target. we select α via validation set performance for each dataset. next, we apply the kl divergence on the predicted scores and the distribution q α . note that, since f (q i , d j ) is a relevance score, for kl divergence we need to first convert it into a probability distribution over candidates using a softmax normalization, let us call this pi (j). afterwards, the label relaxation loss compares the predicted distribution pi (j) with the distribution q α , denoted as p r . since we have two different distributions, instead of using any margin-based loss, it is computed using kl divergence as follows. herein p r can be defined as follows. since the prediction is probabilistic, kl divergence penalizes differences in a way that reflects confidence mismatches, i.e., confident, however, when predictions are wrong, are more penalized than uncertain ones. below we give an example to explain this clearly. example consider (q i , d j + , d j -) that has been judged by multiple annotators. suppose 80% of them preferred d + i,j , while 20% preferred d j -. this uncertainty would be difficult to capture with one-hot labels or a uniform label smoothing approach. rather than forcing the model to match a hard label [1, 0], or smoothing it arbitrarily to something like [0.9, 0.1], label relaxation allows us to model more concisely. let us choose the relaxation parameter as α = 0.2, with which we can define the relaxed set as q 0. @xmath2: p(+) ≥ 0.8, p(-) ≤ 0.2 . therefore, p r (+) = 0.8, p r (-) = 0.2. this implies that if the model predicts any values between 0.8 and 1.0, the associated loss would be considered as 0. otherwise, using the kl-divergence loss, the model is then trained to minimize the divergence. this formulation respects the ambiguity in the supervision and allows the model to output calibrated probabilities that reflect uncertainty, rather than overconfident or artificially smoothed predictions. as a result, label relaxation not only improves robustness to label noise but also enhances the model's ability to represent uncertainty, which is critical in real-world applications such as qa, recommendation, and information retrieval. calibration error in ranking is typically measured using expected calibration error (ece) to evaluate the calibration of the model's probability outputs @xcite . calibration in this context refers to the agreement between predicted probabilities and the actual likelihood of correctness. more specifically, considering document ranking tasks, the goal is to ensure that the probability assigned to a document reflects its actual relevance to the query. a well-calibrated ranking model would assign a probability close to 1 to relevant documents and a probability close to 0 to non-relevant ones. to define it more formally, let us assume r q i be the set of relevant documents for query q i , and ## evaluation datasets and models for evaluation, we used four datasets, namely, (i) aida @xcite , (ii) mintaka @xcite , (iii) lc-quad 2.0 @xcite , and (iv) ms marco @xcite . datasets (i)-(iii) pertain to entity ranking tasks, and the ms marco dataset corresponds to document ranking tasks. the aida dataset contains news articles and entities that are linked to wikipedia. mintaka is generated through crowd workers, wherein the entities in question-and-answer pairs are linked to the wikidata knowledge graph. lc-quad 2.0 (or in short, lc-quad) is also generated through crowd workers, but, contains sparql queries. finally, the ms marco dataset is frequently used for diverse tasks to perform question answering, passage ranking, and document ranking. both lc-quad and ms marco are question-answering datasets. the ranking models are taken from their original implementation given in hugging face, bert 2 , e5 foot_2 . thereafter, using in-batch negative sampling @xcite , we fine-tuned them on the datasets described above. each of the models is trained using the default learning rates and the parameters considering 10 epochs. finally, the evaluation is performed by using the model's embeddings indexed by using faiss indexing api @xcite . note that, for ms marco, while computing the hard negatives, we randomly selected 10,000 negative documents from the whole corpus at a time. finally, for fine-tuning the models, we used a server with 128 gb of ram and an nvidia rtx h100 gpu with 80 gb of ram. label noise in this work, we consider semanticaware label noise, wherein instead of flipping labels randomly, our approach considers a more realistic scenario. more specifically, for a given ratio of noise addition, we intentionally introduce an error by replacing the correct (relevant) document with a non-relevant one that is semantically very similar to the former. this is done by first randomly choosing a subset of queries from the training batch. then, for each selected query, the correct answer is changed and replaced with another candidate that is closest in meaning, based on a similarity score between the original relevant document and all the other candidates. therefore, we simulate noisy supervision by replacing the correct document with a semantically similar but non-relevant one for a subset of queries. we vary the noise proportion across five levels: 0% (no noise) to 5% of the training labels, following a progressive corruption scheme. concretely, at 2% noise, 2% of the queries in the training set have their relevant document replaced. note that although we do not perform human verification to find out the plausibility of the noisy labels, we still ensure their semantic plausibility by selecting replacements based on embedding similarity 4 . additionally, some datasets, for instance, mintaka, originate from multiple human annotators, which in principle could provide empirical relevance distributions. however, the versions we use in our evaluation only provide single canonical labels. for consistency across benchmarks, we therefore did not compare against models trained on empirical annotation distributions. ## conclusion & future directions in this work, we have studied label relaxation considering the neural ranking models in performing document ranking tasks. to this end, we first formally define the label relaxation in the context of the ranking task. afterwards, we integrate it into bi-encoder ranking models. additionally, to find out whether label relaxation can mitigate the impact of label noise in fine-tuning neural ranking models, we conducted extensive evaluations considering 2 different bi-encoder models, 4 different ranking datasets, and 5 different noise levels. we also compare our results to the popular label smoothing calibration approach. the results of our evaluation suggest that label relaxation can indeed be helpful in fine-tuning ranking models when label noise is prevalent in the ranking datasets. however, our findings also suggest that label relaxation is effective on the e5 model, which is extensively weakly supervised pre-trained on the ranking tasks. on the other hand, if a pre-training on the ranking tasks is not performed, the results do not improve. additionally, we find that the ece might not be suitable to measure the calibration of the ranking models under noise and memorization errors could be helpful to get better insights. we believe label relaxation has a lot of potential to build well-calibrated models. we can envisage works that would explore adaptive label relaxation approaches that adjust relaxation based on model confidence or noise estimates, and investigate their effects on model calibration. furthermore, extending relaxation to cross-encoder and llm-based rankers, and studying new measures to compute calibration error in the context of document ranking, could also be a potentially interesting direction.