Perplexity Trap: PLM-Based Retrievers Overrate Low Perplexity Documents
Authors: Haoyu Wang
Sunhao Dai
Haiyuan Zhao
Liang Pang
Xiao Zhang
Gang Wang
Zhenhua Dong
Jun Xu
Ji-Rong Wen
Venue: ICLR 2025 Poster (2025)
Keywords: Source Bias
LLM-Generated Content
Infomation Retrieval
Large Language Models
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We verify the causal effect of document perplexity on estimated relevance scores in PLM-based retrievers to explain the cause of source bias.

Abstract: Previous studies have found that PLM-based retrieval models exhibit a preference for LLM-generated content, assigning higher relevance scores to these documents even when their semantic quality is comparable to human-written ones. This phenomenon, known as source bias, threatens the sustainable development of the information access ecosystem. However, the underlying causes of source bias remain unexplored. In this paper, we explain the process of information retrieval with a causal graph and discover that PLM-based retrievers learn perplexity features for relevance estimation, causing source bias by ranking the documents with low perplexity higher. Theoretical analysis further reveals that the phenomenon stems from the positive correlation between the gradients of the loss functions in language modeling task and retrieval task. Based on the analysis, a causal-inspired inference-time debiasing method is proposed, called **C**ausal **D**iagnosis and **C**orrection (CDC). CDC first diagnoses the bias effect of the perplexity and then separates the bias effect from the overall estimated relevance score. Experimental results across three domains demonstrate the superior debiasing effectiveness of CDC, emphasizing the validity of our proposed explanatory framework. Source codes are available at https://github.com/WhyDwelledOnAi/Perplexity-Trap.

Published as a conference paper at ICLR 2025
PERPLEXITY-TRAF: PLM-BASED RETRIEVERS OVERRATE Low PERPLEXITY DOCUMENTS
Haoyu Wang Sunhao Dai's Haiyuan Zhao', Liang Pang?, Xiao Zhang'
Gang Wang', Zhenhua Dong 3, Jun Xu' Ji-Rong Wen'
GGaoling School of Artificial Intelligence, Renmin University of China, Beijing, China "CAS Key Laboratory of AI Safety, Institute of Computing Technology, Beijing, China Huawei Noah's ' Ark Lab, Shenzhen, China (wanghacyu0922 924, sunhaodai junxu ) @ Bruc.edua
ABSTRACT
Previous studies have found that PLM-based retrieval models exhibit a preference for LLM--genrated content, assigning higher relevance scores these documents even when their semantic quality is comparable to numan-written ones. This phenomenon, known as source bias, threatens the sustainable development of the information access ecosystem. However, the underlying causes of source bias remain unexplored. In this paper, we explain the process of information retrieval with causal graph and discover that PLM-based retrievers learn perplexity features for relevance estimation, causing source bias by ranking the documents with low perplexity higher. Theoretical analysis further reveals that the phenomenon stems from the positive correlation between the gradients of the loss functions language modeling task and retrieval task. Based on the analysis, causal-inspired inference- time debiasing method is proposed, called Causal Diagnosis and Correction (CDC). CDC first diagnoses the bias effect of the perplexity and then separates the bias effect from the overall estimated relevance score. Experimental results across three domains demonstrate the superior debiasing effectiveness of CDC, emphasizing the validity of our proposed explanatory framework 1
INTRODUCTION
The rapid advancement of large language models (LLMs) has driven a significant increase in AI- generated content (AIGC), leading to information retrieval (IR) systems that now index both human- written and LLM- L.ee-genertted contents (Cao et al. 2023 Dai et al., 2024b 2025). However, recent Xu et al.) 2024) have uncovered that Pretrained Language Model
studies (Dai et al. 2024a c (PLM) based retrievers (Guo al. 2022 Zhao et al., 2024) exhibit preferences for LLM JLM-ggnnratedd documents, ranking them higher even when their semantic quality is comparable to human--ritten content. This phenomenon, referred to as source bias, is prevalent among various popular PL.M-based retrievers across different domains (Dai al. 2024a). If the problem is not resolved promptly, human authors' creative willingness will be severely reduced, and the existing content ecosystem may collapse. So it's urgent comprehensively understand the mechanism behind source bias, especially when the amount of online AIGC is rapidly increasing (Burtch et al. 2024 Liu et al. 2024
Existing studies identify perplexity (PPL) a key indicator for distinguishing between LLM- generated and human-written written contents (Mitchell et al. [2023] Bao et al. 2003). Dai et al. (2024c) find that although the semantics of the text remain unchanged, LLM-rewritten documents possess much lower perplexity than their human-written counterparts. However, it's still unclear whether document perplexity has causal impact on the relevance score estimation of PLM-based retrievers (which may lead to source bias), and if so, why such causal impact exists.
In this paper, we delve deeper into the cause of source bias by examining the role of perplexity in PLM-based retrievers. By manipulating sampling temperature when generating with LLMs, we
"Equal contributions.
1 CCoresponding author.
Codes available hhcppes///ghhhbbcc////heeeelllle

---

Published as a conference paper at ICLR 2025
observe negative correlation between estimated relevance scores and perplexity. Inspired by this, construct causal graph where document perplexity plays as treatment and document semantic plays confounder (Figure 2). We adopt two-stage least squares (2SLS) regression procedure (Angrist and Pischke| 2009 Hartford et al. 2017) to eliminate the influence of confounders when estimating this biased effect, the experimental results indicate the effect is significantly negative. Based these findings, the cause of source bias can be elucidated as the unexpected causal effect of perplexity on estimated relevance scores. For semantically identical documents, the documents with low perplexity causally get higher estimated relevance scores from PLM- based retrievers. Since LLM-ge -e..-eerrttedd documents typically have lower perplexity than human-written ones, they receive higher estimated relevance scores and are ranked higher, leading to the presence of source bias.
To further understand why estimated relevance scores of PLM-based retrievers are influenced by perplexity, we provide theoretical analysis for the overlap between masked language modeling (MLM) task and mean-pon peann-polling retrieval task. Analysis in the linear decoder scenario shows that, the retrieval objective's ' gradients are positively correlated to the language modeling gradients. This correlation causes the retrievers to consider not only the document semantics required for retrieval but also the bias introduced by perplexity. Meanwhile, this correlation further explains the trade-off between retrieval performance and source bias observed in previous study (Dai et al.) 2024a): the stronger the ranking performance of the PLM-based retrievers, the greater the impact of perplexity.
Based on the analysis, we propose an inference-time debiasing method called CDC (Causal Diagnosis and Correction). With the proposed causal graph, separate the causal effect of perplexity from the overall estimated relevance scores during inference, achieving calibrated unbiased relevance scores. Specifically, CDC first estimates the biased causal effect of perplexity on a small set of training samples, which is then applied to de-bias the test samples at the inference stage. This debiasing process is inference-time and can be seamlessly integrated into existing trained PLM-based retrievers. We demonstrate the debiasing effectiveness of CDC with experiments across six popular PLM-based retrievers. Experimental results show that the estimated causal effect of perplexity can generalized to other data domains and I. L.Mss, highlighting its practical potential in eliminating source bias.
We summarize the major contributions of this paper as follows:
We construct a causal graph and estimate the causal effect through experiments, demonstrating that PLM-based retrievers causally assign higher relevance scores to documents with lower perplexity, which is the cause of source bias.
We provide a theoretical analysis explaining that the effect of perplexity in PI-M-based retrievers due to the positive correlation between objective gradients of retrieval and language modeling.
We propose CDC for PLM-based retrievers to counteract the biased effect of perplexity, with experiments demonstrating its effectiveness and generalizability in eliminating source bias.
RELATED WORK
With the rapid development of LLMS Zha et al.) 2023) the internet has quickly integrated huge amount of AIGC (Cao et al. 2023 Dai et al. 2024b; 2025). Potential bias may occur when these generated contents are judged by neural networks as a competitor together with human works. For example, Dai et al. (2024c) are the first to highlight a paradigm shift in information retrieval (IR): the content indexed by IR systems is transitioning from exclusively human-writter corpora coexistence of human- wrrtten and LLM-generater corpora. They then uncover an important finding that mainstream neural retrievers based on pretrained language models (PLMs) prefer LLM-geenrrate content, a phenomenon termed source bias (Dai et al. 2024a|c). Xu et al. (2024) further discover that this bias extends to text-image retrieval, and similarly, other works further observe the existence of source bias in other IR scenarios, such recommender systems (RS) (Zhou et al. 2024). retrieval- augmented generation (RAG) (Chen et al.) 2024) and question answering (QA) (Tan et al. 20244). In the context of LLMssaa-judger similar bias is discovered as self-enhancement bias (Zheng et al.)
2024), likelihood bias (Ohi et al.] (2024), and familiarity bias (Stureborg et al. 2024). where LLM overates AIGC when serving as a judge.
Existing works provide intuitive explanations suggesting that this kind of bias may stem from coupling between neural judges and LLMSs DDai et al. 2024c Xu et al., 2024), such similarities in model
2

---

Published as a conference paper at ICLR 2025
Pearson Cocficiint: 4.99
Pearson Cocficcin::
Peurson Cocflicion: 0.85
1.770
Pepledyy
- Setemmoom
0.579
1,690
Prppendy
-0.985
Soore
Peplandy
0.703
1,760
0..77
Pempextry
1,635
00,90
Pemlecity
22000
00,99
1,750
0,373
0.975
2000
0,500
Rolev
1,740
1
1,700
0.970
1,90
1.0
0.0
04
Tempesttune
Tomperttuee
Temperature
(a) DL19
(b) TREC-COVID
(c) SCIDOCS
Figure 1: Perplexity and estimated relevance scores of ANCE on positive query- deremmd pairs in three dataset, where documents generated by LLM rewriting with different sampling temperatures. The Pearson coefficients highlight the significant negative correlation between the two variables.
architectures and training objectives. However, the specific nature of this coupling, how it operates to cause source bias, and why it exists remains unclear. Ohi et al. (2024) find the correlation between perplexity and bias, while our work is the first systematically analyze the effect of perplexity for neural models' preference. Given that both PLMs and LLMs are highly complex neural network models, investigating this question is particularly challenging and difficult.
ELUCIDATING SOURCE BIAS WITH CAUSAL GRAPH
This section first conducts intervention experiments to illustrate the motivation. Subsequently, we construct causal graph explain source bias and demonstrate the rationality of the causal graph.
3.1 MOTIVATION: INTERVENTION EXPERIMENTS ON TEMPERATURE
Previous studies have revealed significant difference in the perplexity (PPL) distribution between LLM-generated content and that PPL might be key indicator for analyzing the cause of source bias (Dai et al. 2024c). To verify whether perplexity causally affects estimated relevance scores, we use LLMs (in following chapters the LLMs we use are Llama2-7B-chal (Touvron et al.) 2023) unless emphasized) to generate documents with almost identical semantics but varying perplexity, where semanties are expected as the only associated variable when retrieval.
-wiiten content (Mitchell et al. 2023 Bao et al. 2023 suggesting
Specifically, we manipulate the sampling temperatures during generation to obtain LLM-gencrated documents with different PPLs but similar semantic content. Following the method of Dai al. (204cc, we use the following simple prompt: "Please rewrite the following text: (human-write written text/", We also recruit human annotators to conduct evaluations to ensure the quality of the generated LLM content. The results, shown in Appendix E.2. indicate that there are fewer quality discrepancies between documents generated at different sampling temperatures compared to the original human- written documents. This ensures the reliability of the subsequent experiments.
We then explore the relationship between perplexity and estimated relevance scores on the corpora generated with different temperatures, where perplexity is calculated by BERT masked language modelling following previous work (Dai et al. 2024c). Figure presents the average perplexity and estimated relevance scores by ANCE across three datasets from different domains. As expected, lower sampling temperatures result in less randomness LLM M.M--geerrtedd content and thus lower PPI.. Meanwhile, we find that documents generated with lower temperatures were also more likely to be assigned higher estimated relevance scores. The Pearson coefficients for the three datasets are all below -0.8, emphasizing the strong negative linear correlation between document perplexity and relevance score. Similar results for other PLM-based retrievers are provided in Appendix E.2.2
Since document semantics remain unchanged during rewriting, the synchronous variation between document perplexity and estimated relevance scores reflects a causal effect. These findings offer an intuitive explanation for source bias: LLM-generated content typically has lower PPL, and since documents with lower perplexity are more likely to receive higher relevance scores, -LM--gnerrted content is more likely be ranked highly, leading to source bias.
3

---

Published as a conference paper at ICLR 2025
3.2 CAUSAL GRAPH FOR SOURCE BIAS
Inspired by the findings above, we propose a causal graph to elucidate source bias (Fan et al. 2022
as illustrated in Figure 2 Let Q denotes the query set and C' denote the corpus. During the inference stage for certain PLM-based retriever, given a query q E Q and a document d E C, the estimated relevance score & R is simultaneously determined by both the golden relevance score R E R and document perplexity Pa E RR.. Note that the fundamental goal of IR is to calculate the similarity between document semantics Ma and query semantics Mg for document ranking, Ry.d + Rg a is considered an unbiased effect, while the influence of Pa + Rqqa is considered as a biased effect. Subsequently, we explain the rationale behind each edge in the causal graph as follows:
First, let the document source Sa is binary variable where Sa = denotes the document is generated by LLM and S 0 denotes the document is written by human. As suggested in (Dai et al. 2024c), LLM- generated documents through rewriting possess lower perplexity than their original documents, even though there is no significant difference in their semantic content. Thus, an edge Sa + Pa exists. This phenomenon can be attributed to two main reasons: (1) Sampling strategies aimed at probability maximization, such as greedy algorithms, discard long-tailed documents during LLM inference. More detailed analysis and verification can be found in (Dai et al. 2024c). (2) Approximation error during LLM training causes the tails of the document distribution to be lost (Shumailov et al. 2023).
Sa Source
Pa Porplecty
Bias Pffect
M Semartic
R
Vrabion Efflect
q.a
Relevance Soore
Ra.d
Belevsacce
M QUary
Next, the document semantics Ma reflect the topics of the document d, including domain, events, sentiment information, and so on. Since documents with different semantic meanings convey different amounts of information, their difficulties in masked token prediction vary. This means that different document semantics lead different document perplexities. For example, colloquial conversations are more predictable than research papers due to their less specialized vocabulary. Thus, the content directly affects the perplexity, establishing the edge Ma + Pa-
Semaniic
Figure 2: The proposed causal graph for explain- ing source bias.
Finally, as retricval models are trained to estimate ground-truth relevance, their outputs are valid approximations of the golden relevance scores, making Ma + Rg, +- M, natural unbiased effect. However, retrieval models may also learn non-causal features unrelated to semantic matching, especially high-di high-dimensiooal features in deep learning. According findings in Section 3.1] document perplexity Pa has emerged as potential non-causal feature learned by PLM-based retrievers, where higher relevance estimations coincide with lower document perplexity. Moreover, Since document perplexity is determined at the time of document generation, which temporally predates the existence of estimated relevance scores, document perplexity should be a cause rather than a consequence of changes in relevance. Hence, biased effect of Pa + Rigd exists.
3.3 EXPLAINING SOURCE BIAS VIA THE PROPOSED CAUSAL GRAPH
Based on the causal graph constructed above, source bias can be explained as follows: Although the content generated by LLMs retains similar semanties the human-wrilten content, LLM--ennrated content typically exhibits lower perplexity. Coincidentally, retrievers learn and incorporate perplexity features into their relevance estimation processes, consequently assigning higher relevance scores to L.MM-generated documents. This leads to the lower ranking of human-written documents.
is worth noting that source bias is an inherent issue in PLM-based retrievers. Before the advent of LLMs, these retrievers had already learned non-causal perplexity features from purely wwritenn corpora. However, because the document ranking was predominantly conducted on human-wrilten corpora, the relationship between PLM-based retrievers and perplexity was not evident. As powerful LLMs have become more accessible, the emergence of LLM--genrateed content has accentuated the perplexity effect. The content generated by LLMs exhibits perceptibly different perplexity distribution compared to human human-written written content. This disparity in perplexity distribution causes documents from different sources to receive significantly different relevance rankings.
4

---

Published as a conference paper at ICLR 2025
Table Quantified causal effects (and corresponding p-value) for document perplexity on estimated
relevance scores via two-stage regression. Bold indicates that the estimate can pass significance test with p-value< 0.05. Significant negative causal effects are prevalent across various PLM-based retrievers in different domain datasets.
Dataset
BERT -9.32 (1e-4)
RoBERTa
ANCE
TAS-B B
Contriever
coCondenser
DL19
-28.15 (2e-12)
-0.52 (9e-3)
-0.96 (le-2)
-0.02 (0.33)
-0.69 (3e-2)
TREC-COVID
-1.69 (2e-2)
2.42 (8e-2)
0.09 (0.21)
-0.48 (6e-3)
-0.05 (7e-7)
-0.32 (8e-3)
SCIDOCS
-2.44 (6c-2)
-6.42 (2e-3)
-0.23 (0.15)
-0.39 (0.10)
-0.02 (0.24)
-0.26 (0.41)
EMPIRICAL AND THEORETICAL ANALYSIS ON THE EFFECT OF PERPLEXITY
In this section, we conduct empirical experiments and theoretical analysis substantiate that PLM- based retrievers assign higher relevance scores to documents with lower perplexity.
4.1 EXPL.ORNNG THE BIASED EFFECT CAUSED BY PERPLEXITY
4.1.1 ESTIMATION METHODS
From the temperature intervention experiments in Section 3.1 we observe a clear negative correlation between document perplexity and estimated relevance scores. Despite human evaluation allows us to largely confirm that document semantics Md generated from different temperatures are almost the same, estimating the biased effect of Pa - Raqd directly is problematic due to inevitable minor variations in document semantics, which, though subtle, are significant in causal effect estimation. From the causal view, to robustly estimate the causal effect of Pa -> Ry.d: the document semantics Ma. query semantics M, and golden relevance scores Ry considered as confounders. Therefore, directly estimating this biased causal effect is not feasible without addressing this confounding factor.
We use 2SLS based on instrumental variable (IV) methods (Angrist and Pischke 2009 Hartford et al., 2017) to more accurately evaluate the causal effect of document perplexity on estimated relevance scores, more details about the method can be found in Appendix [D] According to the causal graph, document source Sa serves IV for estimating the effect of Pa Ry The IV is independent of confounders: query semantics My, document semantics Md. and golden relevance scores Ra
In the first stage of the regression, on document source Sa:
linear regression to predict document perplexity Pa based
Pa = BISa Pa
(I)
where Pa is independent with document source S₂ and therefore depends solely document semantics Mg. As a result, we obtain coefficient 81 and the predicted document perplexity Pa. In the second stage, we substitute Pa with Pa = S1Sa estimate the predicted relevance score Rg from the certain PLM-based based retrievers:
Ry = ByPa + Rq.d.
where residual term Ry represents the part of the estimated relevance scores that can't be explained by document perplexity. Since Pa independent of document semantics Ma, the estimated coefficient B2 can accurately reflect the causal effect of perplexity on estimated relevance scores.
4.1.2 EXPERIMENTAL RESULTS AND ANALYSIS
In this section, we apply the causal effect estimation method described previously to assess the impact of document perplexity Pa on the estimated relevance score Rg
Models. To comprehensively evaluate this causal effect, select several representative PLM-based retrieval models from the Cocktail benchmark (Dai et al. 2024a), including: (1) BERT (Devlin et al.
2 (2) RoBERTa (Liu et al. 2019); ; (3) ANCE (Xiong et al. (2020); (4) TAS-B (Hofstätter et al.
202I); (5) Contriever (Zacarr et al. 2022); (6) coCondenser (Gao and Callan 2022). We employ the
officially released checkpoints. For more details, please refer to Appendix
E.1
5

---

Published as a conference paper at ICLR 2025
Datasets. We select three widely-used IR datasets from different domains ensure the broad applicability of our findings: (1) DL19 dataset (Craswell et al. 2020) for exploring retrieval across miscellaneous domains. (2) TREC-COVID dataset (Voorhees et al. 202T focused biomedical information retrieval. (3) SCIDOCS (Cohan et al. (020) dedicated to the retrieval of scientific scholarly articles. Given that source bias arises from the ranking orders of positive samples from different sources, we only compare the estimated relevance scores of human-written and LLM- generated relevant documents against their corresponding queries.
Results and Analysis. The results across different datasets and different PLM-based retrievers are shown in Table [], As we can see, in most cases, perplexity exhibits a consistently negative causal effect on relevance estimation, with documents of lower perplexity more likely to receive higher relevance scores. Although this causal effect relatively weak, it is statistically significant, with p-values < 0.05 in most instances. We also explore whether this causal effect changes with different sampling temperature. Results in Appendix Table 5 indicate that 32 is robust for temperature changes, that is, this causal effect is independent with generation temperature. This finding is crucial retrieval tasks emphasize the relative ranking of relevance scores rather than their absolute values. Even a slight preferential increase in estimated relevance scores for -LMgeeerrteed content human-writter content will lead consistent trend of higher rankings for LLM--generteed documents by PLM-based retrievers, further confirming the observations in Figure
Finding 1: For PLM- based retrievers, document perplexity has a causal effect on estimated relevance scores. Lower perplexity can lead to higher relevance scores.
4.2 ANALYZING MECHANISM BEHIND THE BIASED EFFECT
4.2.1
WHY PERPLEXITY AFFECTS -BASED RETRIEVERS?
In Section our empirical experiments have confirmed that PLM-based retrievers take perplexity features into account for document retrieval. However, the reason why perplexity-related features play a role, particularly when these models are primarily designed for document ranking, remains unclear. Considering that PLM--asedd retrievers are generally fine-tuned from PLMs on retrieval tasks, we delve into the relationship between the mask language modeling task in the pre-training stage and the mean-pooling document retrieval task in the fine-tuning stage. Our formulation are as follows and explanations can be found in Appendix C.I
Model Architecture. To simplify our analysis, we assume common architecture for PLM--bssed retrievers, consisting of an encoder (t; 0) TLXD + R and a one-layer decoder g(;; W) (=:W), where T denotes the set composed of one-hot vectors, L is the length of query or document, D is the dictionary size, N is the dimension of embedding vector, and o(-) maps real vectors to simplexes. For the ease of qualitative analysis, we replace softmax operation with linear operation, and zW is assumed positive to ensure the well-definition of the probability distribution.
Masked Language Modeling (MLM) Task. The PLM is initially pre-trained on the MLM task with E ( @ logg( ( where @ denotes the Hadamard product, 11, means averages over the length of the documents, [d log (f (dd))] is the expression of CrossEntropy using one-hot vectors.
CrossEntropy loss: Li (d)
Document Retrieval Task. estimates the relevance for given query query-document pairs by computing the dot product of the document embedding vectors dmmb f(d, 9) and query emhedding vectors eb = f(q. 0). Without loss of generality, we assume ||dpmbb]2 ||2 = 1, 1 1= "....,,, T., which means the embeddings of each token is normalized. The loss function can be written as (d,d, q) = -tr((] mmm
In the fine-tuning stage for the document retrieval task, the retrieval model
(emmbb((] ( rmb)) where 1 is the mean pooling operation of the embeddings over the document length L.
With the formulation above, we further explore the theoretical underpinnings of why perplexity influences retrieval performance by examining the gradients of the loss functions for both the MLM task and the document retrieval task, as shown in the following Theorem
Theorem 1. Given the following three conditions:

---

Published as a conference paper at ICLR 2025
Representation Collinearity: the embedding vectors of relevant query-document pairs are collinear after mean pooling, i.e.,
lLxLf(q) x
Semi- OotioOathogonal Weight Matrix: the weight matrix of the decoder semi-orthogonll, i.e.,
WW IN.
Enccoderddecoder Cooperation: fine-tuning does disrupt the corresponding function between encoder and decoder, i.e,
f(d) = gd)
Then there exists a matrix K-
Ak [(1-ki)
ER
Ed (demb Witd which satisfies
In
0L2 ddemb
UL1 Ademb
K
The three conditions made with their rationale are explained in Appendix C.I and the proof of Theorem I can be found in Appendix C.2 From Theorem we observe that the gradients of the two losses of MLM task and the retrieval task have a positive linear relationship.
Note that L (d) actually represent the document perplexity Pa and L2(d, q) actually represent the negative estimated relevance score RR Then we can easily derive the following Corollary, which illustrates how the key conclusion 0C2/ md = K @ 0Li ed in Theorem leads to the biased effect of document perplexity Pa on estimated relevance score Rq.d:
Corollary 1. Consider a Haman--writee document d1 and its LLM-rewritten document d2, they are both relevant with query q. Assume LLM-rewritten documents possess lower perplexity at token level Mitchell et al. 2023).. Let rvee/vee be nutrix-to-row/colume vector operator, L (d) denote the perplexity of the 1-th token the document, (domb), ) denote the embedding of the 1-th token,
- - = (d2) O(dsmb) a(deub); 0dg
vec(d, d2) > 0, 1 1=
where Ist-order approximation of Chain rule is taken as the surrogate function (Grabocka et al. 2019 Nguyen et al. 2009 for C (d). According to Theorem 7 Jand Ist-order approximation of L2 (2(d).
aLi ddgub
adgmb
Ry Ragd -[2((d)) L2(d2)] = -rveo K
vec(di - d2)
Od2
Aki (c) a(dzmb))
Aki
=- Σ
voc(d) d2) = Σ
L(1 k(1-k) (C) (d2)) < 0.
L(1 kn) O(dgmb); ddg
Corollay indicates that human-writted document will receive lower relevance estimation than its LLM--written document, resulting in source bias. It is important to note that our theoretical analysis does cover all situations in reality, we will discuss these limitations in Appendix B
Finding 2:
For PLM-based retrievers, the gradients of MLM and IR loss functions (metrics) possess linear overlap, leading the biased effect of perplexity on estimated relevance scores.
4.2.2 FURTHER VERIFICATION OF THEOREM
Theorem reveals the linear relationship between language modeling gradients and retrieval gradients w.r.t. document embedding vectors dmm For more comprehensive verification for its reliability, we derive Corollay 2 from Theorem l and provide supporting experiments about the corollay. The derivation is similar with that in Corollary 1
Corollary 2. For two retrievers ft; 01), f(t; 02) which share the same PLM, such as BERT. retriever ft; 0:) possesses more powerful language modeling ability than ft; 02), i.e.,
BaED [C (d: 01) FAED (d) 02)) < <0, 1 1,...:
7

---

Published as a conference paper at ICLR 2025
then similar to the Corollary 7 we have
(0L2(d 02) ddemb
adumb
EdED (E2](C2(d;0:)) 01)]
EEED C2 02)]
EdED Ivec
vec(0] -
002
0L1 (d
02) ad
Aki
=B rvec K
vec(O1 -
-22) =E
-
Ci(d;
<0.
dd
002
L(1-k) k))
Note that de [C1 (d; 0)] typical measure of language modeling ability and EdED Ed (d: 0)] reflects the ranking performance, Corollary Saaaaryyjiiiiei inddicates that if a retriever possesses more powerful language modeling ability, its ranking performance will be better.
To offer empirical support for the corollary, we evaluate the language modeling ability of PLM-based retrieval models with different ranking performances. By taking the retrieval model directly as PLM encoder to do MLM task, we calculate the average text perplexity of the retrieval corpus to evaluate their language modeling ability, which offers support for the encoder-decoder corporation assumption at the same time. As illustrated in Figure 3 there is clear correlation between text perplexity and retrieval accuracy (except Contriever). These results, demonstrating that language modeling capabilities are indeed correlated with retrieval performance, strengthen the practical reliability of our assumptions and conclusions as the deductive verification of the above hypothesis we used. This finding also explains why PLM dramatically improve the performance of retrievers over past years.
Combining the previous findings, we can further under- stand the relationship between model retrieval perfor- mance and the degree of source bias. On one hand, if the PLM-based retriever demonstrates better MLM capability, it tends to be more sensitive to document perplexity, which leads more severe source bias (Corollary T. On the other hand, retriever with bet- ter MLM capabilities can also achieve more accurate relevance estimations, leading to better ranking per- formance (Corollary 2 Consequently, PLM-based retrievers encounter trade-off between accuracy in retrieval and the severity of source bias. Specifically, higher ranking performance is associated with more sig- nificant source bias. This relationship has been noted in previous research (Dai al. 2024a), and we the first to offer a plausible explanation for this phenomenon.
caCundiier
Kanking Performance
52
TAS-B
00
50
ANCE
Comsieerr
48
46-
BERT
42
Kolliiia
40
Perplexity
Figure 3: Model perplexity and ranking per- formance (NDCG@3) on averaged results of DL19, TREC-COVID, and SCIDOCS.
Finding 3: Better language modeling improves PLM-based retriever's ranking performance, but also heightens its sensitivity to perplexity, thus increasing source bias severity.
5 CAUSAL-INSPIRED SOURCE BIAS MITIGATION
In this section, we further propose causal-inspired debiasing method eliminate the source bias, which can be naturally derived from our above causal analysis. We then conduct experiments to evaluate the effectiveness of the proposed debiasing method.
5.1 PROPOSED DEBIASING METHOD: CAUSAL DIAGNOSIS AND CORRECTION
In Section and 4 we have constructed a causal graph and estimated the biased effect of perplexity on the final predicted relevance score. Based these insights, propose an inference-tine -ime debiased method via Causal Diagnosis Correction (CDC). The main procedure of CDC lies on two stage:
(i) Bias Diagnosis: Employing the Instrumental Variable method for estimating the bias effect 8 of perplexity Pa to estimated relevance score Rg.d- d- (ii) Bias Correction: Separating the biased effect of document perplexity from the overall estimated relevance scores Rg.d-
8

---

Published as a conference paper at ICLR 2025
Algorithm 1: The Proposed CDC: Debiasing with Causal Diagnosis and Correction
Input: training D, test query Q, test corpus C, estimation budget M
Output: unbiased estimated relevance scores R
// Bias Diagnosis
Initialize the estimation set for estimating biased effect
for training pairs (g. d E D and [D.] M do
Instruct LLM to generate doc de via rewriting the original human- written doc dft
Predict the estimated relevance scores f?! F. for pairs (qs, af*) and (9;, df)
Calculate perplexity p! pe for doc dff and doc df, respectively
Updating the estimation D. D. (F. p8 pM pf)
end
Estimate the biased effect coefficient 82 with 2-stage regression using Eq. 2 on De 10
// Bias Correction
for test query 9: E Q do
12
Predict the estimated relevance scores fa for each pair (ge, de) with de EC Calculate document perplexity ps for each doc de E
13
Debias the original model prediction ftz using Eq. (3), add the calibrated score 7t to R
14
end
return R
Table 2: Performance (NDCG@3) and bias (Relative A Dai et al. 2024c on NDCG@N 3) of different PLM-based retrievers with and without our proposed CDC debiased method on three datasets. Note that a more negative bias metric value indicates greater bias towards LLM-generated documents, while a more positive value indicates greater bias towards human- numan-written documents.
DL19 (In- Domain)
TREC-COVID (Out-of-Demuin)
SCIDOCS (Out-of-
Model
Performance
Bias
Performance
Bias
Performance
Bias
Raw
+CDC
Raw
+CDC
Raw
+CDC
Raw
+CDC
Raw
+CDC
Raw
+CDC
BERT
75.92
77.65
-23.68
3.90
33.72
43.38
-39.38
-18.40
10.80
10.44
-2.85
29.19
Roberta
72.79
71.33
-36.32
4.45
46.31
45.86
48.14
-10.51
8.85
8.24
-30.90
32.13
ANCE
69.41
67.73
-21.03
34.95
71.01
69.94
-33.59
-1.94
12.73
12.31
-1.57
26.26
TAS-B
74.97
75.63
-49.17
-9.97
63.95
62.84
-73.36
-37.42
15.04
14.15
-1.90
23.48
Contriever
72.61
73.83
-21.93
-5.33
63.17
61.35
-62.26
-31.33
15.45
15.09
1.63
col Condenserr
75.50
75.36
-18.99
9.60
70.94
71.07
-6.95
-45.39
13,93
13.79
-5.95
1.06
Specifically, the final calibrated score Rq.a for document ranking can formulated as follows:
Ry Rq. -32Pa.
which can be derived by rearranging Eq. 22. In this formula, R2 independent to document source and perplexity. Therefore, it serves good proxy for semantic relevance ranking.
Specifically, we first take M samples from the training set D construct the estimation set D. for estimating the biased effect $2 (lines 2-8), where M is the estimation budget. To construct the estimation set D., we instruct an LLM generate document dy by rewriting the original human- the retriever predict their relevance scores F and o for the given query and calculate their document perplexities p,d and pf Further, following the practice in Section 4.1 we use two-stage IV regression on the estimation set D. to estimate the biased coefficient Br (line 9). During testing, we use Eq. 3 to correct the original model prediction ft, obtaining the calibrated score i for final document ranking (line 11-15). We summarize the overall procedure of CDC in Algorithm
written document d.f. For these types of samples,
5.2 EXPERIMENTS AND ANALYSIS
To evaluate the effectiveness of CDC, we implement it across various retrievers in simulated-realistic debiasing scenarios, where generated documents are from different domains and LLMs. In this case, we investigate the generalizability of the CDC method at both LLM level and Domain level.
At domain-level employ bias diagnosis the training of DL19 to estimate the biased effect 3 for each retrieval model, and then conduct in-domain and cross-domain evaluation the test sets

---

Published as a conference paper at ICLR 2025
Table 3: Performance (NDCG@3) 3) and bias (Relative A DDai et al. 2024c) on NDCG@3) of the retrievers mixed SciFact corpus from different LLMs. Bias Diagnosis is conducted on DL19 corpus from Llama-2, 2, where CDC performs generalization at both LLM and data-domain levels.
T. L.maa (I- Xonann)
GP
GPT-33 (Oaaaf-Damai))
Mistral (Out-al-Tanmin)
Model
Perfermancc
Bias
Perfomaace
Blas
Perforancc
Bias
Performance
Blas
Raw
+CDC
Rew
-CDCC
Row
+CDCC
Raw
+CTCC
Rew
.CDC
Raw
+CDC
Raw
+CTX:
Row
+CTC
DLRT
3508
12.37
6.75
16,47
35.75
3.59
6.01
15.97
15.27
18108
35.11
35,05
0.73
13.07
RoBERTa
-29.54
38.53
57.70
4.52
39.17
38,00
-35,39
14:00
37.28
-17.95
16.78
ANCE
4.50
42.67
3.28
42.76
-13.50
-8.59
TAS. B
15.04
-7.96
52.12
52.44
00.55
52.83
52.90
5.68
-5.11
Contriever
55.37
-2.87
1.07
-4.4
56.17
-743
-..13
-2.30
CCCoddeseer
-12.98
-9.26
48.57
48.91
6.04
48.59
48.81
-1.00
5.30
49.57
49.92
-5.90
-0.76
of DL19, TREC-COVID, SCIDOCS. Note that only 128 samples (i.e., estimation budget M 128)
are used for bias diagnosis, this sample size is sufficient for effective results. More detailed settings can be found in Appendix E.I The averaged results over five different seeds are reported in Table 2
As we can see, using the estimated biased coefficient of in-domain retrieval data, our debiasing CDC successfully mitigates or even reverses the retrieval models' preference towards human-written documents without fine-tuning retrievers. Meanwhile, this estimated biased coefficient demonstrates generalizability across out-of-domain datasets. The majority of the retrieval performance degradation was generally less than 2 percentage points, revealing that our debiasing CDC has acceptable impact on ranking performance, see detailed significance test in Appendix Table In addition, the mean and standard deviation of performance and bias after CDC debiasing for the five sampling sessions is provided in Appendix T indicating the robustness of CDC to training samples.
We also find that the debiasing results may vary across different retrievers. Specifically, CDC has more significant effects on vanilla models like BERT while exhabits lower impacts on stronger retrievers such as Contriever. We offer the following analysis to explain such observations. Stronger retrievers are developed using more sophisticated contrastive learning algorithms, which enhance their abilities to differentiate between highly relevant documents and the others. In this way, it may be more challenging for CDC corrections to alter the initial rankings. So a more aligned or model-specific approach could potentially enhance the debiasing process.
Considering that the web content may be generated by diverse LLMs, we expand our evaluations to assess the generalizability of the CDC method across corpora generated by different LLMs, including Llama (Tovroon et al., 2003), GPT-4 (Achiam et al. (023), GPT--355, and Mistral (Jiang et al.) (023). Due to the cost of computing resources, we conducted experiments on a smaller dataset SciFact, which is also used in previous works (Dai et al. 2024c a). In this setup, CDC used Llama's 's rewritten DL19 documents to estimate 33 and subsequently correct retrieval results on SciFact corpora mixed with each LLM separately. The results displayed in Table coonfirm that CDC is capable to generalize across various LLMs and maintain high retrieval performance while effectively mitigating bias.
In summary, these empirical results validate the feasibility of our proposed debiasing method by effectively reducing the biased impact of document perplexity on model outputs. And this method can be integrated efficiently into dual-encoder architerctures used in ANN search by pre-computing and indexing query- -ndeepndnd document perplexity with embeddings. Moreover, A2 can be adjusted according to specific requirements, where larger absolute value of 8g leads to further preference for numan-written texts albeit at the potential cost of ranking performance degradation. More discussion about the open question that "Should we debias toward numan--ritter contents?" is in Appendix A.
CONCLUSION
This paper aims to explain the phenomenon of source bias where PLM-based retrievers overrate low- perplexity documents. Our core conclusion is that PLM-based retrievers use perplexity features for relevance estimation, leading to source bias. To verify this, we conducted two-stage IV regression and found a negative causal effect from perplexity to relevance estimation. Theoretic analysis reveals that the gradient correlation between language modeling and retrieval tasks contributes to this causal effect. Based on the analysis, a causal-inspired inference-time debiasing method called CDC is proposed, Experimental results verified its effectiveness terms of debiasing the source bias.
10

---

Published as a conference paper at ICLR 2025
7 ACKNOWLEDGEMENTS
This work was funded by the National Key R&D Program of China (2023YFA1008704)) the National Natural Science Foundation of China (62472426, 62276248, 62376275). the Youth Innovation Promotion Association CAS under Grants (2023111), fund for building world-class universities (disciplines) of Renmin University of China, the Fundamental Research Funds for the Central Universities, PCC@ RUC, and the Research Funds of Renmin University of China (RUC24QSDL013) Work partially done at Engineering Research Center of Next-Generation Intelligent Search and Recommendation, Ministry of Education.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.06:: 2023.
Joshua D Angrist and Jörn-Steffen Pischke. Mostly harmless econometrics: An empiricist's compan- ion. Princeton university press, 2009.
Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang. and Yue Zhang. Fast-detectgpt: Efficient machine-generated
zero-shot detection of machine- generated text via conditional probability curvature. arXiv preprint arXiv:2310.05 130, 2023.
Gordon Burtch, Dokyun Lee, and Zhichen Chen. Generative degrades online communities. Communications of the ACM, 67(3)400-4,, 2024.
Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S Yu, and Lichao Sun. A comprehen- sive survey of ai-generated content (aigc): A history of generative ai from gan to chatgpt. arXiv preprint arXiv:2303.0m 04226, 2023.
Xiaoyang Chen, Ben He, Hongyu Lin, Xianpei Han, Tianshu Wang, Boxi Cao, Le Sun, and Yingfei Sun. Spiral of silences: How is large language model killing information retrieval?-a case study on open domain question answering. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. 2024.
Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel S Weld. Specter: Document-level representation learning using citation-informed transformers. arXiv preprint arXiv:2004.07110, 2020.
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M Voorhees. Overview of the trec 2019 deep learning track. arXiv preprint arXiv:2003.07821 2020.
Sunhao Dai, Weihao Liu, Yuqi Zhou, Liang Pang, Rongju Ruan, Gang Wang, Zhenhua Dong, Jun Xu, and Ji-Rong Wen. Cocktail: A comprehensive information retrieval benchmark with llm-generated documents integration. Findings of the Association for Computational Linguistics: ACL 2024, 2024a.
Sunhao Dai, Chen Xu, Shicheng Xu, Liang Pang, Zhenhua Dong, and Jun Xu. Bias and unfairness in information retrieval systems: New challenges in the 1lm era. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6437-6447, 2024b.
Sunhao Dai, Yuqi Zhou, Liang Pang, Weihao Liu, Xiaolin Hu, Yong Liu, Xiao Zhang, Gang Wang, and Jun Xu. Neural retrievers biased towards Ilm-generated content. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 526-537, 2024c.
Sunhao Dai, Chen Xu, Shicheng Xu, Liang Pang, Zhenhua Dong, and Jun Xu. Unifying bias and unfaimess information retrieval: New challenges in the Ilm era. In Proceedings of the 18th ACM International Conference Web Search and Data Mining, 2025.
Jacob Devlin, Ming-Wor Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: /8/00 04.005 2018.
11

---

Published as a conference paper at ICLR 2025
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-tram Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pages 41816 2019.
Shaohua Fan, Xiao Wang, Yanhu Mo, Chuan Shi, and Jian Tang. Debiasing graph neural networks via learning disentangled causal substructure. Advances in Neural Information Processing Systems, 35:24934-2499 2022.
Luyu Gao and Jamie Callan. Unsupervised corpus aware language model pre-training for dense passage retrieval. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 2843-2853, 2022.
Tiayyu Gao, Xingcheng Yao, and Danqi Chen. Simese: Simple contrastive learning of sentence embeddings. arXiv preprint 200402200.08881,, 2021.
Charles Goodhart. Problems of monetary management: the uk experience in papers in monetary economics. Monetary Economics, 1, 1975.
Josif Grabocka, Randolf Scholz, and Lars Schmidt-Thiemes Learning surrogate losses. arXiv preprint arXiv:1905.10108 2019.
Jiafeng Guo, Yinqiong Cai, Yixing Fan, Fei Sun, Ruging Zhang, and Xueqi Cheng. Semantic models for the first-stage retrieval: A comprehensive review. ACM Transactions on Information Systems (TOIS), 40(4):1-42 2022.
Jason Hartford, Greg Lewis, Kevin Scyton-Brown, and Matt Taddy. Deep iv: A flexible approach for counterfactual prediction. In International Conference on Machine Learning, pages 1414-1423, PMLR, 2017.
Sebastian Hofstätter, Sheng-Chiel Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. Efficiently teaching an effective dense retriever with balanced topic aware sampling. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 113-122, 2021.
Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. Unsupervised dense information retrieval with contrastive learning. Transactions on Machine Learning Research, 2022.
Kalervo Järvelin and Jaana Kekäläinen. Cumulated sain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS), 20(4):422-446, 2002.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023.
Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. On the sentence embeddings from pre-trained language models, arXiv preprint arXiv:20/ 1.05664, 2020.
Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems, 36, 2024.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint 1tXhv:1907.11692 2019.
Eric Mitchell, Yoonho Lee. Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. machine-generateed
Detectgpt: Zero-shot machine- generated text detection using probability curvature. In International Conference on Machine Learning, pages 24950-24962. PMLR, 2023.
Michael Mitzenmacher and Eli Upfal. Probability and computing:
Randomization and probabilistic techniques in algorithms and data analysis. Cambridge university press, 2017.
12

---

Published as a conference paper at ICLR 2025
Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. Mteb: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316, 2022.
XuanLong Nguyen, Martin J Wainwright, and Michael Jordan. On surrogate loss functions and I-divergences. 2009.
Masanari Ohi, Masahiro Kaneko, Ryuto Koike, Mengsay Loem, and Naoaki Okazaki. Likelihood- based mitigation of evaluation bias in large language models. arXiv preprint (oo::240..198877 2024.
N Reimers. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:/908. 10084, 2019.
Gerard Salton, Anita Wong, and Chung-Shu Yang. A vector space model for automatic indexing. Communications of the ACM, 18(11):613-620 1975.
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Ander- son. The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2305. 17493, 2023.
Rickard Stureborg, Dimitris Alikaniotis, and Yoshi Suhara. Large language models are inconsistent and biased evaluators. arXiv preprint arXiv:2405.01724 2024.
Hexiang Tan, Fei Sun, Wanli Yang. Yuanzhuo Wang, Qi Cao, and Xueqi Cheng. Blinded by generated contexts: How language models merge generated and retrieved contexts for open-domain qa? Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023.
Ellen Voorhees, Tasmeer Alam, Steven Bedrick, Dina Gemner-Fushmanl William R Hersh, Kyle Lo, Kirk Roberts, Ian Soboroff, and Lucy Lu Wang. Trec-covid: constructing a pandemic information retrieval test collection. In ACM SIGIR Forum, volume 54, pages 1-12. ACM New York, NY, USA, 2021.
Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Aroold Overwijk. Approximate nearest neighbor negative contrastive learning for dense text retrieval. arXiv preprint 2000::0000000000 2020.
Shicheng Xu, Danyang Hou, Liang Pang, Jingcheng Deng, Jun Xu, Huawei Shen, and Xueqi Cheng. Ai-generated images introduce invisible relevance bias to text-image retrieval. Proceedings of the 47th international ACM SIGIR conference research and development in information retrieval, 2024.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolci Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.1888 2023.
Wayne Xin Zhao, Jing Liu, Ruiyang Ren, and Ji-Rong Wen. Dense text retrieval based on pretrained language models: A survey, ACM Transactions on Information Systems, 42(4):1-60, 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging Ilm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2024.
Yugi Zhou, Sunhao Dai, Liang Pang, Gang Wang, Zhenhua Dong. Jun Xu, and Ji- Ji-Rong Wen. Source echo chamber: Exploring the escalation of source bias in user, data, and recommender system feedback loop. arXiv preprint a4Xiv:2405.1 17998, 2024.
13

---

Published as a conference paper at ICLR 2025
A DISCUSSION
A.1 SHOULD WE DEBIAS TOWARD HUMAN- WRITTEN CONTENTS?
While we refer to the retrievers' preference for LLM-rewritten content as a "bias", it's crucial to recognize that not all biases are harmful. As illustrated in previous works (Dai et al.) 2024c Zhou et al. 2024), from content creators' perspective, reducing preference toward LLM-rewritlee content helps guarantee sufficient incentives for authors to encourage creativity, and thus sustain healthier content ecosystem. From users' perspective, LLM- documents might possess enhanced quality, such as better coherence, and improved reading experience.
In this work, our debiasing approach is primarily methodological application derived from our causal graph analysis, serving to validate the "perplexity-trap" hypothesis further. At the same time, our framework allows for adjustable preference levels between human-written and LLM--generatee documents, catering to specific practical requirements. This flexibility ensures our approach can be tailored to balance between enhancing information quality and maintaining content provider fairness.
A.2 SHOULD PERPLEXITY BE A CAUSAL FACTOR TO QUERY-DOCUMNT RELEVANCE?
It's one of the assumptions of this work that perplexity should not be a causal factor to query- document relevance. is true that there may be a correlation between perplexity and query-document relevance, e.g., the coherence of a document may also have an impact on relevance. However, there an insurmountable gap between the perplexity of LLM-rewritten documents and human work, because people do not intentionally take PPL into account when writing, but LLMs do generation with perplexity as a goal. We are currently faced with a situation where this perplexity gap has breached the range of human perception of relevance, leading serious source bias even when the rewritten documents share nearly the same semantics with human works, as verified and discussed in previous literature (Dai et al., 2024c). It's just like what Goodhart's Law (Goodhart) 1975) states: "When a measure becomes a target, it ceases to be a good measure." " So perhaps a threshold should be set, and when perplexity is less than the threshold, it should be made independent with relevance.
B LIMITATIONS
This study has several limitations that are important to acknowledge. Data and Experiments. it is recognized that there are numerous other IR datasets that could have been included. Our selection, although limited in scope, was strategic to ensure broad representation across different domains, and we believe that our findings can be generalized to other domains. Secondly, due to the cost associated with human evaluation, we were constrained perform only 6 x 20 evaluations for each dataset, corresponding to six different sampling temperatures. This decision, while pragmatic, may generalize results to other conditions. Thirdly, we have to
Firstly, while our analysis was conducted on three representative datasets,
limit the extent to which admit that impacts of LLM rewriting on semantics indeed lack more consideration although they have been designed possibly credible. Since simulated environment construction is not our main contribution, we have adopted the datasets provided by previous works (Dai et al. (0224b or follow their methodology (Dai al. 2024c) to evaluate the source bias of retrievers. In Dai al. (2024c), the document embeddings are compared using cosine similarity, and more detailed human evaluation was conducted to assess the various impacts LLM rewriting, which indicated no significant changes in document semantics. We will conduct more meticulous semantic checks to pursue more rigorous conclusions possible.
Theoretical Analysis In our theoretical proofs, we made certain assumptions and simplifications. Specifically, we narrow our analysis in PLM-based dual-encoder and mean-pon mean-pooling scenario. These are necessary to achieve mathematical tractability and are grounded in practical considerations, which have been discussed in the previous sections. We believe these assumptions are reasonable and have validated the reliability of our conclusions through experimental verification. For the other scenarios, such as auto regressiee embedding models and CLS-based retrievers, we will explore and discuss them in the future work.
14

---

Published as a conference paper at ICLR 2025
Despite these limitations, we maintain that our work provides valuable insights into the subject matter and serves as a foundation for future research.
C NOTES ON THEORETICAL ANALYSIS
In this section, we provide detailed reasons our assumptions and proof to our proposed theorem in Section
C.I EXPLANATION ON ASSUMPTIONS
Our theoretical analysis based on a of assumptions, to which we're going to offer the reasons. Encoder-Only Retrievers Encoder-only architectures are generally considered more suitable for textual representation tasks, while encoder-decoder and decoder-only models typically used for generative tasks. Thus, encoder-only models have been widely employed for retrieval tasks and have demonstrated effective results. In fact, most of the mainstream dense retrievers listed on the MTEB (Muennighoff et al.) (022) leaderboard are based on encoder-only architectures. Mean-Pooling Strategy. We use of mean pooling for query/doc embeddings in the derivation of Theorem 1, while simplification, differs from the practice of using CLS token embeddings in BERT-like models. From a practical perspective, (weighted) mean pooling embedding outperform CLS token embedding when ranking, which has been widely confirmed in previous works (Dai 2024b; Reimers 2019). From a theoretical perspective, (weighted) mean pooling is able to
ct al. retain more local information about documents, which is important for retrieval tasks, query is regarded related to document when the query is related to a particular sentence in the document. Furthermore, there is literature indicating that CLS token embeddings may not always effectively capture sentence representations, which can be a limitation in retrieval contexts (Li et al. (022)). Representation Collinearity Hypothesis
Representation Collinearity Hypothesis is fundamental assumption long implemented in information retrieval systems (Salton et al. 1975). When measuring relevance scores by calculating dot or cosine similarity, we assume that the best relevant document owns an embedding that is collinear with the query embedding (given that the norm of the document embedding is held constant). In practice, dense retrievers are trained on contrastive learning to maximize the similarity between query and its relevant documents while minimize the similarity of irrelevant documents (Gao et al. Semi-Orthogonal Weight Matrix Hypothesis
2021 Zhao et al. 2024).
W VX satisfies the Semi-Orthogondl
Weight Matrix assumption WWT = IN,, which is necessary to achieve mathematical tractability. Since practical PLMs uses 2-layer MLPs rather than the weight matrix W, this can't 't be verified directly. If we ignore the activation function in the MLPs of BERT, let W W, W2. then 50 N |diag( W WW which suggests that the diagonal elements
N² #WW'|
much
larger than the others. One reasonable intuition is conclusion in high-dimension probabilities which states "for any e > O, there are m 2(e vectors in RN such that any pair of them are nearly orthogonal." (Mitzenmacher and Uptal 2017) Since N 2 768 for commonly-use retrievers, the hypothesis holds with high probability.
Encoder-decoder Cooperation Hypothesis This assumption has certain practical background. The experiment Sectior4 can viewed as a verification of this assumption, where finetuned encoder is used with unfinetuned MLPs to do MLM task. In this setting, the hybrid model recieve relatively low perplexity as PLMs. In practice, the beginning learning rate finetuning retrievers is usually set at 1 which makes retrievers more likely to the conserve of inversion property.
C.2 PROOF OF THEOREM 1
In this section, we provide the proof of Theorem T Note that the three conditions made are naturally satisfied: (1) Representation collinearity a fundamental assumption long implemented in information retrieval systems. (2) Matrix orthogonality is a common and intuitive property of the decoder's weight matrix. (3) Encoder-decode adheres to the original design principles of auto-encoder networks. Then we give the proof as follows:
Proof. Given the following three conditions;
15

---

Published as a conference paper at ICLR 2025
Representation Collinearity: the embedding vectors of relevant query- pairs collinear after mean pooling, i.c.,
1xxif(q) xl)
Orthogonal Weight Matrix: the weight matrix of the decoder is orthogonal, i.e.,
WWT =
Encoder-decoder cooperation: fine-tuning does not disrupt the corresponding function between encoder and decoder, i.e.,
f(d) = (d).
Our goal to prove 202//dmmme K 0d
Note that the two losses are both involved with demb,
0h1 1 11xilo(d ddW
lxxu (dm W) dW
ddmmb
L
OC2 1 Odemb
I lub.
Replacing the softmax operation with linear normalization, let : denote element-wise division,
[o(())] Σn Vin
21, L = L. L.
Considering the following matrix identity,
(AMxx BN*K) @ (cas 1x) = (AMxx @ (cas 12)) BRNK,
it reveals that the gradient of L1 can be rearranged
[oldemb W) dWT demb W
demb
djWT
W d)WT.
k
k
where column vector k, R& satisfies k Ed (d complex. Meanwhile, using mean inequality (also called QM-AM inequality), we can find that
(eembbW)a Wha > 0 because a (d W); is
D Σ demb W N
ki <
1
V (eelmbby
emu
1 <1.
N
According to the orthogonal weight matrix assumption,
Jemb
demb
Jmmb
dmh
W
d
dWT
(d)
(d).
9 (d).
1
1
k 1
k
From the encoder-decode cooperation condition, we obtain
0C1 1xxll ddmmb
Jmmb
1. ki 1 diag 1 kL dumb
f(d) = 1 LXLL cubb
1
KL
kL
Considering the positive query-document pair d, assume their embedding vectors are collinear,
8L2
1
emb A lixidamb
1xxLq
md
L
observe that
AC2
ky 0C, 1 ki. ademb T,
X
diags
ade
Let K
ky 1r, 15, then it holds that
OL2 md
0L, Jdb
K
16

---

Published as a conference paper at ICLR 2025
Treatment
B1
S
Pa
Pa
B2
Instrmmenal Vanable
Pa
Ma
q.a
Outcome
Ra.d
Ma
Confounder
Figure 4: By leveraging IV regression on Sd, Pa is decomposed into causal and non-causal parts. A precise causal effect be obtained from the coefficient of the second-stage regression, i.e., $2.
D INSTRUMENTAL VARIABLE REGRESSION
In statisties, instrumental variable (IV) is used estimate causal effects. The changes of IV induces changes of explanatory variable but keeps error term constant. The basic method to estimate causal effect is 2SLS. the first stage, 2SLS regress explanatory variable on instrumental variable and obtain the predicted values of explanatory variable. In the second stage, 2SLS regress output variable on predicted explanatory variable. Then, the coefficient corresponding to the predicted explanatory variable can be viewed as measure of causal effects.
According to our proposed causal graph, the document source Sa has three properties: (1) It is correlated to the document perplexity Pa. (2) It is independent with Document semantics Ma because we can instruct LLMs to rewrite human documents for any document semantics. (3) only affect the estimated relevance score Ray through document perplexity Pa. Thus, document source Sa can be considered instrumental variable evaluate the causal effect of document perplexity on estimated relevance scores.
As depicted in Figure estimate document perplexity Pa based on document source Sş in the first stage. The results, coefficient 31 and predicted document perplexity Pa Sr are used in the second stage estimated the predicted relevance score Ry via linear regression, where the estimated coefficient 32 is valid measure for the magnitude of the causal effect.
E MORE EXPERIMENTS
E EXPERIMENTAL. DETAILS
Our experiments are all conducted on machines equipped with NVIDIA A6000 GPUs and 52-core Intel(R) Xeon(R) Gold 6230R CPUs at 2.G For better reproducibility, we employ the following officially released checkpoints:
BERT (Devlin et al. 2018 2019) and RoBERTa (Liu et al., 2019) are used in dense retrieval as PLM encoders. We employ the trained models from the Cocktail benchmark (Dai et al.
2024a) The models are available at hhttps:///t http::://hugginggfee
bert www.-----ccsee--m--------- w
and https://hiqqingfaac.. co/ respectively.
IR- BB-Cocktaal//obetttbbbaa masse-meaac mean v3 msmarco
ANCE (Xiong et al. 2020) improves dense retrieval by sampling hard negatives via the Approxi- mate Nearest Neighbor (ANN) index. The model is available at httttts://hgggigggg
co/
Bentence-traie forme htt.://aeeaaataaaaaaaoaooroooraa
TAS-B (Hofstätter al. 2021), leverages balanced margin sampling for efficient query selec- tion. The model is available at htt.tttttttssstggaaaaa......
msma rco-ccooo st bert base-
17

---

Published as a conference paper at ICLR 2025
Table 4: Human evaluation on which document is more relevant to the given query semantically? The numbers in parentheses are the proportion agreed upon by all three human annotators.
DL19
Temperature
Human
LLM
Equal
0.00
0.0% (0.0%)
5% (0.0%)
95% (83.8%)
0.20
0.0%().0%)
5% (0.0%)
95% (94.2%)
0,40
0.0% (0.0%)
0.0% (0.0%)
100% (79.6%)
0.60
0.0% (0.0%)
0.0% (0.0%)
100% (84.6%)
0.80
0.0% (0.0%)
0.0% (0.0%)
100% (94.5%)
1.00
(0.0%)
0.0% (0.0%)
100% (94.5%)
TREC-COVID
Temperature
Human
LLM
Equal
0.00
0.0% (0.0%)
0.0% (0.0%)
100% (84.6%)
0.20
0.0% (0.0%)
0.0% (0.0%)
100% (94.5%)
0.40
0.0% (0.0%)
0.0% (0.0%)
100% (74.6%)
0.60
0.0% (0.0%)
0.0% (0.0%)
100% (94.5%)
0.80
0.0% (0.0%)
0.0% (0.0%)
100% (79.6%)
1.00
0.0% (0.0%)
0.0% (0.0%)
100% (84.6%)
SCIDOCS
Temperature
Human
LLM
Equal
0.00
0.0% (0.0%)
0.0% (0.0%)
100% (84.6%)
0.20
0.0% (0.0%)
0.0% (0.0%)
100% (84.6%)
0.40
0.0% (0.0%)
0.0% (0.0%)
100% (79.6%)
0.60
0.0% (0.0%)
5.0% (0.0%)
95% (83.8%)
0.80
0.0% (0.0%)
0.0% (0.0%)
100% (79.6%)
1.00
0.0% (0.0%)
5% (0.0%)
95% (89.0%
Contriever (Izacard et al. 2022 employs contrastive learning with positive samples generated through cropping and token sampling. The model is available wttttts://huggiiggeett co/ facebook contri eeeer
coCondenser (Gao and Callan 2022) a retriever that conducts both pre-training and supervised fine-tuning. The model is available at /https://hugging co/
sentence trans formers rco bert bert-co-condensori
We follow the metrics proposed by previous work when measuring ranking performance and source bias. For ranking performance, we use NDCG@k (Jävellin and Kekäläinen, 20022. For source bias, we use Relative A NDCG@k (Dai et al. c Xu et al. [024 Zhou et al. 2024), which is
formulated as
RelativeA MetricHuman MetricLLM 100%.
MMerricHaman + MetricLLM)
In CDC debiasing, considering the sample size we conduct bias correction for the top candidates in retrival. Rising up the candidates number leads to less preference for LLM-gennrated documents while ranking performance may drop a little.
E.2
MORE RESULTS OF GENERATED CORPUS WITH VARYING SAMPLING TEMPERATURE
E.2.1 HUMAN EVALUATION
Although L-M-geeerated documents are solely based on their corresponding human documents, still necessary verify that the generated document has the same relevance scores with given query as the original documents. To provide empirical support on the fact that --M-eenertted documents not injected with extraneous information about queries, conduct human evaluation.
We randomly select 20 (query, numan-written document. LLM- LLM--enerated document) triples for each dataset and each sampling temperature. The human annotators who have least Bachelor's 's degrees are asked to evaluate which document is more relevant without knowing document sources. Their results transferred into the "Human", "L.M"", "Equal" options later. The final labels of each
18

---

Published as a conference paper at ICLR 2025
Pearson Cocficiin:: 4096
Pearson Cocfiicion:
Peurson Cocflicion: 0.57
1.770
Pepladyy - Setommsom
-0,05
1,000
Peppenay
-0,403
Peplendy
0.223
Pemlecay
00433
Paploxity
22000
Relevante
0.223
1,750
-0439
2,00
1,740
0.445
1,909
0,22
1.700
0,402
1,90
-0.22
00
04
Tempestiune
Tamperttuue
Temperature
(a) DL19
(b) TREC-COVID
(c) SCIDOCS
Figure 5: Perplexity and estimated relevance scores of Contriever on positive query- query-document pairs in three datasets, where documents are generated by LLM with different sampling temperatures.
1.770
0,7%
-0.144
Store
0.369
Porplxity
1,760
0.728
Peplaxty
+0.13%
Perplasty-
0,350
0.720
1.020
-0.12
1,740
1,700
1,005
0.120
0,339
0.0
0.2
04
0.6
0.5
1.0
0.0
0.2
0.4
0.6
0.5
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Tempertture
Tempersture
Temperatune
DL19
(b) TREC-COVID
(c) SCIDOCS
Figure 6: Perplexity and estimated relevance scores of TAS-B on positive query query-document pairs in three datasets, where documents are generated by LLM with different sampling temperatures.
triple determined by the votes of three different annotators. The results in Table Illlsstrate that documents from different sources possess the same relevance to the corresponding queries, which guarantees the correctness of our controlled variables experiments.
E.2.2 RESULTS WITH MORE PLM-BASED RETRIEVERS
we discover the negative correlation between document perplexity and estimated relevance scores by Contriever. In this section, we demonstrate the replicability of the discovery by providing similar results on TAS-B B and Contriever. As depicted in Figure and Figure O there is a significant negative correlation between document perplexity and estimated relevance scores as sampling temperature changes. Documents with lower perplexity obtain prevalent higher estimated relevance scores across different PLM-based retrievers, further affirming the universality of the phenomenon.
In Section 3.1
E.2.3 MORE RESULTS OF 82 ESTIMATION
In Section 4.1. we estimated the causal effect of perplexity on estimated relevance scores through 2SLS. Since the estimation needs LLM generation, it's natural to explore the hyperparameters related to the generation.
According to the causal graph we proposed, the sampling temperature does affect 31 the first stage of the regression, but is independent with 3z. We explore whether 31 changes in turn affects the value of 32 by using documents generated with different sampling temperature. The & and A obtained from our estimation on the set of rewritten texts with different sampling temperatures are shown in 5 It can be found that under the maximum sampling temperature difference, the variation of 31 is within 15% and the variation of 3 is within 20%, and such variations similar to the errors brought by random sampling, so the variation of the sampling temperature acceptable in the CDC algorithm.
19

---

Published as a conference paper at ICLR 2025
Table The influence of generation temperatures on the magnitude of the causal coefficients B1, 82. The coefficients are estimated from all positive query-document pairs.
D1.19
TREC-COVID
SCIDOCS
Temperature
0.2
0.4
0.6
0.2
0.6
(,(ERT
-7.80
-7.78
-7.77
-7.94
-1.21
-1.20
-1.24
-1.26
-2.29
-2.29
-2.33
-2.46
(RRBBT))
23.557
-23.50
-23.45
-23997
1.73
1.73
1.77
1.80
-6002
-6.04
-6.13
-6.47
(ANCC)
-0.44
-0.44
-0.44
-0.45
0.07
0.07
0.07
0.07
-0.22
-0,22
-0.22
-0.23
(a(TA)B))
-0.80
-0.80
-0.82
-0.34
-0.34
-0.35
-0.35
-0.37
-0.37
-0.37
-0.39
8a (Cootriee)r))
-0.01
-0.01
-0.01
-0.01
-0.03
-0.03
-0.04
-0.04
-0.02
-0.02
-0.02
-0.02
BalcoCondenser)
-0.58
-0.58
-0.58
00,99
-0.23
-0.23
-0.24
-0.24
-0.25
-0.25
-0.25
-0.26
8
-0.44
-0.44
-0.44
-0.43
-0.41
-0.41
-0.40
-0.39
-0.41
-0.40
-0.40
-0.38
E.3 MORE RESULTS OF CDC DEBIASING
In this section, we report more experimental results to provide a more comprehensive analysis of CDC, including robustness analysis with error bar (Table and significance test (Tabel 7
Table 6: Mean and standard deviation of Performance (NDCG@3 @3) and bias (Relative A @Di et al. 2024c) NDCG@* 33 3) of different PLM-based retrievers with our proposed CDC debiased method three datasets in five repetitions.
DL19 (e-Domain)
TREC-COVID (fut-of-
SCIDOCS (Out-of- -ommai)
Model
Performance
Bias
Performance
Bias
Performance
Bias
Mean
Std
Mean
Std
Mean
Std
Mean
Std
Mean
Std
Mean
Std
BERT
77.65
0.89
5,90
4,40
45.88
1.14
-18.40
6.72
10.44
0.19
29.19
9.35
RoBERTa
71.33
0.48
4.45
0.80
45.86
0.78
-10.51
3.58
8.24
0.18
32.13
7.28
ANCE
67.73
0.15
34.95
11.51
69.94
0.77
-1.94
4.63
12.31
0.33
26.26
10.61
TAS-B
75.63
0.24
-9.97
5.25
62.84
0.48
-37.42
3.99
14.15
0.16
23.48
5.84
Contriever
73.83
0.27
-5.33
1.93
61.35
0.73
-31.33
3.22
15.09
0.10
1.63
1.89
coCondenser
75.36
9.60
71.07
-45,39
0.21
1.06
2.79
Table 7: The p-value of significance test conducted on the NDCG@3 and Relative A Dai et al. 2024c) on NDCG@3 with and without CDC debias method, with bold fonts indicating the Performance or Bias can pass significance test with v-value< 0.05. As expected, most Performance DOES NOT pass the significance test while all the Bias DOES pass the significance test.
DL19
TREC-COVID
SCIDOCS
Model
Performance
Bias
Performance
Bias
Performance
Bias
BERT
4.56e-03
5.33e-04
1.87e-03
3.97e-05
1.37c-01
1.47e-03
RoBERTa
7.26c-02
2.33e-04
1.22c-01
5.46e-05
8.48c-01
9.45e-07
ANCE
1.74e-01
4.48c-03
4.6le-01
3.09e-04
1.62e-01
2.25c-05
TAS-B
6.16e-01
3.19e-04
8.58e-01
1.27c-03
6.67e-04
2.16c-04
Contriever
2.77e-01
3,94c-02
2.98e-01
5.03e-04
4.44c-02
7.65c-03
coCondenser
8.82e-01
1.16e-02
5,95e-01
3.81e-04
2.71e-01
1.58e-03
20


Reviews:
[{"review_id": "AXsGr8NQpe", "reviewer": "Reviewer_V53D", "summary": "This paper studies the causes for source bias in PLM-retrievers. It explores the underlying architectural relations between perplexity and relevance scores and proposes an inference-time mitigation method that decouples the perplexity bias from the retrieval task.", "questions": "- How relevant do you think your study will remain given the rise of synthetically trained models, given the effects of low ppl and high relevance on LLM-generated data by majorly human-data trained retrieval models in your experiments?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- A clear technical description of the source bias issue, experimentation for cause and architectural relation, clear presentation of results and limitations.\n- The linear model fit for perplexity given document and semantic variables strengthens the argument presented by the causal graph.\n- Human annotation for verifying synthetic correctness.\n- I appreciate the acknowledgment and inclusion of semantic nuance of documents as another variable in your experiments.\n- The offering of a mitigation method with CDC strengthens the argument presented in your study and is an appreciated contribution.", "weaknesses": "- The focus on small and older models like BERT and RoBerta is slightly exclusive of more recent LLMs and SLMs.\n- Similarly, there is some lack of cross-model experiments for models used to generate data and for the retrieval task, popular closed- and open-source models could demonstrate very different results (A more inclusive experimental setup would be more convincing that your argument generalizes and isn’t overfit to your experiments).\n- There might be a limitation with closed models where training data is unknown for the CDC methodology.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QEnUkNPv2R", "reviewer": "Reviewer_kkqd", "summary": "In this article, the authors propose a thorough study of the \"Perplexity Trap,\" a phenomenon previously observed: when using embeddings from pretrained language models for retrieval tasks, documents written by LLMs tend to be ranked higher. The main hypothesis is that this occurs because generated texts tend to have lower perplexity due to the generative process, which in turn affects document/query similarity. The authors aim to confirm this hypothesis through three experimental studies (one providing a contribution to mitigate this bias) and one theoretical study. All results seem to confirm the perplexity hypothesis.", "questions": "## On the rewriting method\n“Specifically, we manipulate the sampling temperatures during generation to obtain LLM-generated documents with different PPLs but similar semantic content.” and “Since document semantics remain unchanged during rewriting,”. \n-> This is a strong assumption, that is, in my opinion, not verified. Using high temperature could lead to texts with different semantic content. The article lacks an experimental verification using LLM based eval, human eval, LSA/LDA based approaches or Non LLM embedding similarity. The question asked to the human evaluator in E.2.1 only evaluate relevance (author should provide the complete question asked to the annotator) and not semantic equivalence, and evaluates only 20 triplets. Additionally, increasing temperature might lead to hallucination, mechanically decreasing the relevance of the document. These should be evaluated properly. Could you elaborate on this?\n\n## On the theoretical section 4.2 and appendix\n\nNotations should be formally introduced to ease the readability, such as \\textbf{d} or dimensions of matrices (e.g. W). As notations are not that consensual, this is important to ease the reader work. Such grey zones led to several questions/interrogation that I list here. \n\nLine 312, d_{emb} is an L \\times N matrix, and is assumed to have a l2 norm of 1. What does it mean here for a matrix? Are you constraining the spectral norm, which is the formal equivalent of L2 norm for matrices? Could you please clarify. \n\nFollowing the explanations, W is the matrix that maps a vector in \\mathbb{R}^n to the vocabulary space, so of dim N by D. \n1) It cannot be orthogonal but only semi orthogonal as it is not a squared matrix \n2) I don’t understand how the conditions can be verified for most PLM as imposing orthogonality is really hard. Could you clarify? I might have missed something in the formalization of the problem, but it seems to be nowhere explained and motivated. \n\nLast assumption (fine tuning conserve the inversion property) is not discussed and motivated. With large learning rate, I don’t see how this can be verified. \n\nLine 810 is losing the reader. You are summing over N, if d^{emb}W is of dimension L \\times D, so k is a vector of dimension L ? More importantly, I don’t get the justification for it having only positive values. If dW contains non-positive values, linear normalization does not provide guarantee that k is positive and that norm of d is 1 (again, which norm?). Although it relies on undisclosed assumptions that should be made clearer.\n\nQM-AM states that the quadratic mean is greater than the arithmetic mean. K is n times the arithmetic mean, so you can’t say that this is lower than the quadratic mean (it is lower than  n times the QM though). \n\nOverall, I find it misleading to use k, as it is a constant that does not depend from d_emb, that may lead to thinking that k is fixed for all d_emb.\n\nEventually, the sign of k will modify the dependency type and could contradict the assumption of the work.  \n\nAgain, all these might come from a misunderstanding of the nature of matrix W, but anyway, this should be clarified in the documents. \n\n## Minor\n\nANCE is used before being introduced line 264 (e.g. line 145 )\nShouldn’t MLM task loss divided by L only rather than L \\times D ?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "Very important topic to study, including a consensual hypothesis for the “perplexity trap” that was still underexplored\n\nBoth experimental (correlation 3.1 and causality 4.1) and theoretical findings are (quite) convincing explanation of the link between perplexity and IR document relevance\n\nThe proposed method for bias mitigation is relevant, new and effectively reduce bias on several dataset and LLMs.", "weaknesses": "My main concern is the theoretical development in section 4.2. It lacks formalism, some notations are badly introduced or not at all. This leads to some aspect (either details or important things) either difficult to understand and follow, or that seem false. I’ll developpe everything in the questions section below. \n\nThe rewriting approach appears to be sound but lacks experimental support to show that the semantic content was not changed. See a more detailed comment below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RiroB6JFBk", "reviewer": "Reviewer_S3rs", "summary": "The paper studies why PLM-based retrievers are biased towards LLM-generated content and hypothesize that it is due to a causal effect between document perplexity and relevance score estimation. Based on the analysis, the paper further proposes a correction method, CDC, to separate the bias effect. Experiments include controlled experiments to demonstrate the causal effect, and experiments on 3 IR datasets, using existing MLM-based PLM retrievers, where the effect of using CDC is demonstrated.", "questions": "Would not \"document semantic\" be explained away?\n\nPlease list the statistics of each dataset used.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Performing a deeper understanding on the PLM retriever is biased towards AIGC is an interesting topic and can be interesting to some from the IR community. The paper provides reasonable illustrations via causal analysis and proposes a debiasing method accordingly, which is a complete work in terms of the story.\n\n2. The analysis of MLM vs IR loss function is interesting and provides more insights into the problem.", "weaknesses": "1. Limited to PLM-based retrievers. One may argue that a well behaved retriever, or existing retrievers do not possess this issue. Or there're more standard ways to address this such as by standard training. One may argue that this topic is not of general interest to the ICLR community as it is more SIGIR focused, as the setting is specific and not widely acknowledged.  \n\n2. Analysis is limited to MLM, while MLM is becoming less and less relevant. Indeed, the retrievers used in the paper are relatively old (most recent 2022). The proposed method also shows more effect on older PLMs than more recent ones. This again raises the concern about how relevant the paper is to the broader ML community. How about models trained with CLM? \n\n3. The causal analysis and approach leveraged in the paper is conventional. This is not necessarily a \"weakness\" but it does not make the paper a stronger case.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)