Towards Imperceptible Document Manipulations against Neural Ranking Models
Year: 2023

Abstract:
Adversarial attacks have gained traction in order to identify vulnerabilities in neural ranking models (NRMs), but current attack methods often introduce noticeable errors. Moreover, current methods rely heavily on using a well-imitated surrogate NRM to guarantee the attack effect, making them difficult to use in practice. This paper proposes a framework called Imperceptible DocumEnt Manipulation (IDEM) to produce adversarial documents that are less noticeable to both algorithms and humans. IDEM instructs a well-established generative language model like BART to generate error-free connection sentences, and employs a separate position-wise merging strategy to balance between relevance and coherence of the perturbed text. Evaluation results on the MS MARCO benchmark demonstrate that IDEM outperforms strong baselines while preserving fluency and correctness of the target documents. Furthermore, the separation of adversarial text generation from the surrogate NRM makes IDEM more robust and less affected by the quality of the surrogate NRM.

heading: 1 Introduction
text: Adversarial Information Retrieval (AIR) has been a topic of significant attention from the research community (Davison et al., 2006; Leng et al., 2012; Farooq, 2019). It refers to the scenario in which a portion of the collection can be maliciously manipulated, with Adversarial Web Search (Castillo and Davison, 2010) being a typical example. In the context of the Web, this type of manipulation is commonly referred to as black-hat search engine optimization (SEO) or Web spamming, whose goal is to deceive ranking algorithms by artificially inflating the relevance of targeted Web pages, resulting in undeservedly high rankings for those pages (Gyöngyi and Garcia-Molina, 2005). Meanwhile, in the last few years, neural ranking models (NRMs), particularly those that utilize pre-trained
language models (PLMs), have demonstrated remarkable performance across a diverse range of text ranking tasks (Lin et al., 2021). Furthermore, these NRMs have also been implemented in various industrial applications (Lin et al., 2021), such as Web search engines (Zou et al., 2021), to improve the accuracy and relevance of search results.
However, recent studies have revealed the vulnerabilities of NRMs to adversarial document manipulations (Wu et al., 2022; Liu et al., 2022; Wang et al., 2022; Song et al., 2022), that is, small deliberate perturbations in the input documents can cause a catastrophic ranking disorder in the outcome of NRMs. This highlights the need to investigate and identify the potential weaknesses of NRMs before their deployment, in order to ensure their robustness and prevent potential risks. Several manipulation techniques have been proposed to maliciously boost the ranking of low-ranked documents, such as PRADA (Wu et al., 2022) and PAT (Liu et al., 2022). Although these adversarial attack methods have shown the ability to fool NRMs by replacing crucial words, or appending query text or other ad-
6648
versarial tokens, they are still subject to two major limitations. Firstly, existing attack methods tend to introduce grammatical errors, impossible expressions, or incoherent text snippets into the original document, making the attack easy to mitigate, such as by perplexity filters (Song et al., 2020) or grammar checkers (Liu et al., 2022). Secondly, existing attack methods heavily rely on a well-imitated surrogate NRM to produce adversarial text, but this requires a lot of in-domain training data collected by querying the victim NRM, which can be infeasible or even unavailable in real-world situations.
To this end, we propose IDEM, an imperceptible document manipulation framework that aims to produce adversarial documents that are less perceptible to both humans and algorithms. As depicted in Figure 1, a well-established generative language model (GLM), such as BART, is first engineered to generate a series of grammatically correct connection sentences between the query and the target document, a position-wise merging mechanism is then employed to select an optimal connection sentence to be appropriately positioned within the target document. During the generation of connection sentences, we take advantage of the language modeling objective in the GLM without adding any ranking-incentivized objective. This not only helps to produce more natural and fluent text without introducing extra errors that are easy to detect, but also separates the surrogate NRM from the generation process to substantially reduce the dependence of attack performance on the surrogate NRM. Extensive experiments carried out on the widely-used MS MARCO passage ranking dataset indicate that IDEM is able to achieve better attack performance against black-box NRMs than recent baselines, regardless of whether the surrogate NRM is similar to or far from the victim NRM. According to both automatic and human evaluations, the adversarial documents produced by IDEM can maintain semantic fluency and text quality.
Our contributions are three-fold: 1) We propose an imperceptible document manipulation framework, IDEM1, that employs contextualized blankinfilling and coherent merging for the ranking attack. 2) Extensive attack experiments under three types of surrogate NRMs show that IDEM is able to robustly promote the rankings of the target documents. 3) Automatic and human evaluations of text
1We make our code and data publicly available at https: //github.com/cxa-unique/IDEM.
quality indicate that IDEM is capable of generating more natural and fluent adversarial documents.

heading: 2 Problem Statement
text: Task description. Typically, given a query q and candidates D= {d1, d2, · · · , d|D|} collected by a retrieval model (e.g., BM25; Robertson et al., 1995) from the whole corpus, a NRM MV (i.e., the victim NRM) produces relevance scores s(q, di) for all query-document pairs, outputting a re-ranking list L = [d1, d2, · · · , d|D|], wherein s(q, d1) > s(q, d2)> · · ·>s(q, d|D|). In the adversarial ranking attack, an adversarial document is constructed by applying a deliberate perturbation pi to the target document di ∈ L such that it can be ranked higher by the victim NRM. Existing attack methods search and find perturbations on different levels of granularity, such as replacing important words with synonyms (e.g., PRADA) or adding an extra text piece (e.g., PAT). Ideally, adversarial documents should display semantic consistency and fluency to avoid detection. However, there is still room for improvement regarding the imperceptibility of adversarial examples produced by existing methods, such examples can be found in Appendix A.5.
Task setting. For consistency with real-world situations (e.g., black-hat SEO), akin to recent studies (Wu et al., 2022; Liu et al., 2022), this work focuses on the decision-based black-box attack setting, where attackers can only obtain a limited number of queries {qi}mi=1 and their corresponding ranking lists {Li=[di,1, di,2, · · · , di,|D|]}mi=1 with rank positions by querying the victim NRM MV , but have no access to the exact relevance scores, as well as the architecture, parameters, gradients, or training data of the victim NRM. In this setting, a weaksupervised training data TS={(qi, di,j , di,k)}mi=1 is usually collected on basis of {(qi,Li)}mi=1, wherein 1≤j<k≤|D| so that di,j and di,k are considered as relatively positive and negative documents, respectively. A surrogate NRM MS is trained on this TS to imitate the victim NRM MV using a pairwise loss, such as hinge loss (Wu et al., 2022). Besides, existing methods require this surrogate NRM to be functionally similar to the victim NRM, as it determines the direction of adversarial attack (Liu et al., 2022). This means the attack performance of existing methods heavily depends on the surrogate NRM, but it needs lots of in-domain training samples (i.e., m is large) as well as the associated cost of querying the victim NRM, while our proposed
IDEM can achieve reliable attack effect even with an out-of-domain (OOD) surrogate NRM.

heading: 3 Method
text: As outlined in Figure 1 and Algorithm 1, IDEM contains two stages, the first is to obtain a series of connection sentences that can bridge the semantic gap between the query and the target document. After that, these connection sentences are considered at all positions in the inter-sentences of the target document to trade-off the semantic fluency and attacked relevance, and the best one is output as the final adversarial document.

heading: 3.1 Connection Sentences Generation
text: To obtain more natural and fluent connection sentences, we choose to draw support from the wellestablished GLMs, such as BART in our default setting. The BART model (Lewis et al., 2020) has been pre-trained using a text infilling loss function, which enables it to fill in blanks within the context in a more flexible way. Hence, in this section, we use the BART model as an example to illustrate how to engineer GLMs in a blank-infilling pattern to generate connection sentences that include information about both the query and the document.
Specifically, given a query q and a target document di∈L, which is simplified as d from here, we concatenate them with a blank in between as seen in the template as follows.
q ⊕ It is known that ⊕ d (1)
wherein query q ends with appropriate punctuation, and the blank is replaced by a single special token defined and used in the GLM, such as [MASK] token in the BART model. Given both query q and document d as the context, the prefix text “It is known that” serves as a prompt (Liu et al., 2021) to instruct the BART model to output more informative text that is related to both q and d. The choice of prompt words can have an impact on the final attack effect, which is analyzed in Section 4.3.
Afterward, the BART model takes Eq. 1 as the input and fills a variable number of tokens (i.e., sentences with varying lengths) into the blank position. Herein, we do not incorporate extra rankingincentivized objectives into the BART model in order to produce grammatically correct connection sentences well suited to the surrounding text. Besides, we employ the Top-k sampling (Fan et al.,
Algorithm 1 IDEM Input: a query q, a target document d, a surrogate NRM MS for a victim NRM MV Parameter: the number of sentence in the target document |d|, the max number of connection sentence N , sample times K, sample size M , the max word length of connection sentence L Output: an adversarial document dadv

heading: 1: Stage 1: Connection Sentences Generation
text: 2: Let Sc = {} 3: for 1 to K do 4: while |Sc| < N do 5: Sample M connection sentence candi-
dates {su}Mu=1 from the BART model by taking Eq. 1 as the input
6: if the length of su is smaller than L then 7: Keep su in Sc 8: end if 9: end while
10: end for 11: Stage 2: Merging with Original Document 12: for su in Sc do 13: for position index v from 0 to |d| do 14: Get an adversarial candidate dadv⟨u,v⟩ as in
Eq. 2, evaluate its coherence score (Eq. 3) and relevance score (Eq. 4), and compute the weighted merging score (Eq. 5)
15: end for 16: end for 17: Find and save the top-1 ranked dadv⟨u,v⟩ as d adv 18: return dadv
2018) strategy to ensure the diversity of the generated sentences. As depicted in Algorithm 1, we sample multiple times (i.e., K), take M candidates each time and save at most N connection sentences, denoted as Sc = {s1, s2, · · · sN}. We also limit the maximum length of the connection sentence to L words to control the interference in the semantic content of the original document. All connection sentences in Sc are considered in the next stage for the merging with the original document d.

heading: 3.2 Merging with Original Document
text: As Transformer-based NRMs have a positional bias towards the start of document (Jiang et al., 2021; Hofstätter et al., 2021), adding adversarial text pieces at the beginning can usually achieve remarkable attack effect (Wang et al., 2022), but this also has the obvious drawback that the attack can be easily detected (Liu et al., 2022). Thus, to
balance between the attack effect and the fidelity of perturbed document content, we design a positionwise merging strategy to place an appropriate connection sentence at an optimal position within the original target document.
Specifically, given a query q, a target document with multiple sentences d = s1, s2, · · ·, s|d| and a set of connection sentences Sc, we evaluate a large number of candidate adversarial documents with respect to the various combinations of d and Sc. For each connection sentence su ∈ Sc, we merge su with the target document d by placing it at position index v (i.e., an integer from 0 to |d|), obtaining adversarial candidates in the form of Eq. 2.
dadv⟨u,v⟩ =    su ⊕ d v = 0 d1→v ⊕ su ⊕ d(v+1)→|d| 0 < v < |d| d⊕ su v = |d| (2) wherein da→b means the text piece consists of consecutive sentences from sa to sb, and the subscript ⟨u, v⟩ means the connection sentence su is inserted at the index v of the target document d.
Subsequently, we examine all of the candidate adversarial documents dadv⟨u,v⟩ and find out the best one as the final adversarial document dadv. An effective adversarial example should be imperceptible to human judges yet misleading to NRMs (Wu et al., 2022). If the semantics of the added connection sentence differs greatly from the surrounding text in the target document, the resulting content will be incoherent and easily noticed by humans. Thus, with the help of the next sentence prediction (NSP) function in the pre-trained BERT model (Devlin et al., 2019), we define a coherence score on the junction between the connection sentence and the original target document.
Specifically, when the connection sentence su is placed in the middle of the target document d (i.e., 0<v < |d|), both the text pieces before and after the connection sentence are fed into the NSP function to get the coherence score as seen in Eq. 3.
coh(dadv⟨u,v⟩) = 0.5× [fnsp(d1→v, su ⊕ d(v+1)→|d|) + fnsp(d1→v ⊕ su, d(v+1)→|d|)] (3) wherein fnsp(A,B) is the NSP score of text A being followed by text B. Similarly, when su is attached to the beginning or the end of the document d (i.e., v=0 or v= |d|), the coherence score is given as fnsp(su, d) or fnsp(d, su), respectively.
Apart from the semantic coherence, another important factor that impacts the feasibility of attack is the relevance between the query q and the candidate dadv⟨u,v⟩. Herein, due to the black-box setting as described in Section 2, we need a surrogate NRM MS to estimate the relevance score as in Eq. 4.
rel(q, dadv⟨u,v⟩) = MS(q, dadv⟨u,v⟩) (4) wherein the surrogate NRM MS is not required to be heavily ranking-consistent with the victim NRM MV but is only expected to distinguish which connection sentence contains spurious-relevant information (e.g., query keywords). In other words, the surrogate NRM does not participate in the generation of connection sentences, so that the adversarial information is from the BART model rather than the surrogate NRM. Thus, IDEM does not compulsively require a well-imitated surrogate NRM that is trained on a large number of in-domain training samples. As demonstrated later in Section 4.2, even when using an OOD surrogate NRM that is only slightly better than BM25, our proposed IDEM still achieves relatively high attack performance.
Finally, to trade-off the semantic coherence and relevance, as seen in Eq.5, we resort to a weighted sum of them as the merging score for each dadv⟨u,v⟩.
scorem(q, dadv⟨u,v⟩) = α× coh(dadv⟨u,v⟩) + (1− α)× rel(q, dadv⟨u,v⟩)
(5) wherein α is a weight factor, and both scores are transformed to [0, 1] by the min-max normalization before being added together. Out of all the candidates dadv⟨u,v⟩, the one with the highest merging score is chosen as the final adversarial document dadv.

heading: 4 Experiments

heading: 4.1 Experimental Setup
text: Dataset. Akin to previous works (Wu et al., 2022; Wang et al., 2022; Liu et al., 2022), we employ popular MS MARCO passage dataset (Nguyen et al., 2016) for our experiments, which contains about 8.8M passages (seen as documents in this paper) as the corpus. The evaluation data contains a Dev set with 6,980 queries and an Eval set with 6,837 queries. In addition, we use Natural Question (NQ) dataset (Kwiatkowski et al., 2019) that has been processed by Karpukhin et al. (2020) as the OOD training source for the surrogate NRM.
Victim NRM. Akin to Liu et al. (2022), the ‘msmarco-MiniLM-L-12-v2’ model publicly available
at Sentence-Transformers (Reimers and Gurevych, 2019) is used as the representative victim NRM (i.e., MV ) in our study. This victim NRM adopts MiniLM (Wang et al., 2020) as the backbone, and it is fine-tuned on MS MARCO in a cross-encoder architecture (Nogueira and Cho, 2019). Overall, MV achieves highly effective ranking performance on the MS MARCO Dev set as seen in Table 2. The transfer attacks from MV to other types of victim NRMs (e.g., MonoT5) are available in Section 4.2.
Surrogate NRMs. To examine the practicality of ranking attack methods in real-world situations, wherein access to the victim ranking system could be limited or unavailable, we experiment with three kinds of surrogate NRMs: MS1 is trained on all 6,837 Eval queries, MS2 is trained on the randomly sampled 200 Eval queries, and MS3 is trained on the OOD NQ queries. More details of surrogate NRMs can be found in Appendix A.1.
Target documents. Following Wu et al. (2022), we evaluate attack methods on randomly sampled 1K Dev queries with two types of target documents, i.e., Easy-5 and Hard-5, which are sampled from the re-ranked results by the victim NRM on the top1K BM25 candidates. Determining whether a document is “Easy” or “Hard” depends on the difficulty of boosting it into the top-10 or top-50 positions. Specifically, Hard-5 is the five bottom-ranked documents, and Easy-5 is the five documents ranked between [51, 100] in which one document is randomly picked out of every 10 documents.
Details of IDEM. By default, pre-trained BARTBase (Lewis et al., 2020) is instructed to generate connection sentences. The impacts of other GLMs, including pre-trained T5 (Raffel et al., 2020) and fine-tuned GPT-2 (Donahue et al., 2020), are also examined in Section 4.3. In the generation stage, k in sampling strategy is set as 50, and we sample 10 or 50 times (50 ones per time) and save at most 100 or 500 connection sentences with a max length of 12 words (i.e., M=50, K=10/50, N=100/500 and L=12). In the merging stage, we employ the NSP function in pre-trained BERT-Base to evaluate the coherence, and the α in Eq.5 is set as 0.5 (0.1) for Easy-5 (Hard-5) target documents. The impacts of L and N are available in Section 4.3, and the impact of α is analyzed in Appendix A.4.
Compared methods. We compare the following attack methods against NRMs: Query+ (Liu et al., 2022) directly adds the query text at the beginning of the target document. PRADA (Wu et al., 2022)
finds and replaces important words with synonyms in the target document. Brittle-BERT (Wang et al., 2022) and PAT (Liu et al., 2022) append a few tokens at the beginning of the target document, the former only considers the ranking-oriented objective, while the later further considers semantic and fluency constraints. For fair comparisons, PRADA perturbs at most 20 tokens, Brittle-BERT and PAT add at most 12 tokens. More details of these attack baselines are available in Appendix A.2.
Metrics. As for the re-ranking results, we report official MRR@10/1K metrics on the MS MARCO Dev set. Akin to Liu et al. (2022), we calculate the overlap of top-10 (Inter@10) and the Rank Biased Overlap (Webber et al., 2010) (RBO@1K, p is set as 0.7) to measure the ranking consistency between the surrogate and victim NRMs. As for the attack results, we report the percentage of successfully boosted target documents (i.e., attack successful rate, ASR) following Wu et al. (2022). Akin to Liu et al. (2022), we also report the average boosted ranks (Boost) of the target documents, and the percentage of the target documents that are promoted into top-10 (% r≤ 10) and top-50 (% r≤ 50). In addition, we measure the average perplexity (PPL) calculated using a pre-trained GPT-2 model (Radford et al., 2019), where a lower PPL value reflects better fluency of the adversarial documents as suggested by Kann et al. (2018) and Lei et al. (2022).

heading: 4.2 Results
text: IDEM demonstrates the ability to achieve comparable attack performance while not greatly diminishing the semantic fluency. The attack results are summarized in Table 1, where all values are averaged across the 5K target documents for 1K Dev queries. When the query is added to the beginning of the target document (referred to as Query+), a notable increase in ranking can be observed, which is as expected. However, this approach negatively impacts semantic fluency, resulting in a loss of approximately 8 (17) perplexity points on Easy-5 (Hard-5) target documents. Ideally, when the victim NRM is freely accessible, a sufficient amount of surrogate training data is obtainable to train a well-imitated surrogate model, like MS1 in Table 2. As demonstrated in Table 1, when applied in conjunction with MS1 , the use of word-level synonym substitutions (i.e., PRADA) does not provide superior attack performance, but adding adversarial tokens at the beginning of the target documents
(i.e., Brittle-BERT and PAT) yields better attack performance. However, adversarial documents generated by these prior methods, particularly PRADA and Brittle-BERT, exhibit excessively high perplexity (PPL) values. In contrast, our proposed IDEM not only achieves comparable attack performance, exemplified by its superior performance on Easy-5 target documents, but also mitigates the detrimental effects on text fluency, as evidenced by the lower PPL values on its adversarial documents.
IDEM is more robust and much less affected by the surrogate NRM, even when it is an OOD NRM. When access to the victim NRM (i.e., MV ) is restricted, as shown in Table 2, there is a notable discrepancy between the surrogate MS2 and the victim MV , as only a limited number of indomain training samples can be used. When working with MS2 , the attack performances of all methods, particularly PAT and Brittle-BERT, are greatly diminished as can be observed in Table 1. For ex-
ample, when MS1 is changed to MS2 , the attack performance (% r ≤ 10) of Brittle-BERT drops dramatically from 81.3 to 33.2 on Easy-5 target documents, and from 61.5 to 8.98 on Hard-5 target documents. Additionally, the semantic fluency of adversarial documents produced by Brittle-BERT also decreases, as evidenced by an increase in PPL from 107.9 to 131.3 on Easy-5 target documents. Furthermore, when access to the victim NRM is not available, only publicly available OOD training samples can be used, yield a more discrepant surrogate NRM, like MS3 in Table 2. In this situation, as shown in Table 1, the performances of prior attack methods are also not ideal, especially PRADA and PAT. However, when working with both MS2 and MS3 , IDEM demonstrates the best attack results among all baselines while preserving the semantic fluency of target documents. For more details, including the attack efficiency and adversarial examples for different attack methods, please refer to Appendixes A.3 and A.5, respectively.
IDEM generates more general adversarial documents that can be well transferred across different victim NRMs. To examine the transferability of adversarial documents, considering that the target victim NRM is subject to continuous improvement, updates, or even changes, we conduct experiments with three different NRMs as the attack targets: ELECTRA (‘ms-marco-electrabase’; Reimers and Gurevych, 2019), MonoT5 (‘monot5-base-msmarco’; Nogueira et al., 2020)
and ColBERT (Khattab and Zaharia, 2020). The adversarial documents were generated by attacking the MV (‘ms-marco-MiniLM-L-12-v2’) NRM and subsequently used to attack other victim NRMs. Due to space constraints, Table 3 primarily focuses on presenting the cross-attack results of all adversarial target documents generated using the surrogate NRM MS2 in both Easy-5 and Hard-5 sets. As observed in Table 3, regardless of the target victim NRM, the attack performances of different methods exhibit a consistent trend: IDEM > Brittle-BERT > PAT > PRADA. This trend is also in line with the results presented in Table 1, indicating that adversarial documents that display higher aggression towards one NRM are more likely to be effective against other NRMs due to shared characteristics among the models, such as exact matching.

heading: 4.3 Analysis
text: Impact of prompt in IDEM. During the generation of connection sentence, a prompt (in Eq.1) is utilized to guide the BART model to output more informative text. Herein, we examine the impact of prompt on the final attack results of IDEM. As seen in Table 4, four different prompts are analyzed, in which “It is known that” produces better attack results, particularly on Hard-5 target documents, and also generates lower PPL values. In contrast, “The fact is that” and “We know that” perform slightly worse on Hard-5 target documents, and “It is about that” performs even worse. Overall, the choice of prompt has marginal impact on the ASR but mainly

heading: Easy-5

heading: Hard-5

heading: Easy-5

heading: Hard-5
text: affects the degree of promotion in the attack. Position of connection sentence in IDEM. As mentioned in Section 3.2, IDEM can automatically place an adversarial connection sentence within the target documents. As summarized in Table 5, we count the positions (i.e., index v) of connection sentences. When the surrogate NRM is MS1 or MS2 , it is observed that less than 30% of the connection sentences are positioned in the middle (i.e., v≥1). Meanwhile, when the surrogate NRM is MS3 , the distribution of insert positions is found to be more uniform, with only 10-20% of the connection sentences being appended to the beginning (i.e., v=0). These findings indicate that IDEM can place the adversarial text at any positions within the target document, making it harder to be detected.
IDEM based on other GLMs. In addition to evaluating IDEM using BART, we also examine how well IDEM performs when other GLMs with blank-filling capabilities are instructed to generate connection sentences, including T5-Base (Raffel et al., 2020) and ILM (a fine-tuned version of GPT2; Donahue et al., 2020). As presented in Table 6,
our empirical results show that while the BARTbased IDEM is found to be the overall best, all three models achieve high attack success rates, indicating the general applicability of the IDEM approach.
Online grammar checking. In order to examine the extra errors introduced by different attack methods, we employ the popular online grammar checker Grammarly2 to evaluate the quality of adversarial documents. Specifically, we collect 100 adversarial documents (with the same id) produced by each attack method for 50 Dev queries. We report three evaluation metrics given by Grammarly, including the average number of issues in correctness (e.g., spelling, grammar, and punctuation) and suggestions (e.g., wordy or unclear sentences, etc.) in each adversarial document, and the quality score of all adversarial documents. As shown in Table 7, IDEM introduces the fewest issues and obtains the highest quality score among all attack methods, indicating that the adversarial documents produced by IDEM are more machine-imperceptible.
Human evaluation. To further prove that the adversarial documents generated by IDEM are more natural to readers, a human-subject evaluation was conducted to assess the imperceptibility of adversarial text. Specifically, 32 adversarial documents were randomly selected for each attack method,
2https://app.grammarly.com

heading: Method Human-ImperceptibilityAvg. Kappa

heading: Method Linguistic Acceptability Accuracy
text: and 32 original unaltered documents were also selected. All these documents were then mixed and randomly divided into two groups, with each group being evaluated by two annotators who are computer science graduate students with the necessary knowledge to understand the nature of the ranking attack. The annotators were tasked with determining whether the document content had been attacked (0) or was normal (1). We averaged all annotations on 32 samples from 2 annotators as the final imperceptibility score, and also computed Kappa coefficient for the annotation consistency. As can be seen in Table 8, IDEM receives the highest score for human imperceptibility among all attack methods. Additionally, the Kappa values of almost all attack methods are larger than 0.4 (considered as “moderate agreement”), while IDEMN=100 has the smallest Kappa value (i.e., 0.11), which seems reasonable since it is hard to reach an agreement on the more imperceptibly attacked documents.
Mitigation by linguistic acceptability. In an effort to mitigate the attacks, we make additional attempts using a classification model3 trained on the CoLA dataset (Warstadt et al., 2019). The same adversarial documents subjected to online grammar
3https://huggingface.co/textattack/ roberta-base-CoLA

heading: Max Length L
text: checking are evaluated for linguistic acceptability (LA) using this model. In addition to the LA values, the classification accuracy is reported as an indicator of the mitigation effectiveness. As indicated in Table 9, IDEM’s adversarial documents exhibit linguistic acceptability closest to that of the original documents, while Brittle-BERT’s adversarial documents are deemed the most unacceptable. Moreover, this LA model can successfully identify over 50% of the adversarial documents from Query+, PRADA and PAT, and even 97% of the adversarial documents from Brittle-BERT. In contrast, only 27-28% of IDEM’s adversarial documents are filtered out, implying that more than 70% of IDEM’s attacks remain effective against this mitigation. Also, the misclassification rate of this LA model on original documents is only 13%.
The impact of hyper-parameters. We evaluate IDEM with two hyper-parameters to study how they affect the attack performance: the max length (L) and max number (N ) of connection sentences. For L analysis, we maintain N at 100, while for N analysis, we keep L at 12. In Figure 2, the improvement in IDEM’s attack performance becomes less significant as L increases from 12 to 24 compared to that observed from 6 to 12, leading us to select L as 12. Similarly, in Figure 3, IDEM’s performance
improves as N increases, but the time required to generate one adversarial document increases at a faster rate. Hence, we set N to 500 for a balance between attack effectiveness and efficiency.

heading: 5 Related Work
text: The adversarial IR has been studied for an extended period, such as black-hat SEO, which refers to the intentional manipulation of Web pages with the goal of obtaining an unjustified ranking position, resulting in a decline in the quality of search results and an inundation of irrelevant pages (Gyöngyi and Garcia-Molina, 2005). In this context, mainstream research focuses on studying the adversarial manipulations through various aspects, such as detection (Dalvi et al., 2004; Ntoulas et al., 2006), theoretical and empirical analysis (Raifer et al., 2017), robustness of LTR-based ranking functions (Goren et al., 2018), automatic content modification (Goren et al., 2020), and other research directions (Kurland and Tennenholtz, 2022).
Recently, there has been significant progress in NRMs, particularly those leveraging PLMs, which have shown exceptional performance in text ranking (Lin et al., 2021). Concurrently, an increasing number of studies have shed light on the robustness concerns of NRMs in various scenarios, including the presence of query typos or variations (Zhuang and Zuccon, 2021; Penha et al., 2022; Chen et al., 2022), textual noises (Chen et al., 2023), and adversarial attacks (Raval and Verma, 2020; Song et al., 2022). Although current attack methods like PRADA (Wu et al., 2022), Brittle-BERT (Wang et al., 2022), and PAT (Liu et al., 2022) have shown the ability to deceive NRMs successfully, they introduce additional quality issues and often heavily rely on surrogate NRMs for document manipulation. Instead, our proposed IDEM effectively overcomes the limitations of these existing attack methods and showcases remarkable attack performance.

heading: 6 Conclusion
text: In this study, we introduce a document manipulation framework named IDEM, which is engineered to produce adversarial documents that are not easily detected by both humans and algorithms. Our experiments on the MS MARCO dataset show that IDEM can not only achieve a high level of attack performance, but also generate correct and fluent adversarial documents as evaluated by both automatic and human assessments.

heading: Limitations
text: In our experiments, as NRMs with cross-encoder are widely used, we focus on evaluating the textual adversarial robustness during the re-ranking stage and do not currently take into account the effect on the retrieval stage. But actually, in a “first retrieval then re-ranking” ranking paradigm, the attack is effective only when the adversarial documents are passed into the top retrieval results. Meanwhile, dense retrieval (DR) models have been widely studied, and they may also inherit adversarial vulnerabilities due to the basics of PLMs. Besides, due to limitations in our computing resources, we only tested adding adversarial text to relatively short documents (i.e., passage-level), but the document content in real-world applications could be much longer. Therefore, further comprehensive investigations on examining the NRMs with different architectures, the effects of attacks on the retrieval models, and the manipulations on longer documents are left for future work. Finally, it is important to note that mitigation and defense methods against adversarial ranking attacks are currently understudied, making it a significant area for future research.

heading: Ethics Statement
text: In this paper, we investigate the potential vulnerability concerns of the neural information retrieval (IR) systems and propose a document manipulation framework that generates adversarial documents that are not easily detected by both humans and IR systems. We hope that this study could inspire further exploration and design of adversarial ranking defense/detection methods and aid in the development of robust real-world search engines.

heading: Acknowledgements
text: This work is supported by the National Natural Science Foundation of China under Grants no. U1936207 and 62272439.

heading: A Appendix

heading: A.1 Details of Surrogate NRMs
text: As mentioned in Section 4.1, we conduct black-box attacks against the victim neural ranking model (NRM) MV using three types of surrogate NRMs. Since the training data of the victim NRM is typically unavailable, we use Eval queries in the MS MARCO dataset rather than train queries to construct the surrogate training data TS . As we move from MS1 to MS2 , the number of in-domain Eval queries used for ranking imitation decreases from 6,837 to 200, it means the frequency of querying the victim NRM is greatly reduced. For each Eval query qi, we collect all 406 document pairs (di,j , di,k) (1≤ j <k≤29) from the top-29 of the re-ranking list produced by the victim NRM over the top-1K BM25 candidates, and use this to construct the surrogate training data TS as described in Section 2. As a result, the in-domain surrogate training data for MS1 and MS2 contains 81.2 thousand and 2.77 million training triples, respectively. Additionally, MS3 implies the scenario where no in-domain data from the victim NRM is available, and we collect 3.72 million training triples from the NQ dataset as the out-of-domain (OOD) surrogate training data. The surrogate NRMs are based on the pre-trained BERT-Base model and fine-tuned for two epochs with a learning rate of 3e-6 and batch size of 16 for MS1 and MS2 , and one epoch for MS3 with the same learning rate and batch size. It is important to note that the goal of this work is not to develop a new training method for the surrogate NRMs, and thus we directly adopt the hinge loss with a margin of 1 for ranking imitation as per previous work (Wu et al., 2022).

heading: A.2 Details of Attack Baselines
text: In our adversarial attack experiments, we examine the following baseline methods:
Query+ (Liu et al., 2022) is an intuitive baseline that directly appends the query text to the beginning of the target document. Although the query text can be placed at any position in the target document, or even determined by our proposed positionaware mechanism, appending to the beginning usually produces greater attack results due to the positional bias in Transformer-based NRMs (Jiang et al., 2021; Hofstätter et al., 2021). Thus, Query+ acts as a baseline method that does not take into account the invisibility aspect of the attack.
PRADA (Wu et al., 2022) is a Word Substitution
Ranking Attack (WSRA) method, it first finds important words (i.e., sub-word tokens) in the target document according to the gradient magnitude (Xu and Du, 2020), and then greedily replaces them with the synonyms found in a perturbed word embedding space via PGD (Madry et al., 2018). Based on our observations, when attacking different random target samples, PRADA is able to attain attack performance (ASR) that is close to the results reported in its original publication. This is particularly true when the victim NRM has relatively poor ranking performance, due to the obvious fact that it is much easier to attack a weaker victim NRM. Despite this variability, the overall conclusions drawn from our experimentation remain unchanged.
Brittle-BERT (Wang et al., 2022) studies both local (i.e., a particular query-document instance) and global (i.e., an entire workload of queries) ranking attack to cause a large rank demotion or promotion by adding/replacing a small number of tokens. In our work, we only adopt the local setting and add tokens to the beginning of the target document as it usually produces better attack results. Specifically, Brittle-BERT first initializes a few placeholder tokens at the beginning of the target document and then employs HotFlip (Ebrahimi et al., 2018) algorithm to update them as being more adversarial.
PAT (Liu et al., 2022) generates and adds several trigger tokens at the beginning of the target document. In addition to the ranking-incentivized objective, the search objective of PAT is equipped with semantic and fluency constraints using the pretrained BERT model. The surrogate NRMs trained with hinge loss have a one-class prediction layer, but PAT needs a surrogate NRM with a two-class prediction layer, namely, ‘Pairwise BERT’ as denoted in PAT (Liu et al., 2022), so we use the same surrogate training data to obtain ‘Pairwise BERT’ using the default imitation loss in PAT.
In order to evaluate these baselines, we utilize their publicly available implementations and ensure that all settings are consistent with those described in their respective official publications.

heading: A.3 Time Cost of Attack
text: In previous PRADA, Brittle-BERT and PAT, the replacement, selection, and search of tokens are carried out one by one using the surrogate NRM to produce an adversarial document, so it needs a large amount of time to complete the attack process. However, in our proposed IDEM framework,
the adversarial text is first generated via a GLM (e.g., BART), and then combined in the sentence level, which is more efficient than the token level. As seen in Figure 4, we summarize the total time cost of different attack methods in producing 5,000 adversarial documents using one Titan RTX GPU. When at most 100 candidate connection sentences are generated for the position-wise combination, IDEMN=100 only takes 11.3 hours to achieve great attack performance. By comparison, PRADA and PAT consume more time but still perform worse in attack, Brittle-BERT takes a huge time cost (near 800 hours) even though its attack performance is considerable. Additionally, compared with BrittleBERT, IDEMN=500 can produce comparable attack performance but take much less time cost. Therefore, our proposed IDEM method lays equal stress on attack efficiency and performance.

heading: A.4 The Impact of α in the Merging Score
text: In our IDEM framework, after generating a series of connection sentences between the query and the target document, a position-aware merging mechanism is employed to decide the final adversarial document, wherein a coherence score and a rele-
vance score are added together using α as seen in Eq. 5. As shown in Figure 5, we can observe that α in a wide range (from 0 to 0.95) does not affect the attack success rate (ASR) too much on both Easy-5 and Hard-5 target document sets, and the % r ≤ 10 metric starts to decrease at α=0.5 and α=0.1 on Easy-5 and Hard-5 target document sets, respectively. As for the perplexity (PPL) metric (smaller is better), when α increases (more attention on the coherence), PPL value does not change a lot until α reaches about 0.9 to 1. Meanwhile, it can produce adversarial documents with lower PPL than original ones, e.g., when α is 1, the average PPL points on Easy-5 and Hard-5 target document sets are only 31.8 and 43.6, respectively.

heading: A.5 Adversarial Examples
text: To better understand the workings of various attack methods, we show adversarial examples produced by them under three types of surrogate NRMs in Table 10. We can observe that Query+, and BrittleBERT, PAT, IDEM under MS1 promote the target document ranked at 88th into top-10. However, adding query text (i.e., Query+) and unnatural token sequence (i.e., Brittle-BERT and PAT) at the beginning make adversarial documents distinguishable, while the inserted adversarial text by IDEM is more semantically consistent with the original surrounding content. When the surrogate NRM degrades from MS1 to MS2 as not enough in-domain training samples are available, we can see that the ranking of the adversarial document by PRADA decreases from 27th to 65th, and the ranking of the adversarial document by Brittle-BERT also decreases from 2nd to 14th. Furthermore, when an OOD surrogate NRM (i.e., MS3) is used due to the forbidden access to the victim NRM, we can find out that the attack effects of PRADA, BrittleBERT and PAT are greatly suppressed. For example, although PAT under MS2 promotes the target document to the ranking of 2nd, PAT under MS3 even demotes the ranking of the target document from 88th to 107th. In contrast, under both MS2 and MS3 , IDEM robustly promotes the target document into top-10 (i.e., 9th), and the fluency and correctness of adversarial documents are still within an acceptable range. From these adversarial cases, it is evident that IDEM is less dependent on the surrogate NRM and can perform attacks more robustly than previous attack methods, indicating a flexible use condition of IDEM in real-world situations.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Section: Limitations
3 A2. Did you discuss any potential risks of your work? Section: Ethics Statement
3 A3. Do the abstract and introduction summarize the paper’s main claims? Section 1: Introduction
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4.1: Experimental Setup
3 B1. Did you cite the creators of artifacts you used? Section 4.1: Experimental Setup
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? The MS MARCO dataset utilized in our research is freely accessible to the public, without any accompanying licenses or intellectual property restrictions.
7 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? We employ the MS MARCO dataset exclusively for non-commercial research endeavors, adhering to their intended purpose. Similarly, the data generated within our work is strictly intended for research purposes, aiming to foster progress in the field of information retrieval and related domains.
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? In light of the widespread usage of the MS MARCO dataset in information retrieval tasks, we did not specifically examine the textual data contained within it for potential privacy or offensive information.
7 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? The MS MARCO dataset utilized in our research is openly accessible, and comprehensive information about it can be found on its official page.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4.1: Experimental Setup; Appendix A.1
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C 3 Did you run computational experiments? Section 4: Experiments
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Appendix A.3
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Section 4.1: Experimental Setup
7 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? The outcomes presented in our work are based on a single execution due to constraints in our computing resources.
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 4: Experiments; Appendix
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section 4.3: Analysis
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Our human annotation process does not involve such a question.
3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Section 4.3: Analysis
3 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Section 4.3: Analysis
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Our human annotation process does not involve such a question.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Our human annotation process does not involve such a question.

heading: Future Work
text: Therefore, further comprehensive investigations on examining the NRMs with different architectures, the effects of attacks on the retrieval models, and the manipulations on longer documents are left for future work. Finally, it is important to note that mitigation and defense methods against adversarial ranking attacks are currently understudied, making it a significant area for future research