Towards Imperceptible Document Manipulations against Neural Ranking Models (2023)

Abstract: Adversarial attacks have gained traction in order to identify vulnerabilities in neural ranking models (NRMs), but current attack methods often introduce noticeable errors. Moreover, current methods rely heavily on using a well-imitated surrogate NRM to guarantee the attack effect, making them difficult to use in practice. This paper proposes a framework called Imperceptible DocumEnt Manipulation (IDEM) to produce adversarial documents that are less noticeable to both algorithms and humans. IDEM instructs a well-established generative language model like BART to generate error-free connection sentences, and employs a separate position-wise merging strategy to balance between relevance and coherence of the perturbed text. Evaluation results on the MS MARCO benchmark demonstrate that IDEM outperforms strong baselines while preserving fluency and correctness of the target documents. Furthermore, the separation of adversarial text generation from the surrogate NRM makes IDEM more robust and less affected by the quality of the surrogate NRM.

## introduction adversarial information retrieval (air) has been a topic of significant attention from the research community @xcite @xcite . it refers to the scenario in which a portion of the collection can be maliciously manipulated, with adversarial web search @xcite being a typical example. in the context of the web, this type of manipulation is commonly referred to as black-hat search engine optimization (seo) or web spamming, whose goal is to deceive ranking algorithms by artificially inflating the relevance of targeted web pages, resulting in undeservedly high rankings for those pages @xcite . meanwhile, in the last few years, neural ranking models (nrms), particularly those that utilize pre-trained figure 1 : in idem, a generative language model is instructed to generate a group of connection sentences between the query and the target document, then a positionwise merging strategy is applied to select and position an optimal connection sentence within the target document, in order to promote the ranking (from 88th to 9th) but maintain the fluency measured by perplexity (ppl). language models (plms), have demonstrated remarkable performance across a diverse range of text ranking tasks @xcite . furthermore, these nrms have also been implemented in various industrial applications @xcite , such as web search engines @xcite , to improve the accuracy and relevance of search results. however, recent studies have revealed the vulnerabilities of nrms to adversarial document manipulations @xcite @xcite , that is, small deliberate perturbations in the input documents can cause a catastrophic ranking disorder in the outcome of nrms. this highlights the need to investigate and identify the potential weaknesses of nrms before their deployment, in order to ensure their robustness and prevent potential risks. several manipulation techniques have been proposed to maliciously boost the ranking of low-ranked documents, such as prada @xcite and pat @xcite . although these adversarial attack methods have shown the ability to fool nrms by replacing crucial words, or appending query text or other ad-versarial tokens, they are still subject to two major limitations. firstly, existing attack methods tend to introduce grammatical errors, impossible expressions, or incoherent text snippets into the original document, making the attack easy to mitigate, such as by perplexity filters @xcite or grammar checkers @xcite . secondly, existing attack methods heavily rely on a well-imitated surrogate nrm to produce adversarial text, but this requires a lot of in-domain training data collected by querying the victim nrm, which can be infeasible or even unavailable in real-world situations. to this end, we propose idem, an imperceptible document manipulation framework that aims to produce adversarial documents that are less perceptible to both humans and algorithms. as depicted in figure 1 , a well-established generative language model (glm), such as bart, is first engineered to generate a series of grammatically correct connection sentences between the query and the target document, a position-wise merging mechanism is then employed to select an optimal connection sentence to be appropriately positioned within the target document. during the generation of connection sentences, we take advantage of the language modeling objective in the glm without adding any ranking-incentivized objective. this not only helps to produce more natural and fluent text without introducing extra errors that are easy to detect, but also separates the surrogate nrm from the generation process to substantially reduce the dependence of attack performance on the surrogate nrm. extensive experiments carried out on the widely-used ms marco passage ranking dataset indicate that idem is able to achieve better attack performance against black-box nrms than recent baselines, regardless of whether the surrogate nrm is similar to or far from the victim nrm. according to both automatic and human evaluations, the adversarial documents produced by idem can maintain semantic fluency and text quality. our contributions are three-fold: 1) we propose an imperceptible document manipulation framework, idem 1 , that employs contextualized blankinfilling and coherent merging for the ranking attack. 2) extensive attack experiments under three types of surrogate nrms show that idem is able to robustly promote the rankings of the target documents. 3) automatic and human evaluations of text 1 we make our code and data publicly available at https: //github.com/cxa-unique/idem . ## problem statement task description. typically, given a query q and candidates @xmath0, d 2 , • • • , d |d| } collected by a retrieval model (e.g., bm25; @xcite from the whole corpus, a nrm m v (i.e., the victim nrm) produces relevance scores s(q, d i ) for all query-document pairs, outputting a re-ranking list in the adversarial ranking attack, an adversarial document is constructed by applying a deliberate perturbation p i to the target document d @xmath1. existing attack methods search and find perturbations on different levels of granularity, such as replacing important words with synonyms (e.g., prada) or adding an extra text piece (e.g., pat). ideally, adversarial documents should display semantic consistency and fluency to avoid detection. however, there is still room for improvement regarding the imperceptibility of adversarial examples produced by existing methods, such examples can be found in appendix a.5. ## method as outlined in figure 1 and algorithm 1, idem contains two stages, the first is to obtain a series of connection sentences that can bridge the semantic gap between the query and the target document. after that, these connection sentences are considered at all positions in the inter-sentences of the target document to trade-off the semantic fluency and attacked relevance, and the best one is output as the final adversarial document. ## related work the adversarial ir has been studied for an extended period, such as black-hat seo, which refers to the intentional manipulation of web pages with the goal of obtaining an unjustified ranking position, resulting in a decline in the quality of search results and an inundation of irrelevant pages @xcite . in this context, mainstream research focuses on studying the adversarial manipulations through various aspects, such as detection @xcite , theoretical and empirical analysis @xcite , robustness of ltr-based ranking functions @xcite , automatic content modification @xcite , and other research directions @xcite . recently, there has been significant progress in nrms, particularly those leveraging plms, which have shown exceptional performance in text ranking @xcite . concurrently, an increasing number of studies have shed light on the robustness concerns of nrms in various scenarios, including the presence of query typos or variations @xcite @xcite , textual noises @xcite , and adversarial attacks @xcite . although current attack methods like prada @xcite , brittle-bert @xcite , and pat @xcite have shown the ability to deceive nrms successfully, they introduce additional quality issues and often heavily rely on surrogate nrms for document manipulation. instead, our proposed idem effectively overcomes the limitations of these existing attack methods and showcases remarkable attack performance. ## conclusion in this study, we introduce a document manipulation framework named idem, which is engineered to produce adversarial documents that are not easily detected by both humans and algorithms. our experiments on the ms marco dataset show that idem can not only achieve a high level of attack performance, but also generate correct and fluent adversarial documents as evaluated by both automatic and human assessments.