Breaking the Reviewer: Assessing the Vulnerability of Large Language Models in Automated Peer Review Under Textual Adversarial Attacks (2025)

Abstract: Peer review is essential for maintaining academic quality, but the increasing volume of submissions places a significant burden on reviewers. Large language models (LLMs) offer potential assistance in this process, yet their susceptibility to textual adversarial attacks raises reliability concerns. This paper investigates the robustness of LLMs used as automated reviewers in the presence of such attacks. We focus on three key questions: (1) The effectiveness of LLMs in generating reviews compared to human reviewers. (2) The impact of adversarial attacks on the reliability of LLM-generated reviews. (3) Challenges and potential mitigation strategies for LLM-based review. Our evaluation reveals significant vulnerabilities, as text manipulations can distort LLM assessments. We offer a comprehensive evaluation of LLM performance in automated peer reviewing and analyze its robustness against adversarial attacks. Our findings emphasize the importance of addressing adversarial risks to ensure AI strengthens, rather than compromises, the integrity of scholarly communication.

## introduction peer review is crucial for examining the quality and integrity of academic publications. however, recent studies @xcite @xcite @xcite highlighted a rapid increase in manuscript submissions, placing a significant burden on reviewers. increasing specialization within fields further complicates the process, making it harder for reviewers to provide insightful comments on topics outside their expertise. consequently, some reviews are less comprehensive or insightful than they might * corresponding author. in response to these challenges, there is growing interest in leveraging automated systems to assist in the peer review process. advances in natural language processing (nlp) and machine learning have opened promising avenues for developing tools that can augment human efforts in review generation. early work by @xcite proposed an automated review generation process that combines traditional bibliometric analysis with bert-based extractive summarization to identify key contributions in manuscripts. more recently, @xcite ; @xcite conducted a comprehensive evaluation of llms on automatic paper reviewing tasks, exploring the reliability of llms as potential reviewers. despite the benefits, the integration of llms into the peer review process raises concerns. one pressing issue is the vulnerability of llmgenerated reviews to textual adversarial attacks. given that llms heavily rely on textual input, simple manipulations of the content could significantly distort their assessments. this vulnerability poses a security risk, especially if reviewers, intentionally or unintentionally, use mainstream llms to generate reviews without rigorous oversight. in this paper, we investigate the robustness of llms as scientific paper reviewers. specifically, we address three research questions: to address these questions, we begin by evaluating the performance of leading llms, including closed-source models (e.g., gpt-4o, gpt-4o-mini), as well as open-source models (e.g., llama-3.3-70b, mistral-small-3.1), on two core peer review tasks: review generation and score prediction. our evaluation demonstrates the models' capability to produce coherent and fluent reviews across multiple assessment aspects. we then apply common textual adversarial attack techniques to test the robustness of the llm's evaluations. our findings reveal significant vulnerabilities, as the llm's assessments can be easily distorted through simple manipulations of the input text. our contributions are threefold: we conduct an in-depth evaluation of llm performance in aspect-based automatic paper reviewing, comparing outputs with human reviewers in the review generation task. ## llm vulnerability on paper reviews in response to the growing use of llms in peer review @xcite @xcite , we investigate their ability to automatically generate scientific reviews and examine their robustness against textual adversarial attacks, which are subtle edits that preserve meaning but may influence model outputs. in section 3.1, we simulate a realistic review workflow in which reviewers employ llms in a zero-shot manner to produce aspect-tagged reviews and scores with minimal manual effort. to evaluate the resilience of this process, we introduce attack focus localization in section 3.2, a method that identifies effective textual adversarial attack candidates in extremely long inputs via longest common subsequence (lcs) matching. in section 3.3, we integrate this localization algorithm with existing adversarial perturbations (e.g., typos or pattern-based edits at the character, word, or sentence level), which can potentially bias the llm's output (see fig. 1 ). ## defenses and limitations while recent studies have proposed using aigenerated text detectors (e.g., gptzero, openai classifier) to identify llm-authored reviews or adversarially manipulated inputs, such detectionbased defenses face fundamental limitations when confronting the vulnerabilities in the peer-review process revealed in this work. ## conclusion we present a framework to evaluate the capabilities and robustness of large language models (llms) in automated peer-review tasks, addressing the growing burden on reviewers. using mainstream llms, we perform review generation and score prediction, demonstrating their ability to provide coherent, insightful feedback. to assess robustness, we apply adversarial techniques to manipulate paper content and observe their impact on predicted scores, revealing vulnerabilities where subtle input changes affect reliability. we also propose attack focus localization to identify document regions most vulnerable to attacks, offering finer-grained insights into llm weaknesses. although such manipulations remain difficult to detect, exploring mitigation strategies-such as inference-time monitoring or human-in-the-loop verification-represents a promising avenue to improve the trustworthiness of llm-assisted peer review.