Title: MedGuards: Multi-Agent System for Reliable Medical Error Detection and Correction
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: Multi-agent system, in-context learning, error detection and correction

Abstract: Accurate error detection and correction is critical for medical scenarios, as even minor factual mistakes in generated text can pose serious risks to patient safety. Existing methods for error detection and correction, including automated checks and heuristic-based approaches, do not generalize well across unseen datasets. Moreover, widely used evaluation metrics, such as token-level similarity measures, such as ROUGE and BLEURT, do not reliably reflect clinical correctness, particularly when key medical entities are involved but not detected. In this paper, we propose MedGuards as a medical safety guardrail, which is a new framework that treats medical error detection and correction as a multi-agent in-context learning task. Specialized agents independently detect, localize, and correct errors, while a confidence-guided arbitration mechanism resolves disagreements using reasoning traces and uncertainty estimates. This design enhances interpretability, robustness, and adaptability, without requiring further training of the base Large Language Models (LLMs). Additionally, we introduce the Keyword-Prioritized Correction Score (KPCS), an evaluation metric that assigns greater weights to critical clinical entities, providing a more accurate assessment of safety than conventional metrics. Experiments across four multilingual medical datasets demonstrate significant improvements by the proposed framework across several metrics, with the aim of enabling safer deployment of LLMs in real-world healthcare applications. To enable reproducibility, we make our code publicly available at https://anonymous.4open.science/r/MedGuards-52F2/.