S ilver S peak: Evading AI -Generated Text Detectors using Homoglyphs (2025)

Abstract: The advent of Large Language Models (LLMs) has enabled the generation of text that increasingly exhibits human-like characteristics. As the detection of such content is of significant importance, substantial research has been conducted with the objective of developing reliable AI-generated text detectors. These detectors have demonstrated promising results on test data, but recent research has revealed that they can be circumvented by employing different techniques. In this paper, we present homoglyph-based attacks (‘A’ → Cyrillic ‘А’) as a means of circumventing existing detectors. We conduct a comprehensive evaluation to assess the effectiveness of these attacks on seven detectors, including ArguGPT, Binoculars, DetectGPT, Fast-DetectGPT, Ghostbuster, OpenAI’s detector, and watermarking techniques, on five different datasets. Our findings demonstrate that homoglyph-based attacks can effectively circumvent state-of-the-art detectors, leading them to classify all texts as either AI-generated or human-written (decreasing the average Matthews Correlation Coefficient from 0.64 to -0.01). Through further examination, we extract the technical justification underlying the success of the attacks, which varies across detectors. Finally, we discuss the implications of these findings and potential defenses against such attacks.

## introduction llms have soared in popularity in a wide variety of domains as their text generation capabilities become increasingly humanlike @xcite . for instance, it is estimated that the percentage of arxiv articles whose abstract has been revised by chatgpt is around 35% @xcite . while llms can prove beneficial @xcite , there is growing concern about their potential misuse @xcite @xcite . thus, a number of approaches to detect ai-generated text have been proposed, including zero-shot classifiers @xcite @xcite @xcite , binary classifiers @xcite @xcite , and watermarking techniques @xcite @xcite @xcite . at the same time, research has been conducted on methods for circumventing ai-generated text detectors. some popular techniques include paraphrasing @xcite , watermark stealing @xcite , substitution-based in-context example optimization @xcite , reinforcement learning @xcite or space infiltration @xcite . in this paper, we study an alternative technique dr. capy cosmos, a capybara unlike any other, astounded the scientific community with his groundbreaking. . . dr. capy cosmos, a capubara unlike anu οther, astοunded the scientific community with his groundbreak ng. . . @xcite rewrite(text) → text ′ tokenize @xcite tokenize(text ′ ) prediction: ai prediction: human based on homoglyphs. homoglyphs are visually similar characters with different encodings (e.g., latin 'a' and cyrillic 'a') @xcite . this allows us to generate rewritten versions of any given text that can evade ai-generated text detectors (figure 1 ). @xcite identified the usage of homoglyphs as a potential avenue for evading ai-generated text detectors. however, to the best of our knowledge, no study has yet conducted a comprehensive evaluation of the effectiveness of this approach across diverse datasets and detectors, nor has it provided insights into the technical justification of homoglyph-based attacks, a gap that we aim to fill in this paper. our main contributions are: we introduce them as a way to evade ai-generated text detectors. ## methods in this section, we delineate our experimental approach, along with a description of the detectors and datasets employed. we make our code and datasets publicly available at https://github.com/ acmcmc/silverspeak , under cc by-sa 4.0 and odc-by licenses. furthermore, we ensured that our study adheres to the intended usages of the detectors and datasets presented, for which we include licensing information below. ## results we summarize our experimental results in appendices a and b, with the raw results available in our released datasets. the results correspond to a single run, as we confirmed that the scores obtained are identical across multiple executions. it should be noted that some conventional metrics employed to assess the efficacy of detectors may prove to be deceptive in this particular setting. for example, figure 13e shows a confusion matrix where the f1 score is 0.67, but the detector is classifying almost all examples as ai-generated. we argue that the matthews correlation coefficient (mcc) is better suited (in this case, 0.08), placing greater emphasis on class balance @xcite . therefore, we use it as our main metric, and advise caution when interpreting the results based on other metrics in the appendices. mcc yields values from -1 (inverse correlation) to 1 (perfect corre-lation), with 0 representing no correlation. ## discussion in this section, we discuss the results obtained from the experiments conducted on different ai-generated text detectors using homoglyph-based attacks. then, we analyze the effectiveness of the attacks and their technical justifications. ## conclusion this paper demonstrated that homoglyphbased attacks can evade state-of-the-art ai-generated text detectors. we performed a systematic evaluation of the effectiveness of these attacks on seven different detectors and five different datasets. our results showed that homoglyph-based attacks are very effective, to the point that their mcc drops to around 0 (no correlation) in all of them, albeit at different replacement percentages. we then analyzed the internal mechanisms of the detectors to provide a technical justification for the effectiveness of the attacks. furthermore, we have publicly released our implementation and datasets, which we hope will facilitate further research on ai-generated text detection algorithms. the effectiveness of these attacks adds to the existing evidence that existing ai-generated text detectors are unfit for purpose, highlighting the need for more robust detection mechanisms. the proposed attacks can be employed to assess the resilience of future ai-generated text detectors and to develop more effective solutions.