Towards Real-World Writing Assistance: A C hinese Character Checking Benchmark with Faked and Misspelled Characters (2024)

Abstract: Writing assistance aims to improve the correctness and quality of input texts, with character checking being crucial in detecting and correcting wrong characters. In the real world where handwriting occupies the vast majority, characters that humans get wrong include faked characters (i.e., untrue characters created due to writing errors) and misspelled characters (i.e., true characters used incorrectly due to spelling errors). However, existing datasets and related studies only focus on misspelled characters that can be represented by computer text encoding systems, thereby ignoring faked characters which are more common and difficult. To break through this dilemma, we present \textbf{Visual-C} ^3 , a human-annotated \textbf{Visual} \textbf{C} hinese \textbf{C} haracter \textbf{C} hecking dataset with faked and misspelled Chinese characters. To the best of our knowledge, Visual-C ^3 is the first real-world visual and the largest human-crafted dataset for the Chinese character checking scenario. Additionally, we also propose and evaluate novel baseline methods on Visual-C ^3 . Extensive empirical results and analyses show that Visual-C ^3 is high-quality yet challenging. As the first study focusing on Chinese faked characters, the dataset and the baseline methods are publicly available at https://github.com/THUKElab/Visual-C3.

## introduction with texts on the internet growing explosively every day, writing assistance that is to improve the correctness and quality of texts is becoming increasingly important @xcite @xcite , and has received more and more attention from researchers. in the field of writing assistance, the character checking task aims to detect and correct wrong characters in the given text and occupies a crucial position, as it ensures the correctness of the minimum atom (i.e., original: 人生就象(elephant)一场x戏 correct : 人生就像(like)一场游戏 trans. : life is like a game the characters) of texts @xcite . large amounts of research are devoted to chinese character checking, which is also well known as chinese spell checking or chinese spelling correction (csc) @xcite . in this work, we also focus on the scene of chinese character checking. since chinese character checking is a daily application closely related to human life, to promote its progress and development, we must consider the real-world application needs of humans for it. therefore, a natural question arises: what are the types of erroneous chinese characters that humans would produce during the writing process? based on the observation of human writing habits, it is well known that there exist two main types of chinese characters that humans get wrong in the real world, namely faked characters (错字) and misspelled characters (别字) @xcite . as illustrated in figure 1 , the misspelled character itself is a character that exists but is used incorrectly, the faked character is a non-existent character caused by incorrect writing (e.g., wrong use of radicals or wrong number of strokes). authoritative chinese linguistics studies @xcite have shown that faked characters appear more frequently than misspelled characters in the process of people's daily use of chinese characters, and faked characters are often more difficult to de-tect than misspelled ones because faked characters are often caused by some very slight stroke errors. although faked characters are more common and challenging in the real world, researchers have not paid enough attention to how to handle the faked characters. the main reason for this dilemma is that the existing csc data resources are all textbased. the main drawback of single text-modal data is its inability to represent characters beyond those encoded by computers. fake characters are non-existent in computer text encoding systems. hence, the traditional csc datasets cannot represent faked characters, and the existing csc models proposed cannot hold onto more complex and real scenarios. at this point, a pressing and significant problem is how to expand and develop data resources for chinese character checking to facilitate the automatic detection and correction of faked characters by models. inspired by the enthusiasm to handle the faked characters, we propose to extend chinese character checking to the visual modality, as images are the most direct form to represent the faked characters. we construct a large-scale human-annotated visual chinese character checking dataset, visual-c 3 , which consists of 10,072 sentences represented by images and 12,019 wrong characters (including 5,670 misspelled and 6,349 faked characters) manually annotated by well-trained annotators. to the best of our knowledge, visual-c 3 is the first real scene-oriented dataset that contains both faked and misspelled characters. furthermore, to give future research on visual-c 3 more possibilities, in addition to annotating sentence-level information (i.e., the golden sentence without error characters corresponding to the original content of the input image), we also annotate each image at the character level and provide the position and type information of each character on the image. rich annotation information makes visual-c 3 suitable for various nlp, cv, or multimodal studies. based on visual-c 3 , we design the benchmark tasks in which the model inputs an image containing sentences with wrong characters and outputs the correct sentence without wrong characters corresponding to the input image in the form of text. through this task, visual-c 3 effectively assesses the detection and correction ability of chinese character checking methods, especially for faked characters. to verify the quality and challenge of visual-c 3 , we design and implement two baseline methods with different paradigms and evaluate them on visual-c 3 . extensive experiments and detailed analyses demonstrate that visual-c 3 is high-quality yet challenging. at the same time, the baselines also provide insightful and promising future directions. hopefully, we believe that the emergence of visual-c 3 could promote the research of writing assistance to better adapt to the intelligence needed in the real world. ## models and baselines to reflect the usability of visual-c 3 and provide reference ideas for future research on visual-c 3 , we design two baselines, namely ocr-based and clip-based methods, as illustrated in figure 3 . ## conclusion in this paper, we pay attention to the faked characters, which have never been focused on in previous works. to empower machines to automatically process the faked characters, we construct visual-c 3 , a large-scale visual chinese character checking dataset with faked and misspelled characters. furthermore, we design two baseline methods with different ideas. in particular, we first propose the idea of using image-text retrieval to detect the faked characters in the images. experimental results and detailed analyses indicate that our proposed baselines are effective and visual-c 3 is challenging and of great research value.