Venue: NAACL 2025

# QSpell 250K: A Large-Scale, Practical Dataset for Chinese Search Query

Spell Correction

 Dezhi Ye\({}^{1}\) Haomei Jia\({}^{2}\) Junwei Hu\({}^{1}\) Bowen Tian\({}^{1}\)

Jie Liu\({}^{1}\) Haijin Liang\({}^{1}\) Jin Ma\({}^{1}\) Wenmin Wang\({}^{2}\)

###### Abstract

Chinese Search Query Spell Correction is a task designed to autonomously identify and correct typographical errors within queries in the search engine. Despite the availability of comprehensive datasets like Microsoft Speller and Webis, their monolingual nature and limited scope pose significant challenges in evaluating modern pre-trained language models such as BERT and GPT. To address this, we introduce **QSpell 250K**, a large-scale benchmark specifically developed for simplified Chinese Query Spelling Correction. QSpell 250K offers several advantages: 1) It contains over 250K samples, which is ten times more than previous datasets. 2) It covers a broad range of topics, from formal entities to everyday colloquialisms and idiomatic expressions. 3) It includes both Chinese and English, addressing the complexities of code-switching. Each query undergoes three rounds of high-fidelity annotation to ensure accuracy. Our extensive testing across three popular models demonstrates that QSpell 250K effectively evaluates the efficacy of representative spelling correctors. We believe that QSpell 250K will significantly advance spelling correction methodologies. The accompanying data and code will be made publicly available1.

Footnote 1: [https://github.com/dz1109/CQSpell](https://github.com/dz1109/CQSpell)

## 1 Introduction

Query Spelling Correction is essential for enhancing the efficacy of search engines by identifying and rectifying errors in user queries Sharma et al. (2023); Yang et al. (2022). A misspelled search query can yield irrelevant results, thereby diminishing the user's ability to obtain satisfactory outcomes Gong et al. (2019); Fourney et al. (2017); Gupta et al. (2019). For instance, given the query "vivu positioning display offline," "vivu" in Figure 1 should be corrected to "vivo". Should the model fail to rectify this, the search results would include "vivu," failing to meet the user's needs. By identifying common spelling errors, search engines can be better equipped to handle these inaccuracies by suggesting corrections or automatically adjusting queries.

The field of query spelling correction has garnered considerable interest Li (2020), as evidenced by initiatives such as the Microsoft Speller Challenge (MSC) Wang and Pedersen (2011) and the wealth of research on the participating methodologies and their subsequent refinements. The MSC provides a corpus of approximately 6,000 annotated queries, of which 16% contain errors. To address the issue of limited data scale, Webis Hagen et al. (2017) compiled a more extensive collection of 54,772 queries, with 16.74% marked for spelling inaccuracies. While these datasets have propelled advancements in the domain of query spelling correction, they predominantly cater to English, with minimal efforts extended to other

Figure 1: The display format of query corrections on Google involves the search engine automatically correcting a misspelled query to the appropriate term, while simultaneously notifying the user with the prompt “Showing results for”.



languages, such as Chinese. Existing datasets like SIGHAN13 (Wu et al., 2013), SIGHAN14 (Yu et al., 2014), and SIGHAN15 (Tseng et al., 2015) provide a Chinese Spelling Correction corpus collected from a computer-based test of Chinese as a foreign language. However, these datasets are primarily aimed at long texts in spelling exams, rather than user searches in the web domain. Additionally, the scale of the datasets is small, with each dataset containing approximately 1,000 entries. The MCSSet (Jiang et al., 2022) introduces a Chinese corpus focus on medical domain, which unfortunately limits the assessment of error correction models in non-medical contexts. Consequently, there is a pressing need for a comprehensive Chinese Search Query Spelling Benchmark.

Furthermore, in the realm of real-world search engines, users frequently engage in code-switching, interspersing multiple languages within a single search, such as "[ipap]\(\backslash\)[u] 

approaches have employed Statistical Machine Translation-based models to address the contextual limitations inherent in error modelingHasan et al. (2015). In our study, we classify spelling correction models into three principal categories according to their architectural framework. Decoder-only models Zhang et al. (2023b), represented by the pre-trained GPT2, are adaptable for sequence generation tasks through fine-tuning. Encoder-Decoder models Pande et al. (2022); Zhang et al. (2023a); Kuznetsov and Urdiales (2021),, such as T5 Kakkar et al. (2023), are adept at encoding queries and subsequently generating the correct targets. Text edit models Mallinson et al. (2022), like KSTEM Ye et al. (2023), reconceptualize the sequence generation challenge as a sequence tagging task, with the objective of diminishing latency. Although these models have demonstrated enhanced performance on the MSC dataset, there is still an absence of a rigorous benchmark for QSC.

## 3 Chinese Search Query Spell Correction Benchmark

### Task Definition

Given an incorrect query \({\mathbf{x}}=\{x_{1},x_{2},\ldots,x_{i}\}\), and a correct query \({\mathbf{y}}=\{y_{1},y_{2},\ldots,y_{j}\}\), the Chinese search query spelling correction task can be defined as \(f:{\mathbf{x}}\rightarrow{\mathbf{y}}\), where \(f\) denotes the model to automatically convert the query \({\mathbf{x}}\) to another query \({\mathbf{y}}\). It should be noted that the length of sentences \({\mathbf{x}}\) and \({\mathbf{y}}\) may not be equal, reflecting the presence of missing or redundant errors in real-world scenarios.

### Query Sampling

The first stage of the construction of QSpell 250K is to collect error query candidates. In a real-world search engines, the proportion of error queries is relatively small. In other words, if we do random sampling, most of the queries we get are correct.

To build a large-scale query spelling correction benchmark, we sample 5,000,000 queries with 2 up to 40 characters from the query log in an industrial web search engine. Queries that are excessively lengthy or brief are filtered out. These 5,000,000 queries do not constitute the final dataset for annotation. Rather, they represent the initial set of raw data that requires filtering and screening.

* Step 1. We collect the query log from January 2023 to December 2023 and compute the query search frequency and click rate (query click number / query search frequency). These data were collected from an industrial web search engine.
* Step 2. We remove queries that include personal information, toxic topics. In addition, we further filter out queries with more than 40 or less than 2 characters.
* Step 3. We remove queries with the top search frequency and click rate. A simple assumption is that high-frequency queries are less likely to contain errors. In addition, if the user cannot find a satisfactory result, the click action will not occur.
* Step 4. We select candidates to be annotated after corpus matching and perplexity (PPL) value filtering, which is calculated by the language model 2. A query exhibiting a lower PPL score typically signifies a higher likelihood of occurrence according to the language model, suggesting a more coherent and grammatically aligned construction with the anticipated linguistic patterns. Hence, it can be inferred that queries with lower perplexity are generally characterized by fewer spelling errors.

Footnote 2: [https://github.com/xu-song/bert-as-language-model](https://github.com/xu-song/bert-as-language-model)

\begin{table}
\begin{tabular}{c c l r r} \hline
**Language** & **Category** & **Typos** & **Text** & **Translation** \\ \hline  & Phonetic & \multirow{6}{*}{
\begin{tabular}{l} [7pt] \\ Visual \\ Order \\ Missing \\ Redundant \\ \end{tabular} } & How to cure cough with kumquat \\  & & Visual & \multicolumn{1}{r}{} & The Lotus Ann of Huaiju Drama \\  & & Order & \multicolumn{1}{r}{} & Yue Yunpen’s comedy \\  & Missing & \multicolumn{1}{r}{} & \multicolumn{1}{r}{} & Arena Of Valor Liu Bei \\  & Redundant & \multicolumn{1}{r}{} & \multicolumn{1}{r}{} & Feitian Moutai identification method \\ \hline  & Phonetic & iphone & \multicolumn{1}{r}{} & iphone & \multicolumn{1}{r}{} \\  & Visual & vaccuum seal & \multicolumn{1}{r}{} & vacuum seal & \multicolumn{1}{r}{} \\ English & Order & leaves & \multicolumn{1}{r}{} & leaves & \multicolumn{1}{r}{} \\  & Missing & \multicolumn{1}{r}{} & \multicolumn{1}{r}{} & Kamen Rider amaons \\  & Redundant & windows10t & \multicolumn{1}{r}{} & windows10t & \multicolumn{1}{r}{} \\ \hline \end{tabular}
\end{table}
Table 1: Examples of different types of edits in QSpell 250K that involve both Chinese and English languages.

 



### Query Annotation

After automatically filtering the data, we manually annotated the remaining data, referred to as QSpell 250K. To encourage high-quality marking, we assign each query to three random annotators for independent annotation. Their submissions are then aggregated and sent to a random senior reviewer as the final judge. In addition, annotators can use any tool they want to support their work, such as search engines. Besides, to avoid persistent labeling mistakes, the annotation process is conducted in batches, and slight adjustments to the annotation standard are allowed at this stage. In this way, we can detect problems in the actual labeling process.

* Step 1. Crowdsourced annotation. During the annotation phase, the annotators are required to first examine whether the word itself contains any errors. If there are no errors, they proceed to assess the word's contextual appropriateness. Additionally, annotations utilize web search engines such as Google, Baidu, and references like Wikipedia to cross-validate the judgments. By adopting this approach, the annotations will not be biased to fit into one specific context.
* Step 2. Senior reviewer judgment. After a crowdsourced annotation of a batch is completed, it is sent to senior reviewer to judge whether it meets our annotation standard. This process repeats until the annotation accuracy rate reaches 90%.
* Step 3. Final quality assurance. Each batch of annotated queries that pass the first round of verification is sent to quality inspector for a second round of verification. The quality

\begin{table}
\begin{tabular}{l c c c c c c} \hline
**Dataset** & **Volume** & **Error Ratio** & **Lang** & **Error Type** & **Length** & **Field** \\ \hline MSC & 5,892 & 19\% & English & 4 & Short & Web \\ qSpell & 6,000 & 16\% & English & 4 & Short & Web \\ Webis & 54,772 & 16\% & English & 4 & Short & Web \\ \hline SINGHAN13 & 700/1,000 & 20\% & Chinese & 2 & Long & Specific \\ SINGHAN14 & 3,437/1,062 & 75\% & Chinese & 2 & Long & Specific \\ SINGHAN15 & 2,339/1,100 & 64\% & Chinese & 2 & Long & Specific \\ \hline QSpell 250K & 200,000/50,000 & 51\% & Chinese,English & 4 & Short & Web \\ \hline \end{tabular}
\end{table}
Table 2: The comparison of QSpell 250K and existing spell correction datasets. QSpell 250K, both in terms of data volume and data characteristics, provides an excellent complement to existing datasets.

Figure 3: The annotation process of QSpell 250K benchmark.

Figure 2: Feature distribution of QSpell 250K including text length, topic and error type. Additionally, the distribution of original query frequency is 10% hot, 30% torso, and 60% long-tail. This is because the higher the frequency of a query, the lower the likelihood of it containing errors.



inspector randomly check 30% of the queries and send unqualified queries back to senior reviewers along with the reasons for rejection. The quality inspector possesses a solid educational background and is proficient in using various search tools.

### Analysis and Comparison

In this section, we introduce the features of QSpell 250K from multiple perspectives and compare them with existing datasets.

**Basic feature** We show a comparison of our basic features with the existing datasets in Table 2. QSpell 250K comprises 250,000 queries, of which 50% are misspelled. It is evident that both the volume of our data and the proportion of errors exceed those of previous datasets by more than four-fold, signifying that QSpell 250K presents a more challenging task. QSpell 250K encompasses both Chinese and English, featuring a code-switching characteristic that previous datasets did not possess.

**Topic distribution** In a real-world application, user input often covers a variety of topics. For the convenience of analysis, we divide our data into 16 topics. Figure 2(b) depicts the proportion of our dataset for each topic. These topics are determined by annotators during the annotation process. Due to the diversity of topic distribution, our dataset also poses new challenges to the task of Query Spelling Correction.

**Error type** To enhance the coherence of the dataset with real-world scenarios, we incorporate these errors into QSpell 250K. Figure 2(c) shows the proportion of each error type. From the figure, it can be observed that approximately 75.8% of the errors are phonetically similar errors. This phenomenon may be attributable to the phonetic tendencies inherent in the Chinese language.

## 4 Evaluation

### Datasets Processing

We randomly split QSpell 250K into a training set (200K), and a test set (50K) with a ratio of 10:1. In order to better fit the actual application scenarios of error correction and objectively measure the effect of the model, QSpell 250K contains both correct queries and error queries, the ratio is close to 1:1. If all the data in the training set need to be corrected, then the model will assume by default that all the input data are wrong.

### Benchmark Models

Large Language Models (LLMs), such as ChatGPT and GPT-4 Brown et al. (2020); OpenAI (2023), have brought about a revolution in natural language processing, showcasing strong zero-shot and few-shot generalization capabilities. In this paper, we aim to evaluate the effectiveness of ChatGPT as a zero-shot learner for spelling correction. Specifically, we utilize the gpt-4-turbo model in Chat mode. To explore the efficacy of large language models in query spelling correction, we conduct supervised instruction tuning on the Qwen2.5 with size from 0.5B to 7B Yang et al. (2024). Additionally, to more clearly present the performance metrics of existing datasets, we have also documented the results of state-of-the-art (SOTA) models Sun et al. (2024).

### Benchmark Metrics

We utilize Precision, Recall, and F1 Score as our evaluation metrics Hasan et al. (2015); Ye et al. (2023). For each query \(q\) within the set \(Q\), the spell correction approach predicts a result \(G(q)\). For queries that require no correction, the corrector simply outputs the original query. Subsequently, we compare the model-generated results with the standard corrections \(S(q)\) provided by the corpus.

### Parameter Settings

Our experiments are conducted with Pytorch. For hyperparameter tuning, the learning rate is set to 3e-6, the max sequence length is set to 512, the up is 0.02 and the linear decay is 1.0. All experiments are conducted on the NVIDIA Tesla H100 with 80GB memory. For each model, we obtained the average from five experiments. This approach ensures a fairer comparison and mitigates the impact of random events. The prompt we used is as follows: As a query spelling error correction model, your task is to automatically detect and correct query spelling errors in the query. If the query does not contain errors, output the original query. The input query is: {}. The output query is: {}

### Benchmark Experiments

Table 3 reveals the main results of our experiments. From the experimental results we have the following observation: 1) QSpell-250K demonstrates superior practicality compared to Webis and



SIGHAN datasets. Our experiments with prompted LLMs reveal that QSpell-250K achieves better performance in error correction tasks. Additionally, with the increase in LLM model parameters, there is a smooth growth in performance on the QSpell 250K dataset without abrupt changes. This enhanced performance suggests that QSpell's samples better reflect real-world scenarios, as modern LLMs already possess strong error-correction capabilities. 2) The LLM performed good on QSpell 250K and Webis, but it showed poor results on the SIGHAN dataset. This may be attributed to the smaller sample size of SIGHAN, which makes it difficult for the LLM to transition to downstream tasks. Additionally, since Sighan is collected from a computer-based test of Chinese as a foreign language, it contains numerous rare error corrections, which also contribute to the suboptimal performance of LLMs on the Sighan dataset. 3) Off-the-shelf LLMs perform poorly in spell correction tasks and require fine-tuning. As the size of the model parameters increases, the performance of the LLM improves significantly. Overall, the experimental results indicate that the performance of the existing models on QSpell 250K falls short of our expectations, even with a substantial amount of training data.

### Case Study

To verify the problems of existing models, we further analyze errors that cannot be handled in all baseline models.

Firstly, QSpell 250K requires more domain knowledge. For example, the correct query for "Shift" (Who plotted to kill Cai Mao) should be "Shift" (Who plotted to kill Cai Mao). "Shift"(Cai Mao) is a role name in the Romance of the Three Kingdoms, which is a famous Chinese novel.

Secondly, QSpell 250K requires greater context understanding. There are many queries with multiple error points in QSpell 250K. In such texts, the context of each error points contains at least one misspelled character, which brings noise information. For example, "Shift" (Yi et al., 2019) is misspelled, and the correct query is "Shift"(The method of ChengDu noodles served with soy sauce).

Thirdly, QSpell 250K requires multilingual understanding capabilities. For example, "windows" (Xu et al., 2018; Chen et al., 2017), contains Chinese and English errors. The correct query should be "windows" (Whi et al., 2019) #x"(windows screen flip). To rectify such errors, the model must possess the capability to represent a multitude of languages effectively.

Overall, it is still very challenging to use existing models in a general application and correct these kinds of error.

## 5 Conclusion

In this paper, we present a Large-scale, naturalistic benchmark for Chinese Search Query Spelling Correction (QSpell 250K), which is collected from a real-world application. Compared with existing datasets like Microsoft Specler Challenge and SIGHAN, QSpell 250K supports more reliable evaluation due to the following features: 1) a variety of error patterns, 2) large scale, 3) code-switching. In addition, we conduct experiments on several representative spelling correction methods. The experiments have demonstrated that QSpell 250K is more challenging. At last, as shown by our experiments, the current Query Spelling Correction is not a "solved" problem and has much room for improvement. We hope our benchmark will benefit future research.

\begin{table}
\begin{tabular}{l c c c|c c c c|c c c} \hline \hline \multirow{2}{*}{**Model Type**} & \multicolumn{4}{c|}{**QSpell 250K**} & \multicolumn{4}{c}{**SIGHAN**} & \multicolumn{4}{c}{**Webis**} \\ \cline{3-10}  & **Model** & **P** & **R** & **F1** & **P** & **R** & **F1** & **P** & **R** & **F1** \\ \hline \multirow{2}{*}{**Prompt**} & GPT4 & 0.7409 & 0.3146 & 0.4416 & 0.6395 & 0.4060 & 0.4967 & 0.3896 & 0.4418 & 0.4140 \\  & Qwen2.5 7B & 0.4906 & 0.2980 & 0.3708 & 0.3868 & 0.2328 & 0.2906 & 0.2155 & 0.3581 & 0.2691 \\ \hline \multirow{4}{*}{**FT**} & 7B & 0.8391 & 0.5750 & 0.6824 & 0.7835 & 0.3455 & 0.4795 & 0.5096 & 0.4867 & 0.4979 \\  & 3B & 0.7380 & 0.3984 & 0.5175 & 0.3337 & 0.1565 & 0.2131 & 0.3725 & 0.3778 & 0.3751 \\  & 1.5B & 0.7015 & 0.3266 & 0.4457 & 0.2018 & 0.0861 & 0.1207 & 0.2734 & 0.3540 & 0.3085 \\  & 0.5B & 0.6184 & 0.3109 & 0.4138 & 0.1718 & 0.0546 & 0.0829 & 0.2406 & 0.2539 & 0.2471 \\ \hline SOTA & BERT & - & - & - & 0.7803 & 0.7873 & 0.7880 & - & - & - \\ \hline \hline \end{tabular}
\end{table}
Table 3: The performance of baselines on QSpell 250K, SIGHAN and Webis. For each model, we obtained the average from five experiments.

 



## 6 Limitations and Ethical Considerations

Data Collection for QSpell 250K: During the collection of QSpell 250K, we employ multiple methods aimed at ensuring user privacy, collecting only the users' search query information. Additionally, the data we gathered includes only Chinese and English. Since it does not encompass other languages, our experiments might not be easily generalizable to other search environments. Furthermore, the data originates from a Chinese search engine, representing a specific cultural and linguistic context, and does not reflect the global population.

Annotation of QSpell 250K: For annotating QSpell 250K, we utilized a mixed approach of crowdsourcing and senior reviewer annotations to ensure the quality of the annotations. During the annotation process, annotators could only see the queries and had no access to user information. Additionally, the annotators underwent multiple rounds of training to ensure the accuracy of the annotations. Although we made every effort to remove queries containing harmful intent during the annotation process, there may still be queries with potential risks remaining. In order to protect user privacy, we refrained from accessing the context of user queries.

In summary, we hope that QSpell 250K will foster development in the field of spell correction.

## References

* Ahmad and Kondrak (2005) Farooq Ahmad and Grzegorz Kondrak. 2005. Learning a spelling error model from search query logs. In _Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing_, pages 955-962.
* Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. _Advances in neural information processing systems_, 33:1877-1901.
* Chen et al. (2007) Qing Chen, Mu Li, and Ming Zhou. 2007. Improving query spelling correction using web search results. In _Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLPCoNLL)_, pages 181-189.
* Duan et al. (2012) Huizhong Duan, Yanen Li, ChengXiang Zhai, and Dan Roth. 2012. A discriminative model for query spelling correction with latent structural svm. In _Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning_, pages 1511-1521.
* Fourney et al. (2017) Adam Fourney, Meredith Ringel Morris, and Ryen W White. 2017. Web search as a linguistic tool. In _Proceedings of the 26th International Conference on World Wide Web_, pages 549-557.
* Ganjisaffar et al. (2011) Yasser Ganjisaffar, Andrea Zilio, Sara Javanmardi, Inci Cetindil, Manik Sikka, Sandeep Katumalla, Narges Khatib, Chen Li, and Cristina Lopes. 2011. qspell: Spelling correction of web search queries using ranking models and iterative correction. In _Spelling Alteration for Web Search Workshop_, page 15.
* Gao et al. (2010) Jianfeng Gao, Chris Quirk, et al. 2010. A large scale ranker-based system for search query spelling correction. In _The 23rd International Conference on Computational Linguistics_.
* Gong et al. (2019) Hongyu Gong, Yuchen Li, SUma Bhat, and Pramod Viswanath. 2019. Context-sensitive malicious spelling error correction. In _The World Wide Web Conference_, pages 2771-2777.
* Gupta et al. (2019) Jia Gupta, Zhen Qin, Michael Bendersky, and Donald Metzler. 2019. Personalized online spell correction for personal search. In _The World Wide Web Conference_, pages 2785-2791.
* Hagen et al. (2017) Matthias Hagen, Martin Pothast, Marcel Gohsen, Anja Rathgeber, and Benno Stein. 2017. A large-scale query spelling correction corpus. In _Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval_, pages 1261-1264.
* Hasan et al. (2015) Sasa Hasan, Carmen Heger, and Saab Mansour. 2015. Spotling correction of user search queries through statistical machine translation. In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_, pages 451-460.
* Jiang et al. (2022) Wangjie Jiang, Zhihao Ye, Zijing Ou, Ruihui Zhao, Jianguang Zheng, Yi Liu, Bang Liu, Siheng Li, Yujiu Yang, and Yefeng Zheng. 2022. Mcscset: A specialist-annotated dataset for medical-domain chinese spelling correction. In _Proceedings of the 31st ACM International Conference on Information & Knowledge Management_, pages 4084-4088.
* Kakkar et al. (2023) Vishal Kakkar, Chinmay Sharma, Madhurra Pande, and Surender Kumar. 2023. Search query spell correction with weak supervision in e-commerce. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track)_, pages 687-694.
* Kuznetsov and Urdiales (2021) Alex Kuznetsov and Hector Urdiales. 2021. Spelling correction with denoising transformer. _arXiv preprint arXiv:2105.05977_.
* Li et al. (2006) Mu Li, Muhua Zhu, Yang Zhang, and Ming Zhou. 2006. Exploring distributional similarity based models for query spelling correction. In _Proceedings of the 21st

International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics_, pages 1025-1032.
* Li (2020) Yanen Li. 2020. Query spelling correction. _Query Understanding for Search Engines_, pages 103-127.
* Mallinson et al. (2022) Jonathan Mallinson, Jakub Adamek, Eric Malmi, and Aliaksei Severyn. 2022. Edit5: Semi-autoregressive text editing with t5 warm-start. In _Findings of the Association for Computational Linguistics: EMNLP 2022_, pages 2126-2138.
* OpenAI (2023) OpenAI. 2023. Gpt-4 technical report. _Preprint_, arXiv:2303.08774.
* Pande et al. (2022) Madhura Pande, Vishal Kakkar, Manish Bansal, Surender Kumar, Chinmay Sharma, Himanshu Malhotra, and Praneet Mehta. 2022. Learning-to-spell: Weak supervision based query correction in e-commerce search with small strong labels. In _Proceedings of the 31st ACM International Conference on Information & Knowledge Management_, pages 3431-3440.
* Sharma et al. (2023) Sanat Sharma, Josep Valls-Vargas, Tracy Holloway King, Francois Guerin, and Chirag Arora. 2023. Contextual multilingual spellchecker for user queries. _arXiv preprint arXiv:2305.01082_.
* Sun et al. (2024) Changxuan Sun, Linlin She, and Xuesong Lu. 2024. Two issues with chinese spelling correction and a refinement solution. In _Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 196-204.
* Sun et al. (2012) Xu Sun, Anshumali Shrivastava, and Ping Li. 2012. Fast multi-task learning for query spelling correction. In _Proceedings of the 21st ACM international conference on Information and knowledge management_, pages 285-294.
* Tseng et al. (2015) Yuen-Hsien Tseng, Lung-Hao Lee, Li-Ping Chang, and Hsin-Hsi Chen. 2015. Introduction to sighan 2015 bake-off for chinese spelling check. In _Proceedings of the Eighth SIGHAN Workshop on Chinese Language Processing_, pages 32-37.
* Wang and Pedersen (2011) Kuansan Wang and Jan Pedersen. 2011. Review of msr-bing web scale speller challenge. In _Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval_, pages 1339-1340.
* Wu et al. (2013) Shih-Hung Wu, Chao-Lin Liu, and Lung-Hao Lee. 2013. Chinese spelling check evaluation at sighan bake-off 2013. In _Proceedings of the Seventh SIGHAN Workshop on Chinese Language Processing_, pages 35-42.
* Yang et al. (2024) An Yang, Baosong Yang, Beichen Zhang, Binguan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 technical report. _arXiv preprint arXiv:2412.15115_.
* Yang et al. (2022) Fan Yang, Alireza Bagheri Garakani, Yifei Teng, Yan Gao, Jia Liu, Jingvun Deng, and Yi Sun. 2022. Spelling correction using phonetics in E-commerce search. In _Proceedings of the Fifth Workshop on e-Commerce and NLP (ECNLP 5)_, pages 63-67, Dublin, Ireland. Association for Computational Linguistics.
* Ye et al. (2023) Dezhi Ye, Bowen Tian, Jiabin Fan, Jie Liu, Tianhua Zhou, Xiang Chen, Mingming Li, and Jin Ma. 2023. Improving query correction using pre-train language model in search engines. In _Proceedings of the 32nd ACM International Conference on Information and Knowledge Management_, pages 2999-3008.
* Yu et al. (2014) Liang-Chih Yu, Lung-Hao Lee, Yuen-Hsien Tseng, and Hsin-Hsi Chen. 2014. Overview of sighan 2014 bake-off for chinese spelling check. In _Proceedings of The Third CIPS-SIGHAN Joint Conference on Chinese Language Processing_, pages 126-132.
* Zhang et al. (2023a) Jingfen Zhang, Xuan Guo, Sravan Bodapati, and Christopher Potts. 2023a. Multi-teacher distillation for multilingual spelling correction. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track_, pages 142-151.
* Zhang et al. (2023b) Xiaowu Zhang, Xiaotian Zhang, Cheng Yang, Hang Yan, and Xipeng Qiu. 2023b. Does correction remain an problem for large language models? _arXiv preprint arXiv:2308.01776_.