QS pell 250 K : A Large-Scale, Practical Dataset for C hinese Search Query Spell Correction (2025)

Abstract: Chinese Search Query Spell Correction is a task designed to autonomously identify and correct typographical errors within queries in the search engine. Despite the availability of comprehensive datasets like Microsoft Speller and Webis, their monolingual nature and limited scope pose significant challenges in evaluating modern pre-trained language models such as BERT and GPT. To address this, we introduce QSpell 250K , a large-scale benchmark specifically developed for Chinese Query Spelling Correction. QSpell 250K offers several advantages: 1) It contains over 250K samples, which is ten times more than previous datasets. 2) It covers a broad range of topics, from formal entities to everyday colloquialisms and idiomatic expressions. 3) It includes both Chinese and English, addressing the complexities of code-switching. Each query undergoes three rounds of high-fidelity annotation to ensure accuracy. Our extensive testing across three popular models demonstrates that QSpell 250K effectively evaluates the efficacy of representative spelling correctors. We believe that QSpell 250K will significantly advance spelling correction methodologies. The accompanying data and code will be made publicly available.

## introduction query spelling correction is essential for enhancing the efficacy of search engines by identifying and rectifying errors in user queries @xcite . a misspelled search query can yield irrelevant results, thereby diminishing the user's ability to obtain satisfactory outcomes @xcite @xcite . for instance, given the query "vivu positioning display offline," "vivu" in figure 1 should be corrected to "vivo". should the model fail to rectify this, the search results would include google involves the search engine automatically correcting a misspelled query to the appropriate term, while simultaneously notifying the user with the prompt "showing results for". the field of query spelling correction has garnered considerable interest @xcite , as evidenced by initiatives such as the microsoft speller challenge (msc) @xcite and the wealth of research on the participating methodologies and their subsequent refinements. the msc provides a corpus of approximately 6,000 annotated queries, of which 16% contain errors. to address the issue of limited data scale, webis @xcite compiled a more extensive collection of 54,772 queries, with 16.74% marked for spelling inaccuracies. while these datasets have propelled advancements in the domain of query spelling correction, they predominantly cater to english, with minimal efforts extended to other languages, such as chinese. existing datasets like sighan13 @xcite , sighan14 @xcite provide a chinese spelling correction corpus collected from a computer-based test of chinese as a foreign language. however, these datasets are primarily aimed at long texts in spelling exams, rather than user searches in the web domain. additionally, the scale of the datasets is small, with each dataset containing approximately 1,000 entries. the mc-scset @xcite introduces a chinese corpus focus on medical domain, which unfortunately limits the assessment of error correction models in non-medical contexts. consequently, there is a pressing need for a comprehensive chinese search query spelling benchmark. furthermore, in the realm of real-world search engines, users frequently engage in code-switching, interspersing multiple languages within a single search, such as "ipap插入u盘无反应"("ipap insertion of usb drive unresponsive). this widespread practice introduces complex challenges for current correction models, which, when trained in a single language, falter in the presence of multilingual inputs. thus, spelling correction models must be adept at understanding and processing multiple languages. however, mainstream datasets like msc, sighan have scarcely included this linguistic phenomenon, underscoring an acute need for a comprehensive dataset that can aid in the evolution of spell correction models capable of handling such linguistic diversity. to catalyze progress in query spelling correction (qsc), we present a novel benchmark named qspell 250k, a comprehensive large-scale dataset. the volume of qspell 250k is four to ten times that of its predecessors, amassing a total of 250,000 meticulously annotated queries. besides, the queries are cleaned for personal data. remarkably, over 12% of the queries in qspell 250k feature code-switching (contains both english and chinese.), mirroring the linguistic intricacies encountered in real-world contexts. the dataset predominantly comprises queries sourced from an actual search engine, capturing an extensive spectrum of subjects and newly coined internet phenomena. qspell 250k encompasses five primary categories of errors: phonetic, orthographic, scrambled, omitted, and superfluous characters. the specific error types within qspell 250k are enumerated in the main contributions of our benchmark are summarized as follows: ## conclusion in this paper, we present a large-scale, naturalistic benchmark for chinese search query spelling correction (qspell 250k), which is collected from a real-world application. compared with existing datasets like microsoft speller challenge and sighan, qspell 250k supports more reliable evaluation due to the following features: 1) a variety of error patterns, 2) large scale, 3) code-switching. in addition, we conduct experiments on several representative spelling correction methods. the experiments have demonstrated that qspell 250k is more challenging. at last, as shown by our experiments, the current query spelling correction is not a "solved" problem and has much room for improvement. we hope our benchmark will benefit future research.