TCM - SD : A Benchmark for Probing Syndrome Differentiation via Natural Language Processing (2022)

Abstract: “Traditional Chinese Medicine (TCM) is a natural, safe, and effective therapy that has spread and been applied worldwide. The unique TCM diagnosis and treatment system requires a comprehensive analysis of a patient’s symptoms hidden in the clinical record written in free text. Prior studies have shown that this system can be informationized and intelligentized with the aid of artificial intelligence (AI) technology, such as natural language processing (NLP). However, existing datasets are not of sufficient quality nor quantity to support the further development of data-driven AI technology in TCM. Therefore, in this paper, we focus on the core task of the TCM diagnosis and treatment system—syndrome differentiation (SD)—and we introduce the first public large-scale benchmark for SD, called TCM-SD. Our benchmark contains 54,152 real-world clinical records covering 148 syndromes. Furthermore, we collect a large-scale unlabelled textual corpus in the field of TCM and propose a domain-specific pre-trained language model, called ZYBERT. We conducted experiments using deep neural networks to establish a strong performance baseline, reveal various challenges in SD, and prove the potential of domain-specific pre-trained language model. Our study and analysis reveal opportunities for incorporating computer science and linguistics knowledge to explore the empirical validity of TCM theories.”

## introduction as an essential application domain of natural language processing (nlp), medicine has received remarkable attention in recent years. many studies have explored the integration of a variety of nlp tasks with medicine, including question answering @xcite , machine reading comprehension @xcite , dialogue @xcite , named entity recognition @xcite , and information retrieval @xcite . meanwhile, numerous datasets in the medical domain with different task formats have also been proposed @xcite @xcite . these have greatly promoted the development of the field. finally, breakthroughs in such tasks have led to advances in various medical-related applications, such as decision support @xcite and international classification of disease (icd) coding @xcite . however, most existing datasets and previous studies are related to modern medicine, while traditional medicine has rarely been explored. compared to modern medicine, traditional medicine is often faced with a lack of standards and scientific explanations, making it more challenging. therefore, it is more urgent to adopt methods of modern science, especially nlp, to explore the principles of traditional medicine, since unstructured texts are ubiquitous in this field. tcm, as the representative of traditional medicine, is a medical system with a unique and complete theoretical basis formed by long-term medical practice under the influence and guidance of classical chinese materialism and dialectics. unlike modern medicine, in which medical professionals assign treatments according to disease type, tcm practitioners conduct in-depth analyses based on evidence collected from four diagnostics methods-inspection, auscultation and olfaction, interrogation, and palpation-to determine which type of syndrome (zheng, 证) the patient experiencing. different treatment methods are then adopted according to the type of syndrome. therefore, patients with the same disease may have different syndromes and thus receive different treatments, while patients with different diseases may have the same syndrome and thus undergo the same treatment. these concepts are called "treating the same disease with different therapies (同病异治)" and "treating different diseases with the same therapy (异病同治)," respectively, which are the core methods upheld by tcm. for the example shown in figure 1 , patients a and b have the same disease-dysmenorrhea-but one is influenced by cold while the other is driven by qi stagnation (which is a specific concept in tcm). thus, different therapies would be assigned. however, patient c suffered from angina pectoris but shared the same syndrome as patient b. therefore, they would be treated with similar therapies. thus, the syndrome, instead of the disease, can be regarded as the primary operating unit in the tcm medical system, which not only effectively summarizes the patients' symptoms but also determines the subsequent treatment. in this process, known as syndrome differentiation, the inferencing task of deciding which syndrome is associated with a patient based on clinical information, is a vital pivot of the tcm medical system. in recent years, with the discovery of artemisinin @xcite and the beneficial clinical manifestations of tcm to treat covid-19 @xcite , tcm has increasingly attracted attention. there have been some studies in which nlp techniques were used to explore sd tasks @xcite @xcite @xcite ), but the development has been significantly hindered by the lack of large-scale, carefully designed, public datasets. therefore, this paper aims to further integrate traditional medicine and artificial intelligence (ai). in particular, we focus on the core task of tcm-syndrome differentiation (sd)-to propose a highquality, public sd benchmark that includes 54,152 samples from real-world clinical records. to our best knowledge, this is the first time that a textual benchmark has been constructed in the tcm domain. furthermore, we crawled data from the websites to construct a tcm domain text corpus and used this to pre-train a domain-specific language model called as zy-bert (where zy came from the chinese initials of tcm). the experiments and analysis of this dataset not only explored the characteristics of sd but also verified the effectiveness of domain-specific language model. our contributions are summarized as follows: ## preliminaries to facilitate the comprehension of this paper and its motivation and significance, we will briefly define several basic concepts in tcm and analyze the differences between tcm and modern medicine. ## related works there are three main streams of work related to this manuscript: medical dataset, natural language processing in syndrome differentiation and domain specific pre-trained language model. ), machine reading comprehension (mrc), de-identification (de-id.), disease name recognition (dnr), natural language inference (nli), recognizing question entailment (rqe), and question answering (qa). ## benchmark and methods the tcm-sd benchmark that we collected contains over 65,000 real-world chinese clinical notes. ## experiments we selected the multi-class classification task as the primary form of sd to directly compare the performances of the existing models against the tcm-sd dataset, and used the accuracy and macro-f1 as evaluation metrics. specifically, the chief complaint and medical history were concatenated as the inputs, ## conclusions this paper introduced a meaningful task, sd, in tcm and its connection with nlp and presented the first public large-scale benchmark of sd: tcm-sd. furthermore, a knowledge corpus supporting the model understanding and the large-scale tcm domain corpus for pre-training were constructed. moreover, one domain-specific pre-training language model named as zy-bert was proposed. the experiments on this dataset demonstrated the challenges, the inadequacy of existing models, the importance of knowledge and the effectiveness of domain-specific pre-training. this work can greatly promote the internationalization and modernization of tcm, the proposed benchmark and associated baseline models provide a basis for subsequent research.