Exploring semantic information in disease: Simple Data Augmentation Techniques for Chinese Disease Normalization
Authors: Wenqian Cui
Xiangling Fu
Shaohui Liu
Xien Liu
Ji Wu
Venue: Submitted to ICLR 2023 (2023)
Keywords: Data Augmentation
Medicine
Disease
Disease Normalization
Deep Learning
Natural Language Processing
Representation Learning

Abstract: Disease is a core concept in the medical field, and the task of normalizing disease names is the basis of all disease-related tasks. However, due to the multi-axis and multi-grain nature of disease names, incorrect information is often injected and harms the performance when using general text data augmentation techniques. To address the above problem, we propose a set of data augmentation techniques that work together as an augmented training task for disease normalization, which is called Disease Data Augmentation (DDA). Our data augmentation methods are based on both the clinical disease corpus and standard disease corpus derived from ICD-10 coding. Extensive experiments are conducted to show the effectiveness of our proposed methods. The results demonstrate that our method can have up to 3\% performance gain compared to non-augmented counterparts, and they can work even better on smaller datasets.

Under review as a conference paper at ICLR 2023
Exploring semantic information in disease: Simple Data Augmentation Techniques for Chinese Discase Normaliza- tion
Anonymous authors Paper under double-blind review
Abstract
The discase is concept in the medical field, and the task of normalizing discase names is the basis of all discasc-related tasks. However, due to the multi-axis and multi-grain nature of disease names, incorrect information is often injected and harms the performance when using general text data augmentation techniques. To address the above problem, propose a set of data augmentation techniques that work together as an augmented training task for disease normalization. Our data augmentation methods are based on both the clinical discase corpus and standard discase corpus derived from ICD-10 coding. Extensive experiments are conducted show the effectiveness of our proposed methods. The results demonstrate that our methods can have up to 3% performance gain compared to non-augmented counterparts, and they work even better smaller datasets.
Introduction
The discase is central concept in medical text processing problems. One of the most important tasks, i.c. discase normalization, uses discases as both input and output to match the diagnoses terms used in clinical documents to standard names in ICD coding. The disease normalization lask mainly faces the following three challenges. First, different writing styles. The writing styles of the diseases can be diversified, where different doctors have different writing habits, so single disease might result in thousands of versions of names. Second, data scarcity, where some discases may not be covered in the training set, which often leads to few-shot or zcro-shot scenarios. For example, in the Chinese discase normalization dataset CHIP-CDN, there are 40472 discases to classify, but only data of 3505 diseases (i.e. less than 10% of all diseases) are provided in the training set. Figure illustrates the data scarcity problem in CHIP-CDN dataset. Third, semantics density. The length of disease names is usually short, which makes every character carries huge semantic information. The meanings the diseases are very different from each other even if they share a lot of common characters, and a single change in characters could result in dramatic change in semantic meaning. For instance, " 6% K.E (Common iliac artery dissection)" and " & Sh J (Common carotid artery dissection)" are only different in one character, but the positions of those diseases are very distinct, from the upper half of the body part to the lower half.
Among all the challenges we discussed, data scarcity is the biggest one, since other problems usually can be solved by providing larger datasets for models to learn. common way to address the data scarcity problem is through data augmentation. There are mumerous data augmentation hodhhs methods for general corpora such as synonym replacement or back translation. Wei & Zou (2019) has shown that simple text data augmentation methods can be effective for text classification problems. However, because of the unique structure of disease names (i.e. semantics density), general text data augmentation methods do not work well on them, and sometimes even hurt the overall performance. For example, if random deletion Wei & discase |H E uh ml 3 # (Obstructive Sleep Apnoca)" and results in " H E il Bie EE (Obstruetive Sleep)", that would dramatically change the meaning of that disease
Zou (2019) is performed
and makes it become another disease. Admittedly, general

---

Under review as a conference paper at ICLR 2023
0.40
3300
3000
0999
2009
Data scarcity problem in CHIP- CDN dataset. The blue line represents the overall amount of discases in ICD coding classified by the first coding letter, and the red line represents the number of diseases provided by the CHIP-CDN training set.
Figure 1:
data augmentation methods may be able to address the challenge of different writing styles, as performing random operations on texts can be seen as a way to emulate different writing behaviors. However, due to the above reasons, general data augmentation methods tend to hurt performance, which is demonstrated in our experiments. Therefore, designing data augmentation methods specific to disease corpus is necessary. To bridge this gap, we propose a set of discasc-oricnted data augmentation methods to address this problem.
As with other disease-related tasks, disease normalization can be thought text matching, from clinical names to standard names in ICD coding. Therefore, the key to this task is for the model to learn great encoding that contains enough similar information for each discase. For instance, the model needs to tell that " 1 A agenesis)" and AT MM. it H (Common iliac artery dissection)" and AM Z E (Common carotid artery dissection)" are not, despite that they both share lot of common characters.
process of
(Left renal (Congenital renal agenesis)" )" are the same discase while
Our methods are based on the following two assumptions. First, disease names have the property of structural invariance. A disease name consists of several different types of key elements, such as location, clinical manifestations, etiology, pathology, etc. In the pair of clinical discase and standard ICD discaso, the specified elements can correspond in most cases. Therefore, we can replace a specific clement between the pair of clinical discase and standard ICD discase the same time to generate new pairs. The matching relationship of the newly generated clinical disease and the ICD standard disease pairs can still be maintained. We screened the generated standard ICD diseases to ensure that they belonged to the correct label and that the pairs are effective. It should be noticed that replacing components could derive a new clinical disease name that turns out to be fake (i.e. the discase actually does not exist), but the key point here is to make models learn the necessary semantic association within the disenses. Second, labels in the disease normalization task have transitivity properties. In specific, a more specified description of an object can be comprised into larger group where the descriptions are more coarse, e.g. yellow chair also chair. In the ICD coding system, there are also different and clear granularitiss of discases. Therefore, we can treat the fine-grained discase as their -ranind upper discase by assigning them father labels.
Normally, a data augmentation method generates new data and trains them along with the existing data, without altering the training paradigm. However, the discase normalization task assigns each discase a unique label, while our methods augment the labels. Therefore, the traditional training paradigm input disease in the dataset may get different labels, which will make the model difficult to train due to label confusion. To overcome this problem, we treat the data augmentation operation as pre-training task (we call it augmented training) prior to the original task, so that the model can first learn the necessary semantic information within diseases and then leverage that information when fine-tuning the actual normalization dataset.
still applied to our augmentation methods, a same
2

---

Under review as a conference paper at ICLR 2023
Additionally, both unnormalized discase names from the tasks and standard ICD names of the diseases can be used as inputs in the data augmentation process. A unique advantage of using standard ICD names to perform data augmentation
pre-training task is that the model can get the whole picture of the disease-related information from TCD coding, which includes all classes of diseases, even before the actual training of the downstream task. Therefore, with all those information injected, the model can perform much stronger on smaller datasets where lots of class labels are not able to be seen in the training set.
To the best of our knowledge, we are the first to explore the semantic components and information within disease names. We believe the research on discase name enhancement has high research value and can benefit various downstream tasks, To summarize our contributions:
We propose a set of data augmentation methods for the Chinese disease normaliza-
tion tasks.
Experiments validate that general data augmentation methods have the potential to impair the disease normalization task. However, our method has obvious perfor- mance gain on the task based on various baseline models.
We also analyze the reasons why the proposed method is effective.
2 Background
ICD coding. ICD, the acronym of the International Classification of Diseases, is in- ternational unified classification of diseases developed by the World Health Organization, and ICD-10 is the 10th version of ICD coding which is used in work. The coding is a combination of letters and numbers, which classifies diseases according to their etiology, pathology, clinical manifestations, and anatomical locations, that they form hierarchical coding structure. ICD also adopts a multi-grain fashion where coarso-grained discase are followed by fine-grained discases.
Disease normalization task. In clinical practice, doctors will fill in the name of the disease according to clinical diagnosis standards along with their own writing habits, which makes a single disease name hundreds of versions. The disease normalization task is to match disease names written in different styles into a single standard name provided by ICD coding. After the disease normalization process, researchers can perform further operations upon the normalized names to rcalize all kinds of functions used in wise medical applications. The task can be formalized into the following operation: X -> Y, where X represents the clinical disease namnss and represents the slandard ICD names.
NER. NER stands for Named Entity Recognition, which is a common task in Natural Lan- guage Processing. Tt aims to identify entities that have practical values and their locations from unstructured texts. The classification of these entities may include persons, organi- zations, locations, etc. In this work, we use an NER tool trained by ourselves to identify elements in disease names in order to perform data augmentation. Additionally, that any NER tool that can identify elements in discase names should be fine, and our work mainly focus on the data augmentation methods.
argue
3 Related Work
In this section, we first introduce related works of data augmentation, then we introduce medical data-driven research works that are similar to ours.
3.1 Data Augmentation
Data augmentation is technology to synthesize data based existing data way to expand the amount of dataset. is often used when the amount of data not enough, and it can also act as regularizer to prevent the model from overfitting the training set.
3

---

Under review as a conference paper at ICLR 2023
Unlike images, where it is relatively easy to augment data well keep the semantic information intact, data angmentation in texts is more difficult, due to its unstructured form Ng et al. (2020). Many works focus on angmentations directly the input: |Wei & Zon (2019) propase four simple augmentation methods base on character-level noise injection, which are replacement, insertion, swap, and deletion. Their methods quite straightaway and effective, but the augmentation results may cause unwanted noise by not following the grammar rules. Back translation, augments data by translating the original text to second language and then translating it back. This method can keep the semantic mcaning well of the original text, but the augmented results are lack of diversity and sometimes restricted by the translation tool. In order to make the augmented data more realistic, Kim et al. (2022) leverages lexicalized probabilistic contexl-free grammars lo capture the intricate compositional structure of natural language and then perform word replacements. This method yields good results, but grammar- based methods for general text are difficult to generalize to specialized areas, such as medicine.
There are also methods that leverage pre-trained language mnguage models, to perform data augmen- tation. Ng et al (2020) use MLMobjectivvn in BERT Devlin et al (2018) to mask out some words and then regenerate itt_ttWWtt Win et al, (2019) also uses MLM task as well as changing the segment ids to class labels. Kumar et al. (2020) compares three kinds of data augmenta- tion hods using a conditional pre-trained model, namely auto-encoder, auto-regressive, and seq2seq. A problem with these methods is that the semantic meaning of the original sentence may change after several MLM replacements.
Semi-supervised learning can also way to perform data augmentation by leveraging the vast amount of unlabeled data. Berthelot et al (2019) uses MixUp to guess the low-entropy labels of the augmented data and then mixes the labeled and unlabcled data to derive loss term, and Xie et al, (2020) performs data augmentation on unlabeled data for consistency training- However, we only focus on augmenting the data itself instead of semi-supervised learning objectives in this work.
3.2 Data approaches on medical data
While most researches focus on the effect of data augmentation on general text data, there are also works that try to explore the possibility of data augmentation operations on medical text data. In this section, we mainly introduce data augmentation on medical text data and other related rescarch works.
There are works that focns on the synonym replacement in medical terms. Falis et al (2022) and Abdollahi et al (2021) leverage Unified Medical Language System (UMLS) to find medical synonyms to perlorm replacements after certain medical are identified in classification texts. Focusing on the ICD-coding task, Falis et al (2022) also replaces both the medical terms in raw texts and the classification label to get new training data. While their works mainly focus on replacing the whole medical term, we investigate the possibility of replacing the components of the medical terms by exploring the semantic structures within them.
Additionally, Ansari et al. (2021) investigates the performance of EDA, conditional pre- trained language models and back translation perform augmentation on social media texts for mental health classification. WWan al. (2020a) proposes Segment Reordering as data, augmentation technique to keep the medical semantic meaning intact. WWan et al. (2020b) use pre-trained language models fine-tuned on General Semantic Textual Similarity (STS-G) data to generate psoudo-labels on medical STS data, and then perform iterative training.
4 Methods
In this section, we introduce the details of our proposed data augmentation methods and the overall pipeline. Since the significance of data augmentation is to inject the model with extra knowledge, the key point is to explore the components and relations in diseases so that the model can have broad sense of the internal structures of the diseases. Therefore,
4

---

Under review as a conference paper at ICLR 2023
Diseasa A
Dissase
Disease
Dicease
(WER)
parti
others
par12
others
part
others
12
perti
others
part2
others
parti
others
part2
(Repleno))
(Relace
(Rod)
parti
others
pant
others
parti
others
part2
porti
others
part
others
othe part
pant2
(Dvnormatted neme)
(Stnddard namme)
(Uhnoomalledd namr)
(Standard name)
(N55)
Disnase
AR1
AR2
003.901:
IF x
(Diip rxluced ool
600
F
R /E/ Oreg Inssod monomees
602.903:
G. 1 Geghulemeeeigg mewigeel
55 f1 8
# Daray Fnduood cndes (o)
Coupp renoral swerssse
C03-907: 18.18 32 (acate eeeringss
3 15 M Orup peternstnn
name)
MGA-code
MGA-ppsition
Figure 2: Illustration of our proposed data augmentation methods. The upper half of the figures is the illustration of Axis replacement methods, and the lower half is the illutration of Multi- GraiiGGrann Aggregation.
we leverage the multi-axis and multi-grain nature of the discascs to design all of the data augmentation methods.
First of all, the disease names are composed of several elements, which include but are not limited to etiology, pathology, clinical manifestations, anatomical location, chronicity, degree type, etc. For ease of expression, we merge and select from all those elements into three main categories, which are disease center, anatomical location and disease quality. This shows the multi-axis nature of the diseases.
Disease center, which may include etiology and pathology, is the minimal word that describes the nature of a discase. It defines the main category of discase, such as "disorders" for "Other disorders of the eye with mec".
Disease Center:
Anatomical Location:
Anatomical Location is part of the human body that have actual mcaning in anatomy. It indicates which part of the human body is ill.
Disease Quality: The quality of disease which indicates the subtype of the disease, such as "Drug-indueed" for "Drug-inde induced peripheral neuropathy".
With these three axis words, all kinds of discase names can be combined by them.
Second, a discase can be described by multiple granularities. An upper discase is a coarse- defined discase and lower disease is a fine-grained discase. The ICD coding contains lots of upper-lower disease pairs by assigning them different. lengths of code. For example, in
"ICD-10 Beijing Clinical Version 601", the disease name of code "A18.2" " H4 mi # MEiA # (Peripheral Tuberculous Lymphadenitiss" and "A18.201" is 1 t (Inguinal lymph node tuberculosis)". "Peripheral Tuberculous Lymphadenitis" is a coarso-defined discase due to not specifying a single anatomical location. Additionally, coarso-defind discase can contain multiple fino-grainod discases in ICD coding.
In our intuition, although the disease can only be called the same if all of its components are the same, it is necessary for the model to learn which diseases are more similar than others. Therefore, we define the following data augmentation moth
5

---

Under review as a conference paper at ICLR 2023
4.1 Data Augmentation
We perform data augmentation by assigning pseudo-labels to discases to describe their relationships that they can form new pair of diseases, and those pairs to perform augmented Training of disease normalization lasks. We divide our methods into two main categories: Axis-word Replacement and Multi-grain Aggregation. We call our proposed disease name data augmentation method DDA. Figure y illustrates the overall pipeline of our methods.
We assume that disease names have the property of structural invariance, which means a name derived by replacing an axis-word a discase to another one with the same type also makes sense. Since there are often matches of Axis-words between an nnnormaliized-standde
Axis-word Replacement (AR):
disease pair in the disease normalization Lask, replacing the corresponding Axis- word in the clinical name with the standard name in the pair at the same time can ensure that the newly-generated pair will still match. To locate all sxrswword in the disease, leverage Named Entity Recognition (NER) tool trained by ourselves The entity type includes but is not limited to discase center, anatomical location, and discase quality. We note that the NER tool is just for the use of locating axis-words, and it can be replacod by any modules that can achicve the same function.
We leverage both the ICD ICD-coding and the disease normalization training set to perform axis- word replacement. The detailed descriptions of each category of axis-word replacements are as follows:
ARI: ART is illustrated in the top left corner of Figure First, select a pair of diseases (disease A and disease B) that shares one or more axis (partl in figure) but is different in another axis (part 2 in figure). Then, replace the part 2 in disease A to be the same part2 in discase B. (Note: discase A can be chosen from any sources, but discase B can only be chosen from the standard ICD-coding list it serves as the label of a disease normalization pair.)
ARl-posotion: Perform ARI by fixing the disease center and replacing the anatomical location.
ARl-center: Perform ART by fixing the anatomical location and replacing the disease center.
ARl-quality: Perform ARI by fixing both the disease center and the anatomical location and replacing the disease quality.
AR2: AR2 illustrated in the top right corner of Figure 2 First, select a pair of umnormalized-standde diseases from the disease normalization training set. Let the unnormalized disease be disease A, and the standard disease be disoase B. Then, find disease C from ICD-coding list that shares one or more axis (partl) but is different in another axis (part2). Finally, replace part2 in disease to be the same part2 in disease C, so that the replaced disease A and disease C can form new discase normalization pair.
AR2-position: Perform AR2 by fixing the discase center and replacing the anatomical location.
AR2-center: Perform AR2 by fixing the anatomical location and replacing the discase center.
AR2-quality: Perform AR2 by fixing both the discase center and the anatomical location and replacing the disease quality.
Multi-Grain Aggregation (MGA):
We assume that labels in the disease normalization task have transitivity properties. In specific, a more specified description of an object can be comprised into larger group where the descriptions are more coarse. In the ICD coding system, there are also clear granularities of diseases. The maximum length of code that can be shared between hospitals 6, and the multi-grain structure contains 3-digit, 4-digit, and 6-digit codes. We observe that the semantic mcaning between discases that share the first
We will open source the code of our experiment along with the NER tool for disease names Github.

---

Under review as a conference paper at ICLR 2023
3-digit code but are different in the 4th-digit code can be quite different, but the meaning would be a lot similar the discases share the first 4-digit code. Therefore, We implement MGA augmentation using the following method.
we leverage the multi-grain nature of the TCD coding by assigning the label of a 6-digit disease to its corresponding 4-digit disease. We call the method "aggregation" because normally 4-digit disease can be matched to several 6-digit diseases, so the model can learn which diseases are similar. MGA-code is illustrated
MGA-code:
in the left bottom of Figure 2
MGA-codel: The 6-digit diseases are directly derived from the ICD-coding list.
MGA-code2: The 6-digit diseases are derived from the diseases in CHIP-CDN training set whose labels are 6-digit ICD disease.
Apart from the ICD coding, anatomical locations also follow a hierarchical structure, where several smaller positions can be grouped together to form larger position. Thus, we search for diseases in ICD coding that share the same center and one position is the upper position of another one, and we grouped the classification labels of the lower position discases to their upper position discases.
MGA-position:
MGA-position is illustrated the right bottom of Figure 2. (Note: the upper position diseases must come from the slandard ICD-coding list.)
MGAA-ositionn: The lower position discases are directly derived from the ICD- coding list.
The lower position discases are derived from the discases in CHIP-CDN training set.
MGGA-position2:
(Note: In the human body, we call location the upper position to another position if that location covers a larger area than another. In order to find the upper or lower positions of a position, we construct position tree document where the anatomical positions in the human body are organized into a tree data structure. We use the constructed position tree to recognize the upper and lower relations above. The same goal be achieved with other sources containing knowledge bases of human anatomy.)
4.2 Training Process
Taking the augmented data train the disease normalization task.
Fine-tuning the original disease normalization dataset.
Experiments
5.1 Dataset
We evaluate the effectiveness of our data augmentation methods on a Chinese disease nor- malization dataset called CHIP-CDN. CHIP-CDN originates in the CHIP-2019 competition and was collected in A_ Chinese Riomedical Language Understanding Evaluation Bench- mark called CBLUE Zhang et al. (2021). The datasct contains 6000 unnormalized-stanee disease pairs in the training set, 1000 pairs in the dev set, and 2000 pairs in the test set.
5.2 Experimental Setup
We evaluate, our methods on three basclines: BILSTM Sak ct al. (201 4 and BERT-basc Devlin et al. (2018), CDN-Baseline(Trom CBLUE Zhann et al. (2021). For BILSTM, we use two BILSTM layers followed by a MLP layer to perform classilication. For BERT- based models, we use the CLS vector to perform, classification. For CDN-Baseline we use the original model provided by its git repository which follows a "recall-match" two step training approach based on pre-trained language models. The choose of the baseline models is to demonstrate the effectiveness of our method under different types of models and training

---

Under review as a conference paper at ICLR 2023
Table Comparison of the devset accuracy (%) for the choice of different data augmentation methods on various baseline models.
Model
BILSTM
BERT-base
CDN-Baseling
trainset
0.455
0.558
0.577
trainset+EEDA
0.451
0.519
0.561
trainset++B11
0,466
0.556
0,578
Frainset++DDAA
0.485
0.578
0.592
settings. In specific, we verify the effectiveness of DDA to a train-from-scrated model using a BILSTM model, verify the effectiveness to models with pre-trained knowledge using the BERT-base model, and we verify the effectiveness to complex models using CDN- -DN-BBseline model.
For the BILSTM model and BERT-basc model, we use accuracy to judge the model perfor- mance. In our evaluation, treat this discase normalization multi-class classification rather than multi-label classification task despile that there are few data samples that single unnormalized disease is matched to several standard diseases. Hence, if an unnormal- ized disease is matched to several standard diseases, this data sample is considered correctly predicted as long as one of the standard diseases is correctly predicted. We design the ex- periments in this way to simplify the model as much as possible to more clearly illustrate the effectivennsss of DDA. For CDN-Bascline, we stick to the scttings CBLUE Zhang al. (2021), which use the F1 as the evaluation metric, use BERT-base as the bascline model, and use the two step training paradigm provided by CBLUE for better comparison.
To ensure fairness, we use the exact same parameter settings for the same model. In par- ticular, for CDN-Baseline, we use almost the same parameter settings as CBLUE's git. repository, including random seed numbers.
Additionally, we use devset for performance comparison, since the label of test set the CHIP-CDN dataset is not given. For all experiments, we keep the best performing result as the final score.
5.3 Results
The results are shown in Table [ The trainset in the table represents CHIP-CDN training set. From top to bottom, the performance of different models using different data aug- mentation methods is represented. Among them, BT the back-translation data augment method and DDA is the semantic-basee discase name data augmentation method proposed by us. The experimental results demonstrate that although EDA and back back-translation in- crease diversity, they both hurt performances in some settings (especially for FDA). How- ever, DDA improves the performance in every settings. Clearly, DDA avoids the problem of EDA, and its effect is much better than BT.
We observe that the performances improve for models above after applying the DDA methods, showing the effectiveness of our proposed methods. For the BILSTM model, the relative performance improvement reaches 6%. We further observe that there is more per- formance gain BILSTM than BERT- models and CDN-Baseline probably because the knowledge in pre-trained language models has already covered some of the similar in- formation, but our proposed method further improve their performance, showing the effectiveness of DDA.
5.4 Ablation Study
In this section, we evaluate the effectiveness every data augmentation methods on BIL- STM, BERT-base models and CDN-Baseline. As we propose two types of data augmentation methods, we evaluate them by taking out these methods one by one to see the resulting performances. The results are shown in Table We observe that removing data gener-
the youdao translation tool and the URL is

---

Under review as a conference paper at ICLR 2023
Table Ablation study for CHIP-CDN dataset. We remove our proposed data augmenta- tion methods once at a time and evaluate, the results.
Strategy
BILSTM
BERT-base
CDN-Baseline
DDA full
0.485
0.578
0.592
AR
0.467
0.568
0.588
MGA
0.455
0.558
0.577
ated by either types methods would lead to performance degradation, thus proving the effectiveness of every method that propose.
5.5
Smaller datasets experiments
06
07
04
0.2
01
10%
20%
30%
40%
50%
607
70%
80%
50%
100%
556
10%
20%
305
40%
506
60%
70%
90%
90%
107%
LSTM with
BERT
-E-BERT
BILSTM
BERT-Dase
Figure 3: Performance comparison on smaller datasets for BILSTM and BERT-basc. The smaller datasets are derived by randomly sampling the original CHIP-CDN training set, and the devset. CHIP-CDN stays the same.
We also evaluate the performance improvements CHIP-CDN since the data scarcity problem is more severe in smaller datasets. We evaluate the training set whose sizes range from 5%, to 100% of the CHIP-CDN training set size. For the convenience of training, for augmented training in this setting, we only leverage standard disease names in ICD-coding. No data from disease normalization training set are used.
smaller datasets that derived from
We draw curves to illustrate the comparison on whether to use our proposod methods or not, which is shown in figure 3 When the size of the training set increase, both curves steadily improve. We also notice that the performance gain higher when the size of the training set is smaller.
Conclusion
In this paper, we propose two main types of data augmentation methods for Chinese disease normalization tasks based on two hypothesis respectively, where the disease names have the property of structural invariance, and the labels in discase normalization task have the tran- sitivity properties. Our data augmentation methods explore the semantic information and the relation information in diseases, and are adopted in augmented training fashion to avoid introducing misinformation. Experimental results show that our DDA method can better solve the three main challenges in discase normalization task, namely description diversity, data scarcity, and semantics density. Compared to EDA and back-transe necktrranslation methods, our method has obvious advantages on the disease normalization task. Furthermore, we prove that our data augmentation methods work even better smaller datasets.

---

Under review as a conference paper at ICLR 2023
References
Mahdi Abdollahi, Xiaoying Gao, Yi Mei, Shameek Ghosh, Jinyan Li, and Michael Narag. Substituting clinical features using syuthetic medical phrases: Medical text data augmen- tation techniques. Artificial Intelligence in Medicine, 120:102167, 2021.
Gunjan Ansari, Muskan Garg, and Chandni Saxena. Data augmentation for mental health classification on social media. arXiv preprint arXiv:2112.10064, 2021.
David Berthelot, Nicholas Carlini, lan Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach semi-supervised learning, Advances in neural information processing systems, 32, 2019.
Jacob Devlin, Ming-Wo Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre- training deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04005, 2018.
Matúš Falis, Hang Dong, Alexandra Birch, and Beatrice Alex. Horses to zebras: Ontology- guided data augmentation and synthesis for ied-9 coding. In Proceedings of the 21st Workshop Biomedical Language Processing, pp. 389 2014 2022.
Hazel Kim, Daecheol Woo, Seong Joon Oh, Jeong- Won Cha, and Yo-Sub Han. Alp: Data augmentation using lexicalized pefgs for few-shot text classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 10894-10902, 2022.
Varun Kumar, Ashutosh Choudhary, and Funah Cho. Data augmentation using pre-trained transformer models. arXiv preprint arXiv:200.002245, 2020.
Nathan Ng. Kyunghyun Cho, and Marzyeh Ghassemi. Ssmba: Self-supervised mani- fold based data augmentation for improving out-of-domain robustness. arXiv preprint arXiv:000..10195, 2020.
Haşim Sak, Andrew Senior, and Françoise Beaulays. Long short-term memory based recur- rent neural network architect ures for large vocabulary speech recognition. arXiv preprint arXiv:1402.1128, 2014.
Yuxia Wang, Fei Liu, Karin Verspoor, and Timothy Baldwin. Evaluating the utility of model configurations and data augmentation on clinical semantic textual similarity. In Proceedings of the 19th SIGBioMed workshop on biomodical language processing, pp.
105 111, 2020a.
Yuxia Wang, Karin Verspoor, and Timothy Baldwin. Learning from unlabelled data. for clinical semantic textual similarity. In Proceedings the 3rd Clinical Natural Language Processing Workshop, pp.
2020b.
Jason Wei and Kai Zou. Eda: Easy data augmentation techniques for boosting performance on text classification tasks, arXiv preprint arXiv:1901.11196, 2019.
Xing Wu, Shangwen Lv, Liangjun Zang, Jizhong Han, and Songlin Hu. Conditional bert contextual augmentation. In International conference on computational science, pp. 84- 95. Springer, 2019.
Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. Unsupervised data aug- mentation for consistency training. Advances in Neural Information Processing Systems, 33:6256 6268, 2020.
Ningyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang, Lei Li, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Fei Huang, et al. Cblue: A chinese biomedical language understanding evaluation benchmark. arXiv preprint arXiv:2106.08087. 2021.
10

---

Under review as a conference paper at ICLR 2023
Table 3: Statistics of the resulting augmented data.
Method
AR1
AR2
AR3
AR4
AR5
AR6
MGA1
MGA2
MGA3
443916
202587
98130
360900
18373
56431
23275
8870
3045
Table 4: Hyperparameter settings for all three baseline models
Model
stage
learning rate
epoch
BILSTM
augmented training
1e-4
BILSTM
fine-tune
1e-4
100
BERT
augmented training
1e-5
1
BERT
fine-tune
le-4
5
CDN-Baselin
augmented training
5e-6
1
CDN-Baseline
fine-tune
5e-5
3
A
Appendix
A.1
data augment result statics
The table 3 is all the statistical results of the data we obtained using MGA and AR data augmentation methods
A.2
Hyperparameter settings
Table shows the hyperparameter settings of our choices. For different methods, the way of parameter setting is different. For models that use word2vec initialization or random initialization parameters, the training augmented data can be regarded training task, and a large learning rate and a large number of iterations can be set to make the training sufficient. For models that backbone, a small learning rate and a small number of training iterations should be set to avoid the catastrophic forgetting of valuable information in the pre-trained models.
special pre-
pre-trained model (i.c. BERT) as the
For each baseline model, we first train on the augmented dataset (Augmented Training), and then fine-tune on CHIP-CDN dataset. For the CDN-Baseline model, we Chineso-bert- wwm as the pre-training model, and the training method provided by CBLUE. For the DDA method, we first use the augmented dataset train for I epoch with learning rate of 5e-6 and then tune on CHIP-DDN. The hyperparameter of the num negative sammle_s_mpleeeeee 3+3 and the recall k is (The explanation of hyperparamoter num gegative_sample and recall can be found in their github repository).
A.3 Analysis
In table E. the first row represents the distribution of the number of times the label appears in the training set. The other two rows represent the label distribution of the two types of the augmented data. The statistical result shows that the data DDA can effectively improve the labels that appear less frequently (the number of times < 3) and the labels that do not appear at all (did not appear) in the training set. This is beneficial for addressing the data scarcity problem of disease normalization tasks and the diversity of disease names. This is the direct reason why DDA works. As for EDA and BT, they can only increase the number of labels that are already appeared the training set, which only solve the problem of expression diversity, Hence, their abilities are limited.
A.4 Case Study
We give a real example of the augmentation results generated by different data augmentation methods. We observe that the semantic meaning of the EDA-generated result dramatically
We will open the augmentation code and the angmented result Github.
11

---

Under review as a conference paper at ICLR 2023
Table 5: Analysis the label distribution of DDA result.
Data trainset
all
did not appear
occurs<=3
occurs>3
37645
34396
2870
634
DDA-AR
11518
9532
1497
489
DDA-MGA
8478
7315
915
246
Table 6: Case Study, Compare the generated result through three data augmentation meth- ods.
Model
text
label
original
R +
X #
Traumatic Arthropathy of the Ankle
ankle injury
EDA
WUiitt X 1
traumatic joint disease
ankle injury
BT
:UC (ij
ER (
Traumatic Ankle Injury
ankle injury
DA-AR
* 13 Tr
Traumatic Arthropathy of the Knee E 1 1
knee injury knee injury
DA-MGA
Bl (5 8t
Traumatic Arthropathy of the Ankle
Aukle Sprains and Strain Injuries
changes due the property of semantics density, and changes the key information within the disease by losing the anatomical location. The results generated by BT is more realistic, but this method cannot generate samples beyond the original label scope, and it also suffers from the restrictions of the translation tools. As for proposed method DDA (last two lines in the table), can not only increase the diversity of the input, but also generates data where their labels are never appeared in the training set, that sparse labels can be trained more thoroughly.
A.5 Future work
So far, have only proved the effectiveness of DDA method, but no experimental analysis is done to explore the internal mechanisms of why it is so effective. Moreover, to further avoid the injection of misinformation, we believe designing loss function terms to effectively select more valuable data from the data augmentation results can be a promising direction. We aim perform researches on those topics the future.
12


Reviews:
[{"review_id": "zreZoBkWT9Z", "reviewer": "Reviewer_Ujvh", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": " This paper introduces a new data augmentation method for Chinese disease normalization dataset after analysing the unnormalized disease name and standard disease name. The main contribution is a novel data augmentation method adapted to a new dataset, which consists of axis-word replacement and multi-grain aggregation.\n\n", "summary_of_the_review": "The paper focus a intereting problem although the problem seems not a core problem in ICLR community. The method seems sound but I think it is not properly compared with various existing DA methods", "strength_and_weaknesses": "## Strength:\n- The paper proposes an interesting method to augment dataset, it achieave good performance compared with other augmentation method.\n- It demonstrated that EDA and BT methods are harmful  on CDN datasets.\n\n## Weakness:\n- It would be nice if the authors could basically check how many ratios of  wrong labels brought by the proposed DA and existing one, either using expert annotation  or automatic evaluation. This could be tested by using only a few examples.\n\n- There are no details about how many augmented examples used for baseline DA methods including EDA and BT. This should be carefully compared for a fair comparison.\n\n- Only a small-scaled Chinese dataset was used. It would be better if the authors could evidences their method across many datasets\n\n- The task is only for Chinese language, making it too narrow to fit ICLR.\n\n- The authors should detail the motivaiton to use EDA and BT as the baselines. Since there are many other  DA methods in NLP, see https://arxiv.org/abs/2110.01852 \n\n\n\n\n## minor issue:\n- The font and the reference format seems wrong", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "O7VLyWOxuOG", "reviewer": "Reviewer_6zpd", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The proposed and evaluated data augmentation techniques for normalizing disease names in Chinese. The techniques were derived from disease corpora extracted from an existing dataset of authentic clinical text and the standardized ICD-10 coding system. Experimental results demonstrated a fair 3% performance gain with potential for further gains on smaller datasets.", "summary_of_the_review": "The paper has more weaknesses than strengths and hence, I recommend its rejection.", "strength_and_weaknesses": "* Experimental results demonstrated a fair 3% performance gain with potential for further gains on smaller datasets.\n* Medical natural language processing is societally important.\n* Although the evaluation results demonstrated performance gains, no statistical significance testing, confidence intervals, or effect sizes were reported in the paper. However, the authors reported on an ablation study whose merit should be noted.\n* The novelty and clarity of the study - or minimally their clear and convincing argumentation - can be questioned.\n* Ethical considerations of this study were insufficiently addressed.\n* Unfortunately, the paper was formatted using an incorrect template; it did not look like an ICLR 2023 paper and hence, should be desk rejected. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "F9R2eRK77M", "reviewer": "Reviewer_iwv2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors have proposed a data augmentation framework which helps in improving the performance of existing disease normalization models. The framework is compared against two existing methodologies and in smaller dataset setting as well.", "summary_of_the_review": "Currently, the work needs some more effort on the presentation and proof-reading. The experimentation and training procedure needs to be more clearly written and experiments need to be a bit more thorough in terms of number of runs. Some more baselines would also improve the quality of work.\nAs I mentioned earlier, I believe that the work is important and necessary but I would be much more suitable for a clinically oriented workshop or conference such as ML4H, MLHC, LOUHI, BioNLP or clinicalNLP. \n", "strength_and_weaknesses": "Strength: \n1. The authors have focused on an important problem of disease normalization.\n2. The proposed framework improves the performance of the existing normalization models. \n\nWeaknesses [Questions to the authors]: \n1. The paper needs some proof-reading and formatting. Some common formatting issues are highlighted below:\n    - The starting double quotes in latex need to be `` as opposed to \" since they appear inverted during pdf built. \n    - All the tables would look a bit better with the top and bottom margins would look a bit cleaner.\n    - Para 1 of the introduction consists of multiple sentences with grammatical errors as well. \n2. The authors have mentioned the work of Falis et al [2022] and Abdollahi et al [2021] for data augmentation using UMLS but have not compared their methodologies for data augmentation.\n3. The information regarding the NER tool are completely missing from the paper. On which data was the NER model from section 4.1 trained and what was its performance? Since the performance of the framework heavily depends on the model as well. \n4. Is there a validity study done to see if the axis rotation method would not create diseases which are not anatomically or medically correct or significant?\n5. Section 4.2 needs some more details regarding the training procedure. Were the experiments done using multiple seeds? If yes, could you please share the average and std for the results? \n6. Details in Section 5.2 are confusing. Para 2 of the section mentions accuracy as the evaluation metric but also mentions that F1-score was used for evaluation according to the previous work. Also, was macro or micro F1-score used for evaluation? \n7. Table 1 mentioned that the result is of devset accuracy. Are all the experiments performed on devset? And is the testset accuracy or F1-score not reported?\n\n-------------------------------------\nRecommendation to the authors\n-------------------------------------\nI believe that the work is important and necessary but I would also like to add that it would be much more suitable for a clinically oriented workshop or conference such as ML4H, MLHC, LOUHI, BioNLP or clinicalNLP. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "CoApquniZLT", "reviewer": "Reviewer_ethR", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper describes authors work related to set of data augmentation techniques that work together as an augmented training task for disease diagnosis normalization.Various techniques are described which show incremental improvement in disease diagnosis normalization against ICD-10 classification using CHIP-CDN dataset. The key to their work are described methods which augment the labels.", "summary_of_the_review": "The paper addresses an important challenge faced in healthcare data,especially with disease diagnosis. Abbreviations,different standards, different languages and different coding nomenclature(ICD vs others), make it difficult to find the correct diagnosis in the text. This is a challenge from not just label point of view but also from any other modeling aspect.\nThe authors have shown through their work that their novel techniques can show incremental benefit in overcoming this challenge.\nBut it is not clear, whether this is generalizable or not.", "strength_and_weaknesses": "Strengths:\n- Authors work does demonstrate incremental benefit(3%) over prior related work.\n- Clearly,this is valuable works there is significant variability in disease diagnosis as documented by clinicians.\n\nWeakness: \n- Incremental benefit is limited.\n- Some of the grammar could be improved.\n- Unclear if the Chinese language have the exact translation for comparison.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject