A Multi-Task Biomedical Named Entity Recognition Method Based on Data Augmentation (2024)

Abstract: “The rapid development of artificial intelligence has led to an explosion of literature in the biomed-ical field, and Biomedical Named Entity Recognition (BioNER) can quickly and accurately iden-tify key information from unstructured text. This task has become an important topic to promotethe rapid development of intelligence in the biomedical field. However, in the Named EntityRecognition (NER) of the biomedical field, there are always some problems of unclear boundaryrecognition, the underutilization of hierarchical information in sentences and the scarcity of train-ing data resources. Based on this, this paper proposes a multi-task BioNER model based on dataaugmentation, using four data augmentation methods: Mention Replacement (MR), Label-wisetoken Replacement (LwTR), Shuffle Within Segments (SiS) and Synonym Replacement (SR)to increase the training data. The syntactic information is extracted by incorporating the inputsentence into the Graph Convolutional Network (GCN), and then the tag information encodedby BERT is interacted through a co-attention mechanism to obtain an interaction matrix. Subse-quently, NER is performed through boundary detection tasks and span classification tasks. Com-parative experiments with other methods are conducted on the BC5CDR and JNLPBA datasets,as well as the CCKS2017 dataset. The experimental results demonstrate the effectiveness of themodel proposed in this paper.”

## introduction named entity recognition (ner) aims to identify named entities of specific meaning from the text and classify them into pre-defined entity types. in specific areas, such as biomedicine, ner needs to identify entity types, such as chemicals, diseases, genes, and proteins. ner plays a major role in critical information extraction and provides an important basis for other downstream tasks. in deep learning methods, the forms of biomedical entities is more complex, usually composed of multiple words, and there are many abbreviations, whose forms are changeable and lack of clear rules to follow, which makes it difficult to determine the physical boundaries of entities and increases the difficulty of identification. moreover, well-annotated datasets in the medical field are relatively scarce, further reducing the identification effect. biomedical named entity recognition (bioner) also faces nested overlapping entities, leading to further increased difficulty in identification. limited use of syntactic information in corporcorpus in bioner methods, which also leads to the problem of poor results. based on the above challenges, this paper presents a multi-task bioner model based on data augmentation, with the following main contributions: ## related work ner is one of the important indicators to evaluate the effect of information extraction. this paper introduces a data augmentation-based multi-task bioner model. next, data augmentation, syntactic analysis, and multi-task methods will be detailed to get a more comprehensive understanding of the study. ## method first, the model uses four methods for data augmentation, with input data and label information processed through bert encoding, and syntactic information extracted using gcn. new input data and label information, along with syntactic information via an attention interaction matrix, are then applied to the boundary detection and span classification tasks for entity recognition. the bioner task is transformed from a sequence annotation task into a multi-task framework, specifically addressing the joint tasks of boundary detection and span classification, which solves the problems of unclear boundary recognition and entity nesting. syntactic analysis is performed on the dataset, and gcn is used to encode the syntactic analysis graph, integrating it into the multi-task framework. this allows the model to fully learn the syntactic components in the text and their interrelationships, thereby improving the performance of ner. the overall architecture of the model is shown in figure 1 . 3.1 data augmentation methods in this paper, four methods of mr, lwtr, sis and sr are used to change the input data, and these four methods are used for data augmentatio. mention replacement (mr):for each entity in the input sentence, a binomial distribution is used to randomly decide whether it should be replaced. the binomial distribution represents the resulting probability distribution of a series of independent repeated binary experiments. for example, in figure 2, the chemical type entity "22-oxacalcitriol [b-chem i-chem i-chem]" was replaced by "puromycin aminonucleoside [b-chem i-chem]", an entity of the disease type "secondary hyperparathyroidism [b-dise idisorder]" was replaced by "cauda equina syndrome [b-dise i-dise i-disorder]", "low bone turnover [b-dise i-dise i-disease]" was replaced by "neurotoxicity [b-disease]", and "renal failure [b-dise idisorder]" was replaced by "neurological deficits [b-dise i-disorder]". the new training data, after entity label-wise token replacement (lwtr): for each word, a binomial distribution is also used to randomly decide whether it should be replaced. if the probability result is replacement, use the tagword distribution (the label-token) distribution built from the original training set to randomly select another word with the same tag and ultimately keep the original tag sequence unchanged. as shown in figure 3 , six words (suppresses, without, inducing, bone, dogs, and renal) were replaced by other words (aminonucleoside, levels, of, none, haven, hypercalcemia) with the same label as the original word. figure 3 : the label-wise token replacement method shuffle within segments (sis): first, the word sequence is split into small parts of the same label, where each part corresponds to a word or a series of words of the same type. as shown in figure 4 ,secondary hyperparathyroidism, without causing and low bone transformation all three phrases were disrupted, (secondary hyperparathyroidism) were disrupted as (hyperparathyroidism secondary), (without inducing) as (inducing without), and (low bone turnover) as (turnover low bone). figure 4 : the shuffle within segments method synonym replacement (sr): the sr method is similar to lwtr, replacing words with synonyms retrieved using wordnet or chinese wordnet. but the retrieved synonyms may include more than one word, so for bio labels, a simple rule can be derived: if the replaced word is the first word in the entity (i.e., the corresponding label is "b-chem"), we assigned the same label to the first marker of the retrieved multi-word synonyms and assigned "i-chem" to the other words. if the replaced word is in an entity (i.e., the corresponding label is "i-chem"), we assign its label to all words of multi-word synonyms. as shown in figure 5 , (suppresses) replaced with (restrain), (low bone turnover) replaced with (suppression of bone turnover), and (renal failure) replaced with (impaired renal failure). ## summary and outlook this paper proposes a multi-task bioner model based on data augmentation. it employs four data augmentation methods, mr, lwtr, sis and sr, to augment the training data. the model utilizes gcn to extract syntactic features from sentences. these multiple features are then interacted through a coattention mechanism and combined with boundary detection and span classification in a multi-task mode for ner. the experimental results confirm the effectiveness and positive impact of adopting multi-task learning and data augmentation methods. in the future, we will explore transfer learning and large language models. large language models, with more parameters and deeper structures, have the capability to capture richer and more complex data patterns and features. this enables them to learn more abstract and advanced representations, leading to better performance across various tasks. c c l 2 0 2 4