Improving Supervised Drug-Protein Relation Extraction with Distantly Supervised Models (2022)

Abstract: This paper proposes novel drug-protein relation extraction models that indirectly utilize distant supervision data. Concretely, instead of adding distant supervision data to the manually annotated training data, our models incorporate distantly supervised models that are relation extraction models trained with distant supervision data. Distantly supervised learning has been proposed to generate a large amount of pseudo-training data at low cost. However, there is still a problem of low prediction performance due to the inclusion of mislabeled data. Therefore, several methods have been proposed to suppress the effects of noisy cases by utilizing some manually annotated training data. However, their performance is lower than that of supervised learning on manually annotated data because mislabeled data that cannot be fully suppressed becomes noise when training the model. To overcome this issue, our methods indirectly utilize distant supervision data with manually annotated training data. The experimental results on the DrugProt corpus in the BioCreative VII Track 1 showed that our proposed model can consistently improve the supervised models in different settings.

## introduction drug-protein relations are important for drug discovery, metabolic, and drug response modeling, and their textual evidence is important in the development of evidence-based medicine. however, since drug-protein interactions are reported in the literature and the number of relevant articles is rapidly increasing @xcite , it is difficult for pharmacologists to read every single article to determine the interactions. therefore, automatic interaction extraction from text has attracted much attention. the related shared tasks @xcite are being conducted at biocreative, an international workshop that aims to evaluate text mining and information extraction in the biological domain. for drug-protein relation extraction, models using deep learning have achieved high performance. a typical deep learning model takes as input a sentence and the drug and protein mentions in the sentence, and predicts the relationship between the drug and the protein as expressed in the sentence. @xcite extracted the relationships using a large neural network model pretrained on a large biomedical literature (pubmedbert). deep learning models suffer from the problem of the huge cost of manually annotated training data. a distantly supervised learning method has been proposed by @xcite . the method enables the creation of a large amount of training data at low cost. however, this method still has the problem of producing data with incorrect labels, which become noise during training. several methods have been proposed to mitigate the effects of such noisy examples. one of the most commonly-used methods is multi-instance learning @xcite , where the distant supervision data is treated as a bag of instances corresponding to pairs in the database. @xcite proposed a method to train instances with the representation with the highest prediction probability of the target label in the bag. @xcite proposed a method to weight instances in the bag so that correctly labeled instances will have large weights while noisy cases have small weights. @xcite proposed a method of learning with distant supervision data by utilizing some of manually annotated training data to learn the weights. although such methods show performance improvement in the distantly supervised training setting, the performance is still lower than that of the methods trained on manually annotated training data. this study proposes a novel method of using distantly supervised relation extraction models for supervised drug-protein relation extraction. by using the model trained over the easy-to-create distant supervision data, we aim to improve the performance of supervised drug-protein relation extraction while reducing the cost of building additional manually annotated data and the effect of noisy instances in the distant supervision data. our contributions are as follows: ## methods we propose a novel method for extracting drugprotein relations from manually annotated training data. the method uses a model trained on distant supervision data, which we call a distantly supervised model. by utilizing the distantly supervised model, we aim to improve the extraction performance while reducing the influence of noisy instances included in the distant supervision data. in the following sections, we will explain the baseline relation extraction model in section 2.1, the construction of distant supervision data from databases in section 2.2, and the methods for utilizing the distantly supervised model in section 2.3. ## experimental settings in this section, we explain the settings for the data sets, tasks and hyper-parameter tuning. weighting by importance bert bert … ×𝛽 representation 𝒉 𝒅𝒔 𝒉 𝒔𝒗 mixture ×𝛼 prediction：agonist (ⅰ) extract representations from pretrained feature extraction models pretrained with manually annotated training data and not fixed pretrained with distant supervision data and fixed we used the data set from the biocreative vii track 1 -text mining drug and chemical-protein interactions (drugprot) @xcite for the evaluation. this data set is composed of documents annotated with drug mentions, protein mentions, and their relations. the drugprot corpus consists of train, develop, and test. since the annotations for the test data are not publicly available, this study evaluates the model on the development data. in addition, the distant supervision data built by the method in section 2.2 were used to train the model. the number of instances per relation in the drugprot corpus and the distant supervision data are shown in ## results to evaluate the proposed method, we conducted three experiments: evaluation of the performance of extracting drug-protein relations, analysis of prediction results, and comparison of extraction performance on small-scale manually annotated training data. in this section, we describe these three experiments. ## conclusions we aimed to improve the performance of drugprotein relation extraction by creating distant supervision data at low cost and utilizing the model pre-trained on the data while reducing the noise contained in the distant supervision data. we proposed two methods of utilizing distant supervision data. both methods improved the prediction performance from the baseline for relation types with less manually annotated training data. in addition, the method that mixes representations also improved the f-scores for many relation types, some of them have a large amount of manually annotated training data, as well as the micro-averaged f-score, demonstrating the effectiveness of the proposed method. in addition, we showed that the performance im-