YNU - HPCC at S em E val-2023 Task7: Multi-evidence Natural Language Inference for Clinical Trial Data Based a B io BERT Model (2023)

Abstract: This paper describes the system for the YNU-HPCC team in subtask 1 of the SemEval-2023 Task 7: Multi-evidence Natural Language Inference for Clinical Trial Data (NLI4CT). This task requires judging the textual entailment relationship between the given CTR and the statement annotated by the expert annotator. This system is based on the fine-tuned Bi-directional Encoder Representation from Transformers for Biomedical Text Mining (BioBERT) model with supervised contrastive learning and back translation. Supervised contrastive learning is to enhance the classification, and back translation is to enhance the training data. Our system achieved relatively good results on the competition’s official leaderboard. The code of this paper is available at https://github.com/facanhe/SemEval-2023-Task7 .

## introduction with the massive increase of clinical trial reports (ctrs) publications, there are more than 10000 clinical trial reports for breast cancer at present. therefore, it is not feasible for clinical practitioners to timely understand all current literature, so as to provide personalized evidence-based nursing @xcite . in this context, the application of natural language inference (nli) can promote the development of large-scale interpretation and retrieval of medical evidence, and the successful development of the system can greatly promote us to combine the latest evidence to support personalized care @xcite . to this end, the purpose of this task is to use clinical trial data to conduct multiple evidence natural language inference. the task is based on a group of clinical trial reports of breast cancer, statements, explanations, and labels annotated by domain expert annotators. each clinical trial report data of patients consists of four sections: (1) eligibility criteria: a series of conditions that allow patients to participate in the clinical trial. (2) intervention: information about the type, dose, frequency, and duration of the treatment studied. (3) results: number of participants in the trial, result measurement, unit, and results. (4) adverse events: physical signs and symptoms of patients observed during the clinical trial. an example of this task is shown in the semeval-2023 shared task7 consists of two subtasks @xcite ): the main difficulty of subtask 1 is textual entailment. textual entailment describes the reasoning relationship between two texts, in which text 1 is the premise and text 2 is the hypothesis. if the premise can infer the hypothesis, it is an entailment, otherwise, it is a contradiction. the task of textual entailment can also be regarded as the problem of text classification @xcite . the early task of textual entailment is based on the method of text similarity @xcite or text alignment @xcite . with the development and expansion of deep learning, convolutional neural networks (cnn) @xcite and recurrent neural networks (rnn) @xcite have achieved good results in textual entailment. the long short-term memory (lstm) @xcite and application of the attention mechanism have also achieved better results than the cnn and rnn. however, since the introduction of the pre-trained model bi-directional encoder representations from transformers (bert) @xcite , it has raised the task of textual entailment to a new level. to investigate the medical entailment relations, this study proposed to use the biobert model with supervised contrastive learning for subtask 1. different from the conventional bert encoder, biobert @xcite ) was trained on biomedical texts, thus can be more beneficial for the task of clinical text analysis. for the limitation of input length, different truncation strategies were conducted. in addition to the conventional crossentropy loss function, supervised contrastive learning @xcite was further introduced to enhance the classification. in each training batch, the samples with the same label are pulled together while the ones with different labels are pushed away in the semantic spaces for representation learning. on the original training data, back translation is used to enhance the data without changing the semantics, through this method, we doubled the training data. in addition, in order to obtain better results, the system explored another medical text processing model bio_clinicalbert @xcite , and the system also explored using the longformer @xcite ) model to deal with long text, but unfortunately, the experimental effect did not reach the expected effect, perhaps because the individual did not adjust the appropriate parameters. empirical experiments were conducted on the developing set to select the optimal solution for the final submission. the results show that biobert + supervised contrastive learning + back translation has achieved the best performance of 0.665 in terms of the f 1 -score. the final submission for the test set has achieved 0.679 and ranked 12 th place in subtask 1. the remainder of this paper is organized as follows. section 2 describes the model and method used in our system, section 3 discusses the results of the experiments, and finally, the conclusions are drawn in section 4. ## system description this section will describe the architecture of the proposed model in detail. there are several com- e[cls] e1 tokenizer premise adverse events 1: total: 12/32 (37.50%) anaemia 0/32 ... hypothesis all the primary trial participants do not ... [cls] x1 e[sep] ei [sep] y1 en e[sep] ym [sep] t[cls] t1 t[sep] ti tn-1 t[sep] ## conclusion in this paper, we present a system submitted in subtask 1 of the semeval-2023 task 7, which utilizes the pre-trained model biobert to adjust the official baseline model and uses the text classification task to complete the textual entailment task of ctrs. the experimental results show that our proposed system achieves good performance. in addition, in subtask 1, compared with the top-ranked team system, our system still has a lot of room for improvement. in future research, we hope to try other biomedical text-processing models or textlength processing methods to obtain better results.