Puer at S em E val-2024 Task 2: A B io L ink BERT Approach to Biomedical Natural Language Inference (2024)

Abstract: This paper delineates our investigation into the application of BioLinkBERT for enhancing clinical trials, presented at SemEval-2024 Task 2. Centering on the medical biomedical NLI task, our approach utilized the BioLinkBERT-large model, refined with a pioneering mixed loss function that amalgamates contrastive learning and cross-entropy loss. This methodology demonstrably surpassed the established benchmark, securing an impressive F1 score of 0.72 and positioning our work prominently in the field. Additionally, we conducted a comparative analysis of various deep learning architectures, including BERT, ALBERT, and XLM-RoBERTa, within the context of medical text mining. The findings not only showcase our method’s superior performance but also chart a course for future research in biomedical data processing. Our experiment source code is available on GitHub at: https://github.com/daojiaxu/semeval2024_task2.

## introduction clinical trial reports (ctrs) play a crucial role in documenting the methods and results of clinical trials @xcite . it contains a detailed overview of participant circumstances, intervention experiment descriptions, experimental results, and adverse events that happened in the participants. natural language inference is a valuable technique for analyzing experimental data in ctr and interpreting the results. natural language inference is able to analyze logical linkages, consistency, and contradictions in a document. it can assist detect logical relationships in text automatically, identify potential conflict areas fast, and improve decisionmaking accuracy and efficiency. researchers can better gather and analyze clinical trial data by using natural language inference techniques, which promotes medical quality improvement @xcite . the figure 1 shows the example dataset used in this work. the dataset includes two forms of ctr: single and comparison. a single type ctr can retrieve relevant evidence using a primary id. to retrieve two relevant pieces of evidence using comparison type ctr, primary id and secondary id must be used simultaneously. as an illustration, in the first instance, ctr represents "heart-related adverse events were recorded in both the primary trial and the secondary trial." searching for the matching components of the two pieces of evidence reveals that there are heart-related adverse effects, such as supraventricular tachycardia and atrial fibrosis. as a consequence, the first example is labeled as "entailment" @xcite . in a comparable way, in the second example, ctr believes that "patients with clinical stage ii (t2 n1) invasive breast cancer are not eligible for the primary trial." however, the participation conditions in the gathered evidence clearly show that individuals with clinical stage i or ii (t1 or t2, n0 or n1) invasive mammary carcinoma match the criteria. as a result, the second case is labeled "contradiction" @xcite . in the quest to push the frontiers of biomedical natural language understanding, semeval-2024 task 2 has emerged as a critical arena for testing the efficacy of ai models in parsing complex medical texts @xcite . engaging with this challenge, our work utilizes biolinkbert to set new benchmarks in the safety and accuracy of clinical trial inference @xcite @xcite . this endeavor not only underscores the significance of developing robust nli systems but also highlights our commitment to contributing meaningful innovations to the biomedical domain @xcite @xcite . through this paper, we aim to share our methodologies, findings, and the implications they hold for the broader field of medical research, hoping to inspire further advancements and collaborative efforts in this vital area of study. we created a number of attempts using the above dataset, and the following additions were contributed to our work: 1) we have designed a new loss function by combining the ideas of cross entropy and contrastive learning. this loss function can flexibly adjust parameters according to actual situations and has strong adaptability. ## conclusion this study has presented a comprehensive analysis of the effectiveness of biolinkbert in enhancing clinical trials. our research has meticulously finetuned the biolinkbert-large model with a novel mixed loss function. the experimental results, particularly the achievement of an f1 score of 0.72, underscore the potential of leveraging advanced pre-trained language models in medical research. our findings suggest that the integration of contrastive learning and cross-entropy loss functions significantly improves the model's performance, indicating a promising direction for future research in biomedical text mining. moreover, the success of this project opens new avenues for exploring the application of language models like biolinkbert in other domains of healthcare and medical research. future work could focus on expanding the dataset, experimenting with different architectures, and exploring the impact of domain-specific adaptations on model performance. this could potentially lead to breakthroughs in how we process, understand, and derive insights from clinical trial reports, ultimately contributing to the advancement of medical science and patient care.