JUST - KM at S em E val-2023 Task 7: Multi-evidence Natural Language Inference using Role-based Double Roberta-Large (2023)

Abstract: In recent years, there has been a vast increase in the available clinical data. Variant Deep learning techniques are used to enhance the retrieval and interpretation of these data. This task deployed Natural language inference (NLI) in Clinical Trial Reports (CTRs) to provide individualized care that is supported by evidence. A collection of breast cancer clinical trial records, statements, annotations, and labels from experienced domain experts. NLI presents a chance to advance the widespread understanding and retrieval of medical evidence, leading to significant improvements in connecting the most recent evidence to personalized care. The primary objective is to identify the inference relationship (entailment or contradiction) between pairs of clinical trial records and statements. In this research, we used different transformer-based models, and The proposed model, “Role-based Double Roberta-Large,” achieved the best result on the testing dataset with F1-score equal to 67.0%

## introduction breast cancer clinical trial reports (ctrs) summarize the results of clinical trials conducted to test new treatments or interventions for breast cancer. these reports typically include information on the study design, patient population, treatment regimens, outcomes (such as response rates and side effects), and overall conclusions. these reports provide a comprehensive and transparent account of the trial results to the scientific community and the general public to inform future research and clinical practice. the results of breast cancer clinical trials play a critical role in the development of new treatments and in improving outcomes for patients with breast cancer @xcite . the number of published (ctrs) has grown significantly in recent years, with over 10,000 ctrs for breast cancer alone. as a result, it has become increasingly more work for clinical practitioners to keep up with all the current literature and provide patient care based on the latest evidence @xcite . natural language inference (nli) has the potential to enhance the analysis and interpretation of ctrs by providing a more automated and consistent approach to extracting information and drawing inferences from these reports @xcite . multi-evidence natural language inference (nli) is a task in natural language processing that involves determining the relationship between two pieces of text, specifically whether the second text is an entailment, contradiction, or neutral concerning the first text. in multi-evidence nli, multiple pieces of evidence are used to make the determination instead of relying on a single text. this can be useful when the relationship between the two texts needs to be clarified or additional context is needed to make an accurate determination @xcite . nli has been approached using various methods, including symbolic logic, knowledge bases, and neural networks. recently, it's become a crucial testing ground for methods using distributed word and phrase representations. these distributed representations excel in capturing similarity-based relations and have successfully modeled basic dimensions of meaning such as evaluative sentiment @xcite . following the substantial success of deep learning (dl) techniques in various artificial intelligence tasks, researchers in the field of natural language processing (nlp) have begun to develop dl-based models to analyze patterns in the natural language data produced by humans @xcite . this research aims to build a deep learning model to identify the inference relationship (entailment or contradiction) between pairs of ctr statements (textual relationship -prompt). the rest of this paper goes as follows: section ii reviews the recent research works related to this study. section iii describes our methodology. section iv reports 1 447 the results by discussing the proposed model and comparative performance. finally, the conclusion is in section v. ## related work deep learning has emerged as a powerful tool for analyzing and interpreting complex medical data and has been widely studied in recent years @xcite @xcite . this section presents a brief review of the related work in this field. the authors @xcite presented a new lstm architecture for natural language inference (nli) that builds on top of a previous neural attention model. the proposed model performs word-by-word matching of the hypothesis and the premise using a match-lstm. the lstm emphasizes important word-level matches and remembers critical mismatches to predict the relationship label. the model is evaluated on the stanford natural language inference (snli) corpus and achieves an accuracy of 86.1%, surpassing state-of-the-art. the authors @xcite presented a new state-of-the-art approach for the stanford natural language inference dataset with a high accuracy of 88.6%. instead of using complex network architectures, they demonstrate that a carefully designed sequential inference model using chain lstms outperforms previous models. they achieve further improvement by incorporating recursive architectures in local inference modeling and inference composition. additionally, syntactic parsing information contributes to the best results, even when added to a powerful model. the authors @xcite proposed a new language model, sem-bert, that combines bert with pre-trained semantic role labeling to explicitly capture contextual semantics. sembert is easy to fine-tune for specific tasks while retaining bert's usability. it outperforms bert and sets new state-of-the-art results on ten reading comprehension and language inference tasks. the authors @xcite presented a novel deep learning architecture, dr-bilstm, for the natural language inference task. they are using the stanford nli dataset. after an enhanced preprocessing step, they achieved new state-of-the-art scores by the dr-bilstm model. dr-bilstm used a different approach from other existing methods, which used a simple reading technique. dr-bilstm can model the relationship between a hypothesis and a premise while encoding and inference in an efficient way using dependent reading. also, the ensemble technique is used to merge the models to enhance predictions. the authors @xcite proposed a multi-branch network that combines knowledge and context information to improve performance. the network has two branches, one for context information that uses multi-level dynamic assisted attention to construct interaction between sentence pairs and another for knowledge information that employs a knowledge-based graph attention network (k-gat) to capture structural information of knowledge and uses an attention mechanism for interaction. additionally, a relation branch captures context and knowledge relations between sentence pairs. the model uses five semantic dependenciesbased knowledge types to minimize redundant external knowledge. the experiments show solid competitive results on three popular nli datasets. ## methodology this section outlines our methodology and proceeds as follows: firstly, the dataset for the task is outlined. then, the different preparing dataset approaches from the ctrs are described. finally, we described the proposed model approach to identify the inference relationship (entailment or contradiction) between pairs of clinical trial records and statements. ## results and discussion we conducted experiments with various models to identify the most suitable ones for our task. the models we tried include bert, roberta-base, roberta-large, tf-idf, and our proposed model, "role-based double roberta-large". the results of our experiments are summarized in to ensure that the model is achieved the best results. different hyperparameters have been tuned. and the best results are achieved by the below ## conclusion in conclusion, this research has explored the task of natural language inference and its applications in nlp. through the analysis of current state-ofthe-art models and evaluation metrics, it has been shown that nli is a challenging and complex task that requires modeling both lexical and semantic information. this term paper has examined using the prehighlight the potential of roberta for solving nli problems. the natural language inference (nli) task is considered difficult due to the inherent ambiguity of language, the complexity of reason-ing required to determine the relationship between two sentences, and the variability of sentence structures and relationships. despite the challenges and limitations faced by nli, it is clear that the task is of great importance for nlp and has numerous potential applications in areas such as text classification, question answering, and dialogue systems. therefore, it is crucial for the research community to continue to advance the development of nli models and explore new approaches for improving their performance.