VTCC - NLP at S em E val-2023 Task 6:Long-Text Representation Based on Graph Neural Network for Rhetorical Roles Prediction (2023)

Abstract: Rhetorical Roles (RR) prediction is to predict the label of each sentence in legal documents, which is regarded as an emergent task for legal document understanding. In this study, we present a novel method for the RR task by exploiting the long context representation. Specifically, legal documents are known as long texts, in which previous works have no ability to consider the inherent dependencies among sentences. In this paper, we propose GNNRR (Graph Neural Network for Rhetorical Roles Prediction), which is able to model the cross-information for long texts. Furthermore, we develop multitask learning by incorporating label shift prediction (LSP) for segmenting a legal document. The proposed model is evaluated on the SemEval 2023 Task 6 - Legal Eval Understanding Legal Texts for RR sub-task. Accordingly, our method achieves the top 4 in the public leaderboard of the sub-task. Our source code is available for further investigation https://github.com/hiepnh137/SemEval2023-Task6-Rhetorical-Roles .

## introduction semeval 2023 task 6 legal eval understanding legal texts @xcite aims to automatically process some intermediate tasks to boost the productivity of legal systems. the task is split into three sub-tasks, which are rhetorical roles (rr), legal named entity recognition (l-ner), and court judgement prediction with explanation (cjpe). the purpose of the rr prediction task is to standardize an unstructured corpus so that it is easier for the machine to automatically understand legal documents. participants are given a dataset including long and unstructured legal documents. each document is segmented into sentences using automatic tools @xcite . segmented units could be classified into 13 labels, including 12 semantic labels and a none label † equal contribution; * corresponding author 1 https://github.com/hiepnh137/semeval2023-task6rhetorical-roles for those not belonging to any semantic role. the rr task is considered as the sequence classification task with single-label multiple classes, where participants are required to predict the suitable labels for a series of consecutive text segments in those 13 labels. traditional approach @xcite uses crf with hand-crafted features to segment the document into seven different roles. @xcite proposed bilstm-crf model with sentence embeddings constructed from sent2vec to label rhetorical roles in indian supreme court documents. a multitask learningbased model developed by @xcite uses label shift information to predict labels. @xcite ) created a rr corpus of english indian legal documents and proposed the model based on scibert-hsln architecture @xcite . despite the success of the aforementioned models, there are still two remaining challenges: long-range dependence between sentences and label ambiguity. it is because legal documents are quite long. therefore, it is hard to represent the contextual information in each sentence. besides, the performance is negatively affected due to the similar labels such as arg_petitioner and arg_respondent, pre_not_relied and pre_relied. in order to address the aforementioned problems, we propose a gnn-based context representation module to exploit the inter-sentence relations for the long-text representation. for similar labels, we use contrastive learning to learn embedding space. we also use multitask learning framework introduced by @xcite to boost the model performance. our final submission for sub-task rr achieved a micro-averaged f1 score of 0.8389 positionings our team in the top 4 on the public leaderboard of the sub-task. the architecture of our model consists of two main components: the label shift prediction (lsp) component and the rhetorical roles prediction (rrp) component as shown in figure 1 . specifically, given a legal document d containing n sentences {s 1 , s 2 , ..., s n }, the label shift prediction task can be seen as an auxiliary task to model relationships between two adjacent sentences and predict whether the labels of them are different (y lsp @xmath0. we model lsp via lsp-bilstm-crf architecture, which has shown promising results @xcite . particularly, supporting s @xmath1, ..., s lsp n ) denotes sentence representations, using lsp task can be formulated as follows: on the other hand, the rhetorical roles prediction task can be defined as the sequence labeling task. our architecture for this task is based on scibert-hsln @xcite , which includes three main modules: sentence encoder, gnn-based context representation module, and output layer. ## conclusion in this paper, we represent our methodology for the rhetorical roles predictions task in semeval 2023 task 6 legal eval understanding legal texts. our approach, which is a combination of using multitask learning, contrastive learning, and the proposed gnn-based context representation module, is proven effective via experiments. moreover, our best result ranks top fourth in the public leaderboard of that sub-task.