Multi-granular Legal Topic Classification on G reek Legislation (2021)

Abstract: In this work, we study the task of classifying legal texts written in the Greek language. We introduce and make publicly available a novel dataset based on Greek legislation, consisting of more than 47 thousand official, categorized Greek legislation resources. We experiment with this dataset and evaluate a battery of advanced methods and classifiers, ranging from traditional machine learning and RNN-based methods to state-of-the-art Transformer-based methods. We show that recurrent architectures with domain-specific word embeddings offer improved overall performance while being competitive even to transformer-based models. Finally, we show that cutting-edge multilingual and monolingual transformer-based models brawl on the top of the classifiers’ ranking, making us question the necessity of training monolingual transfer learning models as a rule of thumb. To the best of our knowledge, this is the first time the task of Greek legal text classification is considered in an open research project, while also Greek is a language with very limited NLP resources in general.

## introduction in recent years, there has been intensified activity in the adaptation of artificial intelligence technologies to the legal domain @xcite @xcite , in which legal practitioners are required to analyze and review an overwhelming amount of legal data, mostly being plain text documents. this process requires dedication and an extraordinary level of resources, both concerning human resources along with the use of automated techniques to sift rationally through data. however, more sophisticated automated techniques are able to assist legal experts in making obsolete many labour-intensive manual tasks. these techniques are mostly contained in the area of machine learning and natural language processing (nlp). with legal text processing being an emerging subarea of nlp, many relevant applications have been derived such as legal topic classification @xcite legal information extraction @xcite , legal entity recognition @xcite @xcite , court opinion generation and analysis @xcite , legal judgement prediction @xcite @xcite ) and many more. however, current legal nlp studies are mainly focused in english and chinese, with very limited resources being available in other languages. our work focuses on the task of multi-class legal topic classification, where the goal is to identify the relevant thematic topic that represents a document. in our case, thematic topics (categories) are available in a multi-level hierarchy from broader to more specialized ones. the main contributions of our work are listed below: considering that greek is a language with few nlp resources, we anticipate that our study will be a significant contribution for the greek nlp community. to enhance the available nlp resources and foster reproducible results, we make both our code and dataset publicly available. in a more recent and extended version of this study, @xcite evaluated a battery of lmtc methods ranging from rnn-based label-wise attention networks (lwans) to probabilistic label trees (plts) @xcite @xcite and transformerbased models (bert, roberta) @xcite on three english datasets: eurlex57k @xcite , mimic-iii @xcite and amazon13k @xcite . the experimental results show that plt-based methods outperform lwans, while transformer-based approaches surpass state-of-theart in two out of three datasets. furthermore, a new state-of-the-art method is introduced which combines bert and lwan, giving the best results overall. furthermore, the case of few and zero-shot learning is studied with new models that leverage the label hierarchy and yield better results. following literature, we examine traditional machine learning methods and the rnn-based methods used in @xcite and @xcite and investigate whether these methods can perform equally well in multi-class text classification as they do in the multi-label setting. furthermore, we examine several bert-based methods, including multi-lingual models @xcite that have not been studied to date in the context of legal nlp. most of the preceding efforts focus on the english language and limited studies and resources exist considering languages other than english and chinese. while recently there is a new wave of studies on nlp tasks focused on the greek language @xcite @xcite , there is limited work for legal nlp in greek @xcite . to the best of our knowledge, this is the first study on the task of greek legal text classification, where experiments range from traditional machine learning to transfer learning models. ## conclusions and future work we introduced greek legal code (glc), a new publicly available dataset consisting of 47k greek legislation resources. relying on this dataset, we experimented with several classifiers, ranging from traditional machine learning and recurrent models to state-of-the-art transfer learning models. through their performance evaluation, we realized that although traditional machine learning classifiers (e.g., svm-bow) set strong baselines for some of the considered tasks, they fall short against more sophisticated methods. in contrast, rnnbased methods relying on bigrus provide improved overall performance and were competitive to multilingual transformer-based architectures (m-bert, xlm-roberta). beyond doubt, monolingual transformer-based models (greek-bert and greek-legal-bert) lead to state-of-theart results, especially when they are pre-trained on in-domain corpora. nonetheless, more emphasis should be given to the qualitative and quantitative characteristics of the examined datasets. intricacies like class imbalance, data scarcity and diversity apparently need special handling. regarding our study, we noticed that few-shot and especially zero-shot classes need to be properly handled with appropriate methods, as standard classifiers are insufficient. as for the recent trend to develop novel monolingual bertbased models, results show that already established multilingual models are incredibly powerful even in monolingual tasks. while research is on-going and these models are continuously being improved, also taking into consideration the computational costs, it is quite challenging to motivate researchers into making an effort to train monolingual models for medium or small-sized languages; especially when multilingual models can perform equally well or occasionally, even better. in future work, we plan to investigate specialized methods with improved few-shot and zeroshot performance @xcite @xcite that leverage various data properties (e.g., label descriptors and label hierarchy). also, we intend to apply deep learning techniques that take into account the hierarchy of classes in datasets like glc, where there is an underlying taxonomy @xcite @xcite . furthermore, experimenting with similar datasets like that of eu legislation written in greek will allow us to confirm our current conclusions. for example in the future, we could consider cypriot legislation 16 to evaluate the out-of-domain generalization of models trained on greek legislation. finally, our long term goal is to support and encourage further research in nlp for the greek language by publishing novel datasets, introducing and experimenting with state-of-the-art methods.