Joint Entity and Relation Extraction for Legal Documents with Legal Feature Enhancement (2020)

Abstract: In recent years, the plentiful information contained in Chinese legal documents has attracted a great deal of attention because of the large-scale release of the judgment documents on China Judgments Online. It is in great need of enabling machines to understand the semantic information stored in the documents which are transcribed in the form of natural language. The technique of information extraction provides a way of mining the valuable information implied in the unstructured judgment documents. We propose a Legal Triplet Extraction System for drug-related criminal judgment documents. The system extracts the entities and the semantic relations jointly and benefits from the proposed legal lexicon feature and multi-task learning framework. Furthermore, we manually annotate a dataset for Named Entity Recognition and Relation Extraction in Chinese legal domain, which contributes to training supervised triplet extraction models and evaluating the model performance. Our experimental results show that the legal feature introduction and multi-task learning framework are feasible and effective for the Legal Triplet Extraction System. The F1 score of triplet extraction finally reaches 0.836 on the legal dataset.

## introduction automatic extraction of information from legal documents is crucial for legal document analysis and related business processing. the techniques of information extraction are pivotal modules for down-stream judicial applications such as the assistance of reviewing case documents, identification of criminal case facts and auxiliary generation of legal documents. besides, by implementing named entity recognition and relation extraction, the judgment documents can be transformed into several triplets, capturing entity pairs and their interrelations inside the fact description. the structured legal triplets conduce to legal knowledge graph construction, which benefits query capabilities and interpretability in judicial applications. there are great quantities of actual judgment documents released on china judgments online 1 . the abundant information contained in these judgment documents is worthy of in-depth study thanks to the authenticity and typicality of the documents. in this work, we conduct information extraction based on these public legal texts. recently, information extraction techniques have developed rapidly. neural networks for named entity recognition @xcite and deep learning methods applying to relation extraction @xcite are extensively investigated. early methods accomplish the triplet extraction drawing on the idea of pipelining. they first recognize the entities in the texts and then classify each entity pair into predefined relation types. these methods suffer from the error propagation problem caused by the incorrect and redundant entities attained by the previous step. besides, the interactions between the prediction of entities and relations also need to be emphasized. for these challenges, joint learning methods have been intensively carried out. initially, joint models based on neural networks @xcite apply parameter sharing mechanism to extract entities and relations in a single model. however, these models essentially treat entity recognition and relation extraction as two separated steps and do the prediction in a pipeline. the novel tagging scheme @xcite converts the joint extraction task into a sequence tagging problem and decodes the entities and relations all together. but it cannot solve the problem of overlapping triplets since it assigns each token only one tag. @xcite proposes a joint extraction method based on the sequence-to-sequence (seq2seq) model with copy mechanism, which handles the overlapping problem. recently some seq2seq-based methods @xcite improve the performance of joint entity and relation extraction on the accuracy as well as the computational efficiency. in this work, we concentrate on the triplet extraction in chinese legal domain, especially on the drugrelated criminal cases. there are twelve crime types of drug-related crimes, and three of them have the most cases, i.e. drug trafficking, illegal possession of drugs and providing venues for drug users. taking the representation of the facts of drug-related cases into account, we define four relation types, i.e. traffic in, sell drug to, possess and provide shelter for, which cover the crime of the most three drugrelated types according to criminal law of the people's republic of china. specifically, they differ in that traffic in denotes the fact that the suspect deals in drugs and sell drug to denotes that the suspect conducts a drug trade with someone else. it is obvious that these two relations generally share the same head entity in one case, which appears as the overlapping triplets. in consideration of the performance and the solution to overlapping problem, we propose a legal triplet extraction system based on the seq2seq model to accomplish joint entity and relation extraction from the drug-related criminal judgment documents. the system consists of three main components, encoder, decoder and sequence tagging layer. concretely, the encoder converts the source sentences into semantic vectors with legal feature enhancement. we explore the pre-trained language model bert for the encoder in light of its remarkable performance on multiple nlp tasks. the decoder is an efficient network able to solve the overlapping problem, inspired by previous work @xcite . the sequence tagging layer serves as an auxiliary task and assists the encoder in learning the entity boundary information. our main contributions can be summarized as follows: ## legal triplet extraction system in this section, we describe our method to extract the triplets occurring in the fact descriptions of judgment documents. we start with the introduction of the triplet extraction task in chinese legal domain and propose a manually annotated dataset for legal triplet extraction. we introduce each component of the proposed legal triplet extraction system and the training procedure in detail. the architecture of the triplet extraction system is shown as figure 1 . ## conclusion in this paper, we introduce a triplet extraction system to extract the triplets from the unstructured crime judgment documents. we explore the pre-trained language model for the system. the system extracts the entities and the semantic relations jointly with the assistance of legal feature enhancement. in addition, we manually annotate a dataset for information extraction in chinese legal domain, in order to train supervised models and evaluate the model performance. experiments show that the adoptions of legal feature enhancement and multi-task learning framework promote the performance of legal triplet extraction. for future work, we will explore more effective legal features for the legal triplet extraction system. information extraction on other crimes will be carried out as well.