Automatic Charge Identification from Facts: A Few Sentence-Level Charge Annotations is All You Need (2020)

Abstract: Automatic Charge Identification (ACI) is the task of identifying the relevant charges given the facts of a situation and the statutory laws that define these charges, and is a crucial aspect of the judicial process. Existing works focus on learning charge-side representations by modeling relationships between the charges, but not much effort has been made in improving fact-side representations. We observe that only a small fraction of sentences in the facts actually indicates the charges. We show that by using a very small subset (< 3%) of fact descriptions annotated with sentence-level charges, we can achieve an improvement across a range of different ACI models, as compared to modeling just the main document-level task on a much larger dataset. Additionally, we propose a novel model that utilizes sentence-level charge labels as an auxiliary task, coupled with the main task of document-level charge identification in a multi-task learning framework. The proposed model comprehensively outperforms a large number of recent baselines for ACI. The improvement in performance is particularly noticeable for the rare charges which are known to be especially challenging to identify.

## introduction in countries following a civil law system, there exist written statutory laws defining 'charges' (e.g., theft, murder, sexual offence), which one must consider before establishing a particular charge in a given situation. charge identification is a tedious and time-consuming task requiring legal expertise, since the written laws are often described abstractly to encompass the wide-ranging scenarios in which a charge can apply. thus automating the charge identification task can benefit the police and law practitioners. the task of automatic charge identification (aci, also referred to as 'charge prediction') aims to determine the possible charges that might have occurred in a given situation, using automated methods @xcite @xcite . a method for aci takes as input (i) a document containing a textual description of a situation, called 'fact-document' or 'facts', and, (ii) textual descriptions of the charges/charges, available as written laws (statutes) in the legal statues of a country. the method outputs the set of charges/charges committed in the given situation. most efforts in this task have been devoted towards distinguishing between rare and confusing charges, or learning better representations of the charge descriptions. for instance, @xcite designed 10 discriminative attributes to disambiguate confusing charge pairs, while @xcite and @xcite exploit pairwise co-occurrence and hierarchies of legal statutes respectively, to learn better representations for the statute descriptions. however, there has not been much effort in learning better fact-side representations. fact descriptions are usually long and contain a lot of background information -e.g., the relationship between the parties involved, past disputes, etc. -which do not play a significant role in establishing the charges. importantly, a lot of sentences in the fact descriptions are 'noisy' from the perspective of aci, i.e., do not indicate any charge at all (see the example in in the present work, we hypothesize that identifying charges at the sentence-level as an auxiliary task can help to better distinguish indicative sentences from noisy ones, and learn better fact-side representations for the main task of document-level charge identification. to this end, we choose a small fraction of the fact-documents and annotate each sentence with the charges the particular sentence possibly indicates, in consultation with legal experts. we show that such sentence-level annotations, even for a small fraction of fact-documents, can help various aci models to better identify charges for the facts. further, we propose a novel attention-based model with multi-task learning framework which, given the facts of a situation/case, tries to predict the relevant charges for each sentence and the overall situation (i.e., for the entire facts). our model constructs intermediate fact-side sentence representations which are used to match with the charge-side representations to predict sentence-level charges. we also propose a novel scheme of assigning weights to each sentence based on the sentence-level predictions. using these weights, we aggregate the sentence representations to obtain the fact-side document representation, used for document-level charge classification. we conduct experiments on the facts stated in indian supreme court judgment documents to identify charges out of a set of 20 most frequently occurring charges in the indian penal code. we compare the proposed model with several state-of-the-art aci baselines @xcite @xcite . our proposed model provides statistically significant improvements over all the baselines, especially for the rare charges. in brief, our contributions are as follows: (1) different from prior works on aci, we focus on learning better fact-side representations by annotating a small set of documents with sentence-level charges. ## related work early approaches: early attempts for identifying charges or supporting laws involved analyzing legal cases in specific scenarios using mathematical and statistical models @xcite @xcite , where hand-crafted rules were used to make the predicted results interpretable. with the advent of machine learning, researchers began to model aci as a text classification task. for instance, @xcite try to identify cue phrases to improve classification efficiency, while @xcite use n-gram bag-of-words vectors with linear support vector classifier. @xcite use machine learning approaches to label documents, classify the category of the case and predict the judgment. attentive deep learning models: the early methods use manually designed, shallow textual features, which require substantial human effort, as well as domain knowledge. the success of neural models in various legal nlp tasks @xcite has motivated the use of attention-based deep learning models for aci as well. for instance, @xcite dynamically generate representations for charge texts attentively, based on the fact representations. @xcite design 10 discriminative legal attributes, such as whether the charge is intentional, whether there is some physical injury, etc. these facts are encoded and attentively used to predict these attributes and charges as multi-task learning. for identifying the relevant law articles, @xcite propose dpam, which exploits the fact that some articles co-occur frequently due to high semantic similarity in their textual descriptions. whereas, @xcite use the tree-like hierarchies that exist among law articles to first identify the parent-laws and then the children-laws. better representations are learnt for rare children-laws by considering the semantics of the siblings. in one of the rare works on english, @xcite try out different methods, such as bi-gru with attention, hierarchical attention network and bert for identifying charges in judgment cases from the european court of human rights (echr). we consider all the above mentioned neural models as baselines in our work. modeling multiple legal tasks: recently, a few large legal datasets such as the cail judgment prediction dataset @xcite have become available which contains judgments from the supreme people's court of china, and annotations for three sub-tasks -predicting legal articles, charges and prison terms. some works have utilized correlations between these sub-tasks. for instance, @xcite note that an acyclic dependency exists between the sub-tasks, while @xcite introduce a multi-perspective forward prediction and backward verification framework to utilize result dependencies between the sub-tasks. @xcite use graph-based methods to group statutory articles into communities, and use distinguishable features from each community to attentively encode facts. since these methods are geared toward utilizing the correlations between the related sub-tasks (and need training data pertaining to all these tasks), we do not consider them as baselines, since our main focus is only charge identification. ## proposed model for automatic charge identification problem definition: the aci task takes as input (i) a textual description of the facts (of a situation), and (ii) textual descriptions of a set of charges. the fact description can be considered as a sequence of sentences @xmath0, x 2 , . . . , x n ], where each sentence is a sequence of words the set of charges is @xmath1, y 2 , . . . , y c ], and each charge has a corresponding charge description text, which can also be seen as a hierarchical sequence of sentences and words. the main task of documentlevel aci is to identify one or more charges from y , given x and y . the auxiliary task of sentence-level aci (that we consider in this work) is to identify zero or more charges given each x i and y . proposed model: figure 1 gives an overview of our proposed model. we first obtain a fact-side representation (which can be either a representation of the whole fact, or of each sentence in the fact). then we use this fact-side representation to dynamically generate the charge-side representation, and then pass these through a fully connected layer to obtain the final predictions. below, we describe each of these components. fact encoding layer: we use a modified hierarchical attention network (han) @xcite to encode facts, as shown in figure 1a . a han contains two bi-gru + attention sequence encoders: a sentence-level encoder that produces a sentence embedding from words, and a document-level encoder that produces a document embedding from sentences. thus, in our case, each fact sentence x i is encoded to return a single embedding s i for the entire sentence, using a trainable global context vector u f w to calculate attention weights @xcite . the fact text, now a sequence of sentence embeddings [s 1 , s 2 , . . . , s n ], is passed through the document-level encoder with a trainable global context vector u f s to obtain an embedding for the entire document d c . however, as discussed in section 1, a lot of sentences in the fact descriptions are noisy (i.e., do not indicate any charge), and should be assigned low weights while constructing the document embedding. to guide the model towards distinguishing the noisy sentences, we pass the embedding s i to the charge encoder and matching layer (detailed below) to obtain the sentence-level charge probabilities p @xmath2, p i2 , . . . , p ic ]. we then learn a weight w i for each sentence, from sentence-level predictions p i , and create another weighted document representation d p as charge encoding layer: another han is used to generate an embedding for each charge description. but, instead of using global context vectors to learn word and sentence attention weights, these weights are generated dynamically based on the given facts, similar to the approach used by @xcite . this helps the model to selectively attend to important sentences and words in the charge descriptions based on the given fact, and not important items in general. as seen in figure 1b , the input to this component is a fact-side representation f , which can be one of the sentence embeddings s i , or the final document embedding d. using f , we generate two context vectors u @xmath3, which are used for calculating attention weights for the two bi-gru + attn encoders. thereby, we generate charge-side document embedding d j for each charge y j . ## dataset preparation we collected a large number of judgment documents from the supreme court of india containing facts of real legal situations. in this section, we describe the manner in which we extract information and build the datasets from the raw judgment documents. ## experiments and results this section describes our experiments and results. we start by describing the baseline models with which our proposed model is compared.