# RnRTD: Intelligent Approach Based on the Relationship-Driven Neural Network and Restricted Tensor Decomposition for Multiple Accusation Judgment in Legal Cases

Authors: Xiaoding Guo; Hongli Zhang; Lin Ye; Shang Li
Journal: Computational Intelligence and Neuroscience
 (2019)

---

## Abstract

The use of intelligent judgment technology to assist in judgment is an inevitable trend in the development of judgment in contemporary social legal cases. Using big data and artificial intelligence technology to accurately determine multiple accusations involved in legal cases is an urgent problem to be solved in legal judgment. The key to solving these problems lies in two points, namely, (1) characterization of legal cases and (2) classification and prediction of legal case data. Traditional methods of entity characterization rely on feature extraction, which is often based on vocabulary and syntax information. Thus, traditional entity characterization often requires extensive energy and has poor generality, thus introducing a large amount of computation and limitation to subsequent classification algorithms. This study proposes an intelligent judgment approach called RnRTD, which is based on the relationship-driven recurrent neural network (rdRNN) and restricted tensor decomposition (RTD). We represent legal cases as tensors and propose an innovative RTD method. RTD has low dependence on vocabulary and syntax and extracts the feature structure that is most favorable for improving the accuracy of the subsequent classification algorithm. RTD maps the tensors, which represent legal cases, into a specific feature space and transforms the original tensor into a core tensor and its corresponding factor matrices. This study uses rdRNN to continuously update and optimize the constraints in RTD so that rdRNN can have the best legal case classification effect in the target feature space generated by RTD. Simultaneously, rdRNN sets up a new gate and a similar case list to represent the interaction between legal cases. In comparison with traditional feature extraction methods, our proposed RTD method is less expensive and more universal in the characterization of legal cases. Moreover, rdRNN with an RTD layer has a better effect than the recurrent neural network (RNN) only on the classification and prediction of multiple accusations in legal cases. Experiments show that compared with previous approaches, our method achieves higher accuracy in the classification and prediction of multiple accusations in legal cases, and our algorithm is more interpretable.


---

## Body

## 1. Introduction

In contemporary society, the demand for big data assistance in the judgment of legal cases, such as case intelligence research [1] and judgment [2], big data comprehensive supervision, and assistance in handling legal cases, is increasing with the development of big data and artificial intelligence technology. Researchers are committed to creating an “intelligent legal case judgment” project that combines big data and artificial intelligence. Legal case multiaccusation judgment business is an important part of the realization of such a project. Legal case multiaccusation judgment technology fully applies big data and artificial intelligence technology to service judgment making, legal case handling [3], and facilitation of the public. Big data provides judgments with recognized standards for judging legal cases and avoids the occurrence of different judgment results in similar legal cases. Artificial intelligence technology avoids the subjectivity of human beings, performs scientific and accurate analyses of cases from the perspective of cases and laws, and helps judges make objective judgment in legal cases.The solution to using big data and artificial intelligence technology to accurately judge multiple accusations in different legal cases involves two main points, namely, (1) construction of a comprehensive and accurate characterization method of legal cases and (2) realization of a classification and prediction algorithm for multiple accusations involved in a large number of legal cases. Figure1 shows the process of multiaccusation classification for legal cases. Traditional methods of entity characterization are often used to model an entity by tagging it. However, these feature extraction methods are highly dependent on the vocabulary and syntax in the entity data set and require heavy manpower and material resources. The generality of the tagged model is poor. In addition, feature extraction methods based on vocabulary and syntax require strong expert knowledge as support. The resulting entity characterization considerably limits the subsequent classification algorithm, and the algorithm’s accuracy becomes highly volatile.Figure 1
Multiaccusation classification for legal cases.This study proposes an intelligent legal case judgment technique called RnRTD, which is based on the relationship-driven recurrent neural network (rdRNN) and restricted tensor decomposition (RTD). Figure2 shows the framework of our approach. We present legal case data as tensor χ and propose an RTD technique. RTD is less dependent on vocabulary and syntax than traditional feature extraction methods, and it focuses more on extracting the information of potential structures in legal case tensors. RTD maximizes the accuracy of rdRNN by combining text and structural information. RTD maps legal case tensor χ into specified feature space ℤ, which decomposes the original tensor χ into core tensor χ˜ and its corresponding feature matrix set Ck under the restricted condition η in RTD. The obtained core tensor χ˜ represents the tensor structure information that is most helpful in improving the accuracy of the rdRNN classification algorithm. Core tensor χ˜ can be interpreted as the most advantageous feature structure in χ for rdRNN. RTD is an important feature extraction and dimensionality reduction operation. This study uses rdRNN to update and optimize restricted condition η in RTD iteratively so that its feature space ℤ continually approaches an ideal region, thus enabling rdRNN to achieve an optimal effect in the classification of multiple accusations in legal cases.Figure 2
Framework of the proposed RnRTD.Compared with traditional feature extraction methods, RTD obtains legal case characterization containing tensor element and structural information that is more conducive for improving the accuracy of the rdRNN classification algorithm, and it has lower dependence on vocabulary, syntax, and expert knowledge. That is, the RTD legal case characterization model has better universality and fewer requirements on the dataset format in comparison with traditional feature extraction methods. Compared with the direct use of the original legal case tensorχ as the input of the RNN classification algorithm, rdRNN with an RTD layer has a better effect on the classification of multiple accusations in legal cases. The main reason is that rdRNN constantly updates and optimizes RTD restricted condition η, thereby enabling RTD to point to feature space ℤ where rdRNN has the best effect in legal case classification.The main contributions of this study are summarized in the following points:(i)
This study uses a new method of characterizing legal cases. This study expresses a legal case as a tensor and proposes an RTD method that maps the original legal case tensor into a new feature space. RTD extracts the favorable tensor structure and text information for the subsequent classification algorithm from the original legal case tensor. RTD also extracts valuable tensor features and reduces tensor dimensions. The core tensor obtained by RTD is interpreted as the most valuable tensor structure and textual feature information extracted from the original legal case tensor for the rdRNN classification algorithm.(ii)
This study proposes rdRNN, which is a new approach for intelligent judgment of multiple accusations in legal cases. We add a new gate and a similar case list to control the interaction between tensors of legal cases on the basis of the original neural networks. rdRNN is particularly used for the intelligent judgment of multiple accusations in legal cases. It fully considers the impact of the relationship between legal cases on the judgment results of such cases. For example, highly similar legal cases are likely to have similar judgment results and vice versa.(iii)
This study proposes a neural network-based method for the optimization of the restricted tensor. The restricted tensor is a bridge between the RTD algorithm and rdRNN. rdRNN controls the tensor decomposition process by optimizing the restricted tensor, which guides the core tensor along the direction that is most conducive for improving the accuracy of the classification model. We derive the partial derivative of the loss function in rdRNN for the restricted tensor and realize the optimization operation of the neural network for the restricted tensor.Section2 gives the recent research progress on the classification of multiple crimes in legal cases. Section 3 introduces related definitions and the concepts involved in this study. Section 4 introduces the proposed approach for the judgment of multiple accusations in legal cases. Section 5 provides the experimental results and analysis of this study, and Section 6 presents a detailed discussion of the proposed method.


## 2. Related Work

With the advent of the era of big data and the development of artificial intelligence technology [4], the emergence of deep neural networks provides great prospects for accurate classification and prediction [5]. Neural network-based knowledge representation and reasoning methods enable deep learning approaches to be applied to many scenarios [6]. For the legal field, the combination of artificial intelligence and law has become an inevitable trend [7]. However, current research in this area mainly focuses on legal case modeling [8], legal case document retrieval [9], legal consultation question-and-answer systems [10], and legal case similarity reasoning work [2]. Little research has been conducted on the multiaccusation determination of cases in the legal field.Bartolini et al. proposed a semantic annotation method for indexing and retrieving legal texts [11]. The method uses a specific segment extraction and text classification algorithm to automatically semantically mark legal documents. Aleven developed a computational model based on artificial intelligence algorithms and professional legal knowledge [2]. The model determines the correlation between cases based on the context and problem scenarios of the case. Joshi et al. proposed a text mining method for electronic evidence review of legal cases [12]. The method uses semantic topic and text classification technology to repeatedly detect the feature vocabulary in legal documents and then automatically segments and screens the documents, avoiding the manual work of legal analysts.Sulea et al. proposed a legal case judgment system based on SVM classifier [13]. The method uses machine learning techniques to predict the legal field to which the legal case belongs and the outcome of its judgment. By accurately extracting the features of legal cases, the method can roughly predict the specific date of the case. Brninghaus and Ashley proposed a text classification method based on facts of legal cases [14]. The method uses artificial intelligence algorithms and legal background knowledge to predict the outcome of legal cases. The method extracts facts of legal cases, indexes and models them according to the features, and finally completes the classification of legal cases.The critical part for the prediction of legal case judgments is case modeling and case classification. Traditional text modeling methods are based on feature tags, which rely heavily on the syntax and semantic information of the source data. Labeling features requires a lot of manual work and expert knowledge. Therefore, the text classification algorithm formed on this basis is not scalable, and the accuracy is highly volatile.


## 3. Preliminaries

This section introduces the related methods, definitions, and background knowledge involved in this study. Section3.1 presents the basic notations and definitions. Section 3.2 provides a formal representation of the tensor decomposition problem. Section 3.3 introduces the calculation process of forward propagation in bidirectional long short-term memory (Bi-LSTM). Section 3.4 presents a formal description of the problem about intelligent legal judgment to be solved in this study.


### 3.1. Definitions and Notations

This section describes the relevant notations and definitions required in this work. Tensors are actually multidimensional matrices [15], which we represent in Euler script letters, such as χ and ν. We refer to the dimensions as tensor modes and to the number of a tensors modes as order. We describe the scalars in lowercase letters (such as a,b) and the vectors in boldface lowercase letters (such as c, d). We declare the matrices in capital letters, such as A and B. We use AT to represent the transpose of matrix A. We express the identity matrix as I, the identity tensor as τ, and the matrix with all elements of 1 as 1. Table 1 shows all the required notations and definitions.Table 1
Symbols.
Symbol
Definition
χ
,
X
,
x
,
x
Tensor, matrix, vector, scalar
A
T
Transpose of matrixA
χ
i
1
,
…
,
i
N
Thei1,…,iNth entry of χ, same for vectors and matrices
A
:,
i
Theith column of matrix A
∘
Outer product
∗
Elementwise multiplication
⊗
Kronecker product
⊙
Khatri–Rao product
×
n
Then-mode stretch
×
n
Then-mode product
χ
n
Then-mode matricization of tensor χ
χ
F
The Frobenius norm of tensorχDefinition 1 (outer product).
The outer product of vectorsa∈ℝI and b∈ℝJ is denoted as A=a∘b, where A∈ℝI×J and Ai,j=aibj.Definition 2 (elementwise multiplication).
The elementwise multiplication of vectorsa∈ℝI and b∈ℝI is denoted as A=a∗b, where A∈ℝI and Ai=aibi. In another case, the elementwise multiplication of vector a∈ℝI and matrix A∈ℝI×J is denoted as Z=a∗A, where Z∈ℝI×J and Zi,j=aiAi,j.Definition 3 (Kronecker product).
Given vectorsa∈ℝI and b∈ℝJ, their Kronecker product is denoted as S=a⊗b, where S∈ℝIJ and S=a1bT,a2bT,⋯,aIbTT. Given matrices A∈ℝI×J and B∈ℝP×Q, their Kronecker product is denoted as A⊗B.(1)A⊗B=A:,1⊗B:,1A:,1⊗B:,2⋯A:,J⊗B:,Q−1A:,J⊗B:,Q.Definition 4 (Khatri–Rao product).
Given matricesA∈ℝI×R and B∈ℝJ×R, their Khatri–Rao product is denoted as A⊙B, which is calculated by combining the Kronecker product of each corresponding column in A and B, that is(2)A⊙B=A:,1⊗B:,1⋯A:,r⊗B:,r⋯A:,R⊗B:,R.Definition 5 (n-mode matricization).
Given anN-mode tensor χ, χ∈ℝI1×I2×⋯×IN. χ can be matrixed into N forms according to each mode. We denote the n-mode matricization of χ as χn, where χn∈ℝIn×I1⋯In−1In+1⋯IN. χn is obtained by keeping the nth mode unchanged while expanding and concatenating the slices of the remaining modes into a matrix.Definition 6 (Frobenius norm of a tensor).
Given an N-mode tensorχ, χ∈ℝI1×I2×⋯×IN, the Frobenius norm of χ is denoted as(3)χF=∑i1I1∑i2I2⋯∑iNINχi1,i2,…,iN2.Definition 7 (n-mode stretch).
Given anN-mode tensor ν, ν∈ℝJ1×J2×⋯×JN, and a weight matrix W, W∈ℝJn×∏k≠nNJk. The n-mode stretch between ν and W is expressed as υ×nW=κ, where κ∈ℝJ1×J2×⋯×JN.(4)κj1,j2,…,jn,…,jN=Wj1,∏k≠nNjkυj1,j2,…,jn,…,jN.Definition 8 (n-mode product).
Given anN-mode tensor χ∈ℝI1×I2×⋯×IN and a matrix C∈ℝIn×J, their n-mode product is denoted as λ=χ×nC, λ∈ℝI1×⋯×In−1×J×In+1×⋯×IN.(5)λi1,…,in−1,j,in+1,…,iN=∑m=1Inχi1,…,in−1,m,in+1,…,iNCm,j.


### 3.2. Tensor Decomposition

Many tensor decomposition methods, such as PARAFAC and Tucker decomposition, are currently available [15]. As shown in Figure 3, tensor decomposition methods decompose the original tensor into a core tensor and a series of corresponding factor matrices. The essence of tensor decomposition is to approximate the original tensor by using the product of the core tensor and the factor matrices. The mathematical description of tensor decomposition is as follows:Figure 3
Tensor decomposition.Given anN-mode tensor χ, χ∈ℝI1×I2×⋯×IN. The following formula can be obtained by using the tensor decomposition method:(6)χ≈τ×1C1×2C2×⋯×NCN,where τ is the core tensor, τ∈ℝJ1×J2×⋯×JN, and Cn is the corresponding factor matrix set, Cn∈ℝJn×In. Each element in Cn is a column orthogonal matrix. τ and Cn also minimize function φ, where(7)φ=χ−τ∏nN×nCnF.


### 3.3. Bi-LSTM

RNNs have far-reaching implications for the study of sequence data [16]. The nodes between the hidden layers of RNN are connected [17], that is, the input of the hidden layer contains not only the output of the input layer but also the output of the hidden layer at the last moment. In theory, RNN can process sequence data of any length. However, gradient disappearance or gradient explosion often occurs when RNN deals with long-distance dependence, thereby making RNN training difficult. The hidden layers of the original RNN has only one kind of state, which is very sensitive to short-term inputs. Long short-term memory (LSTM) deals with long-distance dependence by increasing the long-term memory state in the original RNN [18].As shown in Figure4, we represent the input value of LSTM at time t as xt, the output value from the previous moment t−1 as ht−1, and the long-term unit state at time t−1 as ct−1. We record the unit status entered at time t as c˜t. The output value of LSTM at time t comprises two parts, namely, the output value of LSTM at current time ht and the unit state of current time ct. LSTM sets up three control gates, which are forget, input, and output, to control the long-term unit state c. The forget gate is used to determine how much of the long-term unit state at the previous moment is retained at the current moment. For example, the forget gate ft at time t determines the weight of ct−1 in the calculation of ct. The input gate is used to determine how much of the input of LSTM is retained in the current long-term unit state. For example, input gate it determines the weight that xt takes while calculating ct. The output gate is used to determine how much the long-term unit state at the current moment contributes to the output of LSTM at the current time. For example, output gate ot determines the influence of the value of ct on ht.Figure 4
Framework of LSTM.The process of forward propagation calculation in LSTM is described as follows:(8)ft=σwf⋅ht−1,xtT+bf,it=σwi⋅ht−1,xtT+bi,c˜t=tanhwc⋅ht−1,xtT+bc,ot=σwo⋅ht−1,xtT+bo.The long-term unit statect at current time t is calculated by ft, it, ct−1 and c˜t, and the final output of LSTM ht is calculated by ot and ct. That is,(9)ct=ft∗ct−1+it∗c˜t,ht=ot∗tanhct,where ht−1 is the output of LSTM at time t−1, xt is the input of LSTM at time t, σ is the sigmoid function, which is our selected activation function in LSTM, c˜t is the unit state input at time t, wf, wi, and wo are the weight matrices of the forget gate ft, the input gate it, and the output gate ot, respectively, and bf, bi, and bo are the bias matrices of ft, it, and ot, respectively. The activation function used in calculating c˜t is the hyperbolic tangent function, where wc is the weight matrix and bc is the bias term.Bi-LSTM is a bidirectional RNN [19]. The unit state of the hidden layer in Bi-LSTM is calculated from the outputs of forward and backward LSTM. We define the output unit state of Bi-LSTM at time t as hBi−LSTMt, the output unit state of forward LSTM as hft, and the output unit state of backward LSTM as hbt. The aforementioned forward propagation formula of LSTM implies that(10)hft=LSTM→hft−1,xt,hbt=LSTM←hbt+1,xt,hBi−LSTMt=hft,hbt.


### 3.4. Problem Description

Problem 1.
We express the legal case as a tensor and classify the legal case according to the judgment result. The category of each legal case is indicated by a scalar, such asr. Given a legal case dataset Ω that contains legal cases with judgment results, Ω=χ1,r1,χ2,r2,⋯,χN,rN. χn represents the nth legal case in the legal case dataset Ω. rn indicates the type of legal judgment result that corresponds to the nth legal case. Our goal is to train a case classification model ϕχ that can classify legal cases based on their judgment results.
In this study, legal cases are represented as three-dimensional tensors. As shown in Figure5, the first dimension represents the basic components of the case, such as the defendant’s statement, the plaintiff’s statement, the public prosecution, and the court’s trial. On this basis, the matrix slice that contains the last two dimensions represents the matrix form of the corresponding legal case component. The matrix slice is composed of the accumulation of word vectors inside the legal case component. Generally, case components are matrixed instead of including the word vectors of all the words in the matrix. We selectively extract words that are valuable for the legal case classification. These words can be divided into two categories. The first category usually includes nouns or pronouns, such as characters, times, places, and objects; the second category usually comprises adjectives, numerals, or verbs, such as the means of committing accusations, the degree of harm, and the number of accusations.Figure 5
Original tensor that represents the legal case.


## 3.1. Definitions and Notations

This section describes the relevant notations and definitions required in this work. Tensors are actually multidimensional matrices [15], which we represent in Euler script letters, such as χ and ν. We refer to the dimensions as tensor modes and to the number of a tensors modes as order. We describe the scalars in lowercase letters (such as a,b) and the vectors in boldface lowercase letters (such as c, d). We declare the matrices in capital letters, such as A and B. We use AT to represent the transpose of matrix A. We express the identity matrix as I, the identity tensor as τ, and the matrix with all elements of 1 as 1. Table 1 shows all the required notations and definitions.Table 1
Symbols.
Symbol
Definition
χ
,
X
,
x
,
x
Tensor, matrix, vector, scalar
A
T
Transpose of matrixA
χ
i
1
,
…
,
i
N
Thei1,…,iNth entry of χ, same for vectors and matrices
A
:,
i
Theith column of matrix A
∘
Outer product
∗
Elementwise multiplication
⊗
Kronecker product
⊙
Khatri–Rao product
×
n
Then-mode stretch
×
n
Then-mode product
χ
n
Then-mode matricization of tensor χ
χ
F
The Frobenius norm of tensorχDefinition 1 (outer product).
The outer product of vectorsa∈ℝI and b∈ℝJ is denoted as A=a∘b, where A∈ℝI×J and Ai,j=aibj.Definition 2 (elementwise multiplication).
The elementwise multiplication of vectorsa∈ℝI and b∈ℝI is denoted as A=a∗b, where A∈ℝI and Ai=aibi. In another case, the elementwise multiplication of vector a∈ℝI and matrix A∈ℝI×J is denoted as Z=a∗A, where Z∈ℝI×J and Zi,j=aiAi,j.Definition 3 (Kronecker product).
Given vectorsa∈ℝI and b∈ℝJ, their Kronecker product is denoted as S=a⊗b, where S∈ℝIJ and S=a1bT,a2bT,⋯,aIbTT. Given matrices A∈ℝI×J and B∈ℝP×Q, their Kronecker product is denoted as A⊗B.(1)A⊗B=A:,1⊗B:,1A:,1⊗B:,2⋯A:,J⊗B:,Q−1A:,J⊗B:,Q.Definition 4 (Khatri–Rao product).
Given matricesA∈ℝI×R and B∈ℝJ×R, their Khatri–Rao product is denoted as A⊙B, which is calculated by combining the Kronecker product of each corresponding column in A and B, that is(2)A⊙B=A:,1⊗B:,1⋯A:,r⊗B:,r⋯A:,R⊗B:,R.Definition 5 (n-mode matricization).
Given anN-mode tensor χ, χ∈ℝI1×I2×⋯×IN. χ can be matrixed into N forms according to each mode. We denote the n-mode matricization of χ as χn, where χn∈ℝIn×I1⋯In−1In+1⋯IN. χn is obtained by keeping the nth mode unchanged while expanding and concatenating the slices of the remaining modes into a matrix.Definition 6 (Frobenius norm of a tensor).
Given an N-mode tensorχ, χ∈ℝI1×I2×⋯×IN, the Frobenius norm of χ is denoted as(3)χF=∑i1I1∑i2I2⋯∑iNINχi1,i2,…,iN2.Definition 7 (n-mode stretch).
Given anN-mode tensor ν, ν∈ℝJ1×J2×⋯×JN, and a weight matrix W, W∈ℝJn×∏k≠nNJk. The n-mode stretch between ν and W is expressed as υ×nW=κ, where κ∈ℝJ1×J2×⋯×JN.(4)κj1,j2,…,jn,…,jN=Wj1,∏k≠nNjkυj1,j2,…,jn,…,jN.Definition 8 (n-mode product).
Given anN-mode tensor χ∈ℝI1×I2×⋯×IN and a matrix C∈ℝIn×J, their n-mode product is denoted as λ=χ×nC, λ∈ℝI1×⋯×In−1×J×In+1×⋯×IN.(5)λi1,…,in−1,j,in+1,…,iN=∑m=1Inχi1,…,in−1,m,in+1,…,iNCm,j.


## 3.2. Tensor Decomposition

Many tensor decomposition methods, such as PARAFAC and Tucker decomposition, are currently available [15]. As shown in Figure 3, tensor decomposition methods decompose the original tensor into a core tensor and a series of corresponding factor matrices. The essence of tensor decomposition is to approximate the original tensor by using the product of the core tensor and the factor matrices. The mathematical description of tensor decomposition is as follows:Figure 3
Tensor decomposition.Given anN-mode tensor χ, χ∈ℝI1×I2×⋯×IN. The following formula can be obtained by using the tensor decomposition method:(6)χ≈τ×1C1×2C2×⋯×NCN,where τ is the core tensor, τ∈ℝJ1×J2×⋯×JN, and Cn is the corresponding factor matrix set, Cn∈ℝJn×In. Each element in Cn is a column orthogonal matrix. τ and Cn also minimize function φ, where(7)φ=χ−τ∏nN×nCnF.


## 3.3. Bi-LSTM

RNNs have far-reaching implications for the study of sequence data [16]. The nodes between the hidden layers of RNN are connected [17], that is, the input of the hidden layer contains not only the output of the input layer but also the output of the hidden layer at the last moment. In theory, RNN can process sequence data of any length. However, gradient disappearance or gradient explosion often occurs when RNN deals with long-distance dependence, thereby making RNN training difficult. The hidden layers of the original RNN has only one kind of state, which is very sensitive to short-term inputs. Long short-term memory (LSTM) deals with long-distance dependence by increasing the long-term memory state in the original RNN [18].As shown in Figure4, we represent the input value of LSTM at time t as xt, the output value from the previous moment t−1 as ht−1, and the long-term unit state at time t−1 as ct−1. We record the unit status entered at time t as c˜t. The output value of LSTM at time t comprises two parts, namely, the output value of LSTM at current time ht and the unit state of current time ct. LSTM sets up three control gates, which are forget, input, and output, to control the long-term unit state c. The forget gate is used to determine how much of the long-term unit state at the previous moment is retained at the current moment. For example, the forget gate ft at time t determines the weight of ct−1 in the calculation of ct. The input gate is used to determine how much of the input of LSTM is retained in the current long-term unit state. For example, input gate it determines the weight that xt takes while calculating ct. The output gate is used to determine how much the long-term unit state at the current moment contributes to the output of LSTM at the current time. For example, output gate ot determines the influence of the value of ct on ht.Figure 4
Framework of LSTM.The process of forward propagation calculation in LSTM is described as follows:(8)ft=σwf⋅ht−1,xtT+bf,it=σwi⋅ht−1,xtT+bi,c˜t=tanhwc⋅ht−1,xtT+bc,ot=σwo⋅ht−1,xtT+bo.The long-term unit statect at current time t is calculated by ft, it, ct−1 and c˜t, and the final output of LSTM ht is calculated by ot and ct. That is,(9)ct=ft∗ct−1+it∗c˜t,ht=ot∗tanhct,where ht−1 is the output of LSTM at time t−1, xt is the input of LSTM at time t, σ is the sigmoid function, which is our selected activation function in LSTM, c˜t is the unit state input at time t, wf, wi, and wo are the weight matrices of the forget gate ft, the input gate it, and the output gate ot, respectively, and bf, bi, and bo are the bias matrices of ft, it, and ot, respectively. The activation function used in calculating c˜t is the hyperbolic tangent function, where wc is the weight matrix and bc is the bias term.Bi-LSTM is a bidirectional RNN [19]. The unit state of the hidden layer in Bi-LSTM is calculated from the outputs of forward and backward LSTM. We define the output unit state of Bi-LSTM at time t as hBi−LSTMt, the output unit state of forward LSTM as hft, and the output unit state of backward LSTM as hbt. The aforementioned forward propagation formula of LSTM implies that(10)hft=LSTM→hft−1,xt,hbt=LSTM←hbt+1,xt,hBi−LSTMt=hft,hbt.


## 3.4. Problem Description

Problem 1.
We express the legal case as a tensor and classify the legal case according to the judgment result. The category of each legal case is indicated by a scalar, such asr. Given a legal case dataset Ω that contains legal cases with judgment results, Ω=χ1,r1,χ2,r2,⋯,χN,rN. χn represents the nth legal case in the legal case dataset Ω. rn indicates the type of legal judgment result that corresponds to the nth legal case. Our goal is to train a case classification model ϕχ that can classify legal cases based on their judgment results.
In this study, legal cases are represented as three-dimensional tensors. As shown in Figure5, the first dimension represents the basic components of the case, such as the defendant’s statement, the plaintiff’s statement, the public prosecution, and the court’s trial. On this basis, the matrix slice that contains the last two dimensions represents the matrix form of the corresponding legal case component. The matrix slice is composed of the accumulation of word vectors inside the legal case component. Generally, case components are matrixed instead of including the word vectors of all the words in the matrix. We selectively extract words that are valuable for the legal case classification. These words can be divided into two categories. The first category usually includes nouns or pronouns, such as characters, times, places, and objects; the second category usually comprises adjectives, numerals, or verbs, such as the means of committing accusations, the degree of harm, and the number of accusations.Figure 5
Original tensor that represents the legal case.


## 4. Our Approach

This study proposes RnRTD for the multiaccusation determination of legal cases. Figure6 shows the RnRTD framework. First, we extract core tensors from the original tensors using the RTD method. The core tensor approximates the restricted tensor in terms of the tensor structure and elements. Second, we use rdRNN to optimize the restricted tensor so that it guides the core tensor along the direction that is most conducive for improving the accuracy of the classification model.Figure 6
Framework of the RnRTD method.


### 4.1. RTD Method

This study proposes a new tensor decomposition method called RTD method. The inputs of the RTD algorithm include the restricted condition tensorη and tensor χ that represents the legal case. The RTD outputs include core tensor χ˜ and its corresponding factor matrix sets, namely, Ck and Dk. RTD decomposes χ into a core tensor χ˜ under the action of the restricted condition η. χ˜ is approximated to η in terms of tensor structure and internal element values. RTD can be interpreted as a mapping of the original tensor χ to the core tensor χ˜. In short, RTD achieves directional decomposition of tensors and extracts vital information from tensors while reducing their dimension. In this study, we define core tensor χ˜ as the most favorable tensor structure and element value information for the subsequent legal case classification algorithm, namely, rdRNN. On this basis, we construct a deep neural network model for RnRTD that is dedicated to legal intelligence judgments.RTD decomposes the original tensor under the restricted condition so that the obtained core tensor constantly approaches the restricted tensor in terms of tensor structure and element value. In Figure7, the formal description of the problems to be solved by the RTD algorithm is shown as Problems 1 and 2.Figure 7
Framework of the RTD method.Problem 2.
Given tensorχ∈ℝI1×I2×⋯×IK, restricted tensor η∈ℝJ1×J2×⋯×JK, and its weight wη, we derive two factor matrix sets, namely, Ck and Dk, Ck∈ℝIk×Hk, Dk∈ℝJk×Hk, that Ck and Dk minimize the following function:(11)ϕ=χ∏kK×kCk−Wη×nη∏kK×kDkF2.
MatrixWη is preset according to the legal case, Wη∈ℝHn×∏k≠nKHk. The elements in sets Ck and Dk are orthogonal matrices, that is, they meet the following conditions. For any elements Ck and Dk in sets Ck and Dk,(12)CkCkT=I,CkTCk=I,DkDkT=I,DkTDk=I.
In this study, we use the alternating least squares (ALS) algorithm to determine the solution of the objective functionϕ. The ALS algorithm can be divided into four steps: (1) randomly pick a variable as a parameter and randomly generate the values of other variables, (2) determine the partial derivative of the loss function ϕ in the specified parameter while fixing the values of other variables, (3) set the partial derivative of ϕ to the specified parameter as zero and calculate the value of the specified parameter, and (4) select another variable as a parameter and return to Step (2). The ALS algorithm continues to iterate Steps (2), (3), and (4) until the error of the loss function ϕ reaches the tolerable upper limit.
Problem 2 needs to be solved using Lemma 1. The specific definition and proof of Lemma 1 are provided as follows.Lemma 1.
Given functiontrαTα=∑i1I1∑i2I2⋯∑inINαi1,i2,…,in2, α∈ℝI1×I2×⋯×IN, α can be a vector, matrix, and tensor. The target function φ=trχ∏kK×kCkTχ∏kK×kCk, where χ∈ℝI1×I2×⋯×IK, Ck∈ℝIk×Hk, and Ck satisfy equation (12). For any element Ck0 in Ck, k0∈1,K, the partial differential of function φ to Ck0 is ∂φ/∂Ck0=εχ∏k≠k0K×kCkTχ∏kK×kCk, where ε is a constant.
The proof of Lemma 1 is shown in Proof 1.Proof 1.
We useμ to represent χ∏k≠k0K×kCk; it can can be derived that φ=trμ×k0Ck0Tμ×k0Ck0. We abbreviate the target function φ=trμCk0TμCk0=trCk0TμTμCk0. We use ν to represent μTμ, and we can get that φ=trCk0TνCk0. According to the function derivation rule, we can obtain the following equation: ∂φ/∂Ck0=∂trCk0TνCk0/∂Ck0=νCk0+νTCk0. Since ν=νT, the calculation formula for the partial derivative of the function φ to Ck0 is(13)∂φ∂Ck0=2νCk0=εχ∏k≠k0K×kCkTχ∏kK×kCk,where ε is a constant, ε=2.
According to the iterative process of the ALS algorithm, the precondition for solving the value ofCk and Dk which minimize the function ϕ in equation (11) using the ALS algorithm is to calculate the value of ∂ϕ/∂Ck0, where k0∈1,K. Equation (11) shows that ∂ϕ/∂Ck0 and ∂ϕ/∂Dk0 are solved in the same manner. Proof 2 provides mathematical proof of the calculation of ∂ϕ/∂Ck0.Proof 2.
We useλ and ϖ to represent χ∏k≠k0K×kCk and wηη∏kK×kDk, respectively. According to formula (11), we can obtain that ϕ=trλ×k0Ck0−ϖTλ×k0Ck0−ϖ. We abbreviate the aforementioned formula as ϕ=trλCk0−ϖTλCk0−ϖ. Then, we can determine the following formula: ∂ϕ/∂Ck0=∂trϖTϖ−2∂trCk0TλTϖ+∂trCk0TλTλCk0/∂Ck0. According to the function derivation rule, we derive that ∂trϖTϖ/∂Ck0=0, ∂trCk0TλTϖ/∂Ck0=λTϖ. In combination with Lemma 1, we can obtain that ∂trCk0TλTλCk0/∂Ck0=2λTλCk0. Finally, the following formula is determined:(14)∂ϕ∂Ck0=−2λTϖ+2λTλCk0.
We set the value of equation (14) to 0 and obtain that(15)λCk0=ϖ.
LetZ=λk0Tϖk0, then Ck0TZ=λCk0k0Tϖk0=ϖk0Tϖk0. By combining Equation (12), we derive that(16)Ck0TZ=ZTCk0,Z=Ck0ZTCk0.
We use the SVD matrix decomposition method to decomposeZ, and find that Z=PSQT. P and Q are orthogonal matrices, P is a left singular matrix, Q is a right singular matrix, and S is a diagonal matrix. After this analysis, the following solution can be obtained:(17)Ck0=PQT.
In summary, according to equations (11)–(17), we can derive a solution of ∂ϕ/∂Ck0 and Ck0 which are described in equations (14) and (17), respectively. ∂ϕ/∂Dk is calculated in the same manner as ∂ϕ/∂Ck. On this basis, we calculate the value of Ck and Dk , which minimize the objective function ϕ in formula (11), by using the ALS algorithm.
Algorithm1 shows the solution of Problem 2 by using ALS algorithm. The inputs of Algorithm 1 are χ which represents one legal case, the restricted tensor η, and its weight wη. In line 2, we randomly initialize the values of Ck, Uk. max_iterations in line 2 represents the maximum number of iterations of ALS algorithm. The function calcu_Z in line 4 corresponds to equation (16). Line 5 and 6 show the calculation process of equation (17).
Another problem to be solved by the RTD algorithm is Problem 3, which is the formal description of the process of tensor decomposition on the original tensor under the action of the restricted tensor and its weight. On the basis of Problem 2, we can obtain factor matrix setsCk and Dk, which minimize the value of function ϕ in formula (11) while satisfying formula (12).Algorithm 1: The solution of Problem 2 by using ALS Algorithm.
Input: Tensorχ which represents the original legal case data, χ∈ℝI1×I2×⋯×IK, the restricted tensor η, η∈ℝJ1×J2×⋯×JK, and its weight wη.
Output: The factor matrix setsCk and Dk, Ck∈ℝIk×Hk, Dk∈ℝJk×Hk.
Initialize the factor matrix setsCk and Dk;
for i=1 to max_iterations do
// First, pick the elements in the factor setCk as variables
for k0=1 to Kdo
Z=calcu_Zχ,η,Ck,Dk,wη,k0;
P,S,QT=SVDZ;
Ck0=PQT;
end
// Then, pick the elements in the factor setDk as variables
for k0=1 to Kdo
Z=calcu_Zχ,η,Dk,Ck,wη,k0;
P,S,QT=SVDZ;
Dk0=PQT;
end
end
returnCk, Dk;Problem 3.
Given a tensorχ∈ℝI1×I2×⋯×IK and factor matrix sets Ck and Dk, Ck∈ℝIk×Hk, Dk∈ℝJk×Hk, Ck and Dk are derived from Problem 2. A core tensor χ˜ is determined, where χ˜∈ℝJ1×J2×⋯×JK and χ˜ minimize the following target function:(18)FRTD=χ∏kK×kCk−χ˜∏kK×kDkF2.
Problem 3 needs to be solved using Lemma 2. The specific definition and proof process of Lemma 2 are as follows.Lemma 2.
Given the target functionψ=trχ˜∏kK×kDk∏kK×kDkTχ˜T, χ˜∈ℝJ1×J2×⋯×JK, Dk∈ℝJk×Hk, each element in Dk satisfies formula (12). Then, the partial derivative of the target function ψ to Dk0 where k0∈1,K is ∂ψ/∂Dk0, ∂ψ/∂Dk0=εχ˜∏kK×kDk∏kK×kDkT, where ε is a constant.Proof 3.
We useκ to represent τ∏kK×kDk; τ is the identity tensor, τ∈ℝJ1×J2×⋯JK. Then, the target function ψ can be rewritten as ψ=trχ˜κκTχ˜T. We use ρ to represent κκT. We can obtain that ∂ϕ/∂χ˜=∂trχ˜κκTχ˜T/∂χ˜=∂trχ˜ρχ˜T/∂χ˜. From the function derivation rule, we can get the following formula: ∂trχ˜κκTχ˜T/∂χ˜=∂trχ˜ρχ˜T/∂χ˜=ρχ˜T+ρTχ˜TT; thus,(19)∂ϕ∂χ˜=2χ˜κκT=εχ˜∏kK×kDk∏kK×kDkT,where ε is a constant, ε=2.
After the aforementioned analysis, Proof 4 gives the solution to Problem 3 and its mathematical proof process while combining Lemma 2 and Proof 3.Proof 4.
We useυ to represent χ∏kK×kCk and γ to represent τ∏kK×kDk, where τ is the identity tensor, τ∈ℝJ1×J2×⋯JK. Then, the function FRTD can be rewritten as FRTD=υ−χ˜γF2, that is FRTD=trυ−χ˜γTυ−χ˜γ. Known by the definition of function tr, trγTχ˜Tυ=trχ˜TυγT. Then, we can get the following equation: ∂FRTD/∂χ˜=∂trυTυ−2∂trχ˜TυγT+∂trχ˜γγTχ˜T/∂χ˜. It can be derived from the function derivation rule and Lemma 2 that(20)∂FRTD∂χ˜=εχ˜γγT−υγT,where ε is a constant, ε=2. Let ∂FRTD/∂χ˜ be zero. We can get the final solution of Problem 3 by combining formula (12).(21)χ˜=χ∏kK×kCk∏kK×kDkT.
Algorithm2 implements RTD by using Algorithm 1. Function TSPA in line 1 represents the implementation of Algorithm 1, and the inputs are χ, η, and wη. Function F_RTD in line 2 shows the calculation of χ˜ using equations (18)–(21). Finally, the core tensor of χ is obtained by using Algorithm 2, which approximates the restricted tensor η on the layer of tensor structure and elements information.Algorithm 2: The restricted tensor decomposition method.
Input: Tensorχ which represents the original legal case data, χ∈ℝI1×I2×⋯×IK, the restricted tensor η, η∈ℝJ1×J2×⋯×JK, and its weight wη.
Output: The core tensorχ˜ which is close to the restricted tensor η in the layer of tensor structure and elements value, χ˜∈ℝJ1×J2×⋯×JK.
// Solving the factor matrix setsCk and Dk using Algorithm 1
C
k, Dk=TSPAχ,η,wη;
χ
˜
=
F_RTD
χ
,
C
k
,
D
k;
returnχ˜;


### 4.2. rdRNN

This study proposes a new RNN called rdRNN. Unlike traditional RNN, rdRNN sets up a new gate based on the bidirectional RNN. The new gate uses the similarity matrix between samples as a parameter of the deep neural network training model. Compared with the original bidirectional RNN, the classification result of rdRNN is more accurate and stable. For the intelligent judgment of legal cases, the original deep neural network method does not consider the correlation between legal cases. This disregard may lead to bias in the final case classification. For example, the verdict of a legal case is inconsistent with the description of the case. To solve this problem, rdRNN fully considers the judgment results of legal cases that are similar to the case to be judged. rdRNN uses these results as a parameter of the deep neural network training model and realizes an efficient and accurate classification of multiple accusations in legal cases.The following section shows the training of rdRNN’s deep neural network while using Rmsprop as its optimization function:(i)
We use the datasetχ˜n,Ln and the restricted tensor η as inputs of rdRNN. χ˜n is the core tensor of χn, which represents the legal case. χ˜n is obtained by the RTD algorithm with χn and η as its inputs. Ln represents the category label of χn according to the judgment result of legal case.(ii)
In this study, we combine rdRNN with the softmax layer to complete the classification of legal cases. For sampleχn, assuming hn is the output vector of rdRNN, the softmax layer implements the mapping of hn to the legal case category Ln.(iii)
We use cross entropycroen_F as the loss function to update rdRNN. rdRNN uses its forward propagation algorithm and error backpropagation formulas to iterate over the values of parameters in neural networks, such as weight matrices wd and bias terms bd that are associated with relationship gate, and restricted tensor η, where d is the number of hidden layers.(iv)
We select Adam as the optimization function of rdRNN, and Rmsprop completes the optimization and calculation of parameterswd, bd, and η by using ∂crosen_F/∂wd, ∂crosen_F/∂bd, and ∂crosen_F/∂η.


#### 4.2.1. Calculation of Forward Propagation in rdRNN

In this study, we fully consider the relationship between legal cases and set up a new gate to complete the classification of legal cases, eliminate contingency errors as much as possible, and avoid inconsistencies between the predicted judgment result and the actual case. Relationship control gatert is used to control the similar relationship between legal cases. rt helps the rdRNN deep neural network make an intelligent judgment by using the judgment results of cases that are similar to the case to be judged.rdRNN can be divided into forward and backward LSTM. These networks do not have obvious differences, except for the opposite propagation direction. In the case of rdRNN forward LSTM propagation network, the formal description of relationship control gatert is as follows:(22)rt=σwrht−1xt+br,where wr and br are the weight matrix and bias term of the relational control gate rt, respectively, σ is the activation function, i.e., the sigmoid function, ht−1 is the output unit state of the neuron at time t−1, and xt is the input value of the neuron at time t.In the forward LSTM network, the output of each neuron at timet is calculated by the following formula:(23)rtftitotc˜t=σnetr,tσnetf,tσneti,tσneto,ttanhnetc˜,t=σσσσtanhwrh,wrxwfh,wfxwih,wixwoh,woxwch,wcx⋅ht−1xt+brbfbibobc,where rt, ft, it, and ot are the relational control, forget, input, and output gates, respectively; c˜t is the unit status of current inputs; netr,t, netf,t, neti,t, and neto,t are the weighted inputs of their corresponding gates at time t; netc,t is the weighted inputs of input state generation function tanh; σ is the activation function, i.e., the sigmoid function, σx=1/1+e−x; tanh is the hyperbolic tangent function, tanhx=ex−e−x/ex+e−x; wr is the weight matrix of relationship control gate rt, wr=wrh,wrx; wf, wi, and wo are expressed in the same manner as wr; and br, bf, bi, and bo are the bias terms of their corresponding activation functions.Subsequently, the unit state of the current momentct is calculated by ft, ct−1, it, and c˜t. The calculation formula is expressed as follows:(24)ct=ft∗ct−1+it∗c˜t.The final output of the forward LSTM neural network at timet is calculated by ot, ct, and rt and the similar list of x. It is described as follows:(25)ht=ot∗tanhct+rt∗1Listx∑x0∈ListxSimx0,xhx0,where Listx is composed of legal cases where the similarity with x is greater than a threshold Max_sim so far. hx0 refers to the output of the forward LSTM neural network that corresponds to legal case x0. Sim⋅ is a function that calculates the similarity between legal cases. In this study, we set function Sim as the weight of the Euclidean distance and the cosine distance between legal cases.(26)Simx,x0=DEuclideanx,x0+wdDcosinex,x0,where DEuclidean and Dcosine refer to the Euclidean distance and the cosine distance between the vectors x and x0, respectively. wd is the weight matrix.


#### 4.2.2. Calculation of Backpropagation in rdRNN

In this section, we describe in detail the backpropagation algorithm of the rdRNN neural network, including the backpropagation of the error along time and the hidden layer. In rdRNN, forward and backward LSTM neural networks have the same principle in the backpropagation algorithm. Therefore, this section mainly uses forward LSTM as an example.Given the error term at timet δt, δt=∂crosen_F/∂ht. Calculation of the backpropagation algorithm of the error term along time is to calculate the value of δt−1=∂crosen_F/∂ht−1. The full derivative formula shows that(27)δt−1=∂crosen_F∂ht∂ht∂ht−1.Equations (23)–(25) show that rt, ft, it, ot, and c˜t are all functions of ht−1. Then, we can obtain(28)δo,t=δt∂ht∂ot∂ot∂neto,t,δf,t=δt∂ht∂ct∂ct∂ft∂ft∂netf,t,δi,t=δt∂ht∂ct∂ct∂it∂it∂neti,t,δc˜,t=δt∂ht∂ct∂ct∂c˜t∂c˜t∂netc˜,t,δr,t=δt∂ht∂rt∂rt∂netr,t,∂neto,t∂ht−1=woh,∂netf,t∂ht−1=wfh,∂neti,t∂ht−1=wih,∂netc˜,t∂ht−1=wch,∂netr,t∂ht−1=wrh.The formula on the left represents the variable declaration, and the formula on the right is calculated from equation (23). According to equations (27) and (28), we can further derive the following formula:(29)δt−1=δo,twoh+δf,twfh+δi,twih+δc˜,twch+δr,twrh,where δo,t=δt∗tanhct∗ot∗1−ot, δf,t=δt∗ot∗1−tanhct2∗ct−1∗ft∗1−ft, δi,t=δt∗ot∗1−tanhct2∗c˜t∗it∗1−it, δc˜,t=δt∗ot∗1−tanhct2∗it∗1−c˜t2. According to relationship control gate rt, equations (23) and (25), we determine that(30)δr,t=δt∗1Listx∑x0∈ListxSimx0,xhx0∗rt∗1−rt.From equation (29), we can finally figure out the calculation method of the error term in rdRNN is passed from the current moment t to any time k.Then, we describe in detail the transmission of error between the hidden layers of rdRNN. The error term of thelth hidden layer in rdRNN is assumed to be the partial derivative of the error function crosen_F versus the weighted input netl. In rdRNN, the input of the lth hidden layer at time t is xtl.(31)xtl=activeFl−1nettl−1,where activeFl−1 denotes the activation function of the l−1th hidden layer in rdRNN and nettl−1 denotes the weighted input of the l−1th hidden layer at time t.Given the error term of the lth hidden layer at time t δtl, δtl=∂crosen_F/∂nettl, the calculation of error propagation between hidden layers is to figure out the value of δtl−1, where(32)δtl−1=∂crosen_F∂nettl−1.According to equations (23), (31), and (32), netr,tl, netf,tl, neti,tl, neto,tl, and netc˜,tl are all functions of xtl, and xtl is a function of nettl−1. Therefore, the full derivative formula shows that(33)δtl−1=∂crosen_F∂netr,tl∂netr,tl∂xtl∂xtl∂nettl−1+∂crosen_F∂netf,tl∂netf,tl∂xtl∂xtl∂nettl−1+∂crosen_F∂neti,tl∂neti,tl∂xtl∂xtl∂nettl−1+∂crosen_F∂neto,tl∂neto,tl∂xtl∂xtl∂nettl−1+∂crosen_F∂netc˜,tl∂netc˜,tl∂xtl∂xtl∂nettl−1.The following formula can be obtained by further calculation:(34)δtl−1=δr,tlwrx+δf,tlwfx+δi,tlwix+δo,tlwox+δc˜,tlwcx∗graactiveFl−1nettl−1,where graactiveFl−1nettl−1 is the derivative of function activeFl−1 at nettl−1.According to equations (27)–(34), we can derive the partial derivative of the loss function crosen_F to the weight matrix set wd and the bias term set bd in rdRNN. Given that ∂crosen_F/∂wrh=∑j=1t∂crosen_F/∂wrh,j=∑j=1t∂crosen_F/∂netr,j∂netr,j/∂wrh,j, we obtain(35)∂crosen_F∂wrx=∑j=1tδr,jhj−1,∂crosen_F∂wfx=∑j=1tδf,jhj−1,∂crosen_F∂wix=∑j=1tδi,jhj−1,∂crosen_F∂wox=∑j=1tδo,jhj−1,∂crosen_F∂wcx=∑j=1tδc˜,jhj−1,∂crosen_F∂br=∑j=1tδr,j,∂crosen_F∂bf=∑j=1tδf,j,∂crosen_F∂bi=∑j=1tδi,j,∂crosen_F∂bo=∑j=1tδo,j,∂crosen_F∂bc=∑j=1tδc˜,j,∂crosen_F∂wrx=δr,jxt,∂crosen_F∂wfx=δf,jxt,∂crosen_F∂wix=δi,jxt,∂crosen_F∂wox=δo,jxt,∂crosen_F∂wcx=δc˜,jxt.


#### 4.2.3. Calculation of the Partial Derivative of Loss Functioncrosen_F to Restricted Tensor η

This study proposes a new intelligent method for judging legal cases called RnRTD, which combines rdRNN and RTD to complete the classification of legal cases. In the process of training the RnRTD neural network, a new problem is involved: updating of the value of the restricted tensorη so that it can continuously approximate the tensor value that is most beneficial for improving the classification accuracy of the RnRTD algorithm.The crux to solving this problem is to calculate the partial derivative of the loss functioncrosen_F to the restricted tensor η, that is, ∂crosen_F/∂η. Directly solving the value of ∂crosen_F/∂η is difficult. We can use the full derivative rule to obtain that(36)∂crosen_F∂η=∑nN∂crosen_F∂χ˜n∂χ˜n∂η.The backward propagation formula of rdRNN shows that∂crosen_F/∂χ˜n=∂crosen_F/∂net1∂net1/∂χ˜n=δ1w0. According to equations (19)–(21), we determine that ∂χ˜n/∂η=∂χn∏kK×kCkn∏k1K×k1Uk1nT/∂η. From equations (14)–(17), we know that Ckn and Dkn are all functions of η. Therefore, the function full derivative rule shows that ∂χ˜n/∂η=∑kK∂χ˜n/∂Ckn∂Ckn/∂η+∑kK∂χ˜n/∂DknT∂DknT/∂η, that is(37)∂χ˜n∂η=∑kKcalCCkn+∑kKcalDDknT,while(38)calCCkn=χn∏k1≠kK×k1Ck1n×k∂Ckn∂η∏k2K×k2DK2nT,calDDknT=χn∏k1K×k1Ck1n×k∂DknT∂η∏k2≠kK×k2DK2nT.Algorithm3 provides the optimization process of RnRTD proposed in this study. max_iter in line 2 represents the total number of training sessions of RnRTD. batch_size in line 6 represents the number of samples per batch while training RnRTD. The RTD tensor decomposition method in line 7 corresponds to Algorithm 2. rdRNN_forwardprop in line 8 and rdRNN_backprop in line 9 represent forward propagation and error backpropagation algorithms for rdRNN, respectively, which are the implementations of Sections 4.2.1–4.2.3. In line 11, we use the Adam algorithm to realize parameter optimization of RnRTD neural network.Algorithm 3: The framework of RnRTD algorithm.
Input:χn,Ln, where χn represents the legal cases and Ln represents the category of legal case corresponding to χn according to judgment results. The size of η, wr, wf, wi, wo, wc, br, bf, bi, bo and bc, where wr=wrh,wrx, wf=wfh,wfx, wi=wih,wix, wo=woh,wox and wc=wch,wcx
Output: The optimal restricted tensorη, parameters of rdRNN wr, wf, wi, wo, wc and br, bf, bi, bo, bc.
Initialize the restricted tensorη, parameters of rdRNN wr, wf, wi, wo, wc and br, bf, bi, bo, bc
for iter=1 to max_iter do
Setbatch_indices to zero;
while batch_indices≤dataset_size do
Set∂crosen_F/∂η, ∂crosen_F/∂wr, ∂crosen_F/∂wf, ∂crosen_F/∂wi, ∂crosen_F/∂wo, ∂crosen_F/∂wc and ∂crosen_F/∂br, ∂crosen_F/∂bf, ∂crosen_F/∂bi, ∂crosen_F/∂bo, ∂crosen_F/∂bc to zero;
for i=batch_indices to batch_indices+batch_size do
χ˜i=RTDχi,η;
L1i=rdRNN_forwardpropχ˜i,wr,wf,wi,wo,wc,br,bf,bi,bo,bc;
∂crosen_F/∂η, ∂crosen_F/∂wr, ∂crosen_F/∂wf, ∂crosen_F/∂wi, ∂crosen_F/∂wo, ∂crosen_F/∂wc, ∂crosen_F/∂br, ∂crosen_F/∂bf, ∂crosen_F/∂bi, ∂crosen_F/∂bo, ∂crosen_F/∂bc+= rdRNN_backpropχ˜i,L1i,Li,η,wr,wf,wi,wo,wc,br,bf,bi,bo,bc;
end
η, wr, wf, wi, wo, wc, br, bf, bi, bo, bc= Adamη,wr,wf,wi,wo,wc,br,bf,bi,bo,bc,∂crosen_F/∂η,∂crosen_F/∂wr,∂crosen_F/∂wf,∂crosen_F/∂wi,∂crosen_F/∂wo,∂crosen_F/∂wc,∂crosen_F/∂br,∂crosen_F/∂bf,∂crosen_F/∂bi,∂crosen_F/∂bo,∂crosen_F/∂bc;
end
end
returnη, wr, wf, wi, wo, wc and br, bf, bi, bo, bc;


#### 4.2.4. Loss Functioncrosen_F and Softmax Layer

In Algorithm3, we use the softmax function softmax to calculate the probability that χn belongs to each type of legal case according to judgment results.Definition 9.
Given a set of samples of legal cases and their corresponding outputs of RnRTDχn,Sn, the probability that χn belongs to each type of legal case is calculated by(39)L1qn=softmaxSn=eSqn∑rQeSrn,where L1qn represents the qth element of L1n.
In this study, cross entropy is used as the loss functioncrosen_F to calculate the error of RnRTD. We define crosen_F as follows:Definition 10.
A set of samples of legal cases and their corresponding legal case typesχn,Ln is given. The predicted legal case category of χn is L1n, which is calculated by RnRTD, and then(40)crosen_FLn,L1n=∑nN∑qQLqnlogL1qn,where N represents the number of samples of legal cases and q represents the dimension of Ln and L1n, that is, the number of types of legal cases.


## 4.1. RTD Method

This study proposes a new tensor decomposition method called RTD method. The inputs of the RTD algorithm include the restricted condition tensorη and tensor χ that represents the legal case. The RTD outputs include core tensor χ˜ and its corresponding factor matrix sets, namely, Ck and Dk. RTD decomposes χ into a core tensor χ˜ under the action of the restricted condition η. χ˜ is approximated to η in terms of tensor structure and internal element values. RTD can be interpreted as a mapping of the original tensor χ to the core tensor χ˜. In short, RTD achieves directional decomposition of tensors and extracts vital information from tensors while reducing their dimension. In this study, we define core tensor χ˜ as the most favorable tensor structure and element value information for the subsequent legal case classification algorithm, namely, rdRNN. On this basis, we construct a deep neural network model for RnRTD that is dedicated to legal intelligence judgments.RTD decomposes the original tensor under the restricted condition so that the obtained core tensor constantly approaches the restricted tensor in terms of tensor structure and element value. In Figure7, the formal description of the problems to be solved by the RTD algorithm is shown as Problems 1 and 2.Figure 7
Framework of the RTD method.Problem 2.
Given tensorχ∈ℝI1×I2×⋯×IK, restricted tensor η∈ℝJ1×J2×⋯×JK, and its weight wη, we derive two factor matrix sets, namely, Ck and Dk, Ck∈ℝIk×Hk, Dk∈ℝJk×Hk, that Ck and Dk minimize the following function:(11)ϕ=χ∏kK×kCk−Wη×nη∏kK×kDkF2.
MatrixWη is preset according to the legal case, Wη∈ℝHn×∏k≠nKHk. The elements in sets Ck and Dk are orthogonal matrices, that is, they meet the following conditions. For any elements Ck and Dk in sets Ck and Dk,(12)CkCkT=I,CkTCk=I,DkDkT=I,DkTDk=I.
In this study, we use the alternating least squares (ALS) algorithm to determine the solution of the objective functionϕ. The ALS algorithm can be divided into four steps: (1) randomly pick a variable as a parameter and randomly generate the values of other variables, (2) determine the partial derivative of the loss function ϕ in the specified parameter while fixing the values of other variables, (3) set the partial derivative of ϕ to the specified parameter as zero and calculate the value of the specified parameter, and (4) select another variable as a parameter and return to Step (2). The ALS algorithm continues to iterate Steps (2), (3), and (4) until the error of the loss function ϕ reaches the tolerable upper limit.
Problem 2 needs to be solved using Lemma 1. The specific definition and proof of Lemma 1 are provided as follows.Lemma 1.
Given functiontrαTα=∑i1I1∑i2I2⋯∑inINαi1,i2,…,in2, α∈ℝI1×I2×⋯×IN, α can be a vector, matrix, and tensor. The target function φ=trχ∏kK×kCkTχ∏kK×kCk, where χ∈ℝI1×I2×⋯×IK, Ck∈ℝIk×Hk, and Ck satisfy equation (12). For any element Ck0 in Ck, k0∈1,K, the partial differential of function φ to Ck0 is ∂φ/∂Ck0=εχ∏k≠k0K×kCkTχ∏kK×kCk, where ε is a constant.
The proof of Lemma 1 is shown in Proof 1.Proof 1.
We useμ to represent χ∏k≠k0K×kCk; it can can be derived that φ=trμ×k0Ck0Tμ×k0Ck0. We abbreviate the target function φ=trμCk0TμCk0=trCk0TμTμCk0. We use ν to represent μTμ, and we can get that φ=trCk0TνCk0. According to the function derivation rule, we can obtain the following equation: ∂φ/∂Ck0=∂trCk0TνCk0/∂Ck0=νCk0+νTCk0. Since ν=νT, the calculation formula for the partial derivative of the function φ to Ck0 is(13)∂φ∂Ck0=2νCk0=εχ∏k≠k0K×kCkTχ∏kK×kCk,where ε is a constant, ε=2.
According to the iterative process of the ALS algorithm, the precondition for solving the value ofCk and Dk which minimize the function ϕ in equation (11) using the ALS algorithm is to calculate the value of ∂ϕ/∂Ck0, where k0∈1,K. Equation (11) shows that ∂ϕ/∂Ck0 and ∂ϕ/∂Dk0 are solved in the same manner. Proof 2 provides mathematical proof of the calculation of ∂ϕ/∂Ck0.Proof 2.
We useλ and ϖ to represent χ∏k≠k0K×kCk and wηη∏kK×kDk, respectively. According to formula (11), we can obtain that ϕ=trλ×k0Ck0−ϖTλ×k0Ck0−ϖ. We abbreviate the aforementioned formula as ϕ=trλCk0−ϖTλCk0−ϖ. Then, we can determine the following formula: ∂ϕ/∂Ck0=∂trϖTϖ−2∂trCk0TλTϖ+∂trCk0TλTλCk0/∂Ck0. According to the function derivation rule, we derive that ∂trϖTϖ/∂Ck0=0, ∂trCk0TλTϖ/∂Ck0=λTϖ. In combination with Lemma 1, we can obtain that ∂trCk0TλTλCk0/∂Ck0=2λTλCk0. Finally, the following formula is determined:(14)∂ϕ∂Ck0=−2λTϖ+2λTλCk0.
We set the value of equation (14) to 0 and obtain that(15)λCk0=ϖ.
LetZ=λk0Tϖk0, then Ck0TZ=λCk0k0Tϖk0=ϖk0Tϖk0. By combining Equation (12), we derive that(16)Ck0TZ=ZTCk0,Z=Ck0ZTCk0.
We use the SVD matrix decomposition method to decomposeZ, and find that Z=PSQT. P and Q are orthogonal matrices, P is a left singular matrix, Q is a right singular matrix, and S is a diagonal matrix. After this analysis, the following solution can be obtained:(17)Ck0=PQT.
In summary, according to equations (11)–(17), we can derive a solution of ∂ϕ/∂Ck0 and Ck0 which are described in equations (14) and (17), respectively. ∂ϕ/∂Dk is calculated in the same manner as ∂ϕ/∂Ck. On this basis, we calculate the value of Ck and Dk , which minimize the objective function ϕ in formula (11), by using the ALS algorithm.
Algorithm1 shows the solution of Problem 2 by using ALS algorithm. The inputs of Algorithm 1 are χ which represents one legal case, the restricted tensor η, and its weight wη. In line 2, we randomly initialize the values of Ck, Uk. max_iterations in line 2 represents the maximum number of iterations of ALS algorithm. The function calcu_Z in line 4 corresponds to equation (16). Line 5 and 6 show the calculation process of equation (17).
Another problem to be solved by the RTD algorithm is Problem 3, which is the formal description of the process of tensor decomposition on the original tensor under the action of the restricted tensor and its weight. On the basis of Problem 2, we can obtain factor matrix setsCk and Dk, which minimize the value of function ϕ in formula (11) while satisfying formula (12).Algorithm 1: The solution of Problem 2 by using ALS Algorithm.
Input: Tensorχ which represents the original legal case data, χ∈ℝI1×I2×⋯×IK, the restricted tensor η, η∈ℝJ1×J2×⋯×JK, and its weight wη.
Output: The factor matrix setsCk and Dk, Ck∈ℝIk×Hk, Dk∈ℝJk×Hk.
Initialize the factor matrix setsCk and Dk;
for i=1 to max_iterations do
// First, pick the elements in the factor setCk as variables
for k0=1 to Kdo
Z=calcu_Zχ,η,Ck,Dk,wη,k0;
P,S,QT=SVDZ;
Ck0=PQT;
end
// Then, pick the elements in the factor setDk as variables
for k0=1 to Kdo
Z=calcu_Zχ,η,Dk,Ck,wη,k0;
P,S,QT=SVDZ;
Dk0=PQT;
end
end
returnCk, Dk;Problem 3.
Given a tensorχ∈ℝI1×I2×⋯×IK and factor matrix sets Ck and Dk, Ck∈ℝIk×Hk, Dk∈ℝJk×Hk, Ck and Dk are derived from Problem 2. A core tensor χ˜ is determined, where χ˜∈ℝJ1×J2×⋯×JK and χ˜ minimize the following target function:(18)FRTD=χ∏kK×kCk−χ˜∏kK×kDkF2.
Problem 3 needs to be solved using Lemma 2. The specific definition and proof process of Lemma 2 are as follows.Lemma 2.
Given the target functionψ=trχ˜∏kK×kDk∏kK×kDkTχ˜T, χ˜∈ℝJ1×J2×⋯×JK, Dk∈ℝJk×Hk, each element in Dk satisfies formula (12). Then, the partial derivative of the target function ψ to Dk0 where k0∈1,K is ∂ψ/∂Dk0, ∂ψ/∂Dk0=εχ˜∏kK×kDk∏kK×kDkT, where ε is a constant.Proof 3.
We useκ to represent τ∏kK×kDk; τ is the identity tensor, τ∈ℝJ1×J2×⋯JK. Then, the target function ψ can be rewritten as ψ=trχ˜κκTχ˜T. We use ρ to represent κκT. We can obtain that ∂ϕ/∂χ˜=∂trχ˜κκTχ˜T/∂χ˜=∂trχ˜ρχ˜T/∂χ˜. From the function derivation rule, we can get the following formula: ∂trχ˜κκTχ˜T/∂χ˜=∂trχ˜ρχ˜T/∂χ˜=ρχ˜T+ρTχ˜TT; thus,(19)∂ϕ∂χ˜=2χ˜κκT=εχ˜∏kK×kDk∏kK×kDkT,where ε is a constant, ε=2.
After the aforementioned analysis, Proof 4 gives the solution to Problem 3 and its mathematical proof process while combining Lemma 2 and Proof 3.Proof 4.
We useυ to represent χ∏kK×kCk and γ to represent τ∏kK×kDk, where τ is the identity tensor, τ∈ℝJ1×J2×⋯JK. Then, the function FRTD can be rewritten as FRTD=υ−χ˜γF2, that is FRTD=trυ−χ˜γTυ−χ˜γ. Known by the definition of function tr, trγTχ˜Tυ=trχ˜TυγT. Then, we can get the following equation: ∂FRTD/∂χ˜=∂trυTυ−2∂trχ˜TυγT+∂trχ˜γγTχ˜T/∂χ˜. It can be derived from the function derivation rule and Lemma 2 that(20)∂FRTD∂χ˜=εχ˜γγT−υγT,where ε is a constant, ε=2. Let ∂FRTD/∂χ˜ be zero. We can get the final solution of Problem 3 by combining formula (12).(21)χ˜=χ∏kK×kCk∏kK×kDkT.
Algorithm2 implements RTD by using Algorithm 1. Function TSPA in line 1 represents the implementation of Algorithm 1, and the inputs are χ, η, and wη. Function F_RTD in line 2 shows the calculation of χ˜ using equations (18)–(21). Finally, the core tensor of χ is obtained by using Algorithm 2, which approximates the restricted tensor η on the layer of tensor structure and elements information.Algorithm 2: The restricted tensor decomposition method.
Input: Tensorχ which represents the original legal case data, χ∈ℝI1×I2×⋯×IK, the restricted tensor η, η∈ℝJ1×J2×⋯×JK, and its weight wη.
Output: The core tensorχ˜ which is close to the restricted tensor η in the layer of tensor structure and elements value, χ˜∈ℝJ1×J2×⋯×JK.
// Solving the factor matrix setsCk and Dk using Algorithm 1
C
k, Dk=TSPAχ,η,wη;
χ
˜
=
F_RTD
χ
,
C
k
,
D
k;
returnχ˜;


## 4.2. rdRNN

This study proposes a new RNN called rdRNN. Unlike traditional RNN, rdRNN sets up a new gate based on the bidirectional RNN. The new gate uses the similarity matrix between samples as a parameter of the deep neural network training model. Compared with the original bidirectional RNN, the classification result of rdRNN is more accurate and stable. For the intelligent judgment of legal cases, the original deep neural network method does not consider the correlation between legal cases. This disregard may lead to bias in the final case classification. For example, the verdict of a legal case is inconsistent with the description of the case. To solve this problem, rdRNN fully considers the judgment results of legal cases that are similar to the case to be judged. rdRNN uses these results as a parameter of the deep neural network training model and realizes an efficient and accurate classification of multiple accusations in legal cases.The following section shows the training of rdRNN’s deep neural network while using Rmsprop as its optimization function:(i)
We use the datasetχ˜n,Ln and the restricted tensor η as inputs of rdRNN. χ˜n is the core tensor of χn, which represents the legal case. χ˜n is obtained by the RTD algorithm with χn and η as its inputs. Ln represents the category label of χn according to the judgment result of legal case.(ii)
In this study, we combine rdRNN with the softmax layer to complete the classification of legal cases. For sampleχn, assuming hn is the output vector of rdRNN, the softmax layer implements the mapping of hn to the legal case category Ln.(iii)
We use cross entropycroen_F as the loss function to update rdRNN. rdRNN uses its forward propagation algorithm and error backpropagation formulas to iterate over the values of parameters in neural networks, such as weight matrices wd and bias terms bd that are associated with relationship gate, and restricted tensor η, where d is the number of hidden layers.(iv)
We select Adam as the optimization function of rdRNN, and Rmsprop completes the optimization and calculation of parameterswd, bd, and η by using ∂crosen_F/∂wd, ∂crosen_F/∂bd, and ∂crosen_F/∂η.


### 4.2.1. Calculation of Forward Propagation in rdRNN

In this study, we fully consider the relationship between legal cases and set up a new gate to complete the classification of legal cases, eliminate contingency errors as much as possible, and avoid inconsistencies between the predicted judgment result and the actual case. Relationship control gatert is used to control the similar relationship between legal cases. rt helps the rdRNN deep neural network make an intelligent judgment by using the judgment results of cases that are similar to the case to be judged.rdRNN can be divided into forward and backward LSTM. These networks do not have obvious differences, except for the opposite propagation direction. In the case of rdRNN forward LSTM propagation network, the formal description of relationship control gatert is as follows:(22)rt=σwrht−1xt+br,where wr and br are the weight matrix and bias term of the relational control gate rt, respectively, σ is the activation function, i.e., the sigmoid function, ht−1 is the output unit state of the neuron at time t−1, and xt is the input value of the neuron at time t.In the forward LSTM network, the output of each neuron at timet is calculated by the following formula:(23)rtftitotc˜t=σnetr,tσnetf,tσneti,tσneto,ttanhnetc˜,t=σσσσtanhwrh,wrxwfh,wfxwih,wixwoh,woxwch,wcx⋅ht−1xt+brbfbibobc,where rt, ft, it, and ot are the relational control, forget, input, and output gates, respectively; c˜t is the unit status of current inputs; netr,t, netf,t, neti,t, and neto,t are the weighted inputs of their corresponding gates at time t; netc,t is the weighted inputs of input state generation function tanh; σ is the activation function, i.e., the sigmoid function, σx=1/1+e−x; tanh is the hyperbolic tangent function, tanhx=ex−e−x/ex+e−x; wr is the weight matrix of relationship control gate rt, wr=wrh,wrx; wf, wi, and wo are expressed in the same manner as wr; and br, bf, bi, and bo are the bias terms of their corresponding activation functions.Subsequently, the unit state of the current momentct is calculated by ft, ct−1, it, and c˜t. The calculation formula is expressed as follows:(24)ct=ft∗ct−1+it∗c˜t.The final output of the forward LSTM neural network at timet is calculated by ot, ct, and rt and the similar list of x. It is described as follows:(25)ht=ot∗tanhct+rt∗1Listx∑x0∈ListxSimx0,xhx0,where Listx is composed of legal cases where the similarity with x is greater than a threshold Max_sim so far. hx0 refers to the output of the forward LSTM neural network that corresponds to legal case x0. Sim⋅ is a function that calculates the similarity between legal cases. In this study, we set function Sim as the weight of the Euclidean distance and the cosine distance between legal cases.(26)Simx,x0=DEuclideanx,x0+wdDcosinex,x0,where DEuclidean and Dcosine refer to the Euclidean distance and the cosine distance between the vectors x and x0, respectively. wd is the weight matrix.


### 4.2.2. Calculation of Backpropagation in rdRNN

In this section, we describe in detail the backpropagation algorithm of the rdRNN neural network, including the backpropagation of the error along time and the hidden layer. In rdRNN, forward and backward LSTM neural networks have the same principle in the backpropagation algorithm. Therefore, this section mainly uses forward LSTM as an example.Given the error term at timet δt, δt=∂crosen_F/∂ht. Calculation of the backpropagation algorithm of the error term along time is to calculate the value of δt−1=∂crosen_F/∂ht−1. The full derivative formula shows that(27)δt−1=∂crosen_F∂ht∂ht∂ht−1.Equations (23)–(25) show that rt, ft, it, ot, and c˜t are all functions of ht−1. Then, we can obtain(28)δo,t=δt∂ht∂ot∂ot∂neto,t,δf,t=δt∂ht∂ct∂ct∂ft∂ft∂netf,t,δi,t=δt∂ht∂ct∂ct∂it∂it∂neti,t,δc˜,t=δt∂ht∂ct∂ct∂c˜t∂c˜t∂netc˜,t,δr,t=δt∂ht∂rt∂rt∂netr,t,∂neto,t∂ht−1=woh,∂netf,t∂ht−1=wfh,∂neti,t∂ht−1=wih,∂netc˜,t∂ht−1=wch,∂netr,t∂ht−1=wrh.The formula on the left represents the variable declaration, and the formula on the right is calculated from equation (23). According to equations (27) and (28), we can further derive the following formula:(29)δt−1=δo,twoh+δf,twfh+δi,twih+δc˜,twch+δr,twrh,where δo,t=δt∗tanhct∗ot∗1−ot, δf,t=δt∗ot∗1−tanhct2∗ct−1∗ft∗1−ft, δi,t=δt∗ot∗1−tanhct2∗c˜t∗it∗1−it, δc˜,t=δt∗ot∗1−tanhct2∗it∗1−c˜t2. According to relationship control gate rt, equations (23) and (25), we determine that(30)δr,t=δt∗1Listx∑x0∈ListxSimx0,xhx0∗rt∗1−rt.From equation (29), we can finally figure out the calculation method of the error term in rdRNN is passed from the current moment t to any time k.Then, we describe in detail the transmission of error between the hidden layers of rdRNN. The error term of thelth hidden layer in rdRNN is assumed to be the partial derivative of the error function crosen_F versus the weighted input netl. In rdRNN, the input of the lth hidden layer at time t is xtl.(31)xtl=activeFl−1nettl−1,where activeFl−1 denotes the activation function of the l−1th hidden layer in rdRNN and nettl−1 denotes the weighted input of the l−1th hidden layer at time t.Given the error term of the lth hidden layer at time t δtl, δtl=∂crosen_F/∂nettl, the calculation of error propagation between hidden layers is to figure out the value of δtl−1, where(32)δtl−1=∂crosen_F∂nettl−1.According to equations (23), (31), and (32), netr,tl, netf,tl, neti,tl, neto,tl, and netc˜,tl are all functions of xtl, and xtl is a function of nettl−1. Therefore, the full derivative formula shows that(33)δtl−1=∂crosen_F∂netr,tl∂netr,tl∂xtl∂xtl∂nettl−1+∂crosen_F∂netf,tl∂netf,tl∂xtl∂xtl∂nettl−1+∂crosen_F∂neti,tl∂neti,tl∂xtl∂xtl∂nettl−1+∂crosen_F∂neto,tl∂neto,tl∂xtl∂xtl∂nettl−1+∂crosen_F∂netc˜,tl∂netc˜,tl∂xtl∂xtl∂nettl−1.The following formula can be obtained by further calculation:(34)δtl−1=δr,tlwrx+δf,tlwfx+δi,tlwix+δo,tlwox+δc˜,tlwcx∗graactiveFl−1nettl−1,where graactiveFl−1nettl−1 is the derivative of function activeFl−1 at nettl−1.According to equations (27)–(34), we can derive the partial derivative of the loss function crosen_F to the weight matrix set wd and the bias term set bd in rdRNN. Given that ∂crosen_F/∂wrh=∑j=1t∂crosen_F/∂wrh,j=∑j=1t∂crosen_F/∂netr,j∂netr,j/∂wrh,j, we obtain(35)∂crosen_F∂wrx=∑j=1tδr,jhj−1,∂crosen_F∂wfx=∑j=1tδf,jhj−1,∂crosen_F∂wix=∑j=1tδi,jhj−1,∂crosen_F∂wox=∑j=1tδo,jhj−1,∂crosen_F∂wcx=∑j=1tδc˜,jhj−1,∂crosen_F∂br=∑j=1tδr,j,∂crosen_F∂bf=∑j=1tδf,j,∂crosen_F∂bi=∑j=1tδi,j,∂crosen_F∂bo=∑j=1tδo,j,∂crosen_F∂bc=∑j=1tδc˜,j,∂crosen_F∂wrx=δr,jxt,∂crosen_F∂wfx=δf,jxt,∂crosen_F∂wix=δi,jxt,∂crosen_F∂wox=δo,jxt,∂crosen_F∂wcx=δc˜,jxt.


### 4.2.3. Calculation of the Partial Derivative of Loss Functioncrosen_F to Restricted Tensor η

This study proposes a new intelligent method for judging legal cases called RnRTD, which combines rdRNN and RTD to complete the classification of legal cases. In the process of training the RnRTD neural network, a new problem is involved: updating of the value of the restricted tensorη so that it can continuously approximate the tensor value that is most beneficial for improving the classification accuracy of the RnRTD algorithm.The crux to solving this problem is to calculate the partial derivative of the loss functioncrosen_F to the restricted tensor η, that is, ∂crosen_F/∂η. Directly solving the value of ∂crosen_F/∂η is difficult. We can use the full derivative rule to obtain that(36)∂crosen_F∂η=∑nN∂crosen_F∂χ˜n∂χ˜n∂η.The backward propagation formula of rdRNN shows that∂crosen_F/∂χ˜n=∂crosen_F/∂net1∂net1/∂χ˜n=δ1w0. According to equations (19)–(21), we determine that ∂χ˜n/∂η=∂χn∏kK×kCkn∏k1K×k1Uk1nT/∂η. From equations (14)–(17), we know that Ckn and Dkn are all functions of η. Therefore, the function full derivative rule shows that ∂χ˜n/∂η=∑kK∂χ˜n/∂Ckn∂Ckn/∂η+∑kK∂χ˜n/∂DknT∂DknT/∂η, that is(37)∂χ˜n∂η=∑kKcalCCkn+∑kKcalDDknT,while(38)calCCkn=χn∏k1≠kK×k1Ck1n×k∂Ckn∂η∏k2K×k2DK2nT,calDDknT=χn∏k1K×k1Ck1n×k∂DknT∂η∏k2≠kK×k2DK2nT.Algorithm3 provides the optimization process of RnRTD proposed in this study. max_iter in line 2 represents the total number of training sessions of RnRTD. batch_size in line 6 represents the number of samples per batch while training RnRTD. The RTD tensor decomposition method in line 7 corresponds to Algorithm 2. rdRNN_forwardprop in line 8 and rdRNN_backprop in line 9 represent forward propagation and error backpropagation algorithms for rdRNN, respectively, which are the implementations of Sections 4.2.1–4.2.3. In line 11, we use the Adam algorithm to realize parameter optimization of RnRTD neural network.Algorithm 3: The framework of RnRTD algorithm.
Input:χn,Ln, where χn represents the legal cases and Ln represents the category of legal case corresponding to χn according to judgment results. The size of η, wr, wf, wi, wo, wc, br, bf, bi, bo and bc, where wr=wrh,wrx, wf=wfh,wfx, wi=wih,wix, wo=woh,wox and wc=wch,wcx
Output: The optimal restricted tensorη, parameters of rdRNN wr, wf, wi, wo, wc and br, bf, bi, bo, bc.
Initialize the restricted tensorη, parameters of rdRNN wr, wf, wi, wo, wc and br, bf, bi, bo, bc
for iter=1 to max_iter do
Setbatch_indices to zero;
while batch_indices≤dataset_size do
Set∂crosen_F/∂η, ∂crosen_F/∂wr, ∂crosen_F/∂wf, ∂crosen_F/∂wi, ∂crosen_F/∂wo, ∂crosen_F/∂wc and ∂crosen_F/∂br, ∂crosen_F/∂bf, ∂crosen_F/∂bi, ∂crosen_F/∂bo, ∂crosen_F/∂bc to zero;
for i=batch_indices to batch_indices+batch_size do
χ˜i=RTDχi,η;
L1i=rdRNN_forwardpropχ˜i,wr,wf,wi,wo,wc,br,bf,bi,bo,bc;
∂crosen_F/∂η, ∂crosen_F/∂wr, ∂crosen_F/∂wf, ∂crosen_F/∂wi, ∂crosen_F/∂wo, ∂crosen_F/∂wc, ∂crosen_F/∂br, ∂crosen_F/∂bf, ∂crosen_F/∂bi, ∂crosen_F/∂bo, ∂crosen_F/∂bc+= rdRNN_backpropχ˜i,L1i,Li,η,wr,wf,wi,wo,wc,br,bf,bi,bo,bc;
end
η, wr, wf, wi, wo, wc, br, bf, bi, bo, bc= Adamη,wr,wf,wi,wo,wc,br,bf,bi,bo,bc,∂crosen_F/∂η,∂crosen_F/∂wr,∂crosen_F/∂wf,∂crosen_F/∂wi,∂crosen_F/∂wo,∂crosen_F/∂wc,∂crosen_F/∂br,∂crosen_F/∂bf,∂crosen_F/∂bi,∂crosen_F/∂bo,∂crosen_F/∂bc;
end
end
returnη, wr, wf, wi, wo, wc and br, bf, bi, bo, bc;


### 4.2.4. Loss Functioncrosen_F and Softmax Layer

In Algorithm3, we use the softmax function softmax to calculate the probability that χn belongs to each type of legal case according to judgment results.Definition 9.
Given a set of samples of legal cases and their corresponding outputs of RnRTDχn,Sn, the probability that χn belongs to each type of legal case is calculated by(39)L1qn=softmaxSn=eSqn∑rQeSrn,where L1qn represents the qth element of L1n.
In this study, cross entropy is used as the loss functioncrosen_F to calculate the error of RnRTD. We define crosen_F as follows:Definition 10.
A set of samples of legal cases and their corresponding legal case typesχn,Ln is given. The predicted legal case category of χn is L1n, which is calculated by RnRTD, and then(40)crosen_FLn,L1n=∑nN∑qQLqnlogL1qn,where N represents the number of samples of legal cases and q represents the dimension of Ln and L1n, that is, the number of types of legal cases.


## 4.2.1. Calculation of Forward Propagation in rdRNN

In this study, we fully consider the relationship between legal cases and set up a new gate to complete the classification of legal cases, eliminate contingency errors as much as possible, and avoid inconsistencies between the predicted judgment result and the actual case. Relationship control gatert is used to control the similar relationship between legal cases. rt helps the rdRNN deep neural network make an intelligent judgment by using the judgment results of cases that are similar to the case to be judged.rdRNN can be divided into forward and backward LSTM. These networks do not have obvious differences, except for the opposite propagation direction. In the case of rdRNN forward LSTM propagation network, the formal description of relationship control gatert is as follows:(22)rt=σwrht−1xt+br,where wr and br are the weight matrix and bias term of the relational control gate rt, respectively, σ is the activation function, i.e., the sigmoid function, ht−1 is the output unit state of the neuron at time t−1, and xt is the input value of the neuron at time t.In the forward LSTM network, the output of each neuron at timet is calculated by the following formula:(23)rtftitotc˜t=σnetr,tσnetf,tσneti,tσneto,ttanhnetc˜,t=σσσσtanhwrh,wrxwfh,wfxwih,wixwoh,woxwch,wcx⋅ht−1xt+brbfbibobc,where rt, ft, it, and ot are the relational control, forget, input, and output gates, respectively; c˜t is the unit status of current inputs; netr,t, netf,t, neti,t, and neto,t are the weighted inputs of their corresponding gates at time t; netc,t is the weighted inputs of input state generation function tanh; σ is the activation function, i.e., the sigmoid function, σx=1/1+e−x; tanh is the hyperbolic tangent function, tanhx=ex−e−x/ex+e−x; wr is the weight matrix of relationship control gate rt, wr=wrh,wrx; wf, wi, and wo are expressed in the same manner as wr; and br, bf, bi, and bo are the bias terms of their corresponding activation functions.Subsequently, the unit state of the current momentct is calculated by ft, ct−1, it, and c˜t. The calculation formula is expressed as follows:(24)ct=ft∗ct−1+it∗c˜t.The final output of the forward LSTM neural network at timet is calculated by ot, ct, and rt and the similar list of x. It is described as follows:(25)ht=ot∗tanhct+rt∗1Listx∑x0∈ListxSimx0,xhx0,where Listx is composed of legal cases where the similarity with x is greater than a threshold Max_sim so far. hx0 refers to the output of the forward LSTM neural network that corresponds to legal case x0. Sim⋅ is a function that calculates the similarity between legal cases. In this study, we set function Sim as the weight of the Euclidean distance and the cosine distance between legal cases.(26)Simx,x0=DEuclideanx,x0+wdDcosinex,x0,where DEuclidean and Dcosine refer to the Euclidean distance and the cosine distance between the vectors x and x0, respectively. wd is the weight matrix.


## 4.2.2. Calculation of Backpropagation in rdRNN

In this section, we describe in detail the backpropagation algorithm of the rdRNN neural network, including the backpropagation of the error along time and the hidden layer. In rdRNN, forward and backward LSTM neural networks have the same principle in the backpropagation algorithm. Therefore, this section mainly uses forward LSTM as an example.Given the error term at timet δt, δt=∂crosen_F/∂ht. Calculation of the backpropagation algorithm of the error term along time is to calculate the value of δt−1=∂crosen_F/∂ht−1. The full derivative formula shows that(27)δt−1=∂crosen_F∂ht∂ht∂ht−1.Equations (23)–(25) show that rt, ft, it, ot, and c˜t are all functions of ht−1. Then, we can obtain(28)δo,t=δt∂ht∂ot∂ot∂neto,t,δf,t=δt∂ht∂ct∂ct∂ft∂ft∂netf,t,δi,t=δt∂ht∂ct∂ct∂it∂it∂neti,t,δc˜,t=δt∂ht∂ct∂ct∂c˜t∂c˜t∂netc˜,t,δr,t=δt∂ht∂rt∂rt∂netr,t,∂neto,t∂ht−1=woh,∂netf,t∂ht−1=wfh,∂neti,t∂ht−1=wih,∂netc˜,t∂ht−1=wch,∂netr,t∂ht−1=wrh.The formula on the left represents the variable declaration, and the formula on the right is calculated from equation (23). According to equations (27) and (28), we can further derive the following formula:(29)δt−1=δo,twoh+δf,twfh+δi,twih+δc˜,twch+δr,twrh,where δo,t=δt∗tanhct∗ot∗1−ot, δf,t=δt∗ot∗1−tanhct2∗ct−1∗ft∗1−ft, δi,t=δt∗ot∗1−tanhct2∗c˜t∗it∗1−it, δc˜,t=δt∗ot∗1−tanhct2∗it∗1−c˜t2. According to relationship control gate rt, equations (23) and (25), we determine that(30)δr,t=δt∗1Listx∑x0∈ListxSimx0,xhx0∗rt∗1−rt.From equation (29), we can finally figure out the calculation method of the error term in rdRNN is passed from the current moment t to any time k.Then, we describe in detail the transmission of error between the hidden layers of rdRNN. The error term of thelth hidden layer in rdRNN is assumed to be the partial derivative of the error function crosen_F versus the weighted input netl. In rdRNN, the input of the lth hidden layer at time t is xtl.(31)xtl=activeFl−1nettl−1,where activeFl−1 denotes the activation function of the l−1th hidden layer in rdRNN and nettl−1 denotes the weighted input of the l−1th hidden layer at time t.Given the error term of the lth hidden layer at time t δtl, δtl=∂crosen_F/∂nettl, the calculation of error propagation between hidden layers is to figure out the value of δtl−1, where(32)δtl−1=∂crosen_F∂nettl−1.According to equations (23), (31), and (32), netr,tl, netf,tl, neti,tl, neto,tl, and netc˜,tl are all functions of xtl, and xtl is a function of nettl−1. Therefore, the full derivative formula shows that(33)δtl−1=∂crosen_F∂netr,tl∂netr,tl∂xtl∂xtl∂nettl−1+∂crosen_F∂netf,tl∂netf,tl∂xtl∂xtl∂nettl−1+∂crosen_F∂neti,tl∂neti,tl∂xtl∂xtl∂nettl−1+∂crosen_F∂neto,tl∂neto,tl∂xtl∂xtl∂nettl−1+∂crosen_F∂netc˜,tl∂netc˜,tl∂xtl∂xtl∂nettl−1.The following formula can be obtained by further calculation:(34)δtl−1=δr,tlwrx+δf,tlwfx+δi,tlwix+δo,tlwox+δc˜,tlwcx∗graactiveFl−1nettl−1,where graactiveFl−1nettl−1 is the derivative of function activeFl−1 at nettl−1.According to equations (27)–(34), we can derive the partial derivative of the loss function crosen_F to the weight matrix set wd and the bias term set bd in rdRNN. Given that ∂crosen_F/∂wrh=∑j=1t∂crosen_F/∂wrh,j=∑j=1t∂crosen_F/∂netr,j∂netr,j/∂wrh,j, we obtain(35)∂crosen_F∂wrx=∑j=1tδr,jhj−1,∂crosen_F∂wfx=∑j=1tδf,jhj−1,∂crosen_F∂wix=∑j=1tδi,jhj−1,∂crosen_F∂wox=∑j=1tδo,jhj−1,∂crosen_F∂wcx=∑j=1tδc˜,jhj−1,∂crosen_F∂br=∑j=1tδr,j,∂crosen_F∂bf=∑j=1tδf,j,∂crosen_F∂bi=∑j=1tδi,j,∂crosen_F∂bo=∑j=1tδo,j,∂crosen_F∂bc=∑j=1tδc˜,j,∂crosen_F∂wrx=δr,jxt,∂crosen_F∂wfx=δf,jxt,∂crosen_F∂wix=δi,jxt,∂crosen_F∂wox=δo,jxt,∂crosen_F∂wcx=δc˜,jxt.


## 4.2.3. Calculation of the Partial Derivative of Loss Functioncrosen_F to Restricted Tensor η

This study proposes a new intelligent method for judging legal cases called RnRTD, which combines rdRNN and RTD to complete the classification of legal cases. In the process of training the RnRTD neural network, a new problem is involved: updating of the value of the restricted tensorη so that it can continuously approximate the tensor value that is most beneficial for improving the classification accuracy of the RnRTD algorithm.The crux to solving this problem is to calculate the partial derivative of the loss functioncrosen_F to the restricted tensor η, that is, ∂crosen_F/∂η. Directly solving the value of ∂crosen_F/∂η is difficult. We can use the full derivative rule to obtain that(36)∂crosen_F∂η=∑nN∂crosen_F∂χ˜n∂χ˜n∂η.The backward propagation formula of rdRNN shows that∂crosen_F/∂χ˜n=∂crosen_F/∂net1∂net1/∂χ˜n=δ1w0. According to equations (19)–(21), we determine that ∂χ˜n/∂η=∂χn∏kK×kCkn∏k1K×k1Uk1nT/∂η. From equations (14)–(17), we know that Ckn and Dkn are all functions of η. Therefore, the function full derivative rule shows that ∂χ˜n/∂η=∑kK∂χ˜n/∂Ckn∂Ckn/∂η+∑kK∂χ˜n/∂DknT∂DknT/∂η, that is(37)∂χ˜n∂η=∑kKcalCCkn+∑kKcalDDknT,while(38)calCCkn=χn∏k1≠kK×k1Ck1n×k∂Ckn∂η∏k2K×k2DK2nT,calDDknT=χn∏k1K×k1Ck1n×k∂DknT∂η∏k2≠kK×k2DK2nT.Algorithm3 provides the optimization process of RnRTD proposed in this study. max_iter in line 2 represents the total number of training sessions of RnRTD. batch_size in line 6 represents the number of samples per batch while training RnRTD. The RTD tensor decomposition method in line 7 corresponds to Algorithm 2. rdRNN_forwardprop in line 8 and rdRNN_backprop in line 9 represent forward propagation and error backpropagation algorithms for rdRNN, respectively, which are the implementations of Sections 4.2.1–4.2.3. In line 11, we use the Adam algorithm to realize parameter optimization of RnRTD neural network.Algorithm 3: The framework of RnRTD algorithm.
Input:χn,Ln, where χn represents the legal cases and Ln represents the category of legal case corresponding to χn according to judgment results. The size of η, wr, wf, wi, wo, wc, br, bf, bi, bo and bc, where wr=wrh,wrx, wf=wfh,wfx, wi=wih,wix, wo=woh,wox and wc=wch,wcx
Output: The optimal restricted tensorη, parameters of rdRNN wr, wf, wi, wo, wc and br, bf, bi, bo, bc.
Initialize the restricted tensorη, parameters of rdRNN wr, wf, wi, wo, wc and br, bf, bi, bo, bc
for iter=1 to max_iter do
Setbatch_indices to zero;
while batch_indices≤dataset_size do
Set∂crosen_F/∂η, ∂crosen_F/∂wr, ∂crosen_F/∂wf, ∂crosen_F/∂wi, ∂crosen_F/∂wo, ∂crosen_F/∂wc and ∂crosen_F/∂br, ∂crosen_F/∂bf, ∂crosen_F/∂bi, ∂crosen_F/∂bo, ∂crosen_F/∂bc to zero;
for i=batch_indices to batch_indices+batch_size do
χ˜i=RTDχi,η;
L1i=rdRNN_forwardpropχ˜i,wr,wf,wi,wo,wc,br,bf,bi,bo,bc;
∂crosen_F/∂η, ∂crosen_F/∂wr, ∂crosen_F/∂wf, ∂crosen_F/∂wi, ∂crosen_F/∂wo, ∂crosen_F/∂wc, ∂crosen_F/∂br, ∂crosen_F/∂bf, ∂crosen_F/∂bi, ∂crosen_F/∂bo, ∂crosen_F/∂bc+= rdRNN_backpropχ˜i,L1i,Li,η,wr,wf,wi,wo,wc,br,bf,bi,bo,bc;
end
η, wr, wf, wi, wo, wc, br, bf, bi, bo, bc= Adamη,wr,wf,wi,wo,wc,br,bf,bi,bo,bc,∂crosen_F/∂η,∂crosen_F/∂wr,∂crosen_F/∂wf,∂crosen_F/∂wi,∂crosen_F/∂wo,∂crosen_F/∂wc,∂crosen_F/∂br,∂crosen_F/∂bf,∂crosen_F/∂bi,∂crosen_F/∂bo,∂crosen_F/∂bc;
end
end
returnη, wr, wf, wi, wo, wc and br, bf, bi, bo, bc;


## 4.2.4. Loss Functioncrosen_F and Softmax Layer

In Algorithm3, we use the softmax function softmax to calculate the probability that χn belongs to each type of legal case according to judgment results.Definition 9.
Given a set of samples of legal cases and their corresponding outputs of RnRTDχn,Sn, the probability that χn belongs to each type of legal case is calculated by(39)L1qn=softmaxSn=eSqn∑rQeSrn,where L1qn represents the qth element of L1n.
In this study, cross entropy is used as the loss functioncrosen_F to calculate the error of RnRTD. We define crosen_F as follows:Definition 10.
A set of samples of legal cases and their corresponding legal case typesχn,Ln is given. The predicted legal case category of χn is L1n, which is calculated by RnRTD, and then(40)crosen_FLn,L1n=∑nN∑qQLqnlogL1qn,where N represents the number of samples of legal cases and q represents the dimension of Ln and L1n, that is, the number of types of legal cases.


## 5. Results

### 5.1. Description of Experimental Data

We use nearly 1.8 million historical legal cases obtained from a Chinese refereeing study network. These legal cases involve more than 200 types of accusations, including theft, intentional assault, smuggling, fraud, and deliberate destruction of public property. Approximately 400,000 cases involve theft, and about 200,000 cases involve intentional assault. The number of accusations involved in each case ranges from 1 to 23.Figure8(a) shows the distribution of various accusations in the legal case data used in this study. The abscissa indicates the accusation index. For example, index 1 corresponds to bribery, and index 2 corresponds to rape. The ordinate indicates the proportion of cases involving the accusation that occupy the overall cases. Figure 8(a) shows that the number of cases involving theft is the highest in the database used in this article.Figure 8
Statistics on allegations in legal cases.
(a)
(b)Figure8(b) shows the distribution of the number of accusations involved in each case. The abscissa indicates the number of accusations involved in the case, and the ordinate indicates the proportion of cases in the corresponding number of accusations. Figure 8(b) shows that the highest number of accusations involves three cases.


### 5.2. Baseline Approaches

Given that multiaccusation judgment based on deep neural network and tensor decomposition is rarely studied, according to the limited tensor decomposition method RTD and the relation-based recurrent neural network rdRNN, we use the following method for a comparison with RnRTD proposed in this study:(i)
This study uses a series of deep neural networks based on convolutional or cyclic neural networks, including TextCNN, TextCNN attention, TextRNN, TextRNN attention, LSTM, and Bi-LSTM, as comparison methods for RnRTD.(ii)
This study uses deep neural networks with only the RTD tensor decomposition layer as comparison methods for RnRTD. Through experiments, we can derive the contribution of the RTD tensor decomposition layer to all deep neural networks.(iii)
This study uses neural networks with only the rdRNN method as comparison methods for RnRTD. Through these comparison experiments, we can derive the contribution of the relationship-based rdRNN strategy to all RNNs.


### 5.3. Data Preprocessing

In this study, the data preprocessing operation can be divided into two parts, namely, the modular representation of legal cases and the construction of the original tensor. Our legal case data preprocessing process can be described as follows:(i)
We organize each case in the legal case database into our preestablished case model, which divides the original case file into the defendant’s statement, the plaintiff’s statement, the content of the public prosecutions allegations, and the court’s judgment.(ii)
We filter and clean the contents of each module in the legal cases, extract the words that are meaningful to our multiaccusation judgment method, and filter out redundant words, stop words, noisy words, and modal particles.(iii)
We train the word-to-vector model to obtain the word vector of the aforementioned vocabulary. Then, we obtain a matrix representation of each case module and derive the tensor representation of the entire legal case.For Step (1), each case module may be spread across different paragraphs, and cases in different regions have different case descriptions. We extract and integrate them separately to arrive at a modular representation of the cases based on the description rules of case documents in each region. For Step (2), the extraction and filtering of the vocabulary in legal cases often requires professional legal background knowledge; otherwise, error filtering will occur. We filter words in legal case modules by using the legal professional vocabulary and the stop word list. For Step (3), word vectors are the basis for the accuracy of the entire deep neural network method. We use a number of Chinese corpus, such as corpus on Baidu Encyclopedia, Zhihu Questions and Answers, Sohu News, and Sina Weibo, to train the word-to-vector model.The tensor representation of legal cases and the subsequent deep neural network classification method require each case to have the same number of words, and the number of words in95% of the cases is below 300. Therefore, we perform a padding operation for cases where the number of words is less than 300. For cases with a vocabulary number greater than 300, we use the TF-IDF weight of the vocabulary to tailor the case vocabulary.


### 5.4. Experimental Hyperparameter Setting

This section describes the hyperparameter settings involved in our proposed method. These settings include the restricted tensorη and weight matrix Wη in RTD and the size of the similar case list in the rdRNN method (i.e., the size of list ∗ in equation (25)).The setting of restricted tensorη directly affects the convergence speed and accuracy of RnRTD. Our experiments show that a large rank of restricted tensor η corresponds to a high accuracy of the subsequent deep neural network algorithms. Conversely, a strong linear relationship between column or row vectors in η results in a low accuracy of the subsequent classification algorithms.In this study, weight matrixWη is used to scale the elements of the last mode in the original tensors. Wη adjusts the weights of certain words in the legal cases. For different accusations, the same vocabulary may have different weights in different types of cases. For example, derailment occupies a large and small weight in cases that involve bigamy and smuggling, respectively.The size of the similar case list in rdRNN is an important indicator that determines the impact of the relationship between cases on the final classification result. If the length of the similar case list is set too long, then it is equivalent to strengthening the weak similarity between cases and weakening the strong similarity between cases. Furthermore, if the length of the similar case list is set too short, then it is equivalent to weakening the weak similarity between cases and strengthening the strong similarity between cases. After many experiments, we set the case similar list length to 50.


### 5.5. Experimental Results and Analysis

This section shows the superiority of the proposed RnRTD method for multiple accusations in legal cases relative to the baseline listed in Section5.2 and provides the corresponding analysis.Figure9 shows a series of experimental results based on Bi-LSTM. The abscissa indicates the number of batch iterations, and the ordinate indicates the accuracy of the multiaccusation judgment methods in legal cases. In contrast with the original Bi-LSTM method and Bi-LSTM with only the rdRNN layer, Bi-LSTM with only RTD achieves stable accuracy at the highest speed as the number of batches increases.Figure 9
Experimental results of RnRTD based on Bi-LSTM.The characteristics of RTD are important factors in the aforementioned phenomenon. On the basis of the restricted tensor η, RTD extracts the tensor elements and structure information that are most relevant to the multiaccusation judgment of legal cases from the original tensor. The weight of the vocabulary unrelated to a particular accusation is considerably weakened, and the weight of the vocabulary associated with a particular accusation is strengthened. The tensor dimension is greatly reduced, and the influence of irrelevant vocabulary on the classification algorithm is reduced. Subsequent neural network algorithms continuously iterate and optimize the restricted tensor and continuously adjust and correct the element values of the core tensor. RTD optimizes the original deep neural network algorithm from the lexical level.In Figure9, as the number of batches increases, the accuracy of Bi-LSTM with only the rdRNN layer becomes ultimately higher than that of Bi-LSTM with only the RTD layer. The reason is that rdRNN fully considers the similarity between different cases and has better discrimination for similar cases expressed by different language description methods. By setting the appropriate similar case list size, rdRNN fully considers cases that are similar to the current case and weighs their corresponding output states according to their similarity. rdRNN corrects and optimizes the original deep neural network from the case level.Figure10 shows the experimental results of the TextRNN-based RnRTD method. Similar to what is shown in Figure 9, TextRNN with only the RTD layer has the highest convergence speed as the number of batches increases compared with the original TextRNN and TextRNN with only the rdRNN layer.Figure 10
Experimental results of RnRTD based on TextRNN.The accuracy of the deep neural network method with only the rdRNN layer is not always higher than that with only the RTD layer. Although the rdRNN layer implements the correction and optimization of subsequent classification algorithms at the case level through the setting of similar case lists, the RTD layer also optimizes classification algorithms at the vocabulary level by setting the restricted tensor. Both methods achieve the final accuracy optimization but have different effects for various contexts. RnRTD combines the advantages of RTD and rdRNN to achieve rapid convergence and high classification accuracy.Table2 provides an experimental comparison of RnRTD methods based on multiple deep neural networks. RnRTD remarkably improves the classification accuracy of original neural networks for the classification of multiple accusations in legal cases. RTD and rdRNN layers also have considerable optimization effects on the original neural networks. RTD is applicable to all deep neural networks and can extract the main information carried by the data at the input layer to realize dimension reduction. rdRNN is an optimization strategy that is suitable for RNNs. It fully considers the similarity between cases within a certain period and optimizes algorithms at the case level. For algorithms based on convolutional neural networks, we remove the relational control gate in rdRNN while retaining the similar case list. Then, the optimization of these algorithms is realized by the rdRNN layer.Table 2
Performance of RnRTD methods based on multiple deep neural networks.
Optimization method
Basic algorithm
TextCNN
TextRNN
TextCNN attention
TextRNN attention
LSTM
Bi-LSTM
GRU
Original method
0.70
0.73
0.74
0.76
0.75
0.77
0.79
Only with RTD
0.73
0.77
0.79
0.82
0.80
0.81
0.83
Only with rdRNN
0.74
0.80
0.82
0.83
0.83
0.84
0.82
With RnRTD
0.80
0.85
0.85
0.90
0.91
0.93
0.91The convolutional neural network is less effective than RNN because legal case data are time series data. In addition, the attention layer only changes the encoding of the input and does not change the structure of neural networks. For the problem of judgment for multiple accusations in legal cases, the attention layer is still difficult to compensate for due to the lack of timing information of TextCNN and the gradient disappearance and gradient explosion of the TextRNN algorithm. From the perspective of the rdRNN layer, GRU has fewer adjustable parameters than LSTM and Bi-LSTM, and optimization on the restricted tensor is relatively limited. Therefore, the Bi-LSTM neural network with RnRTD performs better than other neural network algorithms.


## 5.1. Description of Experimental Data

We use nearly 1.8 million historical legal cases obtained from a Chinese refereeing study network. These legal cases involve more than 200 types of accusations, including theft, intentional assault, smuggling, fraud, and deliberate destruction of public property. Approximately 400,000 cases involve theft, and about 200,000 cases involve intentional assault. The number of accusations involved in each case ranges from 1 to 23.Figure8(a) shows the distribution of various accusations in the legal case data used in this study. The abscissa indicates the accusation index. For example, index 1 corresponds to bribery, and index 2 corresponds to rape. The ordinate indicates the proportion of cases involving the accusation that occupy the overall cases. Figure 8(a) shows that the number of cases involving theft is the highest in the database used in this article.Figure 8
Statistics on allegations in legal cases.
(a)
(b)Figure8(b) shows the distribution of the number of accusations involved in each case. The abscissa indicates the number of accusations involved in the case, and the ordinate indicates the proportion of cases in the corresponding number of accusations. Figure 8(b) shows that the highest number of accusations involves three cases.


## 5.2. Baseline Approaches

Given that multiaccusation judgment based on deep neural network and tensor decomposition is rarely studied, according to the limited tensor decomposition method RTD and the relation-based recurrent neural network rdRNN, we use the following method for a comparison with RnRTD proposed in this study:(i)
This study uses a series of deep neural networks based on convolutional or cyclic neural networks, including TextCNN, TextCNN attention, TextRNN, TextRNN attention, LSTM, and Bi-LSTM, as comparison methods for RnRTD.(ii)
This study uses deep neural networks with only the RTD tensor decomposition layer as comparison methods for RnRTD. Through experiments, we can derive the contribution of the RTD tensor decomposition layer to all deep neural networks.(iii)
This study uses neural networks with only the rdRNN method as comparison methods for RnRTD. Through these comparison experiments, we can derive the contribution of the relationship-based rdRNN strategy to all RNNs.


## 5.3. Data Preprocessing

In this study, the data preprocessing operation can be divided into two parts, namely, the modular representation of legal cases and the construction of the original tensor. Our legal case data preprocessing process can be described as follows:(i)
We organize each case in the legal case database into our preestablished case model, which divides the original case file into the defendant’s statement, the plaintiff’s statement, the content of the public prosecutions allegations, and the court’s judgment.(ii)
We filter and clean the contents of each module in the legal cases, extract the words that are meaningful to our multiaccusation judgment method, and filter out redundant words, stop words, noisy words, and modal particles.(iii)
We train the word-to-vector model to obtain the word vector of the aforementioned vocabulary. Then, we obtain a matrix representation of each case module and derive the tensor representation of the entire legal case.For Step (1), each case module may be spread across different paragraphs, and cases in different regions have different case descriptions. We extract and integrate them separately to arrive at a modular representation of the cases based on the description rules of case documents in each region. For Step (2), the extraction and filtering of the vocabulary in legal cases often requires professional legal background knowledge; otherwise, error filtering will occur. We filter words in legal case modules by using the legal professional vocabulary and the stop word list. For Step (3), word vectors are the basis for the accuracy of the entire deep neural network method. We use a number of Chinese corpus, such as corpus on Baidu Encyclopedia, Zhihu Questions and Answers, Sohu News, and Sina Weibo, to train the word-to-vector model.The tensor representation of legal cases and the subsequent deep neural network classification method require each case to have the same number of words, and the number of words in95% of the cases is below 300. Therefore, we perform a padding operation for cases where the number of words is less than 300. For cases with a vocabulary number greater than 300, we use the TF-IDF weight of the vocabulary to tailor the case vocabulary.


## 5.4. Experimental Hyperparameter Setting

This section describes the hyperparameter settings involved in our proposed method. These settings include the restricted tensorη and weight matrix Wη in RTD and the size of the similar case list in the rdRNN method (i.e., the size of list ∗ in equation (25)).The setting of restricted tensorη directly affects the convergence speed and accuracy of RnRTD. Our experiments show that a large rank of restricted tensor η corresponds to a high accuracy of the subsequent deep neural network algorithms. Conversely, a strong linear relationship between column or row vectors in η results in a low accuracy of the subsequent classification algorithms.In this study, weight matrixWη is used to scale the elements of the last mode in the original tensors. Wη adjusts the weights of certain words in the legal cases. For different accusations, the same vocabulary may have different weights in different types of cases. For example, derailment occupies a large and small weight in cases that involve bigamy and smuggling, respectively.The size of the similar case list in rdRNN is an important indicator that determines the impact of the relationship between cases on the final classification result. If the length of the similar case list is set too long, then it is equivalent to strengthening the weak similarity between cases and weakening the strong similarity between cases. Furthermore, if the length of the similar case list is set too short, then it is equivalent to weakening the weak similarity between cases and strengthening the strong similarity between cases. After many experiments, we set the case similar list length to 50.


## 5.5. Experimental Results and Analysis

This section shows the superiority of the proposed RnRTD method for multiple accusations in legal cases relative to the baseline listed in Section5.2 and provides the corresponding analysis.Figure9 shows a series of experimental results based on Bi-LSTM. The abscissa indicates the number of batch iterations, and the ordinate indicates the accuracy of the multiaccusation judgment methods in legal cases. In contrast with the original Bi-LSTM method and Bi-LSTM with only the rdRNN layer, Bi-LSTM with only RTD achieves stable accuracy at the highest speed as the number of batches increases.Figure 9
Experimental results of RnRTD based on Bi-LSTM.The characteristics of RTD are important factors in the aforementioned phenomenon. On the basis of the restricted tensor η, RTD extracts the tensor elements and structure information that are most relevant to the multiaccusation judgment of legal cases from the original tensor. The weight of the vocabulary unrelated to a particular accusation is considerably weakened, and the weight of the vocabulary associated with a particular accusation is strengthened. The tensor dimension is greatly reduced, and the influence of irrelevant vocabulary on the classification algorithm is reduced. Subsequent neural network algorithms continuously iterate and optimize the restricted tensor and continuously adjust and correct the element values of the core tensor. RTD optimizes the original deep neural network algorithm from the lexical level.In Figure9, as the number of batches increases, the accuracy of Bi-LSTM with only the rdRNN layer becomes ultimately higher than that of Bi-LSTM with only the RTD layer. The reason is that rdRNN fully considers the similarity between different cases and has better discrimination for similar cases expressed by different language description methods. By setting the appropriate similar case list size, rdRNN fully considers cases that are similar to the current case and weighs their corresponding output states according to their similarity. rdRNN corrects and optimizes the original deep neural network from the case level.Figure10 shows the experimental results of the TextRNN-based RnRTD method. Similar to what is shown in Figure 9, TextRNN with only the RTD layer has the highest convergence speed as the number of batches increases compared with the original TextRNN and TextRNN with only the rdRNN layer.Figure 10
Experimental results of RnRTD based on TextRNN.The accuracy of the deep neural network method with only the rdRNN layer is not always higher than that with only the RTD layer. Although the rdRNN layer implements the correction and optimization of subsequent classification algorithms at the case level through the setting of similar case lists, the RTD layer also optimizes classification algorithms at the vocabulary level by setting the restricted tensor. Both methods achieve the final accuracy optimization but have different effects for various contexts. RnRTD combines the advantages of RTD and rdRNN to achieve rapid convergence and high classification accuracy.Table2 provides an experimental comparison of RnRTD methods based on multiple deep neural networks. RnRTD remarkably improves the classification accuracy of original neural networks for the classification of multiple accusations in legal cases. RTD and rdRNN layers also have considerable optimization effects on the original neural networks. RTD is applicable to all deep neural networks and can extract the main information carried by the data at the input layer to realize dimension reduction. rdRNN is an optimization strategy that is suitable for RNNs. It fully considers the similarity between cases within a certain period and optimizes algorithms at the case level. For algorithms based on convolutional neural networks, we remove the relational control gate in rdRNN while retaining the similar case list. Then, the optimization of these algorithms is realized by the rdRNN layer.Table 2
Performance of RnRTD methods based on multiple deep neural networks.
Optimization method
Basic algorithm
TextCNN
TextRNN
TextCNN attention
TextRNN attention
LSTM
Bi-LSTM
GRU
Original method
0.70
0.73
0.74
0.76
0.75
0.77
0.79
Only with RTD
0.73
0.77
0.79
0.82
0.80
0.81
0.83
Only with rdRNN
0.74
0.80
0.82
0.83
0.83
0.84
0.82
With RnRTD
0.80
0.85
0.85
0.90
0.91
0.93
0.91The convolutional neural network is less effective than RNN because legal case data are time series data. In addition, the attention layer only changes the encoding of the input and does not change the structure of neural networks. For the problem of judgment for multiple accusations in legal cases, the attention layer is still difficult to compensate for due to the lack of timing information of TextCNN and the gradient disappearance and gradient explosion of the TextRNN algorithm. From the perspective of the rdRNN layer, GRU has fewer adjustable parameters than LSTM and Bi-LSTM, and optimization on the restricted tensor is relatively limited. Therefore, the Bi-LSTM neural network with RnRTD performs better than other neural network algorithms.


## 6. Discussion

In this study, we propose a new method for multiaccusation judgment in legal cases called RnRTD. RnRTD is a multilabel classification method based on tensor decomposition and RNNs. RnRTD consists of the tensor decomposition method with constraints and relation-driven RNN.We propose a tensor decomposition method with constraints, namely, RTD. We use this method to extract the tensor structure and element information that are most favorable to the subsequent classification algorithm from the original tensor that represents the legal case. RTD continuously corrects and optimizes the values of elements in the core tensor through the weight matrix and restricted tensor; hence, it continues to improve the classification accuracy of the neural network. RTD optimizes neural network classification algorithms at the lexical level. We also propose a relation-driven RNN strategy called rdRNN. Unlike traditional recurrent and LSTM neural networks, rdRNN sets up a new gating switch, that is, the similarity list window. It controls the impact of cases similar to the current case on the output status of the current case. rdRNN optimizes neural network classification algorithms at the case level.According to our experimental results, the RTD layer and the relation-driven cyclic neural network rdRNN have remarkable optimization effects on various deep neural network algorithms. However, no obvious relationship exists between the two. RTD and rdRNN have their own advantages in different contexts. In Figures9 and 10, the accuracy of rdRNN is higher than that of RTD. The accuracy of rdRNN is not always higher than that of RTD. RTD achieves stable accuracy the fastest as the number of batches increases in both figures. The reason is the principal component extraction and dimensionality reduction of RTD itself.RTD is suitable for almost all deep neural networks. It performs principal component extraction and dimensionality reduction on the original data at the input layer. It is similar to traditional principal component analysis methods, such as PCA [20] and SVD [21]. Several decomposition methods [22], such as Tucker and CP [23], are currently used for high-dimensional data. These methods extract the main elements and structural information of the matrix or tensor at the logical level according to the linear relationship of the elements in the matrix or tensor. However, the resulting new matrix or tensor structure is often unexplained. According to the traditional matrix or tensor decomposition method, supervising the completion of the principal component extraction work is difficult.The proposed restricted tensor provides interpretability for the tensor decomposition operation. Under the influence of the restricted condition tensor, RTD retains the information in the original tensor that is beneficial to the subsequent neural network and removes useless information. For the overall classification algorithm, RTD reduces the weight of weak correlation information and improves the influence of strong correlation information on the classification model. In addition, the subsequent deep neural network algorithm will continuously update and optimize the constraints in RTD to guide the core tensor to retain information that is conducive for the classification model. RTD optimizes the classification model at the vocabulary level by combining the weight matrix with the restricted tensor.rdRNN fully considers the similarities between cases and uses it as a factor that influences the output status of the current case. rdRNN optimizes the entire classification model at the case level. Generally, different regions may use different legal case description vocabularies, and rdRNN sets the output status of similar historical cases within a certain period as the reference value of the current case output state by setting a similar case window. Moreover, it sets the weight according to the similarity. RnRTD combines RTD and rdRNN to optimize the classification results from the perspective of case and vocabulary. When we use rdRNN to optimize algorithms based on the convolutional neural network, we remove the relationship control gate, retain only the similar case list in rdRNN, and realize the optimization operation of the rdRNN layer on the neural network.


## 7. Conclusion

In this study, we propose a new method for judging multiple crimes in legal cases, namely RnRTD. RnRTD consists of RTD and rdRNN. RTD is a tensor decomposition method with constraints. RTD decomposes the original tensor that represents a legal case into a core tensor under the guidance of restricted tensor. The resulting core tensor represents the main tensor structure and element information that is most favorable for improving the accuracy of subsequent classification algorithm. We propose the rdRNN algorithm and train it using obtained core tensors. rdRNN guides the tensor decomposition process in RTD by continuously optimizing the restricted tensor and finally makes RTD develop in the direction that is most beneficial to improve the classification accuracy of rdRNN.Nevertheless, this study has several problems. For example, even with the RTD tensor decomposition layer, algorithms based on RNNs usually run very slowly. In our future work, we will attempt to reduce the computational complexity of the algorithm and increase its speed.



---

*Source: 6705405-2019-07-07.xml*