# Legal Judgment Prediction via Heterogeneous Graphs and Knowledge of Law Articles
## 4 Experiments
### Experimental Results  
We chose four metrics (accuracy, macro-precision, macro-recall and macro-F1) to compare the performance of our model with the baseline model on the task. We used a multi-task approach for each baseline model to train the model and select the best set of parameters as the experimental results. All models were trained five times and we selected their average values as the data results used for comparison. Our method is noted as GFDN. Tables 4 and 5 show the experimental results on the datasets CAIL-big and CAIL-small, respectively. It can be concluded that our method was better than the baseline models in all metrics. Compared with LADAN, which is the best performing baseline model, the performance of our model on the CAIL-big dataset was improved by 0.1%, 0.02%, 0.14% and 0.1% and by 0.81%, 0.78%, 0.29% and 0.75% in the CAIL-small dataset, respectively, in terms of accuracy, precision, recall and F1-value in the charge prediction task. In the law article prediction task, our model improved by 0.02%, 0.03%, 0.02% and 0.04% on the CAIL-big dataset and by 0.2%, 0.09%, 0% and 0.03% on the CAIL-small dataset, respectively; in the term of penalty prediction task, our model improved by 0.02%, 0.01%, 0.03% and 0.02% on the CAIL-big dataset and by 0.23%, 0.05%, 0.13% and 0.24% on the CAIL-small dataset, respectively. The main reasons why our model achieved excellent experimental results can be attributed to two points. First, the six kinds of text graphs not only could use the GCN well to obtain text-level feature information but could also obtain inferred information that traditional text feature extraction methods cannot obtain. The graph structure has a natural advantage in inference. Second, the proposed law article distinction extractor could well extract the differences of similar law articles and could also well integrate the information with six kinds of graphs in the graph fusion step. From the analysis in Table 5, we can see that our model also showed excellent performance when dealing with small-sample datasets. The model showed a significant improvement over the LADAN model on CAIL-small. The main reason is that traditional neural network methods require a large number of data as the basis. Our method is based on a graph neural network, which has some advantages in dealing with small-sample datasets according to the experimental results. In predicting the law articles on CAIL-small, our method did not achieve optimal results. The possible reason is that the law articles distinction extractor hurt the model's performance in predicting law articles in a dataset with few samples. In addition, we could also infer, from the two tables, that the number of data in LJP directly determined the performance of the task (all models performed much better on CAIL-big than CAIL-small). The main reason is that the performance of a deep learning model depends heavily on the number of training data and the same is true for the LJP task. \begin{table}
\begin{tabular}{l c} \hline \hline
**Software Name** & **Parameter Description** \\ \hline Operating system & Ubuntu 16.04 \\ Development tool & Pycharm 2020 \\ Version of Python & 3.8 \\ Version of Pytorch & 1.7.0 \\ Version of DGL & 0.3 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Software environment. Since the data gap between Tables 4 and 5 is relatively small, we conducted statistical significance tests and the results are shown in Figure 4. We assumed the null hypothesis as no significant differences between the two groups of data and the alternative hypothesis as a significant difference. As shown in Figure 4, the red dots are the distribution of the samples. The t value fell in the rejection domain, so we rejected the null hypothesis and accepted the alternative hypothesis that there was a significant difference between the two data groups. Then, the Wilcoxon test was also performed in our experiment. In the CAIL-big dataset, we assumed that there were no differences between the experimental results of GFDN and LADAN.