# Legal Judgment Prediction via Heterogeneous Graphs and Knowledge of Law Articles
## 4 Experiments
### Experimental Results
Based on the calculated \(p\)-value (\(p=0.003857<0.005\)), the original hypothesis was rejected; therefore, it can be concluded that there was a significant difference between the two sets of experimental data. Both sets of experiments illustrated that our method was significantly better than the LADAN in terms of performance. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline  & \multicolumn{4}{c}{**Law Articles**} & \multicolumn{4}{c}{**Charges**} & \multicolumn{4}{c}{**Term of Penalty**} \\ \cline{2-13}  & **Acc.** & **MP** & **MR** & **F1** & **Acc.** & **MP** & **MR** & **F1** & **Acc.** & **MP** & **MR** & **F1** \\ \hline TFIDF + SVM & 89.93 & 68.56 & 60.58 & 61.25 & 85.81 & 69.76 & 61.92 & 63.51 & 54.13 & 39.15 & 37.62 & 39.14 \\ FLA & 93.22 & 72.81 & 64.27 & 66.57 & 92.48 & 76.21 & 68.12 & 69.97 & 57.66 & 49.01 & 44.87 & 46.62 \\ CNN & 95.79 & 82.79 & 75.15 & 76.62 & 95.23 & 86.57 & 78.93 & 81.02 & 55.41 & 45.23 & 38.73 & 39.96 \\ RCNN & 95.98 & 82.93 & 75.26 & 77.13 & 95.50 & 87.89 & 79.03 & 81.65 & 55.62 & 45.43 & 38.88 & 40.17 \\ HARNN & 96.01 & 82.99 & 75.58 & 77.38 & 95.62 & 87.93 & 79.27 & 81.79 & 56.11 & 44.21 & 40.57 & 41.87 \\ TOPJUDGE & 95.81 & 84.41 & 74.36 & 76.67 & 95.73 & 87.99 & 79.49 & 81.93 & 57.29 & 47.35 & 42.61 & 44.03 \\ GCN & 95.69 & 84.24 & 74.22 & 76.58 & 95.60 & 87.89 & 79.28 & 81.82 & 57.2 & 47.17 & 42.53 & 43.91 \\ MPBFN & 96.01 & 84.83 & 74.64 & 77.48 & 95.93 & 89.25 & 80.82 & 83.06 & 58.04 & 45.95 & 39.01 & 41.49 \\ LADAN & 96.49 & 85.71 & 80.21 & 81.35 & 96.32 & 88.03 & 82.98 & 84.54 & 59.52 & 51.83 & 45.2 & 46.96 \\ \hline
**GFDN** & **96.51** & **85.74** & **80.23** & **81.39** & **96.42** & **88.05** & **83.12** & **84.64** & **59.54** & **51.84** & **45.21** & **46.99** \\ \hline \end{tabular}
\end{table} 
Table 4: IJP results on dataset CAIL-big. Figure 4: Statistical significance test results between GFDN and LADAN. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline  & \multicolumn{4}{c}{**Law Articles**} & \multicolumn{4}{c}{**Charges**} & \multicolumn{4}{c}{**Term of Penalty**} \\ \cline{2-13}  & **Acc.** & **MP** & **MR** & **F1** & **Acc.** & **MP** & **MR** & **F1** & **Acc.** & **MP** & **MR** & **F1** \\ \hline TFIDF + SVM & 76.52 & 43.21 & 40.12 & 39.68 & 79.81 & 45.86 & 42.72 & 42.77 & 33.32 & 27.66 & 24.99 & 24.64 \\ FLA & 77.72 & 75.21 & 74.12 & 72.78 & 80.98 & 79.11 & 77.92 & 76.77 & 36.32 & 30.81 & 28.22 & 27.83 \\ CNN & 78.61 & 75.86 & 74.6 & 73.59 & 82.23 & 81.57 & 79.73 & 78.82 & 35.2 & 32.96 & 29.09 & 29.68 \\ RCNN & 79.12 & 76.58 & 75.13 & 74.15 & 82.50 & 81.89 & 79.72 & 79.05 & 35.52 & 33.76 & 30.41 & 30.27 \\ HARNN & 79.73 & 75.05 & 76.54 & 74.67 & 83.41 & 82.23 & 82.27 & 80.79 & 35.95 & 34.5 & 31.04 & 31.18 \\ TOPJUDGE & 79.79 & 79.52 & 73.39 & 73.33 & 82.03 & 83.14 & 79.33 & 79.03 & 36.05 & 34.54 & 32.49 & 29.19 \\ GCN & 79.81 & 79.65 & 73.42 & 73.37 & 82.33 & 83.19 & 79.20 & 78.97 & 35.97 & 34.66 & 32.54 & 29.23 \\ MPBFN & 79.14 & 76.03 & 71.1 & 72.21 & 82.16 & 83.15 & 80.82 & 80.06 & 35.76 & 31.72 & 28.31 & 29.56 \\ LADAN & 81.17 & 77.92 & 72.43 & 73.79 & 84.02 & 83.18 & 82.08 & 81.04 & 38.05 & 35.97 & 32.15 & 32.33 \\ \hline
**GFDN** & **81.37** & **78.01** & **72.43** & **73.82** & **84.83** & **83.96** & **82.37** & **81.79** & **38.28** & **36.02** & **32.28** & **32.57** \\ \hline \end{tabular}
\end{table} 
Table 5: IJP results on dataset CAIL-small. In the CAIL-big dataset, there is data imbalance. We plotted the ROC curves and show the corresponding AUC scores because they considers the classification ability for both positive and negative cases. It could still make a reasonable classifier evaluation in such a case. We selected five models from the baseline to compare the AUC with our model GFDN. From Figure 5, we can see that the FLA model had the lowest AUC score because its semantic extractor was not powerful enough to extract the document information fully. Although FLA was designed for the LJP task, it had a lower AUC score than the HARNN model.