SlowGuess/NAACL_2021_OCR huggingface dataset:
Zach Wood-Doughty, Ilya Shpitser, and Mark Dredze. 2018. Challenges of using text classifiers for causal inference. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, volume 2018, page 4586. NIH Public Access.
S. Wright. 1921. Correlation and causation. Journal of Agricultural Research, 20:557-585.
Chaojun Xiao, Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Zhiyuan Liu, Maosong Sun, Yansong Feng, Xianpei Han, Zhen Hu, Heng Wang, et al. 2018. Cait2018: A large-scale legal dataset for judgment prediction. arXiv preprint arXiv:1807.02478.
Nuo Xu, Pinghui Wang, Long Chen, Li Pan, Xiaoyan Wang, and Junzhou Zhao. 2020. Distinguishing law articles for legal judgment prediction. arXiv preprint arXiv:2004.02557.
Liuyi Yao, Sheng Li, Yaliang Li, Mengdi Huai, Jing Gao, and Aidong Zhang. 2018. Representation learning for treatment effect estimation from observational data. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montreal, Canada, pages 2638-2648.
Liuyi Yao, Sheng Li, Yaliang Li, Hongfei Xue, Jing Gao, and Aidong Zhang. 2019. On the estimation of treatment effect with text covariates. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pages 4106-4113. AAAI Press.
Jiji Zhang. 2008. Causal reasoning with ancestral graphs. Journal of Machine Learning Research, 9(Jul):1437-1474.
Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Chaojun Xiao, Zhiyuan Liu, and Maosong Sun. 2018. Legal judgment prediction via topological learning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3540-3549.
Haoxi Zhong, Yuzhong Wang, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. 2020. Iteratively questioning and answering for interpretable legal judgment prediction. AAAI.
Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. 2016. Attention-based bidirectional long short-term memory networks for relation classification. In Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: Short papers), pages 207-212.
A GFCI Algorithm
GFCI (Ogarrio et al., 2016) is a combination of a constraint-based causal discovery algorithm FCI (Spirtes et al., 2013) and a score-based algorithm FGES (Ramsey et al., 2016). Based on the skeleton of FCI, it uses initialization from FGES to improve accuracy and efficiency.
FCI takes sample data and optional background knowledge as input, and guarantees to represent the Markov equivalence class of the true causal DAG. It has two phases: adjacency and orientation. It does not rely on no latent confounder assumption, but it performs relatively poorly, especially on real data.
On the other hand, FGES greedily searches over potential DAGs, and outputs the highest scoring graph it finds. It is fast and accurate when its assumptions are satisfied, but it relies on the condition that there are no latent confounders.
GFCI takes the output of FGES as an initialized graph, and the graph is further augmented by FCI's adjacency phase, in which some adjacencies are removed by conditional independence tests. A similar process happens in the orientation phase. Orientations of FGES are provided as initialization, and further orientations from FCI are applied. Proof in Ogarrio et al. (2016) guarantees the GFCI algorithm outputs a PAG that represents the true causal DAG.
Figure 6 shows the advantage of GFCI. When factor  $A$  is unobserved, score-based algorithms like GES (Chickering, 2002) will wrongly discover edge  $B \rightarrow D$ , while GFCI correctly recognizes the relation between factors  $B$  and  $D$ : there is an unobserved confounder of  $B$  and  $D$ .
When the training set ratio is  $1\%$ , we select  $p = 15$  keywords for each charge, and cluster the keywords of both charges into  $q = 20$  factors; for the training set ratio  $5\%$  and  $10\%$ ,  $p = 25$  and  $q = 30$ ; for the training set ratio  $30\%$  and  $50\%$ ,  $p = 40$  and  $q = 60$ . We use K-means (MacQueen et al., 1967) for clustering. When learning causal relationship, besides cases in the training set, we also use some unlabeled cases of other charges to improve the interpretability of the causal graph. Note that removing these cases will not hurt the performance. We sample  $Q = 5$  causal graphs for each PAG.
For the neural network based models (both the baselines and our proposed models),  $10\%$  of the
(a) True causal graph
Figure 6: Comparison between the output of GES and GFCI.  $A$  is a latent variable, which is also an unobserved confounder of  $B$  and  $D$ . This implies that in fact there is no causal relationship between  $B$  and  $D$  but without the consideration of unobserved confounders, GES tends to be awry in this situation.
(b) GES output
(c) GFCI output
training set is used as the validation set. The models are trained for 30 epochs, and the early stopping mechanism takes effect when the validation loss does not drop for more than 1000 batches. We use Adam optimizer with learning rate 0.001, and the dropout rate is 0.5. We use Tencent AILab Chinese Word Embedding (Song et al., 2018) as word embeddings, and the dimension of each word is 200. The hidden sizes are  $b_{0} = 128$  and  $b_{1} = 64$ . The batchsize is selected from \{4, 8, 16, 32, 64, 128\} considering the dataset size and training set ratio. For Bi-LSTM+Att+Cons,  $\alpha$  is manually tuned within \{0.1, 0.25, 0.5, 1\}, selecting the one with best validation F1. All the hyperparameters are empirically selected and kept the same for different models in the same dataset and the same training set ratio.
To reduce the impact of data imbalance between charges, we apply data augmentation to smaller charges whose cases are 3 times fewer than the biggest charge in that set, regardless of the training ratio. We did not keep them 1:1, in order to reflect their distributions in the real world.
The models are trained on Intel Xeon CPU. It takes about one hour to construct the causal graphs, and a few minutes to perform inference.
C Example of Generated PAG
In this section, we further show part of a Partial Ancestor Graph (PAG) generated by our  $GCI$  framework. The example is shown in Figure 7.
For the edge between Hold People and Hostage, the arrow is  $\rightarrow$ , indicating there is a causal relationship between the factors Hold People and Hostage (the person who was held might become a hostage).
For the edge between Hold Knife and Hold People, the arrow is  $\circ \rightarrow$ , which means there might be an unobserved confounder which causes the two factors, and there might exist causal relationship
Charge Sets	Chains
Personal Injury	trivia → dispute → beat → injury → intentional injury
emotion → disagreement → kill → death → murder
driving a truck → reverse → crush → death → involuntary manslaughter
Violent Acquisition	hold a knife → violence → cash → robbery
necklace → pull → seizure
hostage → threaten → ask for → ransom → kidnapping
F&E	approach → trust → lie → fraud
forge → defraud → fraud
relationship → nude photos → threaten → extortion
E&MPF	work → take advantage of (position/power) → embezzlement
falsely report → arbitrage → embezzlement
take charge of → take advantage of (position/power)→ misappropriate → profit
→ misappropriation of public funds
AP&DD	(use) position → (provide) convenience → abuse of power
complete understand → violate → abuse of power
ignore → result in (bad things) → dereliction of duty
Table 6: Exhibition of causal chains for each charge set.
between the two factors. In the first possible situation, the confounder could be Latent1, the intent to commit a crime. A person who intends to commit a crime may hold a knife and hold people. And in the second possible situation, the person who holds a knife is likely to hold people. Therefore, there is uncertainty in predicting the causal relationship.
For the edge between Hostage and Family, the arrow is  $\longleftrightarrow$ , which means there is an unobserved confounder which causes the two factors, and there does not exist causal relationship between them. For example, the confounder could be Latent2, the intent to obtain properties from the hostage. The intent could not only make the suspect grab the hostage's money, but also contact families to ask for ransom.
D Exhibition of More Causal Chains
Table 6 showcases three chains for each charge set. They depict common patterns of behaviours of defendants that are charged with the crime at the tail of the chains. Note that each chain merely describes one possibility and one aspect of the criminal behaviour, and the chain itself may not be sufficient to initiate such a lawsuit. For example, the chain complete understand  $\rightarrow$  violate  $\rightarrow$  abuse of power of charge set AP&DD shows that the litigant deliberately violated the rules, but he/she will be charged with abuse of power only when his/her action results in major loss of public property.
Figure 7: An example of part of the generated PAG of charge kidnapping in charge set Violent Acquisition. Latent1 and Latent2 are unobserved variables that do not exist in the generated PAG. Detailedly, Latent1 may denote the suspect's intent to commit a crime; and Latent2 may denote the suspect's intent to obtain properties from the hostage.