SlowGuess/ACL_2020_OCR huggingface dataset:
Xu Peng and Barbosa Denilson. 2019. Connecting language and knowledge with heterogeneous representations for neural relation extraction. *NAACL*.
Maria Pershina, Bonan Min, Wei Xu, and Ralph Grishman. 2014. Infusion of labeled data into distant supervision for relation extraction. ACL.
Sebastian Riedel, Limin Yao, , and Andrew D McCallum. 2010. Modeling relations and their mentions without labeled text. ECML/PKDD.
Andrew Slavin Ross, Michael C. Hughes, and Finale. 2017. Right for the right reasons: Training differentiable models by constraining their explanations. *IJCAI*.
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2019. Grad-cam: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision.
Avanti Shrikumar, Peyton Greenside, Anna Shcherbina, and Anshul Kundaje. 2016. Not just a black box: Learning important features through propagating activation differences.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2012. Deep inside convolutional networks: Visualising image classification models and saliency maps. CoRR, abs/1312.6034.
Shikhar Vashishth, Rishabh Joshi, Sai Suman Prayaga, Chiranjib Bhattacharyya, and Partha Talukdar. 2018. Reside: Improving distantly-supervised neural relation extraction using side information. EMNLP.
Guanying Wang, Wen Zhang, Ruoxu Wang, Yalin Zhou, Xi Chen, Wei Zhang, Hai Zhu, and Huajun Chen. 2018. Label-free distant supervision for relation extraction via knowledge graph embedding. EMNLP.
Sarah Wiegrefe and Yuval Pinter. 2019. Attention is not not explanation. NAACL.
Daojian Zeng, Joel R. Tetreault, Kang Liu, Yubo Chen, and Jun Zhao. 2015. Distant supervision for relation extraction via piecewise convolutional neural networks. EMNLP.
Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. Ernie: Enhanced language representation with informative entities. ACL.
6493
A.1 Saliency and (Gradient  $\times$  input)
Assume that a neural model outputs a logit score  $o$  which is a differentiable function and parameterized by  $x \in \mathbb{R}^d$ ,  $\theta$  and etc. The Taylor series of the given function  $o$  near input  $a$  is given by:
$$
o (x) = o (a) + \frac {\partial o}{\partial x} (a) (x - a) + \frac {1}{2 !} \frac {\partial o ^ {2}}{\partial x ^ {2}} (a) (x - a) ^ {2} + \dots . \tag {9}
$$
Approximating the function  $o$  as a linear function, the first order approximation of the Taylor series is given by:
$$
o (x) \approx \frac {\partial o}{\partial x} (a) x + b \tag {10}
$$
Note that  $\frac{\partial o}{\partial x}(a) \in \mathbb{R}^d$ . Therefore for each dimension  $i$  the bigger  $\frac{\partial o}{\partial x}(a)[i]$ , the more (positive or negative) the impact of  $a[i]$  is on  $o$ . The whole impact of  $a$  on  $o$  is given by  $\sum_{i} \frac{\partial o}{\partial x}(a)[i]$  of its absolute value  $\sum_{i} |\frac{\partial o}{\partial x}(a)[i]|$ .
Regarding our task, the logit score of the model for a relation  $k$  is  $o_k$ . For a given sentence  $x_n$ , the amount of positive or negative impact of  $x_n$  on  $o_k$  is approximated by  $\sum_i \left| \frac{\partial o_k}{\partial x}(x_n)[i] \right|$  which is saliency.
The (Gradient  $\times$  input) for a given sentence  $x_{n}$  is equivalent to the linear approximation of  $o_k$  at  $x_{n}$  which is  $\sum_{i}x_{n}[i]\times \frac{\partial o_{k}}{\partial x} (x_{n})[i]$ .
A.2 Ground-truth explanation set.
We annotate the positive bags of the test split of FB-NYT with ground-truth explanations. There are 1950 bags and 6444 sentences. For each pair of (sentence, relation) in a bag, the sentence is either a rationale (supportive) to the relation or it is irrelevant. For example:
Following the annotation of the sentence-relation contributions which is either rationale or irrelevant, we extract a set "expl-eval" (which is going to be used to evaluate the explanation quality of the models) as follows:
exp1-eval  $=$  set()
For each (bag-id, bag):
For each relation label  $k$  given to the bag:
For each pair of rationale  $s^+$  an irrelevant  $s^-$  for  $k$ :
exp1-eval.add((bag-id，k，  $s^+$  ，  $s^{-}$  ）
The size of the generated expl-eval is 1097 tuples of (bag-id,  $k$ , rationale sentence, irrelevant sentence). Please note that the relation label  $k$  is one of the ground-truth labels assigned to bag-id.
6494