SlowGuess/NAACL_2021_OCR huggingface dataset:
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Bill MacCartney and Christopher D Manning. 2014. Natural logic and natural language inference. In Computing meaning, pages 129-147. Springer.
Mark A Musen and Johan Van Der Lei. 1988. Of brittleness and bottlenecks: Challenges in the creation of pattern-recognition and expert-system models. In Machine Intelligence and Pattern Recognition, volume 7, pages 335-352. Elsevier.
Allen Newell and Herbert Simon. 1956. The logic theory machine-a complex information processing system. IRE Transactions on information theory, 2(3):61-79.
Adarsh Prasad, Stefanie Jegelka, and Dhruv Batra. 2014. Submodular meets structured: Finding diverse subsets in exponentially-large structured item sets. In Advances in Neural Information Processing Systems, volume 27, pages 2645-2653. Curran Associates, Inc.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67.
Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain yourself! leveraging language models for commonsense reasoning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4932-4942.
Kyle Richardson, Hai Hu, Lawrence Moss, and Ashish Sabharwal. 2020. Probing natural language inference models through semantic fragments. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8713-8721.
Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5):206-215.
Swarnadeep Saha, Sayan Ghosh, Shashank Srivastava, and Mohit Bansal. 2020. Pover: Proof generation for interpretable reasoning over rules. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 122-136.
Tianxiao Shen, Myle Ott, Michael Auli, and Marc'Aurelio Ranzato. 2019. Mixture models for diverse machine translation: Tricks of the trade. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 5719-5728, Long Beach, California, USA. PMLR.
Raphael Shu, Hideki Nakayama, and Kyunghyun Cho. 2019. Generating diverse translations with sentence codes. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1823-1827.
Oyvind Tafjord, Matt Gardner, Kevin Lin, and Peter Clark. 2019. QuaRTz: An open-domain dataset of qualitative relationship questions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5941-5946, Hong Kong, China. Association for Computational Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008.
Ashwin K. Vijayakumar, Michael Cogswell, Ramprasaath R. Selvaraju, Qing Sun, Stefan Lee, David J. Crandall, and Dhruv Batra. 2018. Diverse beam search: Decoding diverse solutions from neural sequence models. In Proceedings of the AAAI Conference on Artificial Intelligence.
O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. 2015. Show and tell: A neural image caption generator. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3156-3164.
Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merrienboer, Armand Joulin, and Tomas Mikolov. 2015. Towards AI-complete question answering: A set of prerequisite toy tasks. arXiv preprint arXiv:1502.05698.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumont, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45.
Zhengnan Xie, Sebastian Thiem, Jaycie Martin, Elizabeth Wainwright, Steven Marmorstein, and Peter Jansen. 2020. Worldtree v2: A corpus of science-domain structured explanations and inference patterns supporting multi-hop inference. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 5456-5473.
Qiongkai Xu, Juyan Zhang, Lizhen Qu, Lexing Xie, and Richard Nock. 2018. D-page: Diverse paraphrase generation. arXiv preprint arXiv:1808.04364.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369-2380, Brussels, Belgium. Association for Computational Linguistics.
Qinyuan Ye, Xiao Huang, Elizabeth Boschee, and Xi-ang Ren. 2020. Teaching machine comprehension with compositional explanations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 1599-1615.
Mo Yu, Shiyu Chang, Yang Zhang, and Tommi Jaakkola. 2019. Rethinking cooperative rationalization: Introspective extraction and complement control. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4085-4094.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. 2017. Deep sets. In Advances in Neural Information Processing Systems, volume 30, pages 3391-3401. Curran Associates, Inc.
Omar Zaidan, Jason Eisner, and Christine Piatko. 2007. Using "annotator rationales" to improve machine learning for text categorization. In Human language technologies 2007: The conference of the North American chapter of the association for computational linguistics; proceedings of the main conference, pages 260-267.
Luke S. Zettlemoyer and Michael Collins. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, UAI'05, page 658-666. AUAI Press.
Hongming Zhang, Xinran Zhao, and Yangqiu Song. 2020. WinoWhy: A deep diagnosis of essential commonsense knowledge for answering Winograd schema challenge. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5736-5745, Online. Association for Computational Linguistics.
Yan Zhang, Jonathon Hare, and Adam Prugel-Bennett. 2019a. Deep set prediction networks. In Advances in Neural Information Processing Systems, volume 32, pages 3212-3222. Curran Associates, Inc.
Yan Zhang, Jonathon Hare, and Adam Prügel-Bennett. 2019b. Fspool: Learning set representations with featurewise sort pooling. In International Conference on Learning Representations.
QA	Node	Edge	Proof	FA
P	R	F1	P	R	F1	P	R	F1
PROVER (Saha et al., 2020)	99.3	90.0	85.3	86.4	88.6	85.6	86.2	88.0	84.7	85.5	82.1
PROVER-all	99.4	88.4	84.2	85.3	87.9	84.4	85.4	86.5	83.5	84.3	81.1
PROVER-top-p	99.4	34.4	88.6	48.4	34.0	88.0	48.0	33.4	87.3	47.4	00.0
PROVER-top-p-classifier	99.4	86.2	85.1	84.4	85.6	85.1	84.4	84.6	84.2	83.4	78.2
PROVER-top-p-threshold	99.4	85.0	88.4	85.6	84.5	87.8	85.2	83.9	87.1	84.5	78.0
ML-MULTIPROVER	99.3	89.9	89.6	89.3	88.3	88.3	88.0	87.7	87.6	87.3	83.9
IT-MULTIPROVER-seq	99.4	88.5	86.8	87.2	87.4	86.0	86.3	86.7	85.3	85.6	81.4
IT-MULTIPROVER-nec	99.3	90.2	89.7	89.5	89.4	89.1	89.0	89.0	88.6	88.5	85.2
IT-MULTIPROVER	99.5	90.6	90.5	90.1	89.9	90.0	89.5	89.4	89.4	89.0	85.3
Table 5: Comparison of our final MULTIPROVER models with variants of PROVER and other ablations of IT-MULTIPROVER on the validation set of DU5. IT-MULTIPROVER-seq = Iterative-MULTIPROVER with sequential loss. IT-MULTIPROVER-nec = Iterative-MULTIPROVER with no edge conditioning. The final Iterative-MULTIPROVER model outperforms all other models across all metrics.
MULTIPROVER is developed on top of the Hugging Face transformers library (Wolf et al., 2020).<sup>3</sup> Experiments with PROVER (Saha et al., 2020) are performed using their publicly released code and hyperparameters.<sup>4</sup> All MULTIPROVER hyperparameters are chosen based on the best Full Accuracy on the corresponding validation sets. We use RoBERTa-large (Liu et al., 2019) as the pre-trained language model. The batch size and maximum sequence length are set to 8 and 300 respectively. We train all our models for a maximum of 7 epochs using an initial learning rate of  $10^{-5}$ , a weight decay of 0.1 and a dropout probability of 0.1. We use a random seed of 42 across all our experiments. All experiments are performed on one V100 Volta GPU. Batch size and learning rate are manually tuned in the range  $\{8,16\}$  and  $\{10^{-5}, 2*10^{-5}\}$  respectively. For inference, we use PROVER's ILP optimization code, which is modeled using PuLP.<sup>5</sup> In all the datasets, the maximum number of facts and rules corresponding to a context is 25.
Our experiments are conducted on the datasets introduced in Clark et al. (2020). These consist of 5 datasets with synthetic rule-bases, DU0-DU5, a zero-shot test-only dataset called Birds-Electricity and a dataset with human-paraphrased rules called ParaRules. All datasets, except Birds-Electricity,
have their corresponding train, validation and test splits.
DU0-DU5: Each of these consists of 100k questions with synthetic rule-bases and requires reasoning chains up to a maximum depth of  $D$  ( $D = 0, 1, 2, 3, 5$ ). The number of train, validation and test examples in each of the datasets are 70k, 10k and 20k respectively. Further, each question in the datasets is annotated with all possible proofs. The total number of proofs in the DU5 train set range from 1 to 1350, with a mean and median of 1.45 and 1 respectively.
Birds-Electricity: The Birds-Electricity dataset comprises of two test-only datasets where the contexts are about birds and electric circuits. The vocabulary of the entities, attributes and predicates, apart from is() are all new at test time, thus providing a benchmark for testing the generalization capability of the models on out-of-distribution data. Another interesting aspect of this dataset is that all examples are annotated with a unique gold proof.
ParaRules: The ParaRules dataset is one where the facts and rules are paraphrased by humans into more natural language. It consists of a total of 40k questions, with 28k, 4k, and 8k questions in the train, validation and test splits respectively. This dataset tests the model's ability to reason over more complex human-like language. Similar to the synthetic datasets, each example is annotated with all possible proofs.
A.3 Syntax of Proof Graph
Each proof  $\mathcal{P}_i = (\mathcal{V}_i, \mathcal{E}_i)$  is a directed graph, with a set of nodes  $\mathcal{V}_i \subseteq \mathcal{N}$  and a set of edges  $\mathcal{E}_i \subseteq \mathcal{V}_i \times \mathcal{V}_i$ . Each node  $n_i \in \mathcal{N}$  is either a fact  $F \in \mathcal{F}$  or a rule
$^{3}$ https://github.com/huggingface/ transformers  
 $^{4}$ https://github.com/swarnaHub/PRover  
 $^{5}$ https://pypi.org/project/PuLP/  
 $^{6}$ https://rule-reasoning.apps.allenai.org/
p	QA	Node	Edge	Proof
P	R	F1	P	R	F1	P	R	F1	PA
2	99.4	90.5	89.1	89.2	89.3	88.3	88.4	88.8	87.8	87.9	84.4
3	99.3	89.9	89.6	89.3	88.3	88.3	88.0	87.7	87.6	87.3	83.9
4	99.3	89.1	89.2	88.8	87.8	82.0	87.8	87.2	87.5	87.1	83.6
5	99.2	88.6	89.1	88.5	87.2	87.8	87.2	86.6	87.2	86.6	83.1
Table 6: Effect of varying maximum number of proofs (p) on Multilabel-MULTIPROVER. All models are trained on the DU5 training set and evaluated on the corresponding validation set. The proof metrics start to decrease marginally with increase in p.
p	QA	Node	Edge	Proof
P	R	F1	P	R	F1	P	R	F1	PA
2	99.5	90.0	89.0	89.0	89.2	88.4	88.3	88.6	87.8	87.7	84.1
3	99.5	90.6	90.5	90.1	89.9	90.0	89.5	89.4	89.4	89.0	85.3
4	99.5	90.2	89.7	89.5	89.5	89.2	89.1	89.1	88.7	88.6	85.2
5	99.5	90.1	89.6	89.4	89.5	89.2	89.1	89.0	88.6	88.5	85.2
Table 7: Effect of varying maximum number of proofs (p) on Iterative-MULTIPROVER. All models are trained on the DU5 training set and evaluated on the corresponding validation set. Unlike Multilabel-MULTIPROVER, it is significantly robust to variation in p.
$R \in \mathcal{R}$  from the context or a special NAF node, denoting "Negation as Failure". A NAF node in a proof indicates the truthfulness of the negation of statement(s) that cannot be proved using the set of rules (under closed-world assumption). Edges in the graph can be directed either from a fact (or NAF) to a rule or between two rules. An edge from a fact to a rule means that the rule applies on the fact to generate a new fact. Similarly, an edge from a rule  $R_{1} \in \mathcal{R}$  to another rule  $R_{2} \in \mathcal{R}$  implies the application of  $R_{2}$  on the fact generated by  $R_{1}$ . Proofs are either successful or failed. A successful proof is one where the question statement can be logically reached (to be either proved or disproved) using the given rule-base while for failed proofs, no conclusion can be reached, in which case the shallowest branch of the proof tree that fails is generated. For more details and examples of proofs, we refer the readers to prior work (Saha et al., 2020; Clark et al., 2020).
A.4 Ablation Analysis
In Table 5, we compare our baselines PROVER, PROVER-all and PROVER-top-  $p$  variants with our MULTIPROVER models on the validation set of DU5 dataset. Additionally, we also show two ablations of IT-MULTIPROVER - in the first, we replace the Hungarian loss with a sequential loss, which computes the cross-entropy loss of the  $i^{th}$  predicted proof with the  $i^{th}$  gold proof and in the second, we condition the node embeddings on the
previous node embeddings only instead of both node and edge embeddings. All models, except PROVER and PROVER-all, generate a maximum of 3 proofs. PROVER-top- $p$  suffers from a huge drop in proof precision due to the generation of many incorrect proofs. Although carefully choosing the value of  $p$  either by thresholding or through a classifier helps boost the proof precision, PROVER continues to be a superior baseline on this dataset due to a high skew towards single-proof examples. ML-MULTIPROVER improves upon PROVER's proof F1 and full accuracy (FA) which are further bettered by IT-MULTIPROVER, owing to its explicit conditioning mechanism between the proofs. Replacing the Hungarian loss with a sequential loss leads to a significant drop in proof F1, thereby showing the effectiveness of modeling multiple proof generation as a set generation problem. Finally, conditioning the node embeddings on both node and edge embeddings leads to marginal improvement in proof F1. Overall, IT-MULTIPROVER outperforms all other models across all metrics.
A.5 MULTIPROVER with Varying Maximum Number of Proofs
We analyze the effect of varying the maximum number of proofs  $p$  on ML-MULTIPROVER and IT-MULTIPROVER in Table 6 and 7 respectively. All models are trained on the DU5 training set and evaluated on the corresponding validation set. Although all models maintain the QA accuracy, we
Node	Edge	Proof	FA
QA	P	R	F1	P	R	F1	P	R	F1
PROVER-all	98.6	95.9	94.1	94.5	95.4	93.8	94.3	95.3	93.7	94.2	92.3
PROVER-top-p	98.6	39.3	96.6	55.0	38.9	96.0	54.6	38.9	95.9	54.5	00.1
ML-MULTIPROVER	98.9	96.7	96.4	96.4	96.4	96.2	96.2	96.2	96.0	96.0	95.2
IT-MULTIPROVER	98.9	97.3	97.2	97.2	97.2	97.0	97.0	96.8	96.7	96.7	96.1
Table 8: Comparison of models trained on DU3 and ParaRules training sets and evaluated on ParaRules validation set. IT-MULTIPROVER outperforms all other models across all metrics.
Node	Edge	Proof	FA
QA	P	R	F1	P	R	F1	P	R	F1
PROVER-all	98.2	95.3	92.8	93.5	94.7	92.7	93.3	94.4	92.4	93.0	90.5
PROVER-top-p	98.2	38.7	95.9	54.3	38.3	95.5	53.9	38.2	95.3	53.8	00.1
ML-MULTIPROVER	98.3	96.0	95.6	95.7	95.9	95.5	95.6	95.5	95.2	95.2	93.8
IT-MULTIPROVER	98.3	96.8	96.2	96.3	96.5	96.3	96.3	96.2	96.0	96.0	94.5
find that the proof F1 for ML-MULTIPROVER starts to decrease marginally with the increase in  $p$ . Note that this model is trained with padding of empty proof graphs since it generates all  $p$  proofs in parallel. Thus, the amount of padding increases with the increase in  $p$ , thereby leading to a harder learning problem as the model needs to predict more number of empty graphs. IT-MULTIPROVER, on the other hand, is significantly robust to such variations in  $p$ , because it generates proofs iteratively with one empty graph at the end, indicating end of set.
A.6 Evaluation on Human-Paraphrased Rule-Bases
Following PROVER, we also test MULTIPROVER's effectiveness in generating proofs for more human-like complex rule-bases. The ParaRules dataset is constructed by first creating a set of fact groups where each fact group consists of all facts in the theory concerning a particular person and then paraphrasing these fact groups into more complex language. E.g., a fact group "Alan is blue. Alan is rough. Alan is young.", may be re-worded into "Alan is on the young side, but rough. He often feels rather blue." Thus, unlike the DU datasets or the Birds-Electricity dataset where the proof graphs are composed of facts and rules, ParaRules proofs are composed of fact groups and rules. Following past work (Clark et al., 2020; Saha et al., 2020) we train our models combining the DU3 and ParaRules train sets, and evaluate on the ParaRules validation and test set in Tables 8 and 9 respectively. We find that similar conclusions to the DU5 dataset hold for
Table 9: Comparison of models trained on DU3 and ParaRules training sets and evaluated on ParaRules test set. IT-MULTIPROVER outperforms all other models across all metrics.
p	# Parameters	Time/epoch (in hours)
PR	ML	IT	PR	ML	IT
1	361M	361M	488M	5.0	3.4	3.6
2	361M	361M	615M	5.0	3.5	4.0
3	361M	361M	742M	5.0	3.6	4.6
4	361M	361M	869M	5.0	3.7	5.1
5	361M	361M	996M	5.0	3.8	5.7
Table 10: Comparative study of the number of parameters and training time per epoch (in hours) for PROVER-all (PR), ML-MULTIPROVER and IT-MULTIPROVER with varying number of maximum proofs (p).
this dataset as well - ML-MULTIPROVER achieves a better proof F1 and full accuracy than PROVER, which are further improved by IT-MULTIPROVER due to its explicit conditioning mechanism between the proofs.
A.7 Training Time and Size Comparison
Table 10 shows the number of trainable parameters and training times per epoch for the baseline model PROVER and our proposed models, ML-MULTIPROVER and IT-MULTIPROVER across varying number of maximum proofs  $(p)$  per sample. Since ML-MULTIPROVER adopts the same PROVER architecture but with multi-label classification, it has the same number of parameters as PROVER, which also remains unchanged irrespective of the maximum number of proofs. The number of parameters for IT-MULTIPROVER, however, increases with the increase in  $p$  because of the presence of multiple node and edge encoders.
While IT-MULTIPROVER has more parameters than PROVER, our empirical findings reveal that just having a similarly-sized, larger PROVER model will not be sufficient and exploiting the correlations between multiple proofs with a permutation-invariant loss is necessary for the task of generating a set of multiple proofs.
The training time of PROVER is more than that of ML-MULTIPROVER because the former treats each proof as a separate example, causing an increase in the training data size from 70k to 110k. ML-MULTIPROVER is the most time-efficient model and its running time only increases marginally with the increase in  $p$ . This is due to the additional node and edge classifications that the model has to perform corresponding to each extra proof. Unsurprisingly, IT-MULTIPROVER takes longer to train but encouragingly for  $p \leq 4$ , still has a comparable running time to PROVER.