SlowGuess/NAACL_2022_OCR huggingface dataset:
Multiple paths product. For many of the existing training methods which expect a single path leading to the answer as part of the input, it is also possible to make them incorporate multiple possible paths when no path annotation is given. The simplest way is to expand each (question, answer) pair into multiple training instances, each with a different path leading to the same answer, and then apply existing training method treating them as independent instances. This corresponds to the objective  $\prod_{\mathbf{p} \in \mathcal{P}} p(y|\mathbf{p}, q)p(\mathbf{p}|q)$ . This objective has an undesired consequence in practical model training: because of the multiplication operation, the model has to assign equally high probabilities to all given reasoning paths in order to maximize the product of the probabilities. If only some reasoning paths receive high probabilities while others receive low probabilities, the production will still be low. As a consequence, the model cannot differentiate bad reasoning paths from good ones by assigning distinguishable probabilities to them.
Multiple paths marginalization. Our proposed training objective replaces the multiplication operation by the summation operation, and this allows the model to concentrate only on good reasoning paths for each QA pair. It is easy to show that the model tends to assign high probability  $p(\mathbf{p} \mid q)$  to a path  $\mathbf{p}$  when the path leads to few possible answers and therefore the chance of getting the correct answer  $p(y \mid \mathbf{p}, q)$  is high (see 2). Also, using Jensen's inequality, one can show that this marginal probability objective maximizes the answer probability directly which is the learning goal of KBQA task, while the previous one using product operation maximizes a lower bound.
We test different ways of choosing paths and defining training objectives on WQSP and CWQ datasets. We further divide the test samples into two groups, based on whether there exist multiple
Method	Objective	Path p
single ground truth	p(y|p, q)p(p|q)	single ground truth path leading to y
single random	p(y|p, q)p(p|q)	single random path leading to y
multiple product	Πp∈Pp(y|p, q)p(p|q)	all valid paths leading to y
multiple marginal (MRP-QA)	Σp∈Pp(y|p, q)p(p|q)	all valid paths leading to y
Table 4: Different choices of paths and objectives.
WQSP	CWQ
1 path	>1 path	all	1 path	>1 path	all
single ground truth	66.8	69.3	68.1	40.8	49.2	46.4
single random	65.7	64.1	64.8	40.8	46.9	44.9
multiple product	69.1	70.2	69.7	40.9	50.7	47.5
multiple marginal (MRP-QA)	73.0	76.3	74.9	43.7	53.0	49.9
Table 5: We break test set into two groups based on number of paths associated with them and report F1(%)
PQL2H	PQL3H	PQL+
HR-BiLSTM (Yu et al., 2017)	97.5	87.9	92.9
IRN (Zhou et al., 2018)	72.5	71.0	52.9
ABWIM (Zhang et al., 2018)	94.3	89.3	92.6
UHop (Chen et al., 2019)	97.5	89.3	92.3
KV-MemNN* (Miller et al., 2016)	72.2	67.4	-
Our MRP-QA Method-marginal_prob*	98.4	97.8	98.0
Table 6: We report set accuracy  $(\%)$  on PQL. Similar to Table 2, we use  $*$  to highlight the methods which only requires the answer as supervision.
possible paths between the topic entity and the answer based on KB. Table 5 show that our proposed method gives the best performance on both scenarios. The models trained with single path perform consistently worse than those trained with multiple paths. Using random path is worse than using the given ground truth path. Between two models trained using multiple paths, the result shows the advantage of using our proposed objective.
3.4 PathQuestion-Large
In the third set of experiments, we test our MRP-QA model on PATHQUESTION-LARGE (PQL) dataset. This dataset contains synthetic questions generated by templates, and is supported by a very small knowledge base (500,000 times smaller than the full freebase). Not surprisingly, we can see the average performance on this dataset is much better than it is on the other two datasets. Recall that PQL2H and PQL3H represents two subsets with only 2 hops and 3 hops questions respectively. Table 6 shows that our MRP-QA method's performance beats all the other approaches on all three subsets of PQL from  $1\%$  to  $7.8\%$  in terms of test accuracy. Especially the gap between our method to the previous state-of-the-art approach (i.e. UHop)
becomes larger when the number of hops increase from 2 to 3.
Our model requires inference while using the current model to select training samples for next batch in training (see line 6 in Algorithm 1). This EM style training approach helps us filter out bad reasoning paths based on context information. For example, a sample question from WQSP is who was the owner of  $kfc$ ?, the graph search algorithm can easily extract two "correct" paths starting from the topic entity  $kfc$  directing to the ground truth answer Colonel Sanders:  $kfc \rightarrow$  organization.organizationfounders  $\rightarrow$  Colonel Sanders and  $kfc \rightarrow$  advertisingcharacters.product.advertising Characters  $\rightarrow$  Colonel Sanders. However, the second path is totally wrong given that the reasoning path is irrelevant to the given question. Colonel Sanders happens to be the advertising character of  $kfc$ , but this cannot be generalized to other cases. Without using the trained model to filter out this irreverent path, the model may learn incorrect map from who is the owner... to the relation advertisingCharacters. In our experiment, we observe that when we train our model with all reasoning paths generated from DFS algorithm without using this filtering strategy (i.e.  $k_{2} = inf$ ), the F1 score drops  $3.4\%$  as shown in Table 3. This shows the importance of using the filtering strategy.
Next we demonstrate the benefit of maximizing conditional mutual information instead of likelihood. A sample question in WQSP is who did benjamin franklin get married to?. We observe that there are 13 questions are using Benjamin Franklin as the topic entity in the training set, but most of them are related to his invention and none of them is about marriage. With such a strong prior on Benjamin Franklin, our experimental result shows that the model trained with maximum likelihood mistakenly maps this question to a path related to invention, while the model trained with mutual
Question: what state does romney live in? Answer: Massachusetts Topic entity: romney
SINGLE GROUND TRUTH	MULTIPLE PRODUCT	MULTIPLE MARGINAL (OUR)
.89:children	.29:education_institution/ state_province_region	.83:places_lived/ location
.06:governmentPositions/ jurisdiction_of_office	.25:places_lived/ location	.12:governmentPositions/ jurisdiction_of_office
.04:governmentPositions/ office_position_or_title	.25:governmentPositions/ district_represented	.04:governmentPositions/ district_represented
.00:governmentPositions/ district_represented	.01:governmentPositions/ jurisdiction_of_office	.01:place_of_birth/ state
.00:place_of_birth	.01:place_of_birth/ state	.00:education/ degree
.00:jurisdiction_of_office	.01:sibling/ place_of_birth	.00:election_campaigns
Table 7: A running examples from WEBQUESTIONSP dataset. We show the probability  $P(r_0, \dots, r_T | q)$  before the inferred relations. Paths that lead to the correct answers are highlighted in bold. We use / to split two relations. The three columns are corresponding to the results by using different training settings as it is in Table 4. Due to space limit, we only show the partial name of a relation in the example and the probability less than .01 is shown as .00. We do not show  $P(e_0, \dots, e_{T-1} | q)$  because they are not determined by our MRP-QA model.
information makes the correct prediction. Table 3 shows that we get a  $1.8\%$  performance boost by using mutual information.
We further show how generated probabilities look like with different choices of paths and objectives in Table 7. In the given example, only our MRP-QA method outputs the correct path, and one can also find that the top three results correspond to three different but correct reasoning processes. We observe that in many training questions "live in" co-occurs with word "children", which explains why the first model makes wrong prediction. We can see that training with joint objective given a single relation path generates the most sharp relation path distribution, i.e. the gap between the top entity and the second one is larger than that using other objectives. It assigns most probability mass to the top relation path. In this case, the model does not have ability to identify multiple relation paths during inference. The other extreme is that the second model is trained with joint objective and MRPs, which distribute probabilities over many relation paths, hence the model cannot distinguish good relation paths from the bad ones. Between the above two extremes is the proposed marginal objective with MRPs, when the most probable path is assigned the largest probability, while the rest ones still get reasonable probability assignments.
Most of the existing multi-hop KBQA systems approach this task by decomposing it into two subtasks: topic-entity linking and relation extraction. The topic-entity linking gives the system an entry point to start searching, and the relation extraction is used to search relation paths leading to the final answer. Following this track, a straightforward idea is to match the question to a candidate entity/relation directly via calculating the similar
ity between them (Zhang et al., 2018; Yu et al., 2018; Lan et al., 2019). This method is not ideal for multi-hop questions with long paths, because the number of candidate entity-relation combinations grows exponentially as the number of hops increases. To tackle this issue, methods are proposed to decompose the input question into several single-hop questions, and then use existing method to solve each simple question. The decomposition methods are based on semantic parsing (Abujabal et al., 2017; Luo et al., 2018) or templates (Ding et al., 2019). A similar idea is to encode the reasoning information hop by hop, and predict the final answer at the last hop (Miller et al., 2016; Zhou et al., 2018; Chen et al., 2019).
Another line of work has looked at solving KBQA task with only final answer as supervision. Liang et al. (2017) first propose to cast KBQA as a program generation task using neural program induction (NPI) techniques. They learn to translate the query to a program like logical form executable on the KB. As a follow up, Ansari et al. (2019) improves this idea by incorporating high level program structures. Both these NPI models do not require annotated relation path as supervision, but they need some prior knowledge to design the program templates. In other work, Min et al. (2019) recently proposed a latent variable approach which is similar to the one described here, but applied on text-based QA scenarios. The main difference between our work is that our method aims at finding multiple reasoning paths leading to the answer, while their method only focuses on extracting single optimal solution. We employ inference during training to filter our irrelevant paths, while they use it to identify the optimal solution.
Besides knowledge based question answering models, there are also other question answering sysstems without using knowledge base/graph in
formation. For examples, there are end-to-end question-answering models by using reinforcement learning (Wang and Jin, 2019), adversarial learning (Wang et al., 2021b, 2020b), and also the multimodel structure to handle the unknown entities in question answering (Wang et al., 2018, 2020a). Co-reference understanding (Wang et al., 2021a) is also another research direction in designing question-answering systems.
In this paper, We introduce a novel MRP-QA knowledge based question answering system which can leverage information from MRPs. To train our model, we use a marginalized probability objective function. Experimental results show that our model achieve strong performance on popular KBQA datasets.
Abdalghani Abujabal, Mohamed Yahya, Mirek Riedewald, and Gerhard Weikum. 2017. Automated template generation for question answering over knowledge graphs. In Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 3-7, 2017, pages 1191-1200.
Ghulam Ahmed Ansari, Amrita Saha, Vishwajeet Kumar, Mohan Bhambhani, Karthik Sankaranarayanan, and Soumen Chakrabarti. 2019. Neural program induction for KBQA without gold programs or query annotations. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pages 4890-4896.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States., pages 2787-2795.
Zi-Yuan Chen, Chih-Hung Chang, Yi-Pei Chen, Jijnasa Nayak, and Lun-Wei Ku. 2019. Uhop: An unrestricted-hop relation extraction framework for knowledge-based question answering. In Proceedings of the 2019 Conference of the North American
Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 345-356.
Jiwei Ding, Wei Hu, Qixin Xu, and Yuzhong Qu. 2019. Leveraging frequent query substructures to generate formal queries for complex question answering. CoRR, abs/1908.11053.
Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke S. Zettlemoyer. 2017. Allennlp: A deep semantic natural language processing platform.
Google. 2013. Freebase data dumps. https:// developers.google.com/freebase/data.
Gaole He, Yunshi Lan, Jing Jiang, Wayne Xin Zhao, and Ji-Rong Wen. 2021. Improving multi-hop knowledge base question answering by learning intermediate supervision signals. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pages 553-561.
Yunshi Lan, Shuohang Wang, and Jing Jiang. 2019. Knowledge base question answering with topic units. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pages 5046-5052.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting objective function for neural conversation models. In *NAACL HLT* 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016, pages 110-119.
Chen Liang, Jonathan Berant, Quoc V. Le, Kenneth D. Forbus, and Ni Lao. 2017. Neural symbolic machines: Learning semantic parsers on freebase with weak supervision. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pages 23-33.
Kangqi Luo, Fengli Lin, Xusheng Luo, and Kenny Q. Zhu. 2018. Knowledge base question answering via encoding of complex query graphs. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 2185-2194.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781.
Alexander H. Miller, Adam Fisch, Jesse Dodge, Amir Hossein Karimi, Antoine Bordes, and Jason Weston. 2016. Key-value memory networks for directly reading documents. In Proceedings of the 2016 Conference on Empirical Methods in Natural
Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 1400-1409.
Sewon Min, Danqi Chen, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2019. A discrete hard EM approach for weakly supervised question answering. CoRR, abs/1909.04849.
Jiaxin Shi, Shulin Cao, Lei Hou, Juanzi Li, and Hanwang Zhang. 2021. Transfernet: An effective and transparent framework for multi-hop question answering over relation graph. arXiv preprint arXiv:2104.07302.
Haitian Sun, Tania Bedrax-Weiss, and William Cohen. 2019. Pullnet: Open domain question answering with iterative retrieval on knowledge bases and text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2380-2390.
Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhutdinov, and William W. Cohen. 2018. Open domain question answering using early fusion of knowledge bases and text. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 4231-4242.
Alon Talmor and Jonathan Berant. 2018. Repartitioning of the complexwebquestions dataset. CoRR, abs/1807.09623.
Théo Trouillon and Maximilian Nickel. 2017. Complex and holographic embeddings of knowledge graphs: A comparison. CoRR, abs/1707.01475.
Yu Wang and Hongxia Jin. 2019. A deep reinforcement learning based multi-step coarse to fine question answering (MSCQA) system. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, USA, January 27 - February 1, 2019, pages 7224-7232. AAAI Press.
Yu Wang, Yilin Shen, and Hongxia Jin. 2018. A bimodel based RNN semantic frame parsing model for intent detection and slot filling. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 2 (Short Papers), pages 309-314. Association for Computational Linguistics.
Yu Wang, Yilin Shen, and Hongxia Jin. 2020a. A bi-model approach for handling unknown slot values in dialogue state tracking. In 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2020, Barcelona, Spain, May 4-8, 2020, pages 8019-8023. IEEE.
Yu Wang, Yilin Shen, and Hongxia Jin. 2020b. An interactive adversarial reward learning-based spoken language understanding system. In Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China, 25-29 October 2020, pages 1565-1569. ISCA.
Yu Wang, Yilin Shen, and Hongxia Jin. 2021a. An end-to-end actor-critic-based neural coreference resolution system. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2021, Toronto, ON, Canada, June 6-11, 2021, pages 7848-7852. IEEE.
Yu Wang, Yilin Shen, and Hongxia Jin. 2021b. Multi-step spoken language understanding system based on adversarial learning. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2021, Toronto, ON, Canada, June 6-11, 2021, pages 7673-7677. IEEE.
Yi Yang and Ming-Wei Chang. 2016. S-MART: novel tree-based structured learning algorithms applied to tweet entity linking. CoRR, abs/1609.08075.
Wen-tau Yih, Ming-Wei Chang, Xiaodong He, and Jianfeng Gao. 2015. Semantic parsing via staged query graph generation: Question answering with knowledge base. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers, pages 1321-1331.
Wen-tau Yih, Matthew Richardson, Christopher Meek, Ming-Wei Chang, and Jina Suh. 2016. The value of semantic parse labeling for knowledge base question answering. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 2: Short Papers.
Mo Yu, Wenpeng Yin, Kazi Saidul Hasan, Cicero Nogueira dos Santos, Bing Xiang, and Bowen Zhou. 2017. Improved neural relation detection for knowledge base question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pages 571-581.
Yang Yu, Kazi Saidul Hasan, Mo Yu, Wei Zhang, and Zhiguo Wang. 2018. Knowledge base relation detection via multi-view matching. In New Trends in Databases and Information Systems - ADBIS 2018 Short Papers and Workshops, AI*QA, BIGPMED, CSACDB, M2U, BigDataMAPS, ISTREND, DC, Budapest, Hungary, September, 2-5, 2018, Proceedings, pages 286-294.
Hongzhi Zhang, Guandong Xu, Xiao Liang, Tinglei Huang, and Kun Fu. 2018. An attention-based
word-level interaction model: Relation detection for knowledge base question answering. CoRR, abs/1801.09893.
Mantong Zhou, Minlie Huang, and Xiaoyan Zhu. 2018. An interpretable reasoning network for multi-relation question answering. In Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018, pages 2010-2022.