Knowledge-Aware Natural Language Understanding

Natural Language Understanding (NLU) systems need to encode human generated text (or speech) and reason over it at a deep semantic level. Any NLU system typically involves two main components: The first is an encoder, which composes words (or other basic linguistic units) within the input utterances to compute encoded representations, that are then used as features in the second component, a predictor, to reason over the encoded inputs and produce the desired output. We argue that performing these two steps over the utterances alone is seldom sufficient for understanding language, as the utterances themselves do not contain all the information needed for understanding them. We identify two kinds of additional knowledge needed to fill the gaps: background knowledge and contextual knowledge. The goal of this thesis is to build end-to-end NLU systems that encode inputs along with relevant background knowledge, and reason about them in the presence of contextual knowledge. The first part of the thesis deals with background knowledge. While distributional methods for encoding inputs have been used to represent meaning of words in the context of other words in the input, there are other aspects of semantics that are out of their reach. These are related to commonsense or real world information which is part of shared human knowledge but is not explicitly present in the input. We address this limitation by having the encoders also encode background knowledge, and present two approaches for doing so. The first is by modeling the selectional restrictions verbs place on their semantic role fillers. We use this model to encode events, and show that these event representations are useful in detecting newswire anomalies. Our second approach towards augmenting distributional methods is to use external knowledge bases like WordNet. We compute ontologygrounded token-level representations of words and show that they are useful in predicting prepositional phrase attachments and textual entailment. The second part of the thesis focuses on contextual knowledge. Machine comprehension tasks require interpreting input utterances in the context of other structured or unstructured information. This can be challenging for multiple reasons. Firstly, given some task-specific data, retrieving the relevant contextual knowledge from it can be a serious problem. Secondly, even when the relevant contextual knowledge is provided, reasoning over it might require executing a complex series of operations depending on the structure of the context and the compositionality of the input language. To handle reasoning over contexts, we first describe a type constrained neural semantic parsing framework for question answering (QA). We achieve state of the art performance on WIKITABLEQUESTIONS, a dataset with highly compositional questions over semi-structured tables. Proposed work in this area includes application of this framework to QA in other domains with weaker supervision. To address the challenge of retrieval, we propose to build neural network models with explicit memory components that can adaptively reason and learn to retrieve relevant context given a question.