We utilize a stochastic answer network (SAN) to explore multi-step inference strategies in Natural Language Inference. Rather than directly predicting the results given the inputs, the model maintains a state and iteratively refines its predictions. This can potentially model more complex inferences than the existing single-step inference methods. Our experiments show that SAN achieves state-of-the-art results on four benchmarks: Stanford Natural Language Inference (SNLI), MultiGenre Natural Language Inference (MultiNLI), SciTail, and Quora Question Pairs datasets. The natural language inference task, also known as recognizing textual entailment (RTE), is to infer the relation between a pair of sentences (e.g., premise and hypothesis). This task is challenging, since it requires a model to fully understand the sentence meaning, (i.e., lexical and compositional semantics). For instance, the following example from MultiNLI dataset (Williams et al., 2017) illustrates the need for a form of multi-step synthesis of information between premise: "If you need this book, it is probably too late unless you are about to take an SAT or GRE.", and hypothesis: "It's never too late, unless you're about to take a test." To predict the correct relation between these two sentences, the model needs to first infer that "SAT or GRE" is a "test", and then pick the correct relation, e.g., contradiction. This kind of iterative process can be viewed as a form of multi-step inference. To best of our knowledge, all of works on NLI use a single step inference. Inspired by the recent success of multi-step inference on Machine Reading Comprehension (MRC) (Hill et al., 2016; Dhingra et al., 2016; Sordoni et al., 2016; Kumar et al., 2015; Shen et al., 2017; Xu et al., 2018) , we explore the multi-step inference strategies on NLI. Rather than directly predicting the results given the inputs, the model maintains a state and iteratively refines its predictions. We show that our model outperforms single-step inference and further achieves the state-of-the-art on SNLI, MultiNLI, SciTail, and Quora Question Pairs datasets. The natural language inference task as defined here involves a premise P = {p 0 , p 1 , ..., p m−1 } of m words and a hypothesis H = {h 0 , h 1 , ..., h n−1 } of n words, and aims to find a logic relationship R between P and H, which is one of labels in a close set: entailment, neutral and contradiction. The goal is to learn a model f (P, H) → R. In a single-step inference architecture, the model directly predicts R given P and H as input. In our multi-step inference architecture, we additionally incorporate a recurrent state s t ; the model processes multiple passes through P and H, iteratively refining the state s t , before finally generating the output at step t = T , where T is an a priori chosen limit on the number of inference steps. Figure 1 describes in detail the architecture of the stochastic answer network (SAN) used in this study; this model is adapted from the MRC multistep inference literature . Compared to the original SAN for MRC, in the SAN for NLI we simplify the bottom layers and Selfattention layers since the length of the premise and hypothesis is short). We also modify the answer module from prediction a text span to an NLI classification label. Overall, it contains four different layers: 1) the lexicon encoding layer computes word representations; 2) the contextual encoding layer modifies these representations in context; 3) the memory generation layer gathers all information from the premise and hypothesis and forms a arXiv:1804.07888v2 [cs.CL]
REF introduced a stochastic answer network (SAN) for multi-step inference strategies for NLI.
Stochastic Answer Networks for Natural Language Inference
ArXiv
ArXiv
Computer Science