Original: P ROBABILISTIC N EURAL - SYMBOLIC M ODELS FOR I NTERPRETABLE V ISUAL Q UESTION A NSWERING
Revised: P ROBABILISTIC N EURAL - SYMBOLIC M ODELS FOR I NTERPRETABLE V ISUAL Q UESTION A NSWERING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We propose a new class of probabilistic neural-symbolic models for visual question answering (VQA) that provide interpretable explanations of their decision making in the form of programs, given a small annotated set of human programs.
Revised: We propose a new class of probabilistic neural-symbolic models for visual question answering (VQA) that provide interpretable explanations of their decision making in the form of programs, given a small annotated set of human programs.

Original: The key idea of our approach is to learn a rich latent space which effectively propagates program annotations from known questions to novel questions.
Revised: The key idea of our approach is to learn a rich latent space which effectively propagates program annotations from known questions to novel questions.

Original: We do this by formalizing prior work on VQA, called module networks (Andreas, 2016) as discrete, structured, latent variable models on the joint distribution over questions and answers given images, and devise a procedure to train the model effectively.
Revised: We do this by formalizing prior work on VQA, called module networks (Andreas, 2016) as discrete, structured, latent variable models on the joint distribution over questions and answers given images, and devise a procedure to train the model effectively.

Original: Our results on a dataset of compositional questions about SHAPES (Andreas, 2016) show that our model generates more interpretable programs and obtains better accuracy on VQA in the low-data regime than prior work.
Revised: Our results on a dataset of compositional questions about SHAPES (Andreas, 2016) show that our model generates more interpretable programs and obtains better accuracy on VQA in the low-data regime than prior work.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Exciting progress has been made in recent years in deep representation learning (Hochreiter &
Revised: Exciting progress has been made in recent years in deep representation learning (Hochreiter &

Original: Schmidhuber, 1997; LeCun et al., 1998), which has led to advances in artiﬁcial intelligence tasks such as image recognition (Krizhevsky et al., 2012; He et al., 2015), machine translation (Sutskever et al., 2014), visual question answering (Agrawal et al., 2015), visual dialog (Das et al., 2016), and reinforcement learning (Mnih et al., 2015), etc.
Revised: Schmidhuber, 1997; LeCun et al., 1998), which has led to advances in artiﬁcial intelligence tasks such as image recognition (Krizhevsky et al., 2012; He et al., 2015), machine translation (Sutskever et al., 2014), visual question answering (Agrawal et al., 2015), visual dialog (Das et al., 2016), and reinforcement learning (Mnih et al., 2015), etc.

Original: While deep neural networks achieve such impressive performances, many aspects of human cognition such as compositional generalization and reasoning are harder to model with state-of-the art deep learning approaches (Battaglia et al., 2018; Barrett et al., 2018).
Revised: While deep neural networks achieve such impressive performances, many aspects of human cognition such as compositional generalization and reasoning are harder to model with state-of-the art deep learning approaches (Battaglia et al., 2018; Barrett et al., 2018).

Original: Symbolic approaches (Newell & Simon, 1976) on the other hand provide strong compositional and reasoning capabilities which are challenging to model with neural networks (Lake et al., 2017).
Revised: Symbolic approaches (Newell & Simon, 1976) on the other hand provide strong compositional and reasoning capabilities which are challenging to model with neural networks (Lake et al., 2017).

Original: Consequently, a rich line of work in neuro-symbolic processing sought to build AI systems with both strong learning and reasoning capabilities (Valiant, 2003; Evans et al., 2018b; Bader & Hitzler, 2005).
Revised: Consequently, a rich line of work in neuro-symbolic processing sought to build AI systems with both strong learning and reasoning capabilities (Valiant, 2003; Evans et al., 2018b; Bader & Hitzler, 2005).

Original: As we tackle higher-level tasks which involve reasoning, such as visual question answering (Agrawal et al., 2015), planning (Andreas et al., 2016b; Gregor et al., 2016) etc. , it is natural to desire the ability to provide instructions to models to guide them.
Revised: As we tackle higher-level tasks which involve reasoning, such as visual question answering (Agrawal et al., 2015), planning (Andreas et al., 2016b; Gregor et al., 2016) etc. , it is natural to desire the ability to provide instructions to models to guide them.

Original: Symbolic instructions, by their very nature are easier to specify and more interpretable than specifying the parameters of a neural network.
Revised: Symbolic instructions, by their very nature are easier to specify and more interpretable than specifying the parameters of a neural network.

Original: For such high-level tasks, a sensible approach is to specify “what” to do using symbols and learn how to do the task using modern representation learning techniques.
Revised: For such high-level tasks, a sensible approach is to specify “what” to do using symbols and learn how to do the task using modern representation learning techniques.

Original: For the example shown in Figure 1, given a question “ Is a square to the left of a green shape? ”, one can ask the model to reason about the answer by ﬁrst applying a find[green] operator, then find[left] , then And the result together with a find[square] operator in order to predict the answer, in other words specifying “what” are the computations we desire to be executed, in the form of a “program”.
Revised: For the example shown in Figure 1, given a question “ Is a square to the left of a green shape? ”, one can ask the model to reason about the answer by ﬁrst applying a find[green] operator, then find[left] , then And the result together with a find[square] operator in order to predict the answer, in other words specifying “what” are the computations we desire to be executed, in the form of a “program”.

Original: (Johnson et al., 2017; Hu et al., 2017;
Revised: (Johnson et al., 2017; Hu et al., 2017;

Original: Andreas et al., 2016a).
Revised: Andreas et al., 2016a).

Original: The network can then learn “how” to execute such a program from data using deep representation learning (Johnson et al., 2017;
Revised: The network can then learn “how” to execute such a program from data using deep representation learning (Johnson et al., 2017;

Original: Hu et al., 2017; Andreas et al., 2016a).
Revised: Hu et al., 2017; Andreas et al., 2016a).

Original: This paper addresses a very natural desiderata for such neuro-symbolic models, in the context of visual question answering – can we retain the intepretability of “what the network did” expressed in terms of the syntax and lexicon that we are interested in, while specifying minimal teaching examples of “what to do” given an input question?
Revised: This paper addresses a very natural desiderata for such neuro-symbolic models, in the context of visual question answering – can we retain the intepretability of “what the network did” expressed in terms of the syntax and lexicon that we are interested in, while specifying minimal teaching examples of “what to do” given an input question?

Original: [Figure]
Revised: [Figure]

Original: We propose to approach this problem with a treatment of programs z as a latent variable, embedded into a generative model of questions x and answers a given images i as visualized in Figure 2.
Revised: We propose to approach this problem with a treatment of programs z as a latent variable, embedded into a generative model of questions x and answers a given images i as visualized in Figure 2.

Original: This model class has certain desirable properties: ﬁrst, a proper treatment of z as a stochastic latent variable is helpful especially in the presence of a limited number of expert human instructions, as we model the uncertainty associated with z explicitly, leading to more interepretable latent spaces (Section 5); second, the model makes intuitive independence assumptions with regards to the question answering task – conditioned on the program z , questions x are independent of the answers a for an image, meaning that the program must be a sufﬁcient statistic for predicting the answer given a question 1 .
Revised: This model class has certain desirable properties: ﬁrst, a proper treatment of z as a stochastic latent variable is helpful especially in the presence of a limited number of expert human instructions, as we model the uncertainty associated with z explicitly, leading to more interepretable latent spaces (Section 5); second, the model makes intuitive independence assumptions with regards to the question answering task – conditioned on the program z , questions x are independent of the answers a for an image, meaning that the program must be a sufﬁcient statistic for predicting the answer given a question 1 .

Original: Third, we parameterize the p ( a | i , z ) term, which predicts an answer given an image and a program using a popular, related class of question answering models called neural module networks (NMNs) (Hu et al., 2017; Johnson et al., 2017; Andreas et al., 2016a) which previous work has shown leads to better ﬁt between the program (or “what the model is doing”) and the execution (or “how it is doing what it is doing”).
Revised: Third, we parameterize the p ( a | i , z ) term, which predicts an answer given an image and a program using a popular, related class of question answering models called neural module networks (NMNs) (Hu et al., 2017; Johnson et al., 2017; Andreas et al., 2016a) which previous work has shown leads to better ﬁt between the program (or “what the model is doing”) and the execution (or “how it is doing what it is doing”).

Original: Given the close ties of our model to NMNs, we will refer to our model as variational neural module networks (V-NMN) for the rest of the paper.
Revised: Given the close ties of our model to NMNs, we will refer to our model as variational neural module networks (V-NMN) for the rest of the paper.

Original: Apart from our modeling contribution of a probabilistic latent variable, neuro-symbolic model, our key technical contribution is to show how to train this model in the context of visual question answering.
Revised: Apart from our modeling contribution of a probabilistic latent variable, neuro-symbolic model, our key technical contribution is to show how to train this model in the context of visual question answering.

Original: Optimizing such generative models with structured discrete latent variables is an intractable problem to solve in genereal – for this particular case, we formulate training into stages of grounding questions into programs (question coding), learning how to execute them and then with those initializations train the full variational objective.
Revised: Optimizing such generative models with structured discrete latent variables is an intractable problem to solve in genereal – for this particular case, we formulate training into stages of grounding questions into programs (question coding), learning how to execute them and then with those initializations train the full variational objective.

Original: We demonstrate that this stage-wise optimization allows us to successfully learn a probabilistic neural-symbolic model that can answer compositional questions about shapes in an image (Andreas et al., 2016a).
Revised: We demonstrate that this stage-wise optimization allows us to successfully learn a probabilistic neural-symbolic model that can answer compositional questions about shapes in an image (Andreas et al., 2016a).

Original: While the V-NMN model is instantiated for the speciﬁc problem of visual question answering, we believe the tools and techniques we develop are more generally applicable to probabilistic, structured, discrete, interpretable latent spaces for interpretable neuro-symbolic models for planning, instruction following, grounding referring expressions etc.
Revised: While the V-NMN model is instantiated for the speciﬁc problem of visual question answering, we believe the tools and techniques we develop are more generally applicable to probabilistic, structured, discrete, interpretable latent spaces for interpretable neuro-symbolic models for planning, instruction following, grounding referring expressions etc.

Original: We benchmark our model on the SHAPES (Andreas et al., 2016a) dataset with compositionally novel questions and show that our model is able to provide interpretable explanations of “what it is doing” (measured by program correctness) for unseen questions even in the setting where it is given as few as ≈ 20 paired examples of programs and corresponding questions from the dataset, while also getting to high question answering accuracy.
Revised: We benchmark our model on the SHAPES (Andreas et al., 2016a) dataset with compositionally novel questions and show that our model is able to provide interpretable explanations of “what it is doing” (measured by program correctness) for unseen questions even in the setting where it is given as few as ≈ 20 paired examples of programs and corresponding questions from the dataset, while also getting to high question answering accuracy.

Original: We ﬁnd that the proposed approach outperforms a state of the art neural-symbolic model designed for VQA from Johnson et al.
Revised: We ﬁnd that the proposed approach outperforms a state of the art neural-symbolic model designed for VQA from Johnson et al.

Original: (2017), as well as deterministic ablations of our model.
Revised: (2017), as well as deterministic ablations of our model.

Original:  A P ROBABILISTIC N EURAL -S YMBOLIC MODEL FOR VQA
Revised:  A P ROBABILISTIC N EURAL -S YMBOLIC MODEL FOR VQA

Original: For a standard neural-symbolic NMN approach to question answering (Johnson et al., 2017), we are given an input image i and a question x = ( x 1 , · · · , x N ) as a sequence of words x i each encoded using a vocabulary V x , and must produce answers a by planning and executing a series of modular
Revised: For a standard neural-symbolic NMN approach to question answering (Johnson et al., 2017), we are given an input image i and a question x = ( x 1 , · · · , x N ) as a sequence of words x i each encoded using a vocabulary V x , and must produce answers a by planning and executing a series of modular

Original: [Figure]
Revised: [Figure]

Original: computations ( z = ( z 1 , · · · , z N ) ) – i.e. a program of operations ( z i ∈ V z ) to be executed in order to answer the question.
Revised: computations ( z = ( z 1 , · · · , z N ) ) – i.e. a program of operations ( z i ∈ V z ) to be executed in order to answer the question.

Original: Note that z here is the preﬁx serialization of the program and programs themselves often have tree-structured topologies.
Revised: Note that z here is the preﬁx serialization of the program and programs themselves often have tree-structured topologies.

Original: We denote the dataset of ( i , x , a ) triplets as D .
Revised: We denote the dataset of ( i , x , a ) triplets as D .

Original: Additionally, we are provided programs z for some input questions ( x ∈ S ), which specify the notion of grounding desired by an end user.
Revised: Additionally, we are provided programs z for some input questions ( x ∈ S ), which specify the notion of grounding desired by an end user.

Original: Our goal is to provide faithful explanations on unseen questions (along with correct answers) consistent with the speciﬁed grounding.
Revised: Our goal is to provide faithful explanations on unseen questions (along with correct answers) consistent with the speciﬁed grounding.

Original: We accomplish this by modeling z as a latent variable that underlies both question and answer generation given an input image.
Revised: We accomplish this by modeling z as a latent variable that underlies both question and answer generation given an input image.

Original: Intuitively, we expect this will enable us to learn a more meaningful latent representation for questions, as modeling uncertaintly should help better handle the potentially underspeciﬁed mapping between questions and programs provided by the user.
Revised: Intuitively, we expect this will enable us to learn a more meaningful latent representation for questions, as modeling uncertaintly should help better handle the potentially underspeciﬁed mapping between questions and programs provided by the user.

Original: As such, our model factorizes p ( x , z , a | i ) =
Revised: As such, our model factorizes p ( x , z , a | i ) =

Original: p ( x | z ) p ( a | i , z ) p ( z ) as in the graphical model shown in Figure 2, where the prior p ( z ) is learnt by training a neural sequence model on valid strings simulated from known syntax.
Revised: p ( x | z ) p ( a | i , z ) p ( z ) as in the graphical model shown in Figure 2, where the prior p ( z ) is learnt by training a neural sequence model on valid strings simulated from known syntax.

Original: Conceptually, this factorization captures the notion that many differently-worded questions are reducible to identical underlying semantics (captured by a program), making the program space, in a sense, a compression of the question space.
Revised: Conceptually, this factorization captures the notion that many differently-worded questions are reducible to identical underlying semantics (captured by a program), making the program space, in a sense, a compression of the question space.

Original: As mentioned in Section 1 the model also captures the independence of questions from answers for an image given programs, which is also an intuitive desiderata for interpretable neural-symbolic models 2 .
Revised: As mentioned in Section 1 the model also captures the independence of questions from answers for an image given programs, which is also an intuitive desiderata for interpretable neural-symbolic models 2 .

Original: Given the proposed model shown in Figure 2, we must address two tasks to put the model to use for
Revised: Given the proposed model shown in Figure 2, we must address two tasks to put the model to use for

Original: VQA – learning model parameters, and performing inference for a speciﬁc query.
Revised: VQA – learning model parameters, and performing inference for a speciﬁc query.

Original: Learning.
Revised: Learning.

Original: We will always condition on images i but may not observe underlying programs z , so our per-data point training objective is the log-marginal-conditional-likelihood: log p ( x , a | i ) .
Revised: We will always condition on images i but may not observe underlying programs z , so our per-data point training objective is the log-marginal-conditional-likelihood: log p ( x , a | i ) .

Original: Since computing this objective involves an intractable summation over latent programs ( log (cid:80) z p ( x j , z , a j | i j ) ), we use a variational approximation (Jordan et al., 1999). 
Revised: Since computing this objective involves an intractable summation over latent programs ( log (cid:80) z p ( x j , z , a j | i j ) ), we use a variational approximation (Jordan et al., 1999). Speciﬁcally, let - q φ ( z | x ) denote an amortized inference neural network

Revised: parameterized by

Revised: φ ,

Original: Speciﬁcally, let - q φ ( z | x ) denote an amortized inference neural network parameterized by φ , - p θ z ( a | z , i ) be a NMN parametrized by θ z run on image i with program z , - p θ ( x | z ) be a recurrent network  parametrized by θ that maps programs to questions, and - p ( z ) be a prior over programs learned from data or based on program syntax directly 
Revised: which instantiates a sequence to sequence LSTM (Hochreiter Schmidhuber, 1997) model - p θ z ( a | z , i ) be a NMN parametrized by θ z run on image i with program z , - p θ ( x | z ) be a recurrent network (sequence to sequence model) parametrized by θ that maps programs to questions, and - p ( z ) be a prior over programs learned from data or based on program syntax directly , parameter- ized as an LSTM (Hochreiter & Schmidhuber, 1997) sequence model

Original: Then, for datapoints without program supervision, the objective is:
Revised: Then, for datapoints without program supervision, the objective is:

Original: [Equation]
Revised: [Equation]

Original: It is easy to show that elbo( x , a | i ) ≤ log p ( x , a | i ) for β ≥ 1 ; as such, maximizing this objective is a form of approximate maximum likelihood training for distribution parameters θ z , θ , and φ .
Revised: It is easy to show that elbo( x , a | i ) ≤ log p ( x , a | i ) for β ≥ 1 ; as such, maximizing this objective is a form of approximate maximum likelihood training for distribution parameters θ z , θ , and φ .

Original: Additionally, for data points with annotations, we add the likelihood of the program given the question log r φ ( z | x ) to the objective.
Revised: Additionally, for data points with annotations, we add the likelihood of the program given the question log r φ ( z | x ) to the objective.

Original: Note that r shares parameters φ with the inference network q .
Revised: Note that r shares parameters φ with the inference network q .

Original: So our overall objective becomes:
Revised: So our overall objective becomes:

Original: [Equation]
Revised: [Equation]

Original: where the indicator function [[ x ∈ S ]] is 1 if x is in the supervised set and 0 otherwise and α is a  positive scaling factor.
Revised: where the indicator function [[ x ∈ S ]] is 1 if x is in the supervised set and 0 otherwise and α is a linking questions to answer (via attention) operating in low-program supervision regimes often have syntactically poor programs despite leading to correct answers – i.e. the program need not be a sufﬁcient statistic anymore to answer the question. positive scaling factor.

Original: The goal is to learn the desired association of questions with programs from the second term, and to propagate them to other questions using the ﬁrst term.
Revised: The goal is to learn the desired association of questions with programs from the second term, and to propagate them to other questions using the ﬁrst term.

Original: Inference.
Revised: Inference.

Original: At test time, we can answer questions for images by computing p ( a | i , x ) = (cid:80) z p ( a | i , z ) p ( z | x ) using an approximation p q ( a | i , x ) ≈
Revised: At test time, we can answer questions for images by computing p ( a | i , x ) = (cid:80) z p ( a | i , z ) p ( z | x ) using an approximation p q ( a | i , x ) ≈

Original: E z i ∼ q ( z | x ) p ( a | i , z i ) .
Revised: E z i ∼ q ( z | x ) p ( a | i , z i ) .

Original: Challenges.
Revised: Challenges.

Original: While the elbo term in Equation (1) appears similar to a standard ELBO for joint latent variable models (Vedantam et al., 2018; Wang et al., 2016b; Suzuki et al., 2017), there are a few important departures:
Revised: While the elbo term in Equation (1) appears similar to a standard ELBO for joint latent variable models (Vedantam et al., 2018; Wang et al., 2016b; Suzuki et al., 2017), there are a few important departures:

Original: - Firstly, the latent variable z is not a Gaussian or categorical random variable (which admit straightforward reparameterizations (Maddison et al., 2017; Jang et al., 2016)) but rather a sequence of discrete tokens, which complicates the optimization problem in Equation (1). -
Revised: - Firstly, the latent variable z is not a Gaussian or categorical random variable (which admit straightforward reparameterizations (Maddison et al., 2017; Jang et al., 2016)) but rather a sequence of discrete tokens, which complicates the optimization problem in Equation (1). -

Original: Secondly, the conditional log-likelihood for a is parameterized by a NMN that is dynamically assembled based on sample programs drawn from q ( z | x ) .
Revised: Secondly, the conditional log-likelihood for a is parameterized by a NMN that is dynamically assembled based on sample programs drawn from q ( z | x ) .

Original: Depending upon the uncertainty of the program posterior, program tokens may not be consistently grounded across examples, making learning the appropriate computation for the corresponding neural modules difﬁcult.
Revised: Depending upon the uncertainty of the program posterior, program tokens may not be consistently grounded across examples, making learning the appropriate computation for the corresponding neural modules difﬁcult.

Original: - Lastly, our observations x are explained by powerful “autoregressive” sequence models.
Revised: - Lastly, our observations x are explained by powerful “autoregressive” sequence models.

Original: Learning meaningful latent variable models that are informative of observations for such models is difﬁcult (c
Revised: Learning meaningful latent variable models that are informative of observations for such models is difﬁcult (c

Original: f. (Bowman et al., 2016;
Revised: f. (Bowman et al., 2016;

Original: Chen et al., 2016; Alemi et al., 2018)) as powerful decoder models can explain the data using stochasticity from sampling (referred to as autodecoding in Alemi et al.
Revised: Chen et al., 2016; Alemi et al., 2018)) as powerful decoder models can explain the data using stochasticity from sampling (referred to as autodecoding in Alemi et al.

Original: Consequently, it also becomes hard to learn meaningful NMN parameters from the previous step.
Revised: Consequently, it also becomes hard to learn meaningful NMN parameters from the previous step.

Original: In the following sections, we present a training regime and adjustments to the objective in Equation (1) that make optimization more tractable and enable training NMNs with signiﬁcantly reduced questionaligned program supervision.
Revised: In the following sections, we present a training regime and adjustments to the objective in Equation (1) that make optimization more tractable and enable training NMNs with signiﬁcantly reduced questionaligned program supervision.

Original: While instantiated for question answering, it is not difﬁcult to conceive versions of this regime for other problems relevant to probabilistic neural-symbolic models.
Revised: While instantiated for question answering, it is not difﬁcult to conceive versions of this regime for other problems relevant to probabilistic neural-symbolic models.

Original: 2. O PTIMIZING THE ELBO FOR V-NMN
Revised: 2. O PTIMIZING THE ELBO FOR V-NMN

Original: Rather than approaching the joint objective directly, we break ELBO training into three stages:
Revised: Rather than approaching the joint objective directly, we break ELBO training into three stages:

Original: Question coding , where we learn a latent ‘code’ for questions in the latent space of programs, 2.
Revised: Question coding , where we learn a latent ‘code’ for questions in the latent space of programs, 2.

Original: Module training , where we learn the ‘execution’ of the programs from the ﬁrst stage of training, grounding tokens in programs into the operations they perform on images; and 3.
Revised: Module training , where we learn the ‘execution’ of the programs from the ﬁrst stage of training, grounding tokens in programs into the operations they perform on images; and 3.

Original: Joint training , where we train both stages jointly given the ‘warm-starts’ from the other phases.
Revised: Joint training , where we train both stages jointly given the ‘warm-starts’ from the other phases.

Original: Question Coding.
Revised: Question Coding.

Original: The goal of this stage is to learn a code for questions in the latent space of programs – i.e. an informative mapping from questions to programs.
Revised: The goal of this stage is to learn a code for questions in the latent space of programs – i.e. an informative mapping from questions to programs.

Original: Given some set of question-aligned programs, we ﬁnd this coding process effectively propagates these supervised groundings to unsupervised questions, which enable interpretable explanations / groundings of queries to explanations of “what” the model is doing, which is our primary goal.
Revised: Given some set of question-aligned programs, we ﬁnd this coding process effectively propagates these supervised groundings to unsupervised questions, which enable interpretable explanations / groundings of queries to explanations of “what” the model is doing, which is our primary goal.

Original: During question coding, we drop the p θ z ( a | z , i ) term from Eq. 1, yielding the objective
Revised: During question coding, we drop the p θ z ( a | z , i ) term from Eq. 1, yielding the objective

Original: [Equation]
Revised: [Equation]

Original: Recent theory from Alemi et al.
Revised: Recent theory from Alemi et al.

Original: (2018) prescribes changes to the ELBO, setting β < 1 as a recipe to learn representations which avoid learning degenerate latent representations in the presence of powerful decoders.
Revised: (2018) prescribes changes to the ELBO, setting β < 1 as a recipe to learn representations which avoid learning degenerate latent representations in the presence of powerful decoders.

Revised: Essentially, Alemi et al.

Revised: (2018) identify that upto a constant factor, the negative log-marignal likelihood term D = − E z ∼ q φ ( z | x ) [log p θ ( x | z )] , and the kl diveregence term

Revised: R = KL( q φ ( z | x ) , p ( z )) bound the mutual information between the data x and the latent variable z as follows:

Revised: [Equation]

Revised: where H is the entropy of the data, which is a constant.

Original:  This aligns with our goal in question coding, and so we use β < 1 .
Revised: One can immediately notice that the standard elbo = − D − R . This means that for the same value of the elbo one can get different models which make drastically different usage of the latent variable (based on the achieved values of D and R ). Thus, one way to achieve a desired behavior (of say high mutual information I ( x , z ) ), is to set β to a lower value than the standard elbo (c.f. Eqn. 6 in Alemi et al. This aligns with our goal in question coding, and so we use β < 1 .

Original: As this no longer a valid variational lower bound, denote our objective as J ( x ; φ, θ ) from now on.
Revised: As this no longer a valid variational lower bound, denote our objective as J ( x ; φ, θ ) from now on.

Original: We optimize the above objective using the score function estimator (Glynn, 1990), also referred to as REINFORCE (Williams, 1992) applied to the ﬁrst term, with a moving average baseline.
Revised: We optimize the above objective using the score function estimator (Glynn, 1990), also referred to as REINFORCE (Williams, 1992) applied to the ﬁrst term, with a moving average baseline.

Original: Finally, while the KL term (in the reward) encourages the model to generate high-likelihood plausible sequences, we also penalize the model for producing syntactically invalid programs.
Revised: Finally, while the KL term (in the reward) encourages the model to generate high-likelihood plausible sequences, we also penalize the model for producing syntactically invalid programs.

Original: Although gradients for the KL term in Equation (2) can be computed by decomposing the KL for the sequence into per-timestep monte-carlo gradients, we found that having one term with per-timestep gradients and another that only receives sequence-level rewards (the likelihood term) skewed the model towards minimizing the KL, leading to auto-decoding solutions.
Revised: Although gradients for the KL term in Equation (2) can be computed by decomposing the KL for the sequence into per-timestep monte-carlo gradients, we found that having one term with per-timestep gradients and another that only receives sequence-level rewards (the likelihood term) skewed the model towards minimizing the KL, leading to auto-decoding solutions.

Original: Thus, we apply REINFORCE on the entire sequence and compute rewards using the full monte carlo gradient.
Revised: Thus, we apply REINFORCE on the entire sequence and compute rewards using the full monte carlo gradient, i.e. , we do not optimize the KL term separately, but instead maximize: [Equation]

Revised: where ˆ x , ˆ z are the questions and programs we observe in the dataset.

Original: For comparison, the semi-supervised approach in Johnson et al.
Revised: For comparison, the semi-supervised approach in Johnson et al.

Original: (2017) can be seen as optimizing the last term on supervised training data, while our question coding stage optimizes the elbo in addition to the last term.
Revised: (2017) can be seen as optimizing the last term on supervised training data, while our question coding stage optimizes the elbo in addition to the last term.

Original: Our results in Section 5 show that the proposed question coding formulation signiﬁcantly improves performance relative to Johnson et al.
Revised: Our results in Section 5 show that the proposed question coding formulation signiﬁcantly improves performance relative to Johnson et al.

Original: (2017) in terms of explaining “what” the model is doing on test programs.
Revised: (2017) in terms of explaining “what” the model is doing on test programs.

Original: Module Training.
Revised: Module Training.

Original: Having obtained a code for questions in the latent space, we train the parameters of the NMN for question answering.
Revised: Having obtained a code for questions in the latent space, we train the parameters of the NMN for question answering.

Original: The network modules are trained to perform their corresponding tasks.
Revised: The network modules are trained to perform their corresponding tasks.

Original: For example, if the question coding learned to associate the program token v with the question phrase ‘to the left’, then the goal for this second stage is to train the module corresponding to v to actually perform this operation ( e.g. transform[left] ) when executed on images.
Revised: For example, if the question coding learned to associate the program token v with the question phrase ‘to the left’, then the goal for this second stage is to train the module corresponding to v to actually perform this operation ( e.g. transform[left] ) when executed on images.

Original: We do this by optimizing the joint elbo( x , a | i ) in (1) with respect to the parameters θ z of the neural modules.
Revised: We do this by optimizing the joint elbo( x , a | i ) in (1) with respect to the parameters θ z of the neural modules.

Original: We write this objective for each instance as
Revised: We write this objective for each instance as

Original: [Equation]
Revised: [Equation]

Original: Notably, while we write this objective as a function of all neural modular network parameters θ z , for a given instance, only those modules present in the program z ∼ q ( z | x ) are updated.
Revised: Notably, while we write this objective as a function of all neural modular network parameters θ z , for a given instance, only those modules present in the program z ∼ q ( z | x ) are updated.

Original: In practice, we take  argmax decoded programs from q φ ( z | x ) to simplify training, but perform sampling during joint training in the next stage.
Revised: In practice, we take approx. argmax decoded programs from q φ ( z | x ) to simplify training, but perform sampling during joint training in the next stage.

Revised: To be more speciﬁc, we run beam search (Vijayakumar et al., 2018) with a beam size of 1 to do this.

Original: Intuitively, we think of this as a bias vs. variance tradeoff, doing argmax enables us to trade off bias for high-variance from a posterior which (in the pathological case) could match the prior from the ﬁrst stage of training.
Revised: Intuitively, we think of this as a bias vs. variance tradeoff, doing argmax enables us to trade off bias for high-variance from a posterior which (in the pathological case) could match the prior from the ﬁrst stage of training.

Revised: This is also consistent with standard practice for doing inference in sequence models, where beam search is the standard inference tool (and sampling often has issues due to teacher forcing based MLE training, which is how we do semi-supervised learning) (Bengio et al., 2015).

Original: Joint Training.
Revised: Joint Training.

Original: Having trained the question code and the neural module network parameters, we tune all terms jointly, with the following objective, which adds scaling factors α , β and γ , and an extra semi-supervised learning term relative to Equation (1):
Revised: Having trained the question code and the neural module network parameters, we tune all terms jointly, with the following objective, which adds scaling factors α , β and γ , and an extra semi-supervised learning term relative to Equation (1):

Original: [Equation]
Revised: [Equation]

Original: The gamma factor γ controls the scaling of the likelihood term for the answers relative to the question (which has more bits of information), similar scaling factors have been for joint models in Vedantam et al.
Revised: The gamma factor γ controls the scaling of the likelihood term for the answers relative to the question (which has more bits of information), similar scaling factors have been for joint models in Vedantam et al.

Revised: Similar to Equation (4), we optimize the full monte carlo gradient for the above objective as well.

Original: Our full training procedure is shown in Algorithm 1 below.[Table]
Revised: The full training procedure is shown in Algorithm 1 below.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: We ﬁrst explain other related work in neuro-symbolic models, the key differences between our formulation and closely related NMN work (Johnson et al., 2017; Hu et al., 2017; 2018), and then connect our work to other themes and directions of research in literature.
Revised: We ﬁrst explain other related work in neuro-symbolic models, the key differences between our formulation and closely related NMN work (Johnson et al., 2017; Hu et al., 2017; 2018), and then connect our work to other themes and directions of research in literature.

Original: Neural-symbolic models Valiant (2003)
Revised: Neural-symbolic models Valiant (2003)

Original: identiﬁes neuro-symbolic models, which are able to reason as well as learn efﬁciently as one of the key problems in computer science.
Revised: identiﬁes neuro-symbolic models, which are able to reason as well as learn efﬁciently as one of the key problems in computer science.

Original: A rich literature of work exists in this area, around two major threads: ﬁrstly, there is a line of work in building hybrid models  which are hybrids of symbolic reasoning and neural networks (Bader & Hitzler, 2005; H¨olldobler &{+
+}
Revised: A rich literature of work exists in this area, around two major threads: ﬁrstly, there is a line of work in building hybrid models [Table] [Equation] which are hybrids of symbolic reasoning and neural networks (Bader & Hitzler, {+
+}

Revised: ; H ¨ olldobler &

Original: Kalinke), the second thread looks at studying if neural networks can indeed do symbolic reasoning and building models and architectures to achieve this for speciﬁc cases (Evans et al., 2018a; Rockt¨aschel et al., 2015).
Revised: Kalinke), the second thread looks at studying if neural networks can indeed do symbolic reasoning and building models and architectures to achieve this for speciﬁc cases (Evans et al., 2018a; Rockt ¨ aschel et al., 2015).

Original: Our work is distinct in terms of being motivated by interpretability in the low-sample regime rather than getting connectionist/neural networks to explicitly do symbolic manipulations or building hybrid models that do logic. 
Revised: Our work is distinct in terms of being motivated by interpretability in the low-sample regime rather than getting connectionist/neural networks to explicitly do symbolic manipulations or building hybrid models that do logic. More recently, Yin et al.

Revised: (2018), propose a probabilistic latent variable model of structured programs which explain textual observations for tasks like parsing, i.e. , the model from Yin et al.

Revised: (2018) does not capture “how” to execute the programs that it generates.

Original: Connection to prior work on NMNs.
Revised: Our model is more general in the sense that we have a second modality which is the output of what gets executed when the program runs, and we model both jointly in our model, by learning how to execute the program. Work from Yi et al. (2018) also shares the same goal of doing question answering with limited question program supervision while preserving the semantics of human programs. While they to approach this problem by simplifying the p ( a | i , z ) mapping (in context of our model) by ﬁrst converting the image into a table, we approach this goal by modeling a stochastic latent space.

Revised: Thus the two approaches are orthogonal, as one can use our probabilistic latent space in conjunction with their model as well to better share statistics between question-program pairs.

Revised: Overall, compared to these works, our work addresses probabilistic neural-symbolic learning in a more general setting, by learning how to execute programs (on raw-pixels), as well as handling parsing of questions in an end to end manner.

Original:  Our V-NMN model is inspired in its parameterization of program to answer mapping from prior work on visual question answering using neural module networks such as Johnson et al.
Revised: Connection to prior work on NMNs. Our V-NMN model is inspired in its parameterization of program to answer mapping from prior work on visual question answering using neural module networks such as Johnson et al.

Original: ; Hu et al.
Revised: ; Hu et al.

Original: Distinct from these previous approaches, our explicit goal is to preserve the semantics of how a human speciﬁes a task to be broken down in terms of instructions, while previous work was motivated by building inductive biases for reasoning (which turn out to be useful for our model as well).
Revised: Distinct from these previous approaches, our explicit goal is to preserve the semantics of how a human speciﬁes a task to be broken down in terms of instructions, while previous work was motivated by building inductive biases for reasoning (which turn out to be useful for our model as well).

Original: Given this goal, we consrtuct a latent varible model that in some sense embeds previous work.
Revised: Given this goal, we consrtuct a latent varible model that in some sense embeds previous work.

Original: The works from Hu et al.
Revised: The works from Hu et al.

Original: (2017; 2018) are more distant from our work, as they use continuous attention modules, which do not capture independence assumptions of the program being a sufﬁcient statistic for answering the question (which makes them harder to specify completely for a human, as they would have to hallucinate attention parameters).
Revised: (2017; 2018) are more distant from our work, as they use continuous attention modules, which do not capture independence assumptions of the program being a sufﬁcient statistic for answering the question (which makes them harder to specify completely for a human, as they would have to hallucinate attention parameters).

Original: However, apart from our focus on preserving the lexicon/speciﬁcation given by a human, both approaches share the same high level goals of providing interpretable question answering with minimal supervision.
Revised: However, apart from our focus on preserving the lexicon/speciﬁcation given by a human, both approaches share the same high level goals of providing interpretable question answering with minimal supervision.

Original: Beyond Gaussian Latents in Amortized Variational Inference.
Revised: Beyond Gaussian Latents in Amortized Variational Inference.

Original: There has been exciting progress in developing lower-variance estimators for discrete generative models (Tucker et al., 2017; Mnih & Gregor, 2014; Titsias & L´azaro-Gredilla, 2015; Gu & Ling, 2015).
Revised: There has been exciting progress in developing lower-variance estimators for discrete generative models (Tucker et al., 2017; Mnih & Gregor, 2014; Titsias & L´azaro-Gredilla, 2015; Gu & Ling, 2015).

Original: In particular, Maddison et al.
Revised: In particular, Maddison et al.

Original: ; Jang et al.
Revised: ; Jang et al.

Original: (2017) propose the use of the concrete distribution on the probability simplex for providing biased but low variance estimators for discrete latent variable models by considering a reparametrization of samples (which asymptotically approach those from a categorical distribution).
Revised: (2017) propose the use of the concrete distribution on the probability simplex for providing biased but low variance estimators for discrete latent variable models by considering a reparametrization of samples (which asymptotically approach those from a categorical distribution).

Original: The contribution in our work is orthogonal to these approaches, how one can apply such parameterizations to neuro-symbolic models.
Revised: The contribution in our work is orthogonal to these approaches, how one can apply such parameterizations to neuro-symbolic models.

Original: More relevant to our work is Miao & Blunsom (2016) which proposes a discrete sequential latent variable model for sentence summarization, with summaries as the latent variable.
Revised: More relevant to our work is Miao & Blunsom (2016) which proposes a discrete sequential latent variable model for sentence summarization, with summaries as the latent variable.

Original: In contrast to their model, ours is a joint model where the latent space consists of programs, not summaries, which have to explain questions and answers to images.
Revised: In contrast to their model, ours is a joint model where the latent space consists of programs, not summaries, which have to explain questions and answers to images.

Original: Joint Variational Autoencoder Models.
Revised: Joint Variational Autoencoder Models.

Original: Vedantam et al.
Revised: Vedantam et al.

Original: (2018) propose a joint model for images and attributes for “visual imagination”, Suzuki et al.
Revised: (2018) propose a joint model for images and attributes for “visual imagination”, Suzuki et al.

Original: (2017) propose a closely related model and show that one can use the model for conditional generation, while Wang et al.
Revised: (2017) propose a closely related model and show that one can use the model for conditional generation, while Wang et al.

Original: (2016a) propose a deep variational canonical correlation analysis model which learns a joint space between two modalities for denoising and representation learning.
Revised: (2016a) propose a deep variational canonical correlation analysis model which learns a joint space between two modalities for denoising and representation learning.

Original: In contrast to these approaches, our joint model assumes a discrete structured latent variable (instead of a Gaussian), does semi-supervised training, which is possible for us, but not possible when there are no observations from the latent available for supervision, and is targeted towards neural module networks and question answering.
Revised: In contrast to these approaches, our joint model assumes a discrete structured latent variable (instead of a Gaussian), does semi-supervised training, which is possible for us, but not possible when there are no observations from the latent available for supervision, and is targeted towards neural module networks and question answering.

Original: Program Induction and Neural Architecture Search.
Revised: Program Induction and Neural Architecture Search.

Original: In program induction one is typically interested in learning to write programs given speciﬁcations of input-output pairs (Reed & de Freitas, 2015; Kalyan et al., 2018), and optionally language (Neelakantan et al., 2015; Guu et al., 2017) or vision (Gaunt et al., 2017).
Revised: In program induction one is typically interested in learning to write programs given speciﬁcations of input-output pairs (Reed & de Freitas, 2015; Kalyan et al., 2018), and optionally language (Neelakantan et al., 2015; Guu et al., 2017) or vision (Gaunt et al., 2017).

Original: The key difference between program induction and our problem is that program induction assumes the tokens/instructions in the language are grounded to the executor, i.e. , it is assumed that once a valid program is generated there is a black-box execution engine that can execute it, while we learn the execution engine from scratch, and learn the grounding from program symbols to parameters of a neural network for execution.
Revised: The key difference between program induction and our problem is that program induction assumes the tokens/instructions in the language are grounded to the executor, i.e. , it is assumed that once a valid program is generated there is a black-box execution engine that can execute it, while we learn the execution engine from scratch, and learn the grounding from program symbols to parameters of a neural network for execution.

Original: Neural architecture search (Zoph & Le, 2016) is another closely related problem where one is given input output examples for a machine learning task, and the goal is to ﬁnd a “neural program” that does well on the task.
Revised: Neural architecture search (Zoph & Le, 2016) is another closely related problem where one is given input output examples for a machine learning task, and the goal is to ﬁnd a “neural program” that does well on the task.

Original: In contrast, ours can be seen as the problem of inducing programs conditioned on particular inputs (questions), where the number of samples seen per question is signiﬁcantly lower than the number of samples one would observe for a machine learning task of interest.
Revised: In contrast, ours can be seen as the problem of inducing programs conditioned on particular inputs (questions), where the number of samples seen per question is signiﬁcantly lower than the number of samples one would observe for a machine learning task of interest.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Dataset.
Revised: Dataset.

Original: We perform our experiments on the SHAPES dataset (Andreas et al., 2016a).
Revised: We perform our experiments on the SHAPES dataset (Andreas et al., 2016a).

Original: This dataset has been used in previous works (Andreas et al., 2016a;
Revised: This dataset has been used in previous works (Andreas et al., 2016a;

Original: Hu et al., 2017) on visual question answering using neural module networks, and is intended to be a test for compositional generalization.
Revised: Hu et al., 2017) on visual question answering using neural module networks, and is intended to be a test for compositional generalization.

Original: The dataset has 244 unique binary questions ( i.e. having yes/no answers) and 15,616 images in total (Andreas et al., 2016a).
Revised: The dataset has 244 unique binary questions ( i.e. having yes/no answers) and 15,616 images in total (Andreas et al., 2016a).

Original: The dataset also has annotated programs for each of the questions.
Revised: The dataset also has annotated programs for each of the questions.

Original: We use train, val, and test splits of 13,568, 1,024, and 1,024 ( x , z , i , a ) triplets respectively.
Revised: We use train, val, and test splits of 13,568, 1,024, and 1,024 ( x , z , i , a ) triplets respectively.

Original: No questions are repeated across these splits.
Revised: No questions are repeated across these splits.

Original: Further, each question is asked for 64 different images, ensuring that a question can be answered correctly across all these instances only if it is processing the image in a reasonable way (as dictated by the underlying program), as opposed to leveraging any image-question correlations or biases that are unlikely to generalize across images.
Revised: Further, each question is asked for 64 different images, ensuring that a question can be answered correctly across all these instances only if it is processing the image in a reasonable way (as dictated by the underlying program), as opposed to leveraging any image-question correlations or biases that are unlikely to generalize across images.

Original: Thus, the dataset explicitly tests for compositional generalization and “reasoning” in the task of visual question answering.
Revised: Thus, the dataset explicitly tests for compositional generalization and “reasoning” in the task of visual question answering.

Original: The longest questions in the dataset are of length 11 and shortest are of length 4, while the longest programs are of length 6 and shortest programs are of length 4.
Revised: The longest questions in the dataset are of length 11 and shortest are of length 4, while the longest programs are of length 6 and shortest programs are of length 4.

Original: The size of the question vocabulary V x is 14 and the program vocabulary V z is 12.
Revised: The size of the question vocabulary V x is 14 and the program vocabulary V z is 12.

Original: Training.
Revised: Training.

Original: To simulate a data-sparse regime, we restrict the set of question-aligned programs to 5, 10, 15, or 20% of unique questions – such that even at the highest level of supervision, programs for 80% of unique question have never been seen during training.
Revised: To simulate a data-sparse regime, we restrict the set of question-aligned programs to 5, 10, 15, or 20% of unique questions – such that even at the highest level of supervision, programs for 80% of unique question have never been seen during training.

Original: Unless speciﬁed explicitly, we train our program prior using a set of 1848 unique programs simulated from the syntax (more details of this can be seen in the appendix).
Revised: Unless speciﬁed explicitly, we train our program prior using a set of 1848 unique programs simulated from the syntax (more details of this can be seen in the appendix).

Original: In general, it might not be possible to exhaustively enumerate the space of all valid strings to learn the prior.
Revised: In general, it might not be possible to exhaustively enumerate the space of all valid strings to learn the prior.

Original: Thus, we also evaluate the effect of training the prior with varying-sized subsets of the 212 programs present in the training set (without using any corresponding questions or answers) to check how robust overall performance is to the choice of the prior.
Revised: Thus, we also evaluate the effect of training the prior with varying-sized subsets of the 212 programs present in the training set (without using any corresponding questions or answers) to check how robust overall performance is to the choice of the prior (in the appendix).

Original: We ﬁnd sequence-level policy-gradient based training in this setting can be highly variable – with some runs failing spectacularly while others succeed – for both our method and to some extent NMN (Johnson et al., 2017).
Revised: We ﬁnd sequence-level policy-gradient based training in this setting can be highly variable – with some runs failing spectacularly while others succeed – for both our method and to some extent NMN (Johnson et al., 2017).

Original: For fairness and to reduce the impact of stochasticity, we perform the following stage-wise training regime for all models.
Revised: For fairness and to reduce the impact of stochasticity, we perform the following stage-wise training regime for all models.

Original: First, the question-coding stage is run fordifferent random seeds, producing multiple program predictors.
Revised: First, the question-coding stage is run for 5 different random seeds, producing multiple program predictors.

Original: Programs taken from the model that performs the best on the validation set are used to train 10 runs of the module training stage.
Revised: Programs taken from the model that performs the best on the validation set are used to train 10 runs of the module training stage.

Original: Finally, the best of these module networks (based on val) is trained jointly with the program predictor undersettings of α and the max (on val) is chosen.
Revised: Finally, the best of these module networks (based on val) is trained jointly with the program predictor under settings of γ ∈ { 1 , 10 , 100 } and the max (on val) is chosen.

Original: We report mean and variance accuracies on test from this training regime over four random sets of supervisory programs.
Revised: We report mean and variance accuracies on test from this training regime over four random sets of supervisory programs.

Original: Our evaluation metrics are as follows: a) question-coding uses reconstruction accuracy (of getting the  prediction  the  use 
Revised: Our evaluation metrics are as follows: a) question-coding uses reconstruction accuracy (of getting the entire question right) and program prediction accuracy (of matching the ground truth programs) on the training set, since the second stage, module training will use inferences from the ﬁrst stage (on the training set to train modules), b) module training and joint training: visual question answering accuracy, by counting matches to ground truth answers.

Original: Implementation Details :
Revised: Implementation Details :

Original: Following Johnson et al.
Revised: Following Johnson et al.

Original: (2017) and Hu et al.
Revised: (2017) and Hu et al.

Original: (2017), we treat the program
Revised: (2017), we treat the program

Original: z = { z 1 , · · · , z K } as a variable length sequence of tokens with an end of sequence token, which corresponds to the preﬁx traversal of an execution tree.
Revised: z = { z 1 , · · · , z K } as a variable length sequence of tokens with an end of sequence token, which corresponds to the preﬁx traversal of an execution tree.

Original: Given this representational choice, we learn the prior p ( z ) using maximum likelihood training on simulated samples from known syntax of programs.
Revised: Given this representational choice, we learn the prior p ( z ) using maximum likelihood training on simulated samples from known syntax of programs.

Original: An alternative, interesting approach to try would be to use something like Janz et al.
Revised: An alternative, interesting approach to try would be to use something like Janz et al.

Original: (2017), who acquire a generative model more directly from validity annotations.
Revised: (2017), who acquire a generative model more directly from validity annotations.

Original: The log p ( a | z , i ) term in Equation (1) is parameterized by an NMN that is dynamically assembled given the program string, while the generative model p ( x | z ) , and inference ( q ( z | x ) ) terms are both sequence to sequence models.
Revised: The log p ( a | z , i ) term in Equation (1) is parameterized by an NMN that is dynamically assembled given the program string, while the generative model p ( x | z ) , and inference ( q ( z | x ) ) terms are both sequence to sequence models.

Original: Our sequence to sequence models are based on LSTM (Hochreiter & Schmidhuber, 1997) cells.
Revised: Our sequence to sequence models are based on LSTM (Hochreiter & Schmidhuber, 1997) cells.

Original: For the NMN programs, we use syntax similar to that used by Hu et al.
Revised: For the NMN programs, we use syntax similar to that used by Hu et al.

Original: , but we replace their question-attention neural modules with “regular” modules.
Revised: , but we replace their question-attention neural modules with “regular” modules.

Original: For clear comparison we adapt the objectives from the baseline approach, Johnson et al.
Revised: For clear comparison we adapt the objectives from the baseline approach, Johnson et al.

Original: (2017), to our representational choices.
Revised: (2017), to our representational choices.

Original: More details can be found in appendix.
Revised: More details can be found in appendix.

Original:  R ESULTS
Revised:  R ESULTS

Original: We focus on evaluating approaches by varying the fraction of quesiton-aligned programs (which we denote % x ↔ z ) in Table 1.
Revised: We focus on evaluating approaches by varying the fraction of quesiton-aligned programs (which we denote % x ↔ z ) in Table 1.

Original: To put the numbers in context, a baseline LSTM + image model, which does not use module networks, gets an accuracy of 63.0% on Test (see Andreas et al.
Revised: To put the numbers in context, a baseline LSTM + image model, which does not use module networks, gets an accuracy of 63.0% on Test (see Andreas et al.

Original: (2016a)
Revised: (2016a)

Original: ; Table.
Revised: ; Table.

Original: This indicates that SHAPES has highly compositional questions which is challenging to model via conventional methods for Visual Question Answering (Agrawal et al., 2015).
Revised: This indicates that SHAPES has highly compositional questions which is challenging to model via conventional methods for Visual Question Answering (Agrawal et al., 2015).

Original: Overall, we make the following observations:
Revised: Overall, we make the following observations:

Original: - Our V-NMN approach consistently improves performance in data-sparse regimes.
Revised: - Our V-NMN approach consistently improves performance in data-sparse regimes.

Original: While both methods tend to improve with greater program supervision, V-NMN quickly outpaces NMN (Johnson et al., 2017), achieving test accuracies over 20-35% points higher for > 5% program supervision.
Revised: While both methods tend to improve with greater program supervision, V-NMN quickly outpaces NMN (Johnson et al., 2017), achieving test accuracies over 20-35% points higher for > 5% program supervision.

Original: Notably, both methods perform similarly poorly on the test set given only 5% program supervision, suggesting this may be too few examples to learn compositional reasoning.[Table] [Table]
Revised: Notably, both methods perform similarly poorly on the test set given only 5% program supervision, suggesting this may be too few examples to learn compositional reasoning.

Original:  Our question coding stage greatly improves initial program prediction.
Revised: - Our question coding stage greatly improves initial program prediction.

Original: Our V-NMN approach to question coding gets to approximately double the question coding (reconstruction) accuracy as the NMN approach (col 1, question coding).
Revised: Our V-NMN approach to question coding gets to approximately double the question coding (reconstruction) accuracy as the NMN approach (col 1, question coding).

Original: This means that it effectively propagates groundings from question-aligned programs during the coding phase.
Revised: This means that it effectively propagates groundings from question-aligned programs during the coding phase.

Original: Consequently, we also see improved VQA performance after the module and joint training stages which are based on predicted programs
Revised: Consequently, we also see improved VQA performance after the module and joint training stages which are based on predicted programs.

Original: - Question reconstruction is well-correlated with overall accuracy.
Revised: - Question reconstruction is well-correlated with overall accuracy.

Original: Despite the program space being trained directly as a latent code for questions, we ﬁnd question reconstruction accuracy rises with the amount of supervision – conﬁrming that inference in a space of discrete structured latent variables is difﬁcult to do without some supervision to guide the mapping
Revised: Despite the program space being trained directly as a latent code for questions, we ﬁnd question reconstruction accuracy rises with the amount of supervision – conﬁrming that inference in a space of discrete structured latent variables is difﬁcult to do without some supervision to guide the mapping.

Original: Figure 3 shows sample programs for each model.
Revised: Figure 3 shows sample programs for each model.

Original: We see the limited supervision negatively affects
Revised: We see the limited supervision negatively affects

Original: NMN program prediction, with the 5% model resorting to simple Find[X] → Answer structures.
Revised: NMN program prediction, with the 5% model resorting to simple Find[X] → Answer structures.

Original: Interestingly, we ﬁnd that the mistakes made by the V-NMN model, e.g. , green in 5% supervision (top-right) are also made when reconstructing the question (also substituting green for blue).
Revised: Interestingly, we ﬁnd that the mistakes made by the V-NMN model, e.g. , green in 5% supervision (top-right) are also made when reconstructing the question (also substituting green for blue).

Original: Further, when the token does get corrected to blue, the question also eventually gets reconstructed (partially correctly (10%) and then fully (15%)), and the program produces the correct answer.
Revised: Further, when the token does get corrected to blue, the question also eventually gets reconstructed (partially correctly (10%) and then fully (15%)), and the program produces the correct answer.

Original: This indicates that there is high ﬁdelity between the learnt question space, the answer space and the latent space.
Revised: This indicates that there is high ﬁdelity between the learnt question space, the answer space and the latent space.

Original: Effect of the Program Prior p ( z ) .
Revised: Effect of the Program Prior p ( z ) .

Original: Next, we explore the impact of regularizing the program posterior to be close to the prior p ( z ) , for different choices of the prior.
Revised: Next, we explore the impact of regularizing the program posterior to be close to the prior p ( z ) , for different choices of the prior.

Original: Firstly, we disable the KL-divergence term by setting β = 0 in Equation (2) recovering a deterministic version of our model.
Revised: Firstly, we disable the KL-divergence term by setting β = 0 in Equation (2) recovering a deterministic version of our model.

Original: Compared to our full model at 10% supervision, the performance on question coding drops to 12.7% program  prediction accuracy (from 65.45%), while the performance on question reconstruction improves from 87.52% to 93.8%.
Revised: Compared to our full model at 10% supervision, the performance on question coding drops to 12.7% program [Table] prediction accuracy (from 65.45%), while the performance on question reconstruction improves from 87.52% to 93.8%.

Original: Without the KL term, the model focuses solely on reconstruction and fails to learn compositionality in the latent z space, such that supervised grounding are poorly propagated to unsupervised questions as evidenced by the drop in program prediction accuracy.
Revised: Without the KL term, the model focuses solely on reconstruction and fails to learn compositionality in the latent z space, such that supervised grounding are poorly propagated to unsupervised questions as evidenced by the drop in program prediction accuracy.

Original: Note that this ties well to our high-level goal to better leverage the semantics of the lexicon provided by the user for interpretable question answering.
Revised: Note that this ties well to our high-level goal to better leverage the semantics of the lexicon provided by the user for interpretable question answering.

Revised: Beam search vs. sampling for module training.

Revised: As explained in Section 2, we use beam search instead of sampling to optimize Equation (5).

Revised: In this section we empirically study this design choice.

Revised: We generally ﬁnd that in low-supervision settings, where the question coding stage might not produce correct programs (for questions) as frequently, using sampling form the sequence model q ( z | x ) instead of beam search leads to a drop in module training performance.

Original:  Effect of optimizing the true ELBO, β = 1 .
Revised: For example, with 10% supervision, the module training accuracy drops from 81.34 ± 8.61 (with beam search) to 76. ± 8.41 (with sampling). With more supervision the choice of beam search or sampling does not matter, which makes intuitive sense, as the this means that the programs that get sampled tend to be the correct ones more often (that is, the conﬁdence of the model increases with the improvement in correctness). Effect of optimizing the true ELBO, β = 1 .

Original: Next, we empirically validate the claim that approximate inference learned using the ELBO leads to autodecoding behavior as predicted by Alemi et al.
Revised: Next, we empirically validate the claim that approximate inference learned using the ELBO leads to autodecoding behavior as predicted by Alemi et al.

Original: (2018), and that that better program representations for our problem can be found for β < 1 (as shown above).
Revised: (2018), and that that better program representations for our problem can be found for β < 1 (as shown above).

Original: At 15% program supervision, VQA accuracy drops from 96.90% ( ± 0 . 96 ) at β = 0 .to
Revised: At 15% program supervision, VQA accuracy drops from 96.90% ( ± 0 . 96 ) at β = 0 .to

Original: 78.49% ( ± 1.37) at β = 1 (for the empirical prior).
Revised: 78.49% ( ± 1.37) at β = 1 (for the empirical prior).

Original: Question reconstruction drops from 97.43 % ( ± 3 . 42 ) at beta = 0 .
Revised: Question reconstruction drops from 97.43 % ( ± 3 . 42 ) at β = 0 .

Original: 1 , to 43.40 % ( ± 5 . 34 ) at β = 1 , showing auto-decoding behavior (as predicted by theory).
Revised: 1 , to 43.40 % ( ± 5 . 34 ) at β = 1 , showing auto-decoding behavior (as predicted by theory).

Original: Finally, the N2NMN approach (Hu et al., 2017) evaluates their question-attention based module networks in the fully unsupervised setting, getting to 96.19% on TEST.
Revised: Finally, the N2NMN approach (Hu et al., 2017) evaluates their question-attention based module networks in the fully unsupervised setting, getting to 96.19% on TEST.

Original: However, the programs in this case become non-compositional (see Section 3), as the model leaks information from questions to answers via attention, meaning that programs no longer carry the burden of explaining the observed answers.
Revised: However, the programs in this case become non-compositional (see Section 3), as the model leaks information from questions to answers via attention, meaning that programs no longer carry the burden of explaining the observed answers.

Original: This makes the modules less interpretable; which is one of the core motivations of this neural-symbolic class of module networks in the ﬁrst place.
Revised: This makes the modules less interpretable; which is one of the core motivations of this neural-symbolic class of module networks in the ﬁrst place.

Original:  D ISCUSSION AND C ONCLUSION
Revised:  D ISCUSSION AND C ONCLUSION

Original: In this paper we presented a novel, probabilistic neural symbolic model for interpretable visual question answering, that provides explanations of “what” the model is doing on unseen questions given a minimal number of annotated symbolic traces or programs of “what it should do”.
Revised: In this paper we presented a novel, probabilistic neural symbolic model for interpretable visual question answering, that provides explanations of “what” the model is doing on unseen questions given a minimal number of annotated symbolic traces or programs of “what it should do”.

Original: We demonstrate  our formulation provides more interpretable explanations than previousprevious work on visual question answering called neural module networks, on dataset of compositional questions about shapes (Andreas et al., 2016a).
Revised: We demonstrate that our formulation provides more interpretable explanations than previous work on visual question answering called neural module networks, on dataset of compositional questions about shapes (Andreas et al., 2016a).

Original: The key to our approach is a model of programs as a stochastic latent variable, which leads to better sharing of statistics across questions, yeilding a latent space where program annotations for known questions propagate effectively to unknown / novel questions without annotations.
Revised: The key to our approach is a model with programs as a stochastic latent variable, which leads to better sharing of statistics across questions, yielding a latent space where program annotations for known questions propagate effectively to unknown / novel questions without annotations.

Original: In general, the model family we study is quite rich, and also supports other inference queries such as counter factual explanations of what programs could have led to particular answers for a given image.
Revised: In general, the model family we study is quite rich, and also supports other inference queries such as counter factual explanations of what programs could have led to particular answers for a given image.

Original: We hope our work inspires more work on applying such ideas to other tasks requiring compositional reasoning such as planning, navigation etc.
Revised: We hope our work inspires more work on applying such ideas to other tasks requiring compositional reasoning such as planning, navigation etc.

Original: A PPENDIX 
Revised: A PPENDIX . I MPLEMENTATION D ETAILS

Original: We provide more details on the modeling choices and hyperparameters used to optimize the models in the main paper.
Revised: We provide more details on the modeling choices and hyperparameters used to optimize the models in the main paper.

Original: Sequence to Sequence Models: All our sequence to sequence models described in the main paper (Section. 2) are based on LSTM cells with a hidden state of 128 units, single layer depth, and have a word embedding of 32 dimensions for both the question as well as the program vocabulary.
Revised: Sequence to Sequence Models: All our sequence to sequence models described in the main paper (Section. 2) are based on LSTM cells with a hidden state of 128 units, single layer depth, and have a word embedding of 32 dimensions for both the question as well as the program vocabulary.

Original: The when sampling from a model, we sample till the maximum squence length of 15 for questions and 7 for programs.
Revised: The when sampling from a model, we sample till the maximum sequence length of 15 for questions and 7 for programs.

Original: Image CNN: The SHAPES images are of 30x30 in size and are processed by a two layerd convolutional neural network, which in its ﬁrst layer has a 10x10 ﬁlters applied at a stride of 10, and in the second layer, it does 1x1 convolution applied at a stride of 1.
Revised: Image CNN: The SHAPES images are of 30x30 in size and are processed by a two layerd convolutional neural network, which in its ﬁrst layer has a 10x10 ﬁlters applied at a stride of 10, and in the second layer, it does 1x1 convolution applied at a stride of 1.

Original: Channel widths for both the layers are 64 dimensional.
Revised: Channel widths for both the layers are 64 dimensional.

Original: Moving average baseline For a reward R , we use an action independent baseline b to reduce variance, which tracks the moving average of the rewards seen so far during training.
Revised: Moving average baseline For a reward R , we use an action independent baseline b to reduce variance, which tracks the moving average of the rewards seen so far during training.

Original: Concretely, the form for the for the q ( z | x ) network is:
Revised: Concretely, the form for the for the q ( z | x ) network is:

Original: [Equation]
Revised: [Equation]

Original: 5) where, given D as the decay rate for the baseline,
Revised: where, given D as the decay rate for the baseline,

Original: [Equation]
Revised: [Equation]

Original: is the update on the baseline performed at every step of training.
Revised: is the update on the baseline performed at every step of training.

Original: Note that in practice, when estimating the gradient for the variational lower bound, we consider the total derivative (which adds a term having a gradient with respect to R above), since in the case of sample-based optimization of the elbo , the reweard itself is also a function of the variational parameters, via.
Revised: Note that in practice, when estimating the gradient for the variational lower bound, we consider the total derivative (which adds a term having a gradient with respect to R above), since in the case of sample-based optimization of the elbo , the reward itself is also a function of the variational parameters, via.

Original: both the sample used estimate the reward, as well as the reward directly being a function of q ( z | x ) .
Revised: both the sample used estimate the reward, as well as the reward directly being a function of q ( z | x ) .

Original: Hyperparameters for training: We use the ADAM optimizer (Kingma & Ba, 2014) with a learning rate of 1e-3 , a minibatch size of 576, and use a moving average baseline for REINFORCE, with a decay factor of 0.99.
Revised: Hyperparameters for training: We use the ADAM optimizer (Kingma & Ba, 2014) with a learning rate of 1e-3 , a minibatch size of 576, and use a moving average baseline for REINFORCE, with a decay factor of 0.99.

Original: Typical values of β are set to 0.1, α is set to 100 and γ is chosen on a validation set among (0.1, 1.0, 10.0, 100.0).
Revised: Typical values of β are set to 0.1, α is set to 100 and γ is chosen on a validation set among (1.0, 10.0, 100.0).

Original: Simulating programs from known syntax: We follow a simple two-stage heuristic procedure for generating a set of samples to train the program prior in our model.
Revised: Simulating programs from known syntax: We follow a simple two-stage heuristic procedure for generating a set of samples to train the program prior in our model.

Original: Firstly, we build a list of possible future tokens given a current token, and sample a random token from that list, and repeat the procedure for the new token to get a large set of possible valid sequences – we then pass the set of candidate sampled sequences through a second stage of ﬁltering based on constraints from the work of Hu et al. 
Revised: Firstly, we build a list of possible future tokens given a current token, and sample a random token from that list, and repeat the procedure for the new token to get a large set of possible valid sequences – we then pass the set of candidate sampled sequences through a second stage of ﬁltering based on constraints from the work of Hu et al. . A DDITIONAL R ESULTS

Revised: Empirical vs. syntactic priors While our default choice of the prior p ( z ) is from programs simulated from the known syntax, it might not be possible to exhaustively enumerate all valid program strings in general.

Revised: Here, we consider the special case where we train the prior on a set of unaligned , ground truth programs from the dataset.

Revised: Interestingly, we ﬁnd that the performance of the question coding stage, especially at reconstructing the questions improves signiﬁcantly when we have the syntactic prior as opposed to the empirical prior (from 38.39 ± 11.92% to 54.86 ± 6.75% for 5% program supervision).

Revised: In terms of program prediction accuracy, we also observe marginal improvements in the cases where we have 5% and 10% supervision respectively, from 56.23 ± 2.81% to 65.45 ± 11.88%.

Revised: When more supervision is available, regularizing with respect to the broader, syntactic prior hurts performance marginally, which makes sense, as one can just treat program supervision as a supervised learning problem in this setting.