Learning Invariants through Soft Unification
Authors: Nuri Cingillioglu
Alessandra Russo
Venue:  (2020)
Keywords: representation learning
neural networks
unification

Abstract: Human reasoning involves recognising common underlying principles across many examples by utilising variables. The by-products of such reasoning are invariants that capture patterns across examples such as "if someone went somewhere then they are there" without mentioning specific people or places. Humans learn what variables are and how to use them at a young age, and the question this paper addresses is whether machines can also learn and use variables solely from examples without requiring human pre-engineering. We propose Unification Networks that incorporate soft unification into neural networks to learn variables and by doing so lift examples into invariants that can then be used to solve a given task. We evaluate our approach on four datasets to demonstrate that learning invariants captures patterns in the data and can improve performance over baselines.

Under review as a conference paper at ICLR 2020
LEARNING INVARIANTS THROUGH SOFT UNIFICATION
Anonymous authors
Paper under double-blind review
ABSTRACT
Human reasoning involves recognising common underlying principles across many examples by utilising variables. The by-products of such reasoning are invariants that capture patterns across examples such as "if someone went some- where then they are there" without mentioning specific people or places. Humans learn what variables and how to use them a young age, and the question this paper addresses is whether machines can also learn and use variables solely from examples without requiring human pre-enginecring. We propose Unification Net- works that incorporate soft unification into neural networks to learn variables and by doing so lift examples into invariants that can then be used to solve a given task. We evaluate our approach on four datasets to demonstrate that learning invariants captures patterns in the data and can improve performance over baselines.
INTRODUCTION
Humans have the ability to process symbolic knowledge and maintain symbolic thought (Unger & Deacon 1998). When reasoning. humans do not require combinatorial enumeration of exam- ples but instead utilise invariant patterns with placeholders replacing specific entities. Symbolic cognitive models (Lewis) 1999) embrace this perspective with the human mind seen as informa- tion processing system operating on formal symbols such as reading a stream of tokens in natural language. The language of thought hypothesis (Morton & Fodor 1978) frames human thought structural construct with varying sub- uub-components such as "X went to Y By recognising what varies across examples, humans are capable of lifting examples into invariant principles that account for other instances. This symbolic thought with variables is learned at young age through symbolic play (Piaget 200T). For instance child learns that sword can be substituted with a stick (Frost et al. 2004) and engage in pretend play.
Although variables are inherent in models of computation and symbolic formalisms, as in first-order logic (Russell & Norvig 2016), they are pre-engineered and used to solve specific tasks by means of unification assignments that bound variables to given values. However, when learning from data only, being able to recognise when and which symbols should take dif- ferent values, i.e. symbols that can act as variables, is crucial for lifting examples into general principles that are invariant across multiple instances. Figure [hswows the invariant learned by our approach: if someone is the same thing someone else then they have the same colour. With this invariant, our approach can solve all of the training and test examples in task 16 of the bAbl dataset (Weston al. 2016).
X:bennhard is a Y:frog
Z:lily is Y:frog Z:lily is :green
what colour is X:bernhard
Aggreen
Figure 1: Invariant learned for bAbl task 16, basic induction, where X:bernhard denotes vari- able with default symbol bernhard. This single invariant accounts for all the training and test examples.
In this paper we address the question of whether a machine can learn and use the notion of variable, i.e. symbol that can take on different values. For instance, given an example of the form "bernhard is a frog" the machine would learn that the token "bernhard" could be someone else and the token "frog" could be something else. If consider unification selection of the most appropriate value for a variable given a choice of values, we can reframe it as form of attention. Attention models (Bahdanau et al. 2015 Luong et al.) 2015 Chaudhari et al. 2019) allow neural networks to focus, attend to certain parts of the input often for the purpose of selecting relevant portion. Since attention mechanisms are also differentiable they are often jointly

---

Under review as a conference paper at ICLR 2020
learned within task. This perspective motivates idea of unification mechanism that utilises attention and is therefore fully differentiable which we refer to as soft unification.
Hence, propose end-to-end differentiable neural network approach for learning and utilising the notion of a variable that in return can lift examples into invariants used by the network to perform reasoning tasks. Specifically, we (i) propose novel architecture capable of learning and using variables by lifting a given example through soft unification, (ii) present the empirical results of our approach four datasets and (iii) analyse the learned invariants that capture the underlying patterns present in the tasks. Our implementation using Chainer (Tokui et al., 2015) is publicly available at [link removed] (anonymous 1ink provided with submi ciomisssin...
SOFT UNIFICATION
Reasoning with variables involves identifying what variables are, the setting in which they are used as well as the process by which they are assigned values. When the varying components, i.c. vari- ables, of example are identified, the remaining structure be lifted into an invariant which then accounts for multiple other instances.
Definition (Variable). Given a set of symbols S, a variable X is defined as a pair X 2 (x, sa) where sd & is the default symbol of the variable and is a discrete random variable of which the support is S. The representation of variable Φv (X:s)) is equal to the expected value of the corresponding random variable x given the default symbol sd:
(X:sa) = =l=p(((>)]
#E
where S > Rd is a d-dimensional real-valued feature of a symbol 8.
For example, á could be embedding and øv (X:sa) would become weighted sum of symbol embeddings as in conventional attention models. The default symbol of a variable is intended to capture the variable's bound meaning following the idea of referants by Frege (1948). We denote variables using X, Y, A etc. such as X:bernhard where is the name of the variable and bernhard the default symbol shown in Figure !
Definition (Invariant). Given a structure (e.g. list, grid) g over S, an invariant is a pair é (G, eb) where G E G is the invariant example such as tokenised story with tokens as symbols and 1/ S [0, is function representing the degree to which the symbol considered a variable. Thus, the final representation of symbol included in G, (r(s) is:
( (1 (()))(()) (s))vv
the linear interpolation between representation ((s) and variable bound value with itself as the default symbol Φv (X:s).
We adhere to the term invariant and refrain from mentioning rules, unground rules, etc. used in logic- based formalisms, e.g. Muggleton & de Raedt (994).. since neither the invariant structure needs to be rule-like nor the varaales carry logical semantics. This distinction clarified in Section (
Definition 3 (Unification). Given an invariant I and example K G. unification binds the variables in I to symbols in K. Defined as a function : I x g - G, unification binds variables by computing the probability mass functions, in equation]] [ and returns the unified representation using equation 2 The probability mass function of a variable X:sa is:
P(x x s/sa) = softmax (of))ax((o(
support of
(3)
where dv S + Rdd is the unifying feature of symbol and du (K) e k is applied element wise to symbols in K. If g is differentiable, it is referred to as soft unification.
We distinguish e from du emphasise that the unifying properties of the symbols might be different from their representations. For example, ((bernhard) could represent a specific person whereas Φu (bernhard) the notion of someone.
Overall soft unification incorporates 3 learnable components: o, vh, δy which denote the base fea- tures, variableness and unifying features of symbol respectively. Given upstream, potentially
2

---

Under review as a conference paper at ICLR 2020
task specific, network f G - S, an invariant I and an input example K E with a corre- sponding desired output a E S, the following holds:
fo g(T, K) f(K)= =
where f now predicts based on the unified representation produced by g. In this work, we focus on the invariants it produces together with the interaction of f o
3 UNIFICATION NETWORKS
Since soft unification is end-to-end differentiable, it can be incorporated into existing task-specific upstream architectures. We present architectures that model f o using multi-layer percep- trons (MLP), convolutional neural networks (CNN) and memory networks (Weston et al. 2015) to demonstrate the flexibility of our approach. all cases, the dimensional representation of sym- bols are learnable embeddings ((s) O[s)² E with E RISIxd randomly initialised by N/, 1) and [ss) the one-hot encoding of the symbol. The variableness of symbols is a learnable weight the (s) or (w.) where w E g| and is the sigmoid function. We consider every symbol indepen- dently a variable irrespective of its surrounding context and leave further contextualised formulations as future work. The underlying intuition of this configuration is that useful symbol for correct prediction might need to take on other values for different inputs. This usefulness can be viewed the inbound gradient to the corresponding w, parameter and (s) acting as gate. For further model details including the size of the embeddings, please refer to Appendix A
Unification MLP (UMLP) (f: MLP, g: RNN) We combine soft unification into multi-layer perceptron to process fixed length inputs. In this case, the structure G is a sequence of symbols with a fixed length e.g. a sequence of digits 4234. Given an embedded input Φ(k) Rxxdd the upstream MLP computes the output symbol based on the flattened representations f(((k)) voftmax(hEE") where h I2d is the output of the last layer. However, to compute the unifying features Qy. definition 3 g uses a bi-directional GRU (Cho et al. (2014) running over o(k) such that (b)()) = Wy (k) where (k) E IRd the hidden state of the GRU at symbol and Wo E Rdxdd is a learnable parameter. This model emphasises the flexibility around the boundary of o and that the unifying features can be computed in any differentiable manner.
Memory RNN
unification
context attention
Unfiiction RNN
greg is a swan
X:bernhard Yrrrog
brian is lion
Z:lily is Aigreen
julius is a swan
Z:iily is Ydrrog
jullus is white
what colour is Xbbemhard
what colour greg
0
Invariant
Aggreen
Example
while
Iteration
Figure 2: Graphical overview of soft unification within a memory network. Each sentence is pro- cessed by two bi-directional RNNs for memory and unification. At each iteration the context atten- tion selects which sentences to unify and the invariant produces the same answer as the example.
Unification CNN (UCNN) (f: CNN, g: CNN) Given a grid of embedded symbols o(K) R=xhxd where w is the width and h the height, we use a convolutional neural network such that the final prediction f(o(K)) softmax( (Wh + b) E") where h this time the result of global max pooling and W, b are learnable parameters. We also model g using separate convolutional network with the same architecture as and set du (k) = eg (relu(c)(k) (k)) where 61, 02 are the con- volutional layers. The grid is padded with Os to obtain every symbol has a unifying feature. This model conveys how soft unification can be adapted to the specifies of the domain for example by using convolution in spatially structured input.
h d after each convolution such that
Unification Memory Networks (UMN) (f: MemNN, g: RNN) Soft unification does not need to happen prior to f in a o g fashion but can also be incorporated at any intermediate stage multiple times. To demonstrate this ability, we unify the symbols at different memory locations at each iteration of Memory Network (Weston al. 2015). Memory networks handle list of lists
3

---

Under review as a conference paper at ICLR 2020
Table 1: Sample context, query and answer triples and their training sizes per task. For distribution of generated number of examples per task on Sequence and Grid data refer to Appendix B
Dataset
Context
Query
Answer
Training Size
Sequence
8384
duplicate
8
S 1k, S 50
0 0 3
Grid
0 1 6
corner
7
I 1k, S 50
8 5
Mary went to the kitchen.
bAbl
Where is Mary?
kitchen True
1k, 50
Sandra journeyed to the garden.
p(X) + q(X). q(a).
Logic
P(a).
1k, 10k, 50
structure such as a tokenised story as shown in Figure The memory network f uses the final hidden state of bi-directional GRU (outer squares in Figure 2 as the sentence representations to compute a context attention. At each iteration, we unify the words between the attended sentences using the same approach UMLP with another bi-directional GRU (inner diamonds in Figure 2 for unifying features Φu (bernhard) We W (bernhard). Following equation 2 the new unified representation of the memory slot computed and f uses to perform the next iteration. Concretely, 9 produces an unification tensor U R xmx Xxd where M and m is the number of sentences and words in the invariant respectively. and is the number of sentences in the example such that after the context attentions are applied over M and N, obtain <(k) R the unified sentence at that iteration. Note that unlike in the UMLP case, the sentences can be of varying length. The prediction is then softmax haitmaxxWh'" b) where h is the hidden state of the invariant after J iterations. This setup, however, requires pre-training, f such that the context attentions match the correct sentences.
A task might contain different questions such "Where is X?" and "Why did X go Y?". To let the models differentiate hetween questions and potentially learn different invariants, we extend them with a repository of invariants 1 and aggregate the predictions from each invariant. One simple approach is to sum the predictions of the invariants Ei f o g(L, K) used in UMLP and UCNN. Another approach could be to use features from the invariants such as memory representations in the case of UMN. For UMN, we weigh the predictions using bilinear attention 1j based on the Whh, h% To initially form the repository of invariants, we use the bag-of-words representation of the questions and find the most dissimilar ones based their cosine similarity as a heuristic obtain varied examples.
hidden states the first iteration h. and h& such that ' softmax(h,
DATASETS
We use 4 datasets consisting of context, query and an answer (C, C, a): fixed length sequences of symbols, shapes of symbols in a grid, story based natural language reasoning with the bAbl (Weston al. 2016) dataset and logical reasoning represented as logic programs, examples shown in Table with further samples in Appendix
B In each case we use an appropriate model: UMLP for fixed length sequences, UCNN for grid and UMN for iterative reasoning. We use synthetic datasets of which the data generating distributions are known to evaluate not only the quantitative performance but also the quality of the invariants learned by our approach.
Fixed Length Sequences We generate sequences of length 1 = 4 with 8 unique symbols represented as digits to predict (i) a constant, (ii) the head the sequence, (iii) the tail and (iv) the duplicate symbol. We randomly generate 1000 triples and then only take the unique ones to ensure the test split contains unseen examples. The training is then performed over a 5-fold cross validation.
Grid To spatially organise symbols, generate grid of size 3x3 3 with 8 unique symbols organised into 2 x 2 box of identical symbol, a vertical, diagonal or horizontal sequence of length 3, a cross plus shape and triangle. In each task predict (i) the identical symbol, (ii) the head of the sequence, (iii) the centre of the cross or plus and (iv) the corner of the triangle respectively. We follow the same procedure from sequences and randomly generate 1000 discarding duplicate triples.
bAbl The bAbl dataset has become a standard benchmark for evaluating memory based networks. It consists of 20 synthetically generated natural language reasoning tasks (refer Weston et al.
4

---

Under review as a conference paper at ICLR 2020
2016 for task details). We take the 1k English set and use 0.1 of the training set validation. Each token is lower cased and considered a unique symbol. Following previous works (Seo et al. 2017
Sukhbaatar et al. 2015), take multiple word answers also to be unique symbol in S.
Logical Reasoning To demonstrate the flexibility of our approach and distinguish our notion of variable from that used in logic based formalisms, we generate logical reasoning tasks in the form of logic programs using the procedure by Cingillioglu & Russo (2019). The tasks involve learning f(C, True + C Q over 12 classes of logic programs exhibiting varying paradigms of logical reasoning including negation by failure (Clark) 1978). We generate Ik and 10k logic programs per task training with 0.1 as validation and another 1k for testing. We the arity of literals to 1 or 2 using one random character from the English alphabet for predicates and constants, e.g. p(p) and an upper case character for logical variables, e.g. p(x).
EXPERIMENTS
We probe three aspects of soft unification: the impact of unification performance unseen data, the effect of multiple invariants and data efficiency. To that end, we train UMLP and UCNN with and without unification, UMN with pre-training using
3 invariants over either the en- tire training set or only 50 examples. Every model is trained 3 times via back-propagation using Adam (Kingma Ba 2013) on Intel Core 17-6700 CPU using the following objective function:
= Xxx + Au Cnn(s - (), +T Σ (w)(s
(5)
#E3
where Can the negative log- likelihood with sparsity regularisation over a 0.1 discourage the models from utilising spurious number of variables. For UMLP and UCNN, we set Åœ 0, Av = for training just the unified output and the converse for the non-unifying versions. To pre-train the UMN, start with AÆ = 1, Au 0 for 40 epochs then set Au = to jointly train the unified output. For iterative tasks, the at each iteration j and, in the strongly supervised cases, the negative log- log-liiel for the context attentions using provided supporting facts are also added to the objective function. Further details such as batch size and total number of epochs are available in Appendix C
squared error between hidden states (h² h'²)
MLP
CNN
Accuracy
Unification
0.6
No
Yes
Training Size
0.4
1000
500
750 1000
1250
1500
1750 2000
500
750
1250
1500
1750 2000
Iteration
Iteration
Figure 3: Test accuracy over iterations for Unification MLP and Unification CNN models with invariant versus unification. We observe that with soft unification the models achieve higher accuracy with fewer iterations than their plain counterparts on both per task training sizes.
Figure portrays how soft unification generalises better unseen examples in test sets the same sequence or grid never appears in both the training and test sets as outlined in Section over plain models. Despite of o having more trainable parameters than f alone, the models with unification not only maintain higher accuracy in each iteration and solve the tasks in as few as 250 iterations with < 1000 training examples but also improve accuracy by As 0.3 when trained with only S 50 task. We believe soft unification architecturally biases the models towards learning structural patterns which return achieves better results on recognising common patterns symbols across examples. Results with multiple invariants are identical and the models seem to ignore the extra invariants due to the fact that the tasks can be solved with a single invariant and the regularisation applied & zeroing out unnecessary invariants: further results in Appendix [ The fluctuations in accuracy around iterations 750 to 1000 in UCNN are also caused by penalising & which forces the model relearn the task with less variables half way through training.
5

---

Under review as a conference paper at ICLR 2020
Table Aggregate error rates (%) on bAbl Ik for UMN and N2N, GN2N, MemNN by Sukhbaatar et al. 2015), Liu & Perez (2017) and Weston et al. (0155 respectively. Full comparison on individ-
tasks are available Appendix
Training Size Supervision # lnvs Model
lk
50
1k
Weak
|
Strong
Weak
Strong MemNN
3
N2N
GN2N
EntNet
QRN
Mean
19.1
20.5
6.3
27.6
13.9
12.7
29.6
11.3
6.7
# 5%
8
10
4
17
11
15
5
4
Following Tables 2 and 3 we observe a trend of better performance through strong supervision, more data per task and using only invariant. We believe strong supervision aids with selecting the correct sentences to unify and in a weak setting the model attempts to unify arbitrary context sentences often failing follow the iterative reasoning chain. The increase in performance with more data and strong supervision is consistent with previous work reflecting how f gg can be bounded by the efficacy of modelled as a memory network. As result, only in the supervised case do we observe minor improvement over MemNN by 0.7 in Table no improvement in the weak case over comparable memory based networks in Table 3 This dependency on f also limits the ability of f g to learn from 50 examples per task failing 17/20 and 12/12 of bAbI and logical reasoning tasks respectively. The increase in error rate with invariants, we speculate, stems from having more parameters and more pathways in the model rendering training more difficult.
Table 3: Aggregate task error rates (%) on the logical reasoning dataset for UMN and DMN, IMA by Cingillioglu & Russo (2019). Strong supervision, more data and only 1 invariant seem improve the performance of UMN over plain iterative models. Individual task results are in Appendix D
Training Size
lk
10k
50
20k
Supervision
Weak
Strong
Weak
Strong
Weak
Arity Invs Model
2
1
2
1
2
1
2
2
2
2
3
1
3
DMN
IMA
Mean
36.4
39.3
14.3
28.9
21.5
31.8
2.4
12.2
16.0
47.1
21.2
9.1
> 5%
11
7
11
7
10
11
5
ANALYSIS
After training, we can extract the learned invariants by applying threshold on Vs E v0s)) > indicating whether symbol is used as a variable or not. We set 0.0 for all datasets except for bAbl, we use / 0.1. The magnitude of this threshold seems to depend on the amount of regularisation т. equation 5 and the number of training steps along with batch size controlling how much is pushed towards 0. Sample invariants shown in Figure44describe the common patterns present in the tasks with parts that contribute towards the final answer becoming variables. Extra symbols such as iss or travel led do not emerge variables, as shown in Figure we attribute this behaviour to the fact that changing the token travell led to went does not influence the prediction but changing the action, the value of Z:left 'picked' does. However, based on random initialisation, our approach can convert an arbitrary symbol into a variable and let f compensate for the unifications produces. For example, the invariant "X:8 5 2 2" could predict the tail of another example by unifying the head with the tail using ou. equation 3 of those symbols. Further examples are shown Appendix D Pre- Pre-training and consistent invariants compared to immediately training f o g since, we speculate, by equation 4 might encourage gT, K)+ K.
done in UMN seems produce more robust
Interpretability versus Ability A desired property of interpretable models is transparency (Lipton 2018). novel outcome of the learned invariants our approach is that they provide an approx- imation of the underlying general principle present in the data such as the structure of multi-hop reasoning shown in Figure 4e However, certain aspects regarding the ability of the model such as observe
how it performs temporal reasoning, Z:morning as a variable, the overall learned invariant captures nothing about how changing the value of Z:morning alters the behaviour of f. The model might look before after certain time point
still hidden inside f. In Figure 4b) although

---

Under review as a conference paper at ICLR 2020
Y:oohn Z:left the X:football
this Z:morning X:bill went to the Y:school yesterday X:bill journeyed to the A park where was X:bill before the Y:school A Appkk
Y:john travelled to the A:office
where the X:football A:office
(a) bAbl task two supporting facts. The model also learns Z:left since people can also drop or pick up ob- jects potentially affecting the answer.
(b) bAbl task 14, time reasoning. X:bill and Y:school are recognised as variables alongside Z:morning cap- turing when someone went which is crucial to this task.
5 8 6 const
0
X X
0
0
00
Xi (T),
X:8 3.3 head
X:8
X
6
8
05
(:I()) R:x (0), R:x (K) (K),
8 Y:5 tail
Y:5
0 0 0
0
7 0
8 X
Z:1 4.3 Z:1 dup
Z:1
box
centre
corner
Sn ) (Y:)
(d) Successful invariants learned with UMLP using 50 training with UCNN. Variable default sym- amples only shown (C, a...
(c) Successful invariants learned
(e) Logical reasoning task 5 with arity 1. The model captures how S.n could entail X:i in chain.
bols are omitted for clarity.
Figure 4: Invariants learned across the four datasets using the three architectures. For iterative reasoning datasets, bAbl and logical reasoning, they are taken from strongly supervised UMN.
X:bill went somewhere depending what Z:morning binds to. Without the regularising term on (b(s), initially noticed the models using, one might call extra, symbols variables and binding them to the same value occasionally producing unifications such as "bathroom bathroom to the bathroom" and still f predicting, perhaps unsurprisingly, the correct answer bathroom. Hence, regularising di with the correct amount in equation 5 to reduce the capacity of unification seems critical in extracting not just any invariant but one that represents the common structure.
Soft unification from equation reveals three main patterns: one- one-to-ome to-one, one-to-many many-to one bindings as shown in Figure 5) further examples are Appendix Figure captuures what one might expect unification to look like where variables unify with their corresponding counterparts, e.g. X:bernhard with brian and Y:frog with lion. However, occasionally the model can optimise to use less variables and squeeze the required information into single variable, for example by binding Y:bathroom to john and kitchen as shown in Figure 5b We believe this occurs due to the sparsity constraint (s(s) encouraging the model to be conservative possible. Finally, if there are more variables than needed as Figure 5c we observe a many-to-one binding with Y:w and Z:e mapping to the same constant q. This behaviour begs the question how does the model differentiate between p(g) and p(g, q). We speculate the model uses the magnitude of ((w) = 0.037 and (b(e) = 0.042 encode the difference despite both variables unifying with the constant.
the kitchen
Xbemhard brien
lion
John
10
Xa
sandra
Y:w
the
Ze
Ydrog
Yhathroom
(a) bAbl task 16. one-to-one mapping is created between vari- ables X:hemhhrrd with hrian and Y:frog with lion.
(b) bAbl task 6. Only Y:bathroom is recognised variable creating one-to-many binding to capture the same information.
(c) Logical reasoning task 1. An arity predicate is forced to bind with arity I creating many-to-one binding-
Figure 5: Variable bindings produced from equation
Darker cells indicate higher attention values.

---

Under review as a conference paper at ICLR 2020
7
RELATED WORK
Learning an underlying general principle in the form of an invariant often the means for arguing generalisation in neural networks. For example, Neural Turing Machines (Graves et al. 2014) are tested on previously unseen sequences to support the view that the model might have captured the underlying pattern or algorithm. In fact, Weston et al. (2013) claim "MemNNs can discover simple linguistic patterns based on verbal forms such as (X, dropped, Y), (X, took, Y) or (X, journeyed to, Y) and can successfully generalise the meaning of their instantiations." However, this claim is based on the output of ] and unfortunately is unknown whether the model has truly learned such a representation indeed is utilising it. Our approach sheds light to this ambiguity and presents these linguistic patterns explicitly as invariants ensuring their utility through g without solely analysing the output of previously unseen symbols. Although we associate these invariants with exist- ing understanding of the task to mistakenly anthropomorphise the machine, for example by thinking it has learned X:mary as someone, it is important to acknowledge that these are just symbolic pat- terns. In these cases, our interpretations do not necessarily correspond to any understanding of the machine, relating to the Chinese room argument made by Searle 1980).
Learning invariants by lifting ground examples is related to least common generalisation (Reynolds, 1970 by which inductive inference performed on facts (Shapiro) 1981) such as generalising wentmmary,kicheen and went(john.gardee
to went(X.Y). Unlike in predicate logic setting, our approach allows for soft alignment and therefore generalisation between varying length sequences. Existing suurosyybbolic systems (Broda et al. 2002) focus on inducing rules that adhere to given logical semantics of what variables and rules are. For example, LLLP (Evans & Grefenstette] 2018) constructs a network by rigidly following the given semantics of first-order logic. Similarly, Lifted Relational Neural Networks (Sourek et al. (015)) ground first-order logic rules into a neural network while Neural Theorem Provers (Rocktschel & Riedel (017)) build neural networks using backward- chaining (Russell & Norvig 2016) on a given background knowledge base with templates. However, the notion of variable is pre-defined rather than learned with foeus on presenting a practical ap- proach to solving certain problems, whereas our motivation stems from a cognitive perspective.
At first it may seem the learned invariants, Section 6 make the model more interpretable; however, this transparency is not of the model / but of the data. The invariant captures patterns that poten- tially approximates the data generating distribution but still do not know how the model / uses them upstream. Thus, from the perspective of explainable artificial intelligence (XAI) (Adadi & Berrada 2018). learning invariants or interpreting them do not constitute an explanation of the rea- soning model f even though "if someone goes somewhere then they are there" might look like one. Instead, it can be perceived as causal attribution (Miller) (209)) in which someone being somewhere is attributed to them going there. This perspective also relates to gradient based model explanation methods such as Layer-Wir Wise Relevance Propagation (Bach et al. 2015) and Grad-CAM (Selvaraju et al. 2017; Chattopadhay et al. 2018). Consequently, a possible view on v. equation 2 is gra- dient based usefulness measure such that a symbol utilised upstream by f to determine the answer becomes variable similar to how group of pixels in an image contribute more to its classification.
Finally, one can argue that our model maintains a form of counterfactual thinking (Roese 1997) in which soft unification g creates counterfactuals the invariant example to alter the output of towards the desired answer, equation[[] The question where Mary would have been if Mary had gone to the garden instead of the kitchen is the process by which an invariant is learned through multiple examples during training. This view relates to methods of causal inference (Pearl, 2019: Holland 1986 in which counterfactuals are vital as demonstrated in structured models by Pearl 1999)..
8 CONCLUSION
We presented a new approach for learning variables and lifting examples into invariants through the usage of soft unification. Evaluating on four datasets, we analysed how Unification Networks perform comparatively to existing similar architectures while having the benefit of lifting examples into invariants that capture underlying patterns present in the tasks. Since our approach is end-to- end differentiable, we plan to apply this technique to invariants for example in visual question answering.
tasks in order to yield multi-modal
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Amina Adadi and Mohammed Berrada. Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access, 6:52138-52160, 2018, doi: 10.1109/access.00) 2870052.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Miler, and Wojeiech Samek. On pixel-wise explanations for non-linear classifier decisions by layer- wisc relevance propagation. PLOS ONE, (0(7)x00100140 July 2015. doi: 10.1371/journal.pon 0130140.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. ICLR, September 2015.
Krysia B. Broda, Artur S. D' D'Avil Garcez, and Dov M. Gabbay. Newral-Symbolic Learning Systems. Springer London, 2002. ISBN 1852335122.
Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N. Balasubramanian. Grad- CAM++ Generalized gradient-base, visual explanations for deep convolutional networks. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 839 839-847 847. IEEE, IEEE, March 2018. doi: 10.1109/wcv..018 8.00097.
Sneha Chaudhari, Gungor Polatkan, Rohan Ramanath, and Varun Mithal. An attentive survey of attention models. IJCAI, April 2019.
Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decoder approaches. In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation. Association for Compu- tational Linguisties, 2014. doi:
Nuri Cingillioglu and Alessandra Russo. Deeplogic: Towards end-to-end differentiable logical rea- soning. AAAI-MAKE, May 2019.
Keith L. Clark. Negation as failure. In Logic and Data Bases, pp. 293-322. Springer US, 1978. doi: 10.1007/978-1-46633 1-1779978-1144644-344-1-
Richard Evans and Edward Grefenstette. Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research, 61:1-64, Jamuary 2018. doi: 10j1613/jair:5714.
Gottlob Frege. Sense and reference. The Philosophical Review, 57(3):209, May 1948. doi: 10.2307/ 2181485.
Joe L. Frost, Pei-San Brown, John A. Sutterby, and Candra D. Thornoon. The Developmental Benefits Of Playgrounds. Association for Childhood Education International, 2004. ISBN 0871731649,
Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. arXiv:1410.5401 2014.
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska- Barwinska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, Adrià Puigdomènech Badia, Karl Moritz Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen King, Christopher Summerfield, Phil Blunsom, Koray Kavukcuoglu, and Demis Hassabis. Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626): 471-476, October 2016. doi: 10. 1..0088/auure20001.
Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun. Tracking the world state with recurrent entity networks. ICLR, 2017.
Paul W. Holland. Statistics and causal inference. Journal of the American Statistical Association, 91(396):945-966 960, December 1986. doi: 19.108000162445..1966.. 1477744445999989878733444444
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ICLR, 2015.
Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit lyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Romain Paulus, and Richard Socher. Ask me anything: Dynamic memory networks for natural language processing. In ICML. pp. 3378-1387 2016.

---

Under review as a conference paper at ICLR 2020
Richard L. Lewis. Cognitive modeling, symbolic. The MIT encyclopedia the cognitive sciences, pp. 225-527, 1999.
Zachary C. Lipton. The mythos of model interpretability. Communications of the ACM, 61(10):
36 43, September 2018. doi: 10.1 135/333333333
Fei Liu and Julien Perez. Gated end-to-end memory networks. Proceedings of the 15th Confer- ence of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pp. 1-10. Association for Computational Linguistics, 2017. doi: 10.18653/
Thang Luong, Hieu Pham, and Christopher D. Manning. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2015. doi: 10.18653/ v1/d15- 1166.
Tim Miller, Explanation in artificial intelligence: Insights from the social sciences. Artificial Intel- ligence, 267:1-38, February 2019. doi: 1..01//..rtint.2118...
Adam Morton and Jerry A. Fodor. The Language of Thought. volume 75, Philosophy Documenta- tion Center, March 1978. doi: 10.2307/2025426.
Stephen Muggleton and Lue de Raedt. Inductive logic programming: Theory and methods. The Journal of Logic Programming, 19-20:629-679) May 1994. doi: 10.1016/0743-10600 1066(94)) 90035-3.
Judea Pearl. Probabilities of causation: three counterfactual interpretations and their identification. Synthese, 121(1-2):93-19) 1999.
Judea Pearl. The seven tools of causal inference with reflections machine learning. Communica- tions of the ACM, 62(3):54-60) 60, February 2019. doi: 10.1145/3241036.
Jean Piaget. The Psychology of Intelligence. Routledge, August 2001. ISBN 0415254019.
John C. Reynolds. Transformational systems and algebraic structure of atomic formulas. Machine intelligence, 5:135-151, 1970.
Tim Rocktschel and Sebastian Riedel. End-to-end differentiable proving. NIPS, pp. 3791-3803, 2017.
Neal Roese. Counterfactual thinking. Psychological Bulletin, 121(0):133-448, 1997. doi: 1037/0033-2909.122 121....1....13......
Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach (3 Edition). Pearson, November 2016. ISBN 1292153962.
John R. Searle. Minds, brains, and programs. Behavioral and brain sciences, 3(3):417-424 1980. doi: 10.1011/b978-1-44 022-1444..,00077--
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual explanations from deep networks via gradient-basee local- ization. In 2017 IEEE International Conference on Computer Vision (ICCV), pp. 618-626. IEEE, October 2017. doi: 10. 10.1109/cccv.2117.744
Minjoon Seo, Sewon Min, Ali Farhadi, and Hannaneh Hajishirzi. question answering. ICLR, 2017.
Query-reduction networks for
Ehud Y. Shapiro. Inductive inference of theories from facts. Yale University, Department of Com- puter Science, 1981.
Gustav Sourek, Vojtech Aschenbrenner, Filip Zelezny, and Ondrej Kuzelka. Lifted relational neural networks. /508. /28, 2015.
Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. End-to-end memory networks. In NIPS, pp. 2440-2444 2015.
10

---

Under review as a conference paper at ICLR 2020
Seiya Tokui, Kenta Oono, Shohei Hido, and Justin Clayton. Chainer: next-generation open source framework for deep learning. In Proceedings of Workshop on Machine Learning Systems (LearningSys) in The Twenty-ninth Annual Conference Neural Information Processing Sys- lems (NIPS), 2015. URL
wwtps//rrggpaa.//aeeeeeeeeeeeeee Lea Thapers/TearninggSyy. 2
paper_33. 33.
Marshall Unger and Terrence W. Deacon. The Symbolic Species: The Co-Evolution of Language and the Brain, volume 82. Wiley, 1998. doi: 10.2307/329984..
Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. ICLR, 2015.
Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M. Rush, Bart Merrinboer, Armand Joulin, and Tomas Mikolov. Towards ai-complete question answering: A set of prerequisite toy tasks. ICLR, 2016.
Caiming Xiong, Stephen Merity, and Richard Socher. Dynamic memory networks for visual and textual question answering. ICML, pp. 2397-2406, 2016.

---

Under review as a conference paper at ICLR 2020
A MODEL DETAILS
A.1 UNIFICATION MLP & CNN
Unification MLP (UMLP) To model f multi-layer perceptron, we take symbol embeddings of size d = 16 and flatten sequences of length I = 4 into an input vector of size o(k) E R The MLP consists of hidden layers with tanh non-linearity of sizes 2d and d respectively. To process the query, we concatenate the one-hot encoding of the task id to ((k) yielding a final input of size 64 + 4 68. For unification features Φv, we use a bi-directional GRU with hidden size d and an initial state of O. The hidden state each symbol is taken with linear transformation to give (s)s) = W ((s) where (ss) is the hidden state of the biGRU. The variable assignment is then computed an attention over the according to equation 3
Unification CNN (UCNN) We take symbols embeddings of size d = 32 to obtain an input grid 6 ()) 3x33322 Similar UMLP, for each symbol append the task id as one-hot vector to get an input of shape 3 x 3 x (32 + 4). Then f consists convolutional layers with d filters each, relu non-linearity in between the layers. We pad the grid with 2 columns and 2 rows to a x 5 such that the output of the convolutions yield again a hidden output H xd of the same shape. As the final hidden output h, we take a global max pool over H to obtain h E Rd. Unification function is modelled identical to f without the max pooling such that du Kij) H'J where H' the hidden output of the convolutional layers.
kernel size of and stride 1. We
A.2 UNIFICATION MEMORY NETWORKS
Unlike previous architectures, with UMN we interleave g into f. We use embedding sizes of d = 32 and model ff with an iterative memory network. We take the final hidden state of bi-directional GRU, with initial state 0, aas represent the sentences of the context C and query q in d- dimensional vector M; = Φ Ar (C₁) and the query my = 4M (q). The initial state the memory network is h" mgg. At each iteration j:
A tanh( W ((M, h²) b)
8 softmax W ( b)
where A is another d-dimensional bi-directional GRU and p(z; y) = [x: y: o y: (a y) with the element-wise multiplication and [;] the concatenation of vectors. Taking 3 the context attention, we obtain the next state of the memory network:
h
Σ Bl tanh(W (ellWWolee p(M; h²) + b)
and iterate J many times in advance. The final prediction becomes f(C, q) = softmax Wh' +b). All weight matrices W and bias vectors b are independent but tied across iterations.
12

---

Under review as a conference paper at ICLR 2020
B GENERATED DATASET SAMPLES
Table 4: Sample context, query and answer triples from sequences and grid tasks.
Dataset
Task
Context
Query
Answer
Sequence
1488
constant
Sequence
6157
head
Sequence
iii
1837
tail
7
Sequence
iv
3563
duplicate
3
0 0
Grid
0 2
box
8 2
4 0
Grid
ii
0 0
head
4
8
0 0
Grid
iii
1 2
centre
7
0 0
8 0
Grid
iv
5 0
corner 2
2
Table 5: Training sizes for randomly generated fixed length sequences and grid tasks with 8 unique symbols. The reason for Grid task (i) be smaller is because there are 32 combinations of 2 x 2 boxes in a 3 x grid with unique symbols.
Task
Sequences
Grid
704.7 + 12.8
25.6 + 1.8
709.4 + 13.8
623.7 14.1
iii
709.7 + 14.0
768.2 12.5
624.8 = 12.4 iv
795.2 = 10.3
13

---

Under review as a conference paper at ICLR 2020
C TRAINING DETAILS
C.I UNIFICATION MLP & CNN
Both unification models are trained on 5-fold cross-validation over the generated datasets for 2000 iterations with a batch size of 64. We don't use any weight decay and save the training and test accuracies every 10 iterations, as presented in Figure
C.2 UNIFICATION MEMORY NETWORKS
We again batch size of 64 and pre-train for 40 epochs then f together with g for 260 epochs. We use epochs for UMN since the dataset sizes are fixed. To learn alongside f. we combine error signals from the unification of the invariant and the example. Following equation 4 the objective function not only incorporates the negative log-likelihood Lan of the answer but also the mean squared error between intermediate states h and hig at each iteration as an auxiliary loss:
J = (=))((((K)))) AU Lan(s o g(T, K),a)
1 J
Σ 3=1
h2)²
+
Σ ())( SES
(9)
We pre-train by setting À = O for 40 epochs and then set X = For strong supervision we also compute the negative log-likelihood Lall for the context attention Bi, described in Appendix A at each iteration using the supporting facts of the tasks. We apply a dropout of 0.1 for all recurrent neural networks used and only for the bAbl dataset weight decay with 0.001 as the coefficient.
14

---

Under review as a conference paper at ICLR 2020
D FURTHER RESULTS
10
0.9
Test
0.6
0.5
0.4
Iwariants
Figure 6: Results of Unification MLP and CNN increasing number of invariants. There is impact on performance when more invariants per task are given. Upon closer inspection, we noticed the models ignore the extra invariants and only 1. We speculate the regularisation di encourages the models to use single invariant.
Table 6: Individual task error rates on bAbl tasks for Unification Memory Networks.
Supervision Weak
Strong 3
Invs Training Size
3
3
1k
1k
1k
1k
50
0.0
0.0
0.0
0.0
1.4
2
65.6
63.1
0.3
0.7
30.0
3
67.1
62.6
1.0
2.4
39.8
4
0.0
0.0
0.0
0.0
37.0
3.4
4.0
0.8
1.1
26.5
6
0.2
0.6
0.0
0.0
18.4
7
22.0
22.8
10.7
11.3
22.8
8
10.3
8.5
7.4
7.6
24.7
25.7
0.0
0.0
33.8
O.I
2.0
0.0
0.3
32.6
11
0.0
0.0
0.0
0.0
11.9
12
0.0
0.1
0.0
0.0
21.3
2.1
3.7
0.0
0.1
5.8
13.5
0.5
0.1
54.8
15
0.0
0.7
0.0
0.0
0.0
16
55.2
56.2
0.0
0.0
39.7
17
39.2
49.0
51.1
49.3
48.8
4.4
8.0
0.6
0.5
10.4
91.8
89.6
53.9
46.7
90.2
20
0.0
0.0
0.0
0.0
2.7
Mean
19.1
20.5
6.3
6.0
27.6
Std
27.9
27.0
15.6
14.3
21.0
>
5%
8
10
4
4
17
15

---

Under review as a conference paper at ICLR 2020
Table 7: Comparison of individual task error rates (%) the bAbl of the best run. We preferred 1k results if model had experiments published on both 1K and 10k
(Weston et al. 2016) dataset
for data efficiency. References from left to right: (Sukhbaatar et al. 2015) (Liu & Perez 2017)
(Henaff et al. 2017) [Soo et al. 2017) Ours (Xiong et al. 2016)
Graves el al. 2016
Weston
al. 2015 Ours
Kumar
2016
Support
Weak
Strong
Size Model
1k EntNet
10k
1k
10k
N2N
GN2N
QRN
UMN
DMN+
DNC
MemNN
UMN
DMN
0.0
0.0
0.7
0.0
0.0
0.0
0.0
0.0
0.0
0.0
8.3
65.6
0.3
40.3
38.8
69.7
1.2
67.1
1.1
1.8
0.0
2.4
4.8
4
2.8
0,4
1.4
0.7
0.0
0.0
0.0
0.0
0.0
0.0
13.1
1.0
4.6
1.2
3.4
0.5
0.8
2.0
1.1
0.7
7.6
0.0
17.3
17.8
22.3
9.4
22.0
2.4
0.6
15.0
11.3
3.1
8
10.0
12.5
19.2
3.7
10.3
0.0
0.3
9.0
7.6
3,5
13.2
10.7
31.5
0.0
0.1
0.0
0.2
0.0
0.0
0.0
10
15.1
16.5
15.6
0.0
0.1
0.0
0.2
20
0.3
2.5
11
0.9
0.O
12
0.2
0.0
0.8
0.0
0.0
0.0
0.0
0.0
0.0
0.0
13
0.4
0.0
9.0
0.3
2.1
0.0
0.1
0.0
0.I
0.2
14
1.7
1.2
62.9
3.8
19.7
0.2
0.4
1.0
0.1
0.0
15
0.0
57.8
0.0
16
1.3
55.2
45.3
55.1
17
51.0
41.7
46.4
51.8
39.2
4.2
12.0
35.0
49,3
40.6
18
11.1
9.2
8.8
8.8
4.4
2.1
0.8
5.0
0.5
4.7
19
82.8
88.5
90.4
90.7
91.8
0.0
3,9
64.0
46.7
65.5
20
0.0
Mean
13.9
12.7
29.6
11.3
19.1
2.8
3.8
6.7
6.0
6.4
5%
11
Table 8: Individual task error rates (%) on the logical reasoning dataset.
Size
lk
10k
50
20k
Support Arity
Weak
Strong
Weak
Strong
Weak
2
2
1
2
2
2
2
2
# Invs/ Model
3
1
3
3
DMN
IMA
Facts
1.2
0.9
0.0
0.4
0.0
0.0
0.0
0.0
0.0
33.5
0.0
0.0
Unification
0.0
10.3
0.0
10.8
0.0
0.0
0.0
0.0
0.0
41.3
13.0
10.0
Step
50.3
20.0
27.8
0.1
1.3
S.7
50.2
26.0
2.0
Steps
47.5
50.0
5.7
35.0
37.2
47.8
0.0
29.7
28.7
49.9
33.0
5.0
3 Steps
47.6
49.2
10.4
38.7
39.6
45.6
0.0
26.0
26.1
48.3
23.0
6.0
AND
31.3
37.4
10.7
16.4
29.8
29.0
0.2
0.4
1.2
50.0
20.0
5.0
OR 25.2
38.1
21.0
35.0
20.5
30.2
4.4
20.6
17.4
47.6
13.0
3.0
Transitivity
50.0
26.6
39.6
5.0
6.0
49.2
50.0
50.0
Step NBF
46.4
38.7
3.8
28.8
1.1
21.6
0.1
1.1
8.0
47.6
21.0
2.0
2 Steps NBF
48.5
48.9
7.7
39.6
30.4
48.2
0.1
33.4
28.7
50.3
15.0
4.0
AND NBF
51.0
50.1
43.1
48.6
29.4
44.2
0.1
1.3
40.1
49.5
16.0
8.0
OR NBF
51.4
50.8
47.3
47.6
47.8
21.3
27.6
30.5
47.3
25.0
14.0
Mean
36.4
39.3
14.3
28.9
21.5
31.8
2.4
12.2
16.0
47.1
21.2
9.1
Std
18.7
15.9
16.4
14.1
17.1
16.7
6.1
13.2
13.6
4.7
12.3
13.4
> 5%
11
7
10
5
9
12
11
16

---

Under review as a conference paper at ICLR 2020
Xsandra went back to the Y:bathroom is X:sandra in the Y:bathroom
yes
Figure 7: bAbl task 6, yes or no questions. The invariant does not variablise the answer.
X:m Y:e)
+ X:m Yee)
X:a ( Y:w, Z:e)
X:a ( Y:w. Z
X.m (T)
Xmm (c)
x not Z:q A.
+ X:x (Y:z.)
Figure 8: Invariants learned on tasks 1, and 11 with arity and 2 from the logical reasoning dataset. Last invariant on task 11 lifts the example around the negation by failure, denoted as not, capturing its semantics.
3 X 7 Y head
X 0
0 Y
0
6
Y
4 head
010
X
X 3 1 X tail
00Y
0 2
0
0 0 7
X 5 duplicate
head
centre
corner
(a) Invariants with extra varialbes learned with UMLP. (b) Mismatching invariants learned with UCNN.
Figure 9: Invariants learned that not match the data generating distribution from UMLP and UCNN using < 1000 examples to train. In these instances the unification still bind to the the correct symbols order to predict the desired answer; quantitatively we get the same results. Variable default symbols omitted for clarity.
mar
Xv
X9g
Vial
the
Yd
Yo-
Zicsbaal
(a) bAbl task 2. When variable unused in the next iteration, e.g. f:ooobblll, it unifies with random tokens often biased by position.
(b) Logical reasoning task 1. A one-to mme-to-on to-eeoo-e alignment created be- tween predicates and constants.
(c) Logical reasoning task 3. Arity 1 atom forced to bind with arity creates one-to-man to-m mapping.
Figure 10: Further attention maps for equation darker cells indicate higher attention values.
17


Reviews:
[{"review_id": "rkgXIqLJqH", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a neural network approach to variable unification and\nreasoning by example as a way to mimic the human ability to identify invariant\npatterns in examples and then apply them more generally in practice.\nThis general idea of identifying invariates and mapping new instances to\nthem is well motivated by the authors, citing work in philosophy of mind,\ncognitive science, and developmental psychology.\n\nThe authors go on to propose MLP, CNN and Memory Network models\nof unification for sequence, grid, and story reasoning tasks respectively.\nExperiments on the sequence and grid datasets demonstrate the data efficiency\nof this approach. MLP and CNN models with unification achieve near perfect\nperformance in fewer iterations (an order of magnitude fewer in the MLP case!)\n than their non-unification enabled counter parts.\nUnification enabled models also demonstrate high performance in a reduced\ntraining set setting (using only 50 training examples).\nWhile this is encouraging, these are very simple toy tasks.\n\nI also am in doubt as to whether the representation of these problems\ncauses some issues. In the sequence task, one question the models are\ntrying to solve is what symbol is the head or tail of the sequence.\nModeling variables over the sequence of symbols here is, in a sense, the\nwrong object of study. The position of the symbols would need to be\nrepresented, e.g.\n\na b c d\n1 4 3 1\n\nwhere I've represented positions as a-d, and the learned invariant about\nhead questions would be:\n\nX:a b c d\nY:1 4 3 1\n\nAs is, by mapping symbols and not positions to variables, one cannot,\nat the variable level distinguish between the two 1s in the sequence above.\nMy guess is that in practice the bi-GRU model that produces embedding\nfeatures of the symbols in sequence is implicitly representing head/tail\npositioning.\n\nSimilar arguments could be made about the grid example.\n\nI don't find the experiments/analysis on the bAbI dataset very convincing.\nFor instance, in the example given in Figure 4b (reproduced below)\nis shown as an example of\ntemporal reasoning, where a symbol Z is mapped to the\nword morning (a symbol distinguishing a time), and the question asked is\nwhere was Bill before school.\nIf logical reasoning is being used to solve this question, surely the\nsymbol 'before' must also be represented as a variable. Its possible that\nthe model is instead learning a trick about mutual exclusivity, i.e. that\nY:school is the only location symbol not mentioned in question but this\ncould fail as a general strategy.\n\n\nthis Z:morning X:bill went to the Y:school\nyesterday X:bill journeyed to the A:park\nwhere was X:bill before the Y:school\nA:park\n\nFigure 4b\n\nIt would make for a much more interesting paper if the authors took\nexamples such as these and formed counter-factuals to probe the way\nthe models are answering the questions. E.g., transforming the question\nin 4b to \"where was X:bill today\" or \"where was X:bill after school.\"\n\nBecause the authors use soft unification, interpretability is difficult\nto assess. Interpretability is crucial here because to claim that unification and reasoning by logical induction\nis being used to solve tasks, it becomes important to show how the neural networks\nmake their decisions. Given the instances of extra variables and one to many\nmappings on the bAbI dataset it seems very likely that the models are not\nsolving many tasks\nusing unification as it would be possible to learn to use the symbols directly\nto learn to answer. As such, I think these issues are not addressed in the\npaper sufficiently to warrant acceptance.\n\n\nMinor Notes\n\n- In definition 1, the definition of Variable is a little confusing because there are two different senses of the word in use. I understand them to be (1) Variable (X) in the logical template that is intended to be learned and used in problem solving, and\n(2) variable (x) in the neural network model that is a soft asignment of\nthe Variable to a default symbol s. It would be nice if this distinction could\nbe noted or made clearer.\n\n- In the definition 2, in the phrase \"is the invariant example such as a tokenized story\" it might be worth stating that the tokens are the symbols in S.\n\n\n- My understanding is that each unique symbol in the invariate is a potential\nvariable. Does this mean there are no co-referent symbols in the invariate?\nWould be helpful to state whether babi contains co-referent expressions\nand how these might affect the model.\n\n\n- It might be interesting to see how model architecture affects variable\nlearning. For example, does a CNN result in more sensible variable\nassignments  than the mlp on a flattened representation of the grid problem?\n\n\n- What is the strongly supervised case? These are token level annotations I think (at least for babi) but it might be good to specify in more detail what\nthey  are.\n\n- The figure and explanation of the UMN are not very clear. From the figure\nis does not seem that the variables interact with the memory at all. More\nspace could be devoted to this section.\n\nPossibly Relevant Related Work\n\nBrenden Lake. Compositional generalization through metasequence-to-sequence learning. NeurIPS 2019.\n\n\n\n\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "HygP8HETtH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a novel approach for learning invariants that can capture underlying patterns in the tasks through Unification Networks. This effectively allows the machine to learn the notion of `variable`, which is a symbol that can take on different values. \n\nPros:\nThe authors evaluated and presented empirical results on four common benchmark datasets, showing superiority over plain baseline without unification. \nThey further performed analysis on the learned invariants, and verified the sensibility. \nThe paper overall is well written and structured.\n\nCons:\nDespite its superiority over plain baseline, the paper does not provide thorough comparison with other state-of-the-art methods on reasoning related tasks.\n\nSome of the technical details regarding the choice of hyperparameters are missing. For example:\nIn section 6, what’s the rationale of setting $t$ differently for bAbI solely?\nIn Equation 5, how is the sparsity regularization parameter $\\tau$ chosen optimally for a particular task? A bit more discussion on these choices would be helpful. \n\nOverall, this paper presents a seemingly promising architecture capable of learning and using variables, with the caveat for lack of experiments and comparison with other state-of-the-art methods. \n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "BJxGVXNpKB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper explores a very interesting idea: can a model learn what variables are and how to use them? Unfortunately, the paper doesn't seem quite ready: the model description was very hard to follow and it's not clear the approach has found a compelling use case.\n\nI read the paper carefully three times, and try as I might, I simply can't get my head around the entire architecture. The modeling section jumps straight into a series of definitions, without trying to build intuition or provide a worked example. There is an example in Figure 2, but it isn't really explained and I didn't find it helpful. Unification seems to be implemented as a form of attention (or self-attention) where the model can control the degree to which a symbol acts as variable. But the relationship between soft unification and attention isn't really spelled out -- what's the same, what's different? Ultimately it's not clear to me what the model is attending over during soft unification.\n\nThere are various other aspects of the paper that aren't clear:\n- strong vs. weak supervision\n- comparison models DMN and IMA are not introduced at all, and include no references\n- the logical reasoning experiment is not clearly described\n- there is only a cursory conclusion\n\nI am not sure the model has found a compelling use case. On bAbi with weak supervision, the model is worse than the comparison models. It only slightly beats out memory networks with strong supervision. For logical reasoning, it's not clear what it is compared against or if the comparison is fair. The clearest win over standard networks is on the simple synthetic experiments.\n\nFinally, the authors mention the paper has a cognitive science motivation, in that \"Humans learn what variables are and how to use then at a young age\" or that \"symbolic thought with variables is learned...\", taking a strong \"nurture\" stance on the origin of variables. But variables could very well be innate and simply early emerging. Any discussion of the origin of variables in the mind requires more nuance.\n\nI am excited about this research direction, and it could ultimately be a very nice contribution as the work matures. I don't think the paper is ready in its current form.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The main concern raised by reviewers is the limited experiments, which are on simple tasks and missing some baselines to state-of-the-art methods. While the overall approach is interesting, the reviewers found the empirical evidence to be fairly unconvincing. 
Decision: Reject