Dynamic Inference with Neural Interpreters
Authors: Nasim Rahaman
Muhammad Waleed Gondal
Shruti Joshi
Peter Vincent Gehler
Yoshua Bengio
Francesco Locatello
Bernhard Schölkopf
Venue: NeurIPS 2021 Poster (2021)
Keywords: Modular Architectures
Attention Networks
Neural Networks
Abstract Reasoning
Systematic Generalisation.

Abstract: Modern neural network architectures can leverage large amounts of data to generalize well within the training distribution. However, they are less capable of systematic generalization to data drawn from unseen but related distributions, a feat that is hypothesized to require compositional reasoning and reuse of knowledge. In this work, we present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules, which we call _functions_. Inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. The proposed architecture can flexibly compose computation along width and depth, and lends itself well to capacity extension after training. To demonstrate the versatility of Neural Interpreters, we evaluate it in two distinct settings: image classification and visual abstract reasoning on Raven Progressive Matrices. In the former, we show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner. In the latter, we find that Neural Interpreters are competitive with respect to the state-of-the-art in terms of systematic generalization. 

Dynamic Inference with Neural Interpreters
Muhammad Waleed Gondal*:!
Nasim Rahaman", 1,,,,33
Shruti Joshi'
Peter Gehler?
Yoshua Bengiof:2
Francesco Locatello" 1.3
Bernhard Schölkopf
Abstract
Modern neural network architectures can leverage large amounts of data to gen- eralize well within the training distribution. However, they are less capable of systematic generalization to data drawn from unseen but related distributions, feat that hypothesized to require compositional reasoning and reuse knowledge. this work, we present Neural Interpreters, in a self-attention network as a system of modules, which we call functions. Inputs to the model routed through sequence of functions in way that is end-to-end learned. The proposed architecture can flexibly compose computation along width and depth, and lends itself well to capacity extension after training. To demonstrate the versatility of Neural Interpreters, we evaluate in two distinct settings: image classification and visual abstract reasoning Raven Progressive Matrices. In the former, we show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to task in sample efficient manner. In the latter, we find that Neural Interpreters are competitive with respect to the state-of-the-ane in terms of systematic generalization]
architecture that factorizes inference
Introduction
Rule-based programming is the basis of computer science, and builds the foundation of symbolic-AI based expert systems that attempt to emulate human decision-making to solve real-world problems. The process of inference entails channeling information through chain of computational units (e.g.. logical primitives) and culminates in conclusion that answers a given query. Such systems have the advantage that they permit efficient reuse of computational units and enable iterative reasoning over multiple cycles of computation. As simple example, consider the relation parent_ chii_cchhld (A, B), which can be used to construct a new relation sibling(U, (U, V) parent_childcA
U) AND parent_childde V), which in turn be used to construct yet another relation cousin v) V) AND sibling (x, Y), and so on. However, such systems are known to suffer from the knowledge acquisition problem, which is the inability to leverage unstructured data to derive new computational units and improve existing ones [30].
sarent_childde U) AND
In stark contrast to the symbolic paradigm, modern machine learning models excel at absorbing large amounts of unstructured data and yield strong performance many challenging domains, ranging from large-scale image classification language modelling. However, they are relatively rigid in how they share and reuse computation in order to process information: convolutional neural networks, for instance, process the content of an image at every location to arrive at the class label of given image. doing so, they only reuse computational units (here, convolutional filters) laterally at a constant depth i.e., amongst information being processed simultancously. In the same spirit, recurrent neural networks only reuse computational units (here, RNN cells) vertically, i.e., in depth. Such rigidity
* Equal contribution, ordered alphabetically by last name. Mlax--Paack---e Institutee for Intelligent Systems, Tübingen. Mila, Québec. AAmazon Web Services. Work partially done during internship at Amazon Web Services.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

in how computation is reused is believed to be one of the reasons current deep neural networks are less capable of systematically generalizing problems not encountered during training 33 39.
In the present work, we draw inspiration from typed programming languages to develop a self-attention based neural architecture that relaxes this rigidity in computation reuse. The resulting class of models, which we call Neural Interpreters, learns to flexibly use and compose computational units directly from data without additional supervision. Neural Interpreters factorize a self-attention network [58] as a system of computational units that we call functions. The input to the network is set-valued and processed in the network by a dynamically inferred sequence of functions potentially by the same function multiple times, enabling vertical sharing of computational units. This aligns with earlier ideas on independent mechanisms [21] 42 43 46 51: set of mechanisms can be combined and reused in many different ways depending on context (the current input or task), thus factorizing knowledge in independent pieces which can lead to better systematic generalization.
Neural Interpreters have two key benefits over vanilla self-attention networks. First, the modular inductive biases facilitate generalization beyond the training distribution and adaptation to new tasks in sample-efficient manner. Second, but consistent with the notion of factorizing knowledge into approximately independent and composable pieces, the proposed parameterization is (by construction) not only agnostic to the cardinality of the input set, but also to the number of functions. The latter im- plies that given a new task, additional functions can be non-disruptively added and fine-tuned. In other words, knowledge acquired from the prior training tasks can be effectively repurposed for new tasks.
Our primary contributions are follows. (a) We introduce the Neural Interpreter, an attention- based architecture that can be applied on arbitrary set-valued inputs or representations. (b) We quantitatively evaluate the proposed architecture in two distinet problem settings: multi-task image classification and abstract reasoning. In the former setting, we show that Neural Interpreters are capable of sample-efficient adaptation new tasks and can exploit additional capacity added after pre-training. In the latter setting, we demonstrate that Neural Interpreters are capable of out-of-distribution generalization. In partieular, we find that Neural Interpreters outperform the Vision Transformer baseline [3 17], which in turn is competitive in-distribution with the prior state-of-the -fate-of-the-art. In both settings, we find that Neural Interpreters develop the ability to gracefully trade-off performance with compute at inference time [63]. In addition, we include results on toy problem, where we explicitly probe the ability of Neural Interpreters to learn recomposable computational primitives. (c) We ablate over the architectural parameters of Neural Interpreters and qualitatively visualize what the model learns. We find patterns in how the input is routed through the network and that wide range of hyperparameters yield promising results.
2 Neural Interpreters
Scripti
MLP
f
f2
fs
Attention
Scripts
Noural Interprater
f
fa fs
Script:
Attention
Figure I: Leftmost: Overview of the architecture, shown here with image patches as inputs and two CLS tokens with corresponding classification heads. Center Left: The Neural Interpreter, shown here as a stack of three scripts. Center Right: script, shown here as collection of three functions applied over two function iterations. Rightmost: A stack of two Lines of Code (LOCs), spread over three parallel computational streams (one per function) and conditioned by the respective function codes (colored circles). Residual connections are present but not shown in this figure.
In this section, we describe in detail the components of a Neural Interpreter; see Figure for an overview. This architecture can be used as a drop-in replacement for self-attention network, e.g.. Transformers (58). In line with prior work [1] 17 55), we focus on applications to visual data.

---

Input and Output. The imput to the Neural Interpreter is any set with vector-valuee elements (x),, X; E R and the output another set of vectors {y E E with the same cardinality as the input set. We assume in this work that the input set contains vector embeddings of image patches 13] or of entire images [5]. The input set additionally includes one or more learned vectors, called CLS Tokens [66, for which the corresponding outputs interface with their respective classifiers (38]
Scripts. Neural Interpreter can be expressed as stack of na scripts, where a script maps one set of vectors to another with the same number of elements:
(1) script has four components: a type inference module, a type matching mechanism, a set of functions and an interpreter, explained below. The parameters of these components not shared between scripts. Role: Scripts function as independent building blocks that can be dropped in any set-to-set architecture, and Neural Interpreters with single script can perform well in practice.
{yy}} = Neurallnterpreter( (x+)s) = [Script,, o... o (n,, times) o... o Script] ((x+)))
Functions make the computational units in Neural Interpreters; they can be represented as a tuple of vector valued parameters, i.e., fu = (s., C.) where indexes functions. Here, Su referred to as the signature of the function fa (with meaning similar to that in programming languages), and it is normalized vector of dyyp dimensions. The signature vector specifies to the type matching mechanism (see below) what inputs are to be routed to fu- We refer to cu as the code vector of fu, as it instructs interpreter module (shared between functions, see below) how to process the inputs to fu. Role; Functions are vector-valued instructions to other components in the script. They implement the computational units that can be reused in the computational graph.
Functions.
Type Matching and Inference. The type match- ing mechanism (Figure 2 enables the learned rout- ing of set elements through functions, and it pro- ceeds in three steps. First, given set element X;, it is processed by an MLP module that outputs its type vector t,. This module is called the rype in- ference module, and the resulting type vector is an element of the same topological space as function signatures, i.e., a ddyge-dimension Next, given function fu and its signature vector Su E T, the compatibility Cui between the func- tion fu and set element x, is determined by the sine similarity between s,, and t; in TT (larger = more compatible). Finally, if this compatibility is larger than a threshold (r), fu is permitted access to x,. Formally, let j be a set of intermediate representation vectors (indexed by i). With a learnable parameter a and hyper-parameter T, we have:
hypersphere T.
Illustration of the type matching mech- anism. Functions only access set elements whose types in vicinity of their signatures.
Figure 2:
t TypeInferenco(x)) E Ti
dr t;)
(1 S.. tt)
Oui where Öus = exp
dy (Su: ti)
Cui
if dl (Su: t;) > T, else 0.
(3)
E+ E.E, C..
is called the truncation parameter of the kernel and is small scalar for numerical stability. The compatibility matrix C.. E [0, 1] will serve as a modulation mask [46) for the self-attention mech- anism in the interpreter (see below). Role: The type matching mechanism is responsible for routing information through functions. The truncation parameter controls the amount of sparsity in routing.
ModLin Layers and ModMLP. The components described below make use of linear layers conditioned by some code c. Consider linear layer with weight matrix W e Rdd x R and a bias vector b E Re Let x E R denote the input vector to the layer, and c E RRd condition vector, and W. E d. Re learnable matrix. The output y E Rd given as:
y ModLin(x;c) W(x @ LayerNorm (W.)))
where @ denotes element-wise product and we call the resulting layer modulated linear layer, ModLin layer 33. Further, one may stack (say) L such ModLin layers (sharing the same condition or code vector c) interspersed with an activation function (we use GELUs (66) to obtain ModMLP:
y ModMLP(x: c) (ModLing (- :)) Activation
ModLing(::ee (x:c))(x)
(5)
3

---

Role: ModLin layers and the ModMLP can be interpreted as programmable neural modules, where the program is specified by the condition or code vector c.
ModAttn. ModAttn is a conditional variant of the kernel modulated dot product attention (KMDPA) [46. where the key, query and value vectors obtained from ModLin .n layers conditioned by a vector. In our case, this vector is the code vector C. of function fu, and the corresponding key, query and value vectors are computed as follows (with {x,}; as input and h indexing attention heads):
kuhi = ModLingay (x,, c.)
qubi = ModLin (x,, Co)
Vuli = ModLin (x,, cu) (6)
we will encounter Xus. where the extra in the subscript denotes that the set element at index i is specific to the function u; in this case, x; is substituted with X,, in Equation O Next, given the keys, queries and the cunctin-varrablle compatibility matrix C.s- the modulated self-attention weights W Wuhij are given by:
Note that further below (e.g., in Equation 9
]
W Wahiy
Aurti' Kung
W whij
where
Wuntj Cui softmax,
E +
Wunii
dhey
Here, the quantity W uhiy denotes the attention weights in function fu between elements x; and x; at head and the softmax operation normalizes along j; intuitively, information about x; and x, is mixed by fu at head h only if W # O. Now, on the one hand, this can be the case only if both Cui and Cuy are non-zero, i.e., fu is granted access to both variables x; and x, by the typing mechanism. But on the other hand, Ju does not necessarily mix x; and Xj even if both Cui and Cug are non-zero, for the self-attention weights (square brackets) may still be close to zero depending on the context (i.e., the content of x, and xj). Next, the values are linearly mixed using the computed attention weights, which is then processed by a final ModLin layer to yield the output Yus:
Yus ModLin(Fuust cw) where
Yuhi E, Wahiy
Here, Fuish means the head-axis is folded into channels. Role; ModAttn enables interaction between the elements of its input set in multiple parallel streams, one for each function. The query, key, value, and output projectors of each stream are conditioned on the corresponding code vectors, and the interaction between elements in each stream is weighted by their compatibility with the said function.
Line of Code (LOC). An LOC layer is a ModAttn layer followed by a ModMLP layer (Figure [ rightmost),, where both layers share the same condition vector ca. and there are weighted residual connections between the layers. Assuming inputs (Xui)u, Ju, to the LOC, we have:
ب ModAttn (LayerNorm(x...))) Cu. (Coght) h:)
aus = Xui Cusāpi
(9)
bus = ModMLP((LayrrNo((aaa)) cw)
Yui = =ys + Cur
(10)
This parameterization ensures that yoi X,i if Cuu O. In words, if function (fa) is not granted access to variable (x:) by the typing mechanism, it acts as an identity function for this variable. Fur- ther, note that we allow the input set to be indexed only by i; in this case, we assume X x, for all u. Role: A LOC can be thought of as multiple instances of layer in the original transformer architecture (comprising a self-attention and a MIP module with residual connections). applied in parallel streams, one per function. Computations therein are conditioned on the respective code and signature vectors.
Interpreter. The interpreter layer is a stack of ny LOCss sharing the same function codes c, and funition-variable compatibilities Cui. Assuming the input to the interpreter is a set (xih.. we have:
y: = x; + E.. Cui (LOCn, c (.. times)...
((C))((xx))) Cu. {Curls)
(1I)
In words: the interpreter broadcasts a given set element to multiple parallel computational streams, one for each function. After the streams have processed their copy of the input element, the results are aggregated by a weighted sum over the streams, where the weights correspond of the input element with the respective function. Equation 11 can be justified by making two observations. First, if variable x; is not matched with any function by the typing mechanism, is left unmodified by the interpreter, i.e., if C.: 0 for all u. then y; x; This allows signals to be propagated through the interpreter without interference from existing functions, so determined by the type inference module. Second, the additive aggregation over the function index u implies that the overall parameterization of Neural Interpreters does not depend on the number of functions. This allows one to add new function fo simply by including its signature and code (s., c.) as
the compatibility

---

learnable parameters and finetuning these on (say) new problem. Role: instruction executor (one that is shared between functions). Given a set of inputs and an instruction (here, the function code), it executes said instruction to compute the outpur.
The interpreter serves as
Function Iterations in Script. Having expressed the overall model stack of multiple scripts, we are now equipped to describe the computational graph of a single script. A script can be expressed as a recurrent application of the type matching mechanism and the interpreter, where we refer to the composition of the latter two as a Function Iteration (FnIter):
{ = Fnlter {x,}} }}} = Interpreter (x, J. feu (Cujtu.)) where Cuj = TypeMatching(s,, X5)
(12)
(13)
Here, the TypeMatching component encapsulates both type inference and kernel matching, as detailed in Equation 2 A script (cf. Equation 1 can now be expressed as a recurrent application of Fnlter:
{y (FnIter (n, times) ... Fniter)((x)))
(14)
Role: Inside a script, function iterations enable sharing of computational units in depth. Increasing the number of function iterations can increase depth without increasing the number of parameters.
Preventing function signatures and variable types from collapsing on each other. One might ob- tain a scenario where the signatures of all functions and the types of all possible set elements collapse to single point in type-space. This causes all set elements to be routed to all functions, thereby under- mining the inductive bias of modularity. One effective way of preventing this degeneracy is to keep the function signatures fixed at initialization (i.e. high-entropy distribution). This effectively encourages the (learnable) type-inference module to produce diverse types, in order to use all available functions.
In summary, we observe that any input element x; may trace large number of computational paths as it progresses through the model, where a computational path is partially ordered set (poset) of functions (refer to Figure 3 for a visualization of such computational paths). In particular, owing to the fact that we recurrently apply function iterations scripts, this poset may contain repeated functions, enabling weight sharing in depth. Further, the use of an interpreter that is shared between functions but explicitly conditioned (or programmed) by their code vector allows us to remain independent of the number of functions and retain the ability to add more functions after the model has been trained. Future work may investigate manipulating the code vectors themselves, thereby enabling higher-order functions (i.e., functions that manipulate other functions).
3 Related Work
Modularity. Despite recent successes of deep neural networks, their ability to recombine and reuse meaningful units for systematic compositional generalization is shown to be limited [3 33 39) To mitigate this issue, work has been done to modularize deep architectures to enable the reuse and recom- bination of learned modules 9 19 [2 49 553. and connections have been drawn to the principle of Independent Causal Mechanisms 7 22
52). Modular architectures vary in how the modules are learned and laid out during inference. For instance, Neural Module Networks 22] and Neural Event Semantics 8 dynamically compose the modules using natural language parser, whereas neuro-symbolic architectures 22
24 34
46
61 62) typically require entity-centric but use neural components like attention for compositional reasoning. Unlike some of these methods, Neural Interpreters (NI) neither require domain knowledge nor object- centric representations. To obtain strong compositional inductive bias, NIs aim at factorizing the computational graph in terms of functional units which are dynamically recombined in order to solve a particular task. In comparison with SCOFF [23] and NPS [22] which also separate functions, argu- ments and values and enable a similar dynamic and composed application of functions, NIs introduce the notion of interpreter (which enables parameter sharing between functions and extending the set of functions easily) and sophisticated signature and typing mechanisms described in the previous section.
37
40
57
representations [37 57 62
Dynamic Routing of Information. The systematic generalization of modular architectures is shown to depend on the layout of modules 4 48). There exist different frameworks to facilitate
the joint learning of module layout and parameters uses collaborative multi-agent reinforcement learned routing,
19
24
46
50
53 For example, [49]
53
uses trainable gating network.
In this work, we use kernel modulated dot product attention the input types to assign inputs to functions at each step.
46 between function signatures and
5

---

Transformers. Modularity in Transformers (58] is relatively less studied. Repulsive attention [ based on repelling transformer heads to encourage specialization, whereas TIM [4] draws on the principle of independent mechanisms by dividing computation over parallel transformers that interact via competitive top-k attention (24]. In contrast, functions in NIs iteratively acquire set elements with kernel dot-produc" attention, which, if need be, can uniformly activate all the functions for given token, encouraging efficient use of computation. Closer to Neural Interpreters, Switch Transformers (STs) [18] employ mixture of experts framework where experts specialize across tokens, guided by a routing scheme which activates only one expert per token. However, STs focus more on dynamically selecting experts, whereas Neural Interpreters focus dynamically composing available experts. The starting point of this work is the vision transformer (ViT) [3] [7], which applies a standard Transformer directly to image patches. We use the same architectural scaffolding as ViTs, but implement additional inductive biases that facilitate fast adaptation and systematic generalization.
4 Experiments
In this section, we empirically evaluate Neural Interpreters in variety of problem settings. The questions we seek to answer are the following. (a) Can Neural Interpreters learn reusable compu- tational units, given number of training tasks that can in principle be solved with a fixed set of primitives? (Section 1.1 (b) Do Neural Interpreters modularize information in a way that helps fast adaptation ? (Section 4.2) (c) Can Neural Interpreters gracefully handle a larger number of functions (computational units) than they were trained with? (Section 4.2) (d) Do the inductive biases
help with systematic generalization required in abstract reasoning problems? (Section
4.1 Learning Fuzzy Boolean Expressions
In this section, construct a toy problem investigate whether neural interpreters can learn reusable functions when trained on a large number of tasks that share the same underlying building blocks.
Task Definition. Consider set of N scalars between 0 and (inclusive). We now define the following operations on the elements of this set:
NN). ). where v; E [0, 1] is real number
and xz)) = Zyز not(x;) v. = 11 11-14; 245 or or 2rg) 24 3 xz
(15)
Note that the above operations map from [0. 1J2 to [o, 1]: if I, E 1). they reduce to their Boolean namesakes. By combining these primitives, it is possible to construct and sample from family of 22N fuzzy Boolean functions mapping from [0. 1] to [0, 1] (Appendix B). The problem is now set-up follows. We sample 30 random fuzzy Boolean functions of N variables, (fi) :=1 where Si [0, 1js - [0, 1). of which we use 20 for the pre-training phase and reserve 10 for the later adaptation phase. For each function, we sample 163840 points from a uniform distribution over [0, 1]s, of which we use 80% for training, and the remaining for validation. This yields two multi-task regression datasets, one for pre-training and the other for adaptation. A sample from the pre-training dataset comprises a 5D input vector x E [0, 1]8 and the scalar regression targets fi (x), .... (x).
Method. We pre-train Neural Interpreter to regress all 20 functions simultaneously. To do this, we feed it set of 25 elements as input: the first 5 are the components of x (with learnable positional embeddings 44 added in) and the remaining 20 are learned CLS tokens (vectors), one for each function. We attach a (shared) regression head to the the CLS
Table 1: Mean Coefficient of Determination (R²) and StdDev over 10 tasks after training various sets of parameters. Gist: By just allowing the functions to rewire themselves by training only the type-matching parameters, one approaches the performance of tun- ing all remaining parameters, including those of the functions themselves. This suggests that the func- tions have learned recomposable primitives.
output elements corresponding tokens, and train their outputs to regress the respective function. Having pre-trained the model for 20 epochs, we finetune it for an additional 3 epochs on the 10 reserved func- tions {f2..... fso }. For the latter, we always instantiate and train 10 new CLS tokens, cor- responding to the new functions; however, we investigate three settings in which three different sets of parameters remain frozen. In the first setting, all parameters remain frozen, implying that the CLS tokens are the only trainable parameters. In the
Parameter Set
R²
All Parameters (Pretraining)
0.9983 + 0.0005
Finetuning CLS Tokens
0.9202 + 0.0198
+ Type Inference Parameters
0.9857 + 0.0034
+ Remaining Parameters
0.9953 + 0.0013

---

Figure 3: Visualization of computational paths taken by input set elements corresponding to three different sam- ples as they progress through a Neural Interpreter. Colored dots identify functions (same color implies shared parameters). and the weight of the lines denote their compatibility with elements. Gist: There variations (but also similarities) between samples in their constituent set elements are routed through the network.
Figure 4: t-SNE embeddings of the inferred types of set elements as they progress through a Neural Interpreter with two scripts with two function iterations each. The color identifies the closest function in type space, and the progression from left right iterations, suggesting that the input set elements gradually develop type they progress through the network.
the function iterations. Gist: Types more clustered in the later function
second setting, we unfreeze the parameters of the type matching mechanism (function signatures and parameters of the type inference MLP, in addition the CLS tokens). In the third setting, we unfreeze all parameters and finetune the entire model. Additional details in Appendix B
Hypothesis. By finetuning just the type-matching parameters, we only permit adaptation in how information is routed through the network. In other words, we only allow the computational units to rewire themselves in order to adapt to the new task at hand, while preserving their functionality. Now if the computational primitives that are learned during pre-training expect the performance of Neural Interpreters having finetuned just the type-matching parameters approach that obtained by finetuning all the parameters, the latter including ones that determine the functionality of the computational primitives.
recomposable, one would
Results. Table compares the coefficients of determination] (R²2) obtained in each of the investigated finetuning settings. We find that relative to finetuning just the CLS tokens, the performance difference between finetuning all parameters and just the type matching parameters is small. This is in line with expectation, suggesting that Neural Interpreters are indeed capable learning recomposable computational primitives.
4.2 Multi-Task Image Classification
In this section, we evaluate Neural Interpreters multi-task image classification setting. Our goals are (a) to determine whether the inductive bias helps with fast adaptation, (b) to investigate whether the interpreter can indeed function as general instruction executor, intuited in Section [ and (c) to demonstrate that the proposed architecture produces modules that can function autonomously (without additional training objectives that encourage this behaviour). Additional results in Appendix C analyze the effects of varying hyper-parameters
Task Definition. We consider three related datasets sharing the same label semantics, namely SVHN [11), MNISTM [20] and MNIST [35 The images therein are upsampled to shape 32 x 32 (if required), and the resulting image is augmented with RandAugment [14]. Subsequently, the augmented images are split [I3 to 64 patches of shape 4 x 4. In addition, we also use unaugmented
samples from the K-MNIST dataset 12 of Hiragana characters to probe fast-adaptation to new data.
H implies perfect fit; model that regresses to the mean has R²² 0.
7

---

672
Eoseact
504
90 944
15
250
Munber Docie
(a) Fast adaptation
(b) Capacity extension
(c) Dropped functions
Figure 5: Figure Saj Performance of Neural Interpreters (NI) compared to that of Vision Transformer (VIT) that performs equally well the validation set (y-axis), plotted against the number of training samples presented (x-axis). We see Neural Interpreters can adapt faster in the low-data regime. Figure (5) Validation performance (y-axis) of a Neural Interpreter trained with 5 functions but finetuned with various number of functions (x-axis). We see that performance increases with increasing functions, showing that the model does not overfit to the number of functions it was trained with. Figure(5cj Accuracy on the digits dataset as function of the number of dropped functions. We find that the performance degrades gracefully functions are dropped, suggesting that learned functions are autonomous, i.e. they can operate the absence other functions.
Method. We train Neural Interpreters for 100 epochs on the combined digits dataset described above. The input set contains 67 vector valued elements - the first 64 corresponding to linear embeddings of the 4 x patches, and the remaining each CLS token, linear classification head is attached to the corresponding output; given an input sample from a certain dataset, the respective classification head is trained to predict the correct class. To inject position information into the model, we use a variant of the relative positional encoding scheme described in [3], but applied only to the first 64 input elements (corresponding to the image patches). Having pre-trained on the digits dataset, we finetune the model on K-MNIST with varying numbers of samples for 10 epochs. For additional details, please refer to Appendix C
learnable CLS tokens (vectors), one for each dataset. For
Baseline. Vision Transformers (ViT) 13 17 make the natural baseline for Neural Interpreters, given the fact that the former is special case of the latter. The set-up with CLS tokens and classification heads is identical to that of Neural Interpreters. as is the training protocol. We use a hight-weight 8-layer deep model with approximately 1.8M parameters, but ensure that the considered Neural Interpreter model is even lighter, with roughly 0.6M parameters.
Hypotheses. (1) It has been suggested that model that ap- propriately modularizes the learning problem should excel at fast-adaptation
06
Pretrained Functions
new but related data distribution [6].
Accuracy KINIST
0.
If Neural Interpreters obtain a useful modularization of the problem, we should expect it to be able adapt faster the K-MNIST dataset than a non-modular baseline like the ViT. This is because some of the visual primitives present in the digit datasets (e.g., motifs, strokes or shapes) might reoccur in the K-MNIST dataset, which Neural Interpreters should be able to leverage. (2) Recall that in Section 2 we positioned the interpreter as a general purpose instruction executor, capable of being programmed by code vector. If this is indeed the case, and if the interpreter does not over- fit to the functions it was trained with, we should expect the capacity (and therefore performance) to increase as we add and train new functions (i.e., tuples of code and signatures), while keeping all other parameters fixed. (3) functions (modules) are indeed autonomous, we should expect that some of them are removed at test time, the others are still able to maintain performance. Consequently, the accuracy of the Neural Interpreter (trained with all functions) should
0.4
23
Number of Extra Functions
Figure Performance on K-MNIST af- ter adaptation. In one case (orange), func- tions pretrained from the digits dataset are kept intact. In another case (blue), func- tion codes and signatures are randomly sampled before adaptation. We find that the former performs better. Gist: Knowl- edge acquired during pretraining indeed reused during adaptation. This is apparent when comparing with a haseline where this knowledge is destroyed, leading to decreased performance after adaptation. only degrade gracefully as functions are (randomly) removed at test time.
Results. Figure 5a compares the fast-adaptation performance of a Vision Transformer to that of a Neural Interpreter. While both models achieve almost identical performance on the validation set for all datasets (the difference being less than 0.2% in favor of Vision Transformers), we find that Neural Interpreters significantly outperform Vision Transformers at fast-adaptation in the low-data
8

---

regime, and the performance gap is only gradually closed as more data becomes available. This observation supports hypothesis (1). Further, Figure O we establish a baseline where the functions pretrained on the digits dataset are destroyed prior to adaptation. We find that keeping the functions acquired from pretraining intact leads to significantly improved adaptation performance, suggesting that knowledge acquired during pretraining is indeed being reused during adaptation. This lends further support to hypothesis (1). Figure 5b shows the validation performance of a Neural Interpreter that was pre-trained with 5 functions the digits dataset, but tested on the K-MNIST dataset with varying numbers of functions, having finetuned just the function signatures and codes. We find that the performance improves (2). Figure[5cshoow shoss the effect of randomly removing functions on the model accuracy. We observe that the performance all datasets degrades gracefully as more functions are removed, thereby supporting hypothesis (3). In addition, Figure 3 visualizes the computational path taken by input set elements as they progress through the network, verifying that there is diversity in how samples routed through the network, and shows that the routing mechanism discriminates between samples. Figure 4 shows that the input set elements gradually develop type as they progress through the model. Additional figures in Appendix C Ganalyze the effect of varying the number of scripts, number of function iterations, number of functions, kernel truncation parameter, dimension of type space T and freezing the function codes / signatures on the validation performance. The key finding is that, on the one hand, a wide range of hyper-parameet settings leads to performant models; on the other hand, there are patterns in what
functions added at adaptation time. supporting hypothesis
perform consistently well.
4.3 Abstract Reasoning
In this section, our goal is to (a) use visual abstract reasoning tasks to evaluate whether Neural Interpreters are capable of systematic (compositional) generalization, and (b) characterize how Neural Interpreters maintain performance when the amount of compute is reduced at test time.
Task Definition. Progressively Generated Matrices (PGM) 5 is procedurally generated series of datasets based on Raven Progressive Matrices (RPMs) [47,, popular IQ test designed for humans. The datasets, each comprising around 1,2M samples, aim to assess abstract and analogical reasoning capabilities of machine learning models be- yond surface statistical regularities [29] 45). Each sample in a dataset (Figure 7 comprises 8 context panels arranged in an incomplete 3 3 matrix, and 8 candidate answer panels (a panel being an 80 x 80 image). Given access to the context panels, the model must predict which of the candidate images is the most appropriate choice to complete the matrix. The content of the panels in a matrix are related by one or more triples, where triple comprises a logical rule (E progression, XOR, OR, AND, consistent union) applied to an attribute (E size, type, color, position, number) of an object of type (E shapes, lines). There are 8 datasets in the series, of which 7 measure systematic generalization in differ- ent ways, i.e., the test datasets contain samples where the panels are related in ways that are not encountered during training. In this work, consider 6 datasets that probe the compositional reasoning ability of the model, namely: Interpolation, Extrapolation, Held-out (H.O.) triples, H.O. pairs of triples, H.O. Attribute Pairs, and Neutral. We omit H.O. line-type and H.O. shape-type datasets, since they stress the convolu- tional feature extraction instead of compositional reasoning component of the model. Please refer to Appendix D and 5 for additional details.
?
Figure 7: A PGM task. Top: Context panels. Bottom: Candidate panels. WReN and ViT predict the wrong answer (yellow). whereas NI predicts the correct one (green).
Method. Each panel (context and choice) is embedded with a shallow convolutional neural network to obtain an embedding vector. The input to the model is a set of 10 elements, comprising the embeddings of 8 context panels, that of single candidate panel, and CLS token (leamable vector). The model output corresponding to the CLS token is fed as input to the prediction head, which is trained to output score measuring the compatibility of the candidate panel to the context panels. The final prediction is obtained by applying a softmax over the scores of all candidate panels. We note that this set-up resembles the one proposed in [ for WReNs, and refer Appendix D for details.
Baselines. We train Vision Transformers IK 17 and Neural Interpreters with the same architectural scaffolding (described above). Additional basclines include Wild Relational Networks (WReN) (5),

---

Multiplex Graph Networks (MXGNet) [59] and WReNs atop disentangled representations (VAE- WReN) [54]. Regarding the latter, we note that disentangled representations only improves on the feature extraction component of the WReN model; as such, it is entirely complementary to our contribution, which is the abstract reasoning component. Future work may explore replacing the convolutional embeddings with disentangled representations.
Hypothesis. Recall that the test sets of (all but one) PGM datasets contain samples where the panels are not related in a way that is encountered during training. model that able to infer and reason about the underlying compositional structures is therefore more likely be successful. If the inductive biases do indeed enable Neural Interpreters to fac- torize knowledge in to recomposable primitives, we might expect them to be able to dynamically recombine these prim- itives in order to generalize to problems has not encoun- tered during training and thereby excel at this task.
0.8
Regime
0.7
Neurral
Trinle Pairs
0.6
Extrapotatio
0.5
0.5
0.2
011
Number of lterations
Figure 8: Test performance of a Neu- ral Interpreter as a function of the number of function iterations (i.e. the amount of inference- time compute). Gist: Neural In- terpreters are capable of trading-off perfor- mance with compute.
Results. Table 2 tabulates the validation (in-distribution) and test (out-of-distribun) of-distribution) accuracies obtained by all mod- els for the various considered datasets. We make two obser- vations: (1) Neural Interpreters are competitive in terms of test accuracy, outperforming both Vision Transformers and the prior state-of-the art in 4 of 6 datasets. (2) Vision Trans- formers are competitive in-distribution: they outperform the state of the art (excluding Neural Inter- preters) in 5 of 6 datasets in terms of validation accuracy. However, they are outperformed in terms of test accuracy by either Neural Interpreters or other baselines in all datasets. In addition, Figure g shows how the test-set performance varies with the number of function iterations, for a model that was trained with a fixed number of function iterations (8). We note that the number of function iterations is proportional to the amount of computation used by the model. In all cases, the model maintains at least 70% of its original performance while using only half the amount of compute; in most cases, at least 80% of the original performance is maintained. This suggests that Neural Interpreters can function as anytime predictors (63,, which are models that can trade-off performance with compute.
Table 2: Performance (prediction accuracy) of all models on different generalization regimes of PGM datasets. Note that the Val. performance measures in-distribution generalization, and the Test performance measures -att----ditriittionn generalization the corresponding datasets (except for Neutral). Extra. refers to Extrapolation, Aurribtee P. to Attribute Pairs and Triple P. to Triple Pairs.
Regime Model
Neutral
Interpolation Test
Attribute P.
Triple P. Test
Triples
Extra. Val.
Val.
Test
Val.
Val.
Test
Val.
Val.
Test
Test
WReN 5
63.0
62.6
79.0
64.4
46.7
27.2
63.9
41.9
63.4
19.0
69.3
17.2
VAE-WReN [54]
64.8
64.2
70.1
36.8
64.6
43.6
59.5
24.6
MXGNet [59]
67.1
66.7
74.2
65.4
68.3
33.6
67.1
43.3
63.7
19.9
69.1
18.9
ViT
73.3
72.7
89.9
67.7
69.4
34.1
67.6
44.1
73.8
15.9
92.2
16.4
NI (ours)
77.3
77.0
87.9
70.5
69.5
36.6
68.6
45.2
79.9
20.0
91.8
19.4
5
Conclusion
We have presented Neural Interpreters, self-attention based architecture capable of learning system of recomposable computational primitives. Neural Interpreters relax the rigidity in how computation is reused in current neural architectures, and our experiments show that the modular inductive biases it incorporates facilitate systematic generalization and sample-efficient adaptation to new tasks.
There are multiple exciting avenues of future research. One line of work could leverage the capacity extension capability of Neural Interpreters a continual learning setting, thereby enabling continuous integration of knowledge in neural model. Another promising direction entails using Neural Interpreters as a backbone for learning world models, where systematic generalization and out-of- distribution robustness are of paramount importance.
10

---

Acknowledgements
The authors would like to thank Sharath Chandra Raparthy and Sarthak Mittal for providing valuable feedback on the draft. This work was in part supported by the German Federal Ministry of Education and Research (BMBF) through the Tübingen AI Center (FKZ: 01IS18039B). YB acknowedges support from the Canada CIFAR Al Chair Program, and Samsung Electronics Co., Ltd.
References
[1] Bang An, Jie Lyu. Zhenyi Wang. Chunyuan Li, Changwei Hu, Fei Tan, Ruiyi Zhang, Yifan Hu, and Changyou Chen. Repulsive attention: Rethinking multi-head attention as bayesian inference. arXiv preprini
halh:2009.093664 2020.
[2] Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Neural module networks, In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 39-48, 2016.
[3] Ivan Anokhin, Kirill Demochkin, Taras Khakhulin, Gleb Sterkin, Victor Lempitsky. and Denis Korzhenkov. Image generators with annditiionlly-indeeeendde pixel synthesis, 2020.
[4] Dzmitry Bahdanau, Shikhar Murty, Michael Noukhovitch, Thien Huu Nguyen, Harm de Vries, and Aaron Courville. Systematic generalization: What required and can it be learned? In International Conference Learning Representations, 2019.
[5] David G. T. Barrett, Felix Hill, Adam Santoro, Ari S. Morcos, and Timothy Lillicrap. Mcasuring abstract reasoning in neural networks, 2018.
[6] Yoshua Bengio, Tristan Deleu, Nasim Rahaman, Rosemary Ke, Sébastien Lachapelle, Olexa Bilaniuk, Anirudh Goyal, and Christopher Pal. A meta-transfer objective for learning to disentangle causal mechanisms, 2019.
[7] Michel Besserve, Rémy Sun, Dominik Janzing, and Bernhard Schölkopf. A theory of inde- pendent mechanisms for extrapolation in generative models. arXiv preprint arXiv:2004.00184 2020.
[8] Shyamal Buch, Li Fei-Fei, and Noah D. Goodman. Neural event semantics for grounded language understanding. In Transactions of the Association for Computational Linguistics (TACL), 2021.
[9] Michael Chang, Abhishek Gupta, Sergey Levine, and Thomas Griffiths. Automati- cally composing representation transformations as a means for generalization. arXiv preprint 1rXiv:180.004640, 2018.
[10] Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training sulf-supervised vision transformers, 2021.
[11] Anna Choromanska, Mikael Henaff, Michael Mathieu, Gérard Ben Arous, and Yann LeCun. The loss surfaces of multilayer networks. In Artificial intelligence and statisties, pages 92-204.
PMLR, 2015.
[12] Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature, 2018.
[13] Jean-Bapticn Baptiste Cordonnier, Andreas Loukas, and Martin Jaggi. On the relationship between self- attention and convolutional layers. arXiv preprint arXiv: 1.035844 2019.
[14] Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoe V. Le. Randaugment: Practical automated data augmentation with a reduced search space, 2019.
[15] Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Lukasz Kaiser. Uni- versal transformers, 2019.
[16] Jacob Devlin, Ming-Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
[17] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x 16 words: Transformers for image recognition scale, arXiv preprint 2020.
11

---

[18]
William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961, 2021.
[19] Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. arXiv preprint 1eXiv::701.050 2017.
[20] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-aavversarill training of neural networks. The journal of machine learning research, 17(1):209--0330, 2016.
[21] Anirudh Goyal and Yoshua Bengio. Inductive biases for deep learning of higher-level cognition. arXiv preprint arXiv:2011. 2020.
[22] Anirudh Goyal, Aniket Didolkar, Nan Rosemary Ke, Charles Blundell, Philippe Beaudoin, Nicolas Heess, Michael Mozer, and Yoshua Bengio. Neural production systems. arXiv preprint 3xXf::2003.01931 2021.
[23] Anirudh Goyal, Alex Lamb, Phanideep Gampa, Philippe Beaudoin, Sergey Levine, Charles Blundell, Yoshua Bengio, and Michael Mozer. Object files and schemata: Factorizing declarative and procedural knowledge in dynamical systems. arXiv preprini arXiv:2006. 16225, 2020.
[24] Anirudh Goyal, Alex Lamb, Jordan Hoffmann, Shagun Sodhani, Sergey Levine, Yoshua Bengio, and Bernhard Schölkopf. Recurrent independent mechanisms. arXiv preprint arXiv:1909 10893, 2019.
[25] Vineet Gupta, Tomer Koren, and Yoram Singer. Shampoo: Preconditioned stochastic tensor optimization, 2018.
[26] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus), 2020.
[27] Drew A Hudson and Christopher D) Manning. Compositional attention networks for machine reasoning. arXiv preprint arXiv: 1803 803.00367, 2018
[28] Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol Vinyals, and Joao Carreira. Perceiver: General perception with iterative attention, 2021.
[29] Jason Jo and Yoshua Bengio. Measuring the tendency of cnns to learn surface statistical regularities, 2017.
[30] Simon L Kendal and Malcolm Creen. An introduction to knowledge engineering. Springer, 2007.
[31]
Daniel Keysers, Nathanael Schärli, Nathan Scales, Hylke Buisman, Daniel Furrer, Sergii Kashu- bin, Nikola Momchev, Danila Sinopalnikov, Lukasz Stafiniak, Tibor Tihon, et al. Measur- ing compositional generalization: A comprehensive method on realistic data. arXiv preprint arXiv:1912.09713,, 2019.
[32] Louis Kirsch, Julius Kunze, and David Barber. Modular networks: Learning to decompose neural computation. arXiv preprint arXiv: /8/ 1.85249, 2018.
[33] Brenden Lake and Marco Baroni. Generalization without systematicity: On the compositional skills of pepueecc-to-sequeeee Learning, pages 2873-2882. PMLR, 2018.
recurrent networks. In International Conference Machine
[34] Alex Lamb, Di He, Anirudh Goyal, Guolin Ke, Chien-Feng Liao, Mirco Ravanelli, and Yoshua Bengio. Transformers with competitive ensembles of independent mechanisms. arXiv preprint 1rXiv:2103.00336 2021.
[35] Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available:
2, 2010.
[36] Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han. On the variance of the adaptive learning rate and beyond. arXiv preprint
arXiv:1908.0m 03265. 2019.
[37] Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention, 2020.
[38] Lajanugen Logeswaran, Ann Lee, Myle Ott, Honglak Lee, Mare' Aurelio Ranzato, and Arthur Szlam. Few- sequence learning with transformers. arXiv preprint arXiv:2012.09543, 2020.
12

---

[39]
Joao Loula, Marco Baroni, and Brenden M Lake. Rearranging the familiar: Testing composi- tional generalization in recurrent networks. arXiv preprint arXiv:1807.07545, 2018.
[40] Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B Tenenbaum, and Jiajun Wu. The neuro- symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. arXiv preprint arXiv: / 1904. / 25584, 2019.
[41] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[42] G. Parascandolo, N. Kilbertus, M. Rojas-Carilla, Carulla, and B. Schölkopf. Learning independent causal mechanisms. In Proceedings of the 35th International Conference on Machine Learning (ICML), volume 80 of Proceedings of Machine Learning Research, pages 4033 0333-4041. PMLR, 2018.
[43] Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: founda- lions and learning algorithms. The MIT Press, 2017.
[44] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAl blog, 18))9, 2019.
[45]
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks, 2019.
[46]
Nasim Rahaman, Anirudh Goyal, Muhammad Waleed Gondal, Manuel Wuthrich, Stefan Bauer, Yash Sharma, Yoshua Bengio, and Bernhard Schölkopf. S2rms: Spatially structured recurrent modules, 2020.
[47] John C Raven and John Hugh Court. Raven's 's progressive matrices and vocabulary scales, volume 759. Oxford pyschologists Press Oxford, England, 1998.
[48] Clemens Rosenbaum, Ignacio Cases, Matthew Riemer, and Tim Klinger. Routing networks and the challenges of modular and compositional computation. arXiv preprint arXiv: 1904:v:9904.222 12774, 2019.
[49] Clemens Rosenbaum, Tim Klinger, and Matthew Riemer. Routing networks: Adaptive selection of non-linear functions for multi-task learning. arXiv preprint arXiv: 2017.
[50] Sara Sabour, Nicholas Frosst, and Geoffrey Hinton. Dynamic routing between capsules. arXiv preprint
2017.
[51] B. Schölkopf, F. Locatello, S. Bauer, R. Nan Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio. Towards causal representation learning. Proceedings of the IEEE, 2021.
[52] Bernhard Schölkopf, Dominik Janzing, Jonas Peters, Eleni Sgouritsa. Kun Zhang, and Joris Mooij. On causal and anticausal learning. arXiv preprint arXiv: 222066671 2012.
[53] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoe Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-eexprrs layer. arXiv preprint 1701.0070 2017.
[54] Xander Steenbrugge, Sam Leroux, Tim Verbelen, and Bart Dhoedt. Improving generaliza- tion for abstract reasoning tasks using disentangled feature representations. arXiv preprint arXiv:1811.0478. 2018.
[55] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers and distillation through attention, 2021.
[56) Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve, and Hervé Jégou. Going deeper with image transformers, 2021.
[57] Sjoerd Van Steenkiste, Michael Chang, Klaus Greff, and Jürgen Schmidhuber. Relational neural expectation maximization: Unsupervised discovery of objects and their interactions. arXiv preprint arXiv: 10353, 2018
[58] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez. Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. arXiv preprint 7rX.v:1706.03762. 2017.
[59] Duo Wang, Mateja Jamnik, and Pietro Lio. Abstract diagrammatic reasoning with multiplex graph networks. arXiv preprint arXiv:2006. //99, 2020.
[60] Ross Wightman. Pytorch image models.
pytorch- imagem models 2019.
13

---

[61] Yuhuai Wu, Honghua Dong, Roger Grosse, and Jimmy Ba. The scattering compositional learner: Discovering objects, attributes, relationships in analogical reasoning. arXiv preprint arXiv:2007.042/ /2, 2020.
[62] Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B Tenenbaum. Clevrer: Collision events for video representation and reasoning. arXiv preprini arXiv:1910.01442, 2019.
[63] Shlomo Zilberstein. Using anytime algorithms in intelligent systems. Al magazine, 17(3):73-
73, 1996.
14


Reviews:
[{"review_id": "qXkIl4F0too", "reviewer": "Reviewer_zsVd", "summary": "This paper proposes Neural Interpreter (NI), a drop-in replacement for Transformer layers that can be applied on set-valued inputs. The motivation behind the architecture is that common computation units in deep learning such as convolution layer is rigid in how they process the input, so to overcome this, NI leverages self-attention to dynamically determine computation path using its modular functions depending on the given input. The paper shows how NI performs well in transfer-learning and systematic generalization tasks.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality & Significance\n- Neural Interpreter’s way of dynamically composing functions can enable better generalization and its modularity give a way of expanding model - possibly continual learning although not studied in this paper. Similar to recent work Switch Transformer, NI proposes an efficient way of leveraging modular architecture with large number of parameters differing from other modular models such as SCOFF. Although promising, I find the experiments section not too strong as discussed in the next section.\n\nQuality\n- I like the model formulation proposed by NI of how it used functions to modulate and compose computations.\n- Section 4.1 could have baseline models such as the plain transformer model to demonstrate if the inventions in NI is really making the task solvable.\n- Additional experiments on continual learning datasets by continuously adding more functions to NI would strengthen the results. \n- Analyzing or visualizing how modular functions are specialized would be beneficial. Figure 3 shows computation paths, but it is not easy to see what is going on. With a smaller number of functions on a simple task, it would be possible to clearly see if functions in NI can specialize according to the given input type. It would be also good to analyze how the truncation parameter can be used to control sparsity.\n- Figure 5 (bottom) could also compare with ViT for example by adding more layers to ViT.\n- Section 4.3 Table 2, it is hard to tell if the inventions in NI is really beneficial over the ViT baseline, especially because the results are reported from a single run for each setting.\n\nClarity\n- The paper is well-written. Minor comments: notations are a bit confusing. e.g. W_uhij that has four different subscripts merged together.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "kYR_NTzvff", "reviewer": "Reviewer_hiUm", "summary": "The paper proposes a model based on self-attention, factorised in different modules. It processes a sequence of tokens and could be viewed as a generalisation of a Transformer. The key idea is to have different modules (associated with a function) that behave differently but are general and actually share parameters. The core module of ModAttn can be viewed as a self-attention module with dynamic weights that depend on a context / code c. Using n_f learnable codes, each assigned to a ‘function’ leads to n_f modules, each with different functionality. An important aspect is the fact that each function only affects a subset of tokens, just the ones that are close to a type (learnable parameters) of the function. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Pros**:\n\n+ The paper has an important goal, of obtaining models that are factorized in a meaningful way and offer inductive biases towards reusability and fast adaptation.\n\n+ The method is sound and expands on the works using modular networks and self-attention. The method presents novel elements, the idea of functions that act according to similar dynamics (because they share parameters) but are modulated by the interpreter is novel and interesting.\n\n+ The paper presents some interesting results. Especially interesting is to note that adding multiple functions (Figure 5) improves the results, although they are not pre-trained, and the additional parameters should be small.\n \n+ The model obtains good results on a Abstract Reasoning task that require some form of compositional reasoning.\n\n**Cons**: \n\n- I think that the experimental section is not really convincing enough to strongly support all the claims, as summarised below.\n\n- For Multi-Task Image Classification, the proposed NI model adapts faster than the ViT, but this does not mean that it is modular. The real cause of the improved adaptability could be other than modularity. NI model could simply be easier to optimise, without being modular in any way. And the optimisation could be helped by the fact that NI has 3 times less parameters than ViT.\n\n- For Fuzzy Expressions, it is interesting to note that most of the performance gain is obtained when learning the Type Inference Parameters and this hints that the method learns to recompose the previously learned modules. But this is still a somewhat weaker evidence as the compatibility matrix could encode enough information, that helps the tasks, without necessary reusing components.\n\n- “we include results on a toy problem, where we explicitly probe the ability of Neural Interpreters to learn recomposable computational primitives”. What does ‘explicitly probe’ refer to? Although there are hints that suggest the model leans recomposable primitives, this is not explicitly tested. \n\n\n- ““there is diversity in how samples are routed through the network, and shows that the routing mechanism discriminates between samples.” Is there a reason why this diversity is different from the diversity obtained at initialisation? Could we interpret this diversity as having more meaning than what would be obtained by taking the similarities of random projections for example?\n\n\n**Questions**: \nIs there a fundamental difference between using multiple LOC or using multiple scripts? Having both functionalities increases the flexibility of the model, making it more capable of expressing various functions more easily, but is there something beyond this?\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "This is sufficiently tackled.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "12"}, "extra_text": {}}, {"review_id": "fDiczEvvyRE", "reviewer": "Reviewer_5mdn", "summary": "This paper proposes a method for modularizing neural networks inspired by programming functions and compilation. Building on the success of attention and transformer architectures, the neural interpreter (NI) proposes a new architecture built on layered functions, each of which is composed of stacked MLP and attention layers. Functions and input variables both have embeddings, called “signature and type” respectively, that allow NIs to learn specialized functions that operate over a subset of inputs. The NI is applied to a suite of image datasets as well as to Raven matrices, where it is shown to outperform vision transformers.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: \nThe idea is very creative and presents a scalable, continuous approximation to modular components in a neural network. In particular, I found the joint mapping of functions and inputs in an embedding space to an insightful idea that theoretically enables the neural network to compartmentalize information and reuse computation. Overall, I enjoyed the analogy to code interpreters and found the presentation compelling. My primary concern from a modeling perspective is that it was not clear to me why functions are encouraged to specialize to specific inputs e.g. there is not objective encouraging disentanglement or diversity, so why should the functions not all learn similar, or overlapping things -- which would limit the modularity and reusability of the network. \n\nQuality/Significance: \nThe experiments did a good job demonstrating promising results for the NI, although improvements can be made. For instance, in the toy experiment in 4.1, it would be nice to see adaptability to held-out base operations (e.g. XOR). In 4.2, I would like to also compare NI and VTs through more naturalistic images. Consider exploring the Meta-dataset as a collection of realistic images. I would have also liked to see more evidence in Figure 5 on functions learned in pretraining being reused, or more analysis in section 4.3 on why NI is outperformed by WReN in the Attribute P. and Triples datasets.\n\nClarity: \nVery clear writing. The authors do a good job of describing a complex system clearly. I appreciate explicitly denoting changes in notation (e.g. line 123-125), and sentences in Section 2 explaining roles. \n\nQuestions about Section 2:\n- Line 73: are the image embeddings learned as well? Or pretrained?\nIn functions, it seems like you want to disentangle information between the code vector and the type vector. How do you ensure the type vector captures no information about the code and vice versa?\n- In type matching, you want to have function signatures and element types “cluster” naturally such that some functions specialize to some types, etc. To my understanding though, it seems like nothing prevents all the type vectors and function signatures from being close to each other such that all functions have access to all types. Where is diversity / sparsity being enforced? A related question: how do you prevent two functions from learning the same thing?\n- How do you choose tau? If it is too high, I imagine learning will be very slow as C_{u,i} will be 0 for most matches. But at the same time, you want tau to be big in order to specialize functions (i.e., so that functions don’t take everything as input). \n- Do you enforce type embeddings and signature embeddings to live in a compact space (e.g. surface of a hypersphere?), otherwise I imagine you could have issues with unbounded vector magnitudes in training.\nModLin and ModMLP remind me a lot of FiLM (Perez et. al.). Might be worth citing or comparing against.\n- Conditioning attention networks (transformers) on vectors has been studied in the past: popular methods include an adapter network (Houlsby et. al.), FiLM, and a simple task token (TAM, Logeswaran et. al.). I’m curious how ModAttn compares to these. \n- When adding a new function (line 158), do you need to finetune old functions or can you keep them fixed? I could see this either way since existing functions might “take up most of the area in the embedding space”, it becomes harder to add new functions without moving around old functions to make space.\n- I might have missed this but are the conditioning vectors c_u learned? If so, how? Through a network? What are the inputs it gets?\n- Is it possible to come up with a synthetic example of something the neural interpreter can easily capture with its architecture that stacked transformer layers cannot? This would help me clarify when the interpreter should be superior, and why.\n\nQuestions on Section 3:\n- Consider adding a citation to Neural Event Semantics (Buch et. al.) in the related work.\n\nQuestions on Section 4:\n- Building on the toy experiment in Section 4.1, can the neural interpreter adapt to a held out operation? (e.g. it does not get to pretrain on the xor operation and must quickly learn new functions post training). This would be more convincing to show generalizability of modular components.\n- In the multi-task vision experiments, is fine-tuning changing just the type inference parameters or all parameters? Iit would be nice to see that most of the learned functions from pretraining don’t change, hence “reusing modularized knowledge”. Do you know how much information about previous tasks is preserved after adaptation? One could test the performance of the NI on pretrianing tasks after adapting to K-MNIST to see how much information is forgotten. I’m also interested in comparing the orange line in Figure 5 to a NI trained from scratch on K-MNIST (how fast does that learn?).\n- In Table 2, we see in some tasks, WReN / VAE-WReN outperform ViT and NI: do you have a sense of why for Attribute P. and triples, this is true? Is there something fundamental to those tasks that requires a level of disentanglement that NI does not have? Similarly, I imagine NI would benefit greatly from function disentanglement.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I think an explicit limitations section would be helpful to (1) be transparent about any optimization or modeling difficulties in NI and also (2) explore cases in which NI does not perform well. The broader impact statement could be expanded to highlight the importance of modularity of neural networks for maintaining and improvement models. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "Tcr3QZ7tc4t", "reviewer": "Reviewer_8zbe", "summary": "This paper focuses on improving the generalization ability of deep models to the unseen but related distributions to training data, and thus enhancing interpretability. In specific, a Neural Interpreter is proposed, which factorizes the inference of self-attention networks as the dynamic routing within functions. The Neural Interpreter can flexibly compose computation on a per-sample basis, and it is end-to-end trainable. Some interesting experiments including image classiﬁcation and visual abstract reasoning are conducted to validate the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Generally speaking, I think that this is a nice paper. The proposed dynamic Neural Interpreter is novel in concept. The dynamic routing mechanism and the type matching algorithm seem to be natural and flexible. My major concern lies in the experiments.\n\nThe comparison with ViT is presented on some tiny data sets like SVHN and MNIST. However, ViT is designed for much larger scenarios like ImageNet. Therefore, I think that current empirical results are not sufficient enough to validate the superiority of Neural Interpreter.\n\nIn addition, although ViT recently attracts much attention from the research community, CNN is still widely used in many vision tasks. The proposed Neural Interpreter seems to be a pretty flexible framework. It is possible to be applied to CNN?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "My concerns on the limitations of this paper have been presented in \"Main Review\". The authors do not discuss the potential negative societal impact of their work. Adding more related discussions will be helpful.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses the goal of modular information processing in deep networks in an interesting way, inspired by the architecture of function application in a PL interpreter. Everyone agrees that this is an interesting approach, and the analogy to interpreters is useful. The results, especially after the response phase, support the ability of this architecture to generalize in interesting ways. It would be useful in revision to make sure to clarify some key points raised by reviewers (such as what encourages the NI to learn sparse types).
Decision: Accept (Poster)