Attention on Abstract Visual Reasoning
Authors: Lukas Hahne
Timo Lüddecke
Florentin Wörgötter
David Kappel
Venue:  (2020)
Keywords: Transformer Networks
Self-Attention
Wild Relation Networks
Procedurally Generated Matrices

Abstract: Attention mechanisms have been boosting the performance of deep learning models on a wide range of applications, ranging from speech understanding to program induction.  However, despite experiments from psychology which suggest that attention plays an essential role in visual reasoning, the full potential of attention mechanisms has so far not been explored to solve abstract cognitive tasks on image data. In this work, we propose a hybrid network architecture, grounded on self-attention and relational reasoning. We call this new model Attention Relation Network (ARNe). ARNe combines features from the recently introduced Transformer and the Wild Relation Network (WReN). We test ARNe on the Procedurally Generated Matrices (PGMs) datasets for abstract visual reasoning. ARNe excels the WReN model on this task by 11.28 ppt. Relational concepts between objects are efficiently learned demanding only 35% of the training samples to surpass reported accuracy of the base line model. Our proposed hybrid model, represents an alternative on learning abstract relations using self-attention and demonstrates that the Transformer network is also well suited for abstract visual reasoning.

Under review as a conference paper at ICLR 2020
ATTENTION ON ABSTRACT VISUAL REASONING
Anonymous authors Paper under double-blind review
ABSTRACT
Attention mechanisms have been boosting the performance of deep learning mod- els on a wide range of applications, ranging from speech understanding to program induction. However, despite experiments from psychology which suggest that at- tention plays an essential role in visual reasoning. the full potential of attention mechanisms has so far not been explored to solve abstract cognitive tasks on image data. In this work, we propose a hybrid network architecture, grounded on self- attention and relational reasoning. We call this new model Attention Relation Ner- work (ARNe). ARNe combines features from the recently introduced Transformer and the Wild Relation Network (WReN). We test ARNe on the Procedurally Gen- erated Matrices (PGMs) datasets for abstract visual reasoning. ARNe excels the WReN model on this task by 11.28 ppt. Relational concepts between objects are efficiently learned demanding only 35% of the training samples to surpass re- ported accuracy of the hase line model. Our proposed hybrid model, represents an alternative on learning abstract relations using self-attention and demonstrates that the Transformer network is also well suited for abstract visual reasoning.
INTRODUCTION
Psychological models of human intelligence identify different manifestations of intellect. FLUID INTELLIGENCE describes the ability to adapt to new problems and situations without relating to previous learning outcomes (Hagemann et al. 2016). This ability is considered as one of the most important aspects for learning and is essential for solving higher cognitive tasks (Jaeggi et al.) 2008)) In order to excel in this type of intelligence, cognitive capabilies such as figural relations, memory span and inductive thinking are decisive. Fluid intelligence also paves the ground for ABSTRACT REASONING, the ability to use symbols instead of concrete objects. Furthermore, empirical data show that attention and fluid intelligence are strongly interlinked (Stankov 1983) Schweizer 20100] Ren et al. 2013). Subjects that perform bad on attention tasks are also more likely to show deficits in abstract reasoning and fluid intelligence (Ren et al. 2012 2013).
RAVEN'S PROGRESSIVE MATRICES (RPM) an established test method for intelligence, espe- cially fluid intelligence and abstract reasoning (Bilker al. 2012). It is set of non-verbal tests showing several geometric objects arranged to certain implicit rule (Figure I shows an example). The test subject has complete the 3 x 3 matrix by picking object that matches the implicit rule. The PROCEDURALLA ALLY GENER ATED ATED MATRICES (PGMs) (Santoro et al. 2018) dataset is motivated by RPMs and synthetically generated for training neural networks. Its core feature comprises sev- eral relations between objects and one of their attributes, e.g. the number of an object of a certain type in each matrix panel. A single matrix can contain up four such relations simultaneously. A PGM's context and possible answers are shown in Figures Ta and [T, respectively. Importantly, the PGM matrices are presented to a neural network as images such that the task requires a certain level of understanding of geometrical relations. The Wild Relation Network (WReN) uses different combinations of matrix elements as a mechanism for relational reasoning and is currently among the best performing solutions for the PGM dataset. However, despite the close relationship between attention and reasoning, the WReN is lacking a mechanism for attention.
Apart from the visual domain, it was recently shown that attention is also essential for another cogni- tive task that fundamentally relies on abstract reasoning: language understanding. The Transformer, an attention- based neural network, was introduced improve machine translation and transduction Vaswani et al.) 2017). The Transformer embodies a self- attention mechanism to relate parts of its input with each other for writing output. Since this form of attention and abstract reasoning is

---

Under review as a conference paper at ICLR 2020
a
PGM contexts
PGM choices
?
Figure I: Datasets for visual reasoning. a: Sample from the RPM dataset (Bilker et al. 2012)
Context: blue, choices: red. Correct choice: 1. Implicit rule: Subtraction along rows and columns. b, Sample from the PGM dataset (Santoro et al., 2018). contexts (b) and corresponding choices
(c). The ? in (b) should be replaced by correct choice from (c). Correct choice: The hexagon in the last tile. Implicit rule: progression of shape types (number of edges) in each column. Colors added to enhance visualization.
vital for fluid intelligence we hypothesize that a Transformer augmented with relational reasoning capabilities can perform well the PGM task.
Another special trait of human intelligence the ability to infer rules with little supervision. Often, only few samples suffice for people to grasp the idea of tasks such as visual reasoning. In contrast to that, current deep network architectures are trained on huge quantities of data, e.g. the PGM dataset contains more than 1.2 million labelled samples. To see if self-attention also enhances this capability of deep networks we investigate the sample efficacy by training our proposed model only on a fraction of the total available data samples.
Our contribution to the field of machine learning is three-fold:
We introduce the Attention Relation Network (ARNe) that combines features from the WReN and the Transformer network and can be directly trained visual reasoning tasks.
We evaluate ARNe on the PGM task and show that significantly outperforms the current state-of-the art (Steenbrugge et al. 2018) by 11.28 ppt.
We demonstrate that ARNe is very sample-efficient and achieves its peak performance with only 35% of the full PGM dataset.
This paper is organized follows. In Section we discuss related approaches and training datasets. In Section 3 we introduce the ARNe architecture and in Section we show our simulation results. We conclude in Section[5]
RELATED WORK
Language modelling, i.e. predicting the likelihood of future words given set of contextual words, is core task in natural language processing (NLP). While neural ap- proaches to language modelling have some tradition (Bengio et al.) 2003), the success of the word2vec (Mikolov al. 2013) has revived and boosted the interest neural networks for this task. While word2vec defines fixed vectors for words, recently, several approaches were proposed that condition word representations on their context (Howard Ruder 2018 Devlin et al. 2018 Peters et al.) 2018). In particular attention-basse models have emerged as the standard tool for lan- guage modelling as well as several downstream tasks, such as question answering (Raipurkar et al, 2016) and translation (Vaswani al. 2017). Starting with the seminal Transformer (Vaswani al. 2017). follow-up models were successively improved (Devlin et al. (018) and trained on larger text datasets achieving remarkable success across various NLP tasks (Radford al. 2019). Besides the work in language modelling, in concurrent work, Transformer based models have been employed for visual question answering (Li et al. 2019). In this work take inspiration from the model
Language Modelling
2

---

Under review as a conference paper at ICLR 2020
Input Choices
Panel Encoder
Concatenation X;
Equation (2)
Posi Disitional Encoding
Linear Layer
CNN
Layers
Multi-Head
Contexts
Concatenation X
Add
Output
Probabilities
Choice
f
g
possible choices)
MLP layens with ReLUI Drapout
MLP layers with Belli
Layer 1
8
8
Layer
Meta-targets
(12 tinary indicators)
Equation (3)
Equation (2)
Figure 2: Overview of the ARNe network for abstract visual reasoning.
design of the attention Transformer. However, instead of applying in language modelling task, we use it for abstract reasoning.
A Abstract Reasoning Recently, there has been growing interest in abstract reasoning indicated by the introduction of datasets operating on textual data, e.g. babl (Weston al. 2015), well visual data, involving CLEVR (Johnson et al. 2017). FigureQA (Kahou et al. (017) and NLVR (Suhr al. 2017).
While classic recurrent neural networks have often been used for these tasks, it has been found that novel architectures such attention and memory are better suited address abstract reasoning. Attention mechanisms are method to simplify complex information by attending to different lo- cations of the given data. This is an advantageous procedure let neural network determine specific reasoning operation on these selected locations. Memory is needed to preserve informa- tion tasks where multiple consecutive reasoning operations are required. LSTT (Hochreiter & Schmidhuber 1997) for example, the hidden state is protected by gates from being overwritten by new input. Models incorporating attention and memory involve the Neural Turing Machine (Graves) al. 2014)) which evolved into the Differentiable Neural Computer (Graves et al. 2016). Dynamic Memory Networks (Xiong et al., 2016) address both, visual and textual question answering. The re- cently introduced Memory, Attention and Composition (MAC) Network Hudson & Manning (2018) achieved excellent scores in the CLEVR benchmark by using multiple explicit reasoning steps. Re- cently, reasoning module relying on attention Zambaldi al. designed be employed in reinforcement learning framework.
Our work differs by measuring fluid intelligence by understanding visual patterns rather than solving referential expressions (to which part of an image belongs certain word) which is required in CLEVR, Fig NLVR and many visual question answering tasks. The former task is considered in (Santoro et al. 2018) and (Steenbrugge et al. 2018) with which we compare our proposed method.
3 ATTENTION RELATION NETWORK (ARNE)
In this section we describe the Attention Relation Network (ARNe), which combines techniques from language modelling to abstract reasoning. It takes eight contextual panels and eight choice panels from the PGM, of which one fits to abstract relational rule implicitly determined by the context panels. The model output logits of corresponding fitness probability for each choice
3

---

Under review as a conference paper at ICLR 2020
Table 1: Modules and their corresponding parameters of ARNe.
Module
Parameters
Panel Encoder
CNN
2D-Convolution
x 3 x 32, padding stride
Batchnorm
ReLU
Linear layer
Embedding
HW 32 + 512
Transformer
Transformer encoder
x6
see Table 6
g
linear layer
each 512 X 512
x4
ReLU
f
(Dropout) linear layer
0.5 (only for second layer)
x3
512 x 256, 256 x 256 and 256 13
ReLU
panel and logits for corresponding probabilities of the rules embedded in a PGM. The model is illustrated in Figure 2
First, representations for context panels, denoted by x,, and representations for choice panels, de- noted by Cg. are generated using shared Convolutional Neural Network (CNN). x; denotes the i-th context panel feature, C₁ denotes the k-th choice panel feature. This network has the same hy- perparameters as the CNN in the Wild Relation Network (WReN). A one-hot positional encoding indicating one out of nine possible positions within the panel grid is concatenated to each extracted panel feature and the resulting vector is projected to obtain final representation for each panel (see panel encoder and concatenation blocks Figure
Sequences Xx of length N are composed of the context and choice panel representations, Optionally, a learnable deliminiter d which has the same number dimensions as the panel representations can be included between contexts and choice. Thus, sequences of N N 10 elements obtained:
X X2,,X X8, c2)
We generate sequence X for all eight choices. A multi-step attention network inspired by the encoder of the Transformer model (Vaswaní et al., 2017), processes this sequence using the self- attention mechanism for abstract reasoning. All activations are accumulated before the network's output generated by the MLP f.
N O2 (
where g denotes the output of the l-yer MLP (see f and g decoder blocks in Figure 2 The vector E R denotes the logits of the corresponding probability distribution over the choices and the matrix Å 88122 indicates the logits of the corresponding presence probabilities of rules imposed by the panels where å E IR12 is defined as transposed row vector of this matrix. The construction of these meta-targets is explained subsequently.
The network returns matrix E 1 which contains both, logits for the prediction of choice panel and logits the prediction of underlying patterns across the PGM
P1
a
=
(3)
is
a7
08
4

---

Under review as a conference paper at ICLR 2020
Table 2:
PGM accuracy by previous methods and model (ARNe). Accuracy of WReN
ported in (Santoro et al. 2017) and our implementation. MAC model included for comparison.
Model
Accuracy [%]
MAC (our implementation)
12.6
VAE- WReN (Steenbrugge et al. 2018
4
64.2
WReN (Santóro et al. 2017)
10
76.9
WReN (our implementation)
10
79.0
WReN-MAC
79.6
ARNe (our implementation)
10
88.2
Meta-targets the PGM dataset, the layout and appearance of the nine panels follow an implicit rules. Each rule is represented by triplet containing object type, object attribute and relation type. An example rule could be (shape, size, progression) to describe a pattern of triangles with increasing size. Up to four different relational rules (i.e. four triplets) may occur simultaneously in single PGM. Meta-targets are created by aggregating all binary encoded rules with an OR operation resulting in 12 dimensional meta- target vector.
In order to identify correct panels, a successful model should be able to infer the underlying con- struction principle reliably. To enhance rule prediction in the model, incorporate the auxiliary information encoded in the meta-targets in the loss function. Since each choice panel results prediction åg of meta-target, the predictions need be aggregated by over choice panels, implying an OR relation between the elements (Santoro et al. 2018).
Loss The loss L is defined by a weighted combination of finding the right choice p and detection of the correct logical pattern (Santoro et al. 2018):
L - CE(.. p) 8 BCE
àg
1=1
with CE and BCE being cross entropy and binary cross entropy. The parameter controls the influence of the meta-targets, i.e. for 3 0 the meta-targets are ignored. By enforcing correctness of the meta-targets we enable richer gradients that identify better parameters.
Implementation The learning is carried out using adam Kingma & Ba (2014) optimizer with a batch size of 64 and an initial learning rate of 0.5 10-4 using learning rate scheduler with exponential decay. We apply early stopping (Goodfellow et al.) (016)) with patience of three epochs. Parameters of the individual components of our method are presented Table For further details refer the appendix or the implementation of our approach which is available here: http: http//hidee ddde for bl blind review. Will be discl osed upon acceptance
4 EXPERIMENTS
We evaluate ARNe on the PGM dataset (DeepMind 2017). Each PGM input sample consists of 160 160 pixel images. In total, there are 16 panels every sample, eight of which define the context and the remaining eight define possible choices (see Figure Tp. Similar to RPMs, correct panel has be chosen that matches the implicit relational rules encoded the context panels. PGM includes meta-targets in the form of 12-bit feature vectors that denote relation, object and attribute types. The rules that underlie each sample are composed of relational rules, chosen from the set (Progression, AND, OR, XOR, Consistent Union). Figure Dp.c shows example of the Progression rule.
We trained ARNe the 1.2 10 training samples from the PGM dataset where early stopping terminated training after 45 epochs. After training. ARNe detected answer panels with an accuracy of 88. 18% and auxiliary data with 98.72%. Fi-score reached 0.9801. For comparison, in Table 2 results for WReN (Santoro et al., (017)) and VAE WReN (Hudson & Manning 2018)) baseline. These models significantly outperformed by ARNe. It is noteworthy that
included
5

---

Under review as a conference paper at ICLR 2020
Table 3: Meta-target prediction performance.
Meta-target
Accuracy [%]
Precision [%}
Recall [%]
F1- FI-Score [%]
Progression
96.53
90.52
AND
97.76
96.92
94.01
95.44
OR
98.15
98.53
93.99
96.21
XOR
97.59
99.07
91.23
94.99
Consistent Union
96.87
97.65
91.79
94.62
Shape
99.82
99.91
99.81
99.86
99.98
99.98
99.98
99.98
Size
99.93
99.89
99.77
99.83
99.95
99.91
99.98
99.94
Position
99.98
99.91
99.
99.95
Number
Color
98.13
99.01
96.80
97.89
implementation of WReN also outperformed (Santoro et al. 2017 by 2.1 ppt despite careful code validation (see Table and Appendix B).
In addition, we included comparison with the Attention and Composition (MAC) Network (Hud- son & Manning 2018). To the best of our knowledge this is the first time MAC is tested on the PGM dataset. To our surprise MAC did not perform significantly above chance level on this task. We also tested version of MAC that uses a WReN at the input stage, similar to ARNe. We call this augmented variant WReN-MAC. Our first results with this model suggest that also WReN-MAC does not reach the performance of ARNe. We ran tests
reduced PGM dataset that used only 20% of the full training set. WReN-MAC achieved 46.9% test accuracy, about 10% below that of ARNe on the same dataset size (see Figure We are rumning simulations with WReN-MAC on the full dataset but by the time of the submission of this paper these experiments were not concluded. Additional details the implementation of WReN-MAC
provided in Appendinx D
To gain additional insights into the learning behavior of ARNe we clustered the test dataset by the number of relational dependencies. ARNe showed good performance on samples with four (accuracy: 90.54%) and worst for samples with three (accuracy: 82, 74%) relational rules. Next, we evaluated the ability of ARNe predict the meta-targets. Table shows individual performances for each of the 12 meta-targets. ARNe achieved high accuracy in all cate- gories. Detection rates revealed an unconditional accuracy rate of above 90% consistently across all meta-target types.
all types, performing best
Furthermore, we evaluate the generalization abilities of ARNe by the provided samples the PGM dataset. Consistently with the scores on the neutral split, we find our method to outperform the wild relation network Tab.
80
test accuracy [96]
1.2
70
loss
0.8
60
ARNe (ours)
50
WReN
0.4
2
20
40
60
80
100
20
40
60
80
100
dataset ratio [%]]
dataset ratio
Figure 3: Sample efficiency: Test accuracy (a) and loss (b) after training different numbers
of samples (percent of original dataset size). Our implementation of WReN was used here (see Appendix B).

---

Under review as a conference paper at ICLR 2020
Table Generalisation results. No training results for WReN were available (Santoro al. 2018
Values in the training, validation and test columns characterise the model's accuracy.
Model
8
Training (%)
Validation [%]]
Test [%]
WReN
Santoro al. 2018
10
93.60
15.50
A RNe
99.43
98.93
17.76
shape
line
shape
line
PGM sample 15817
PGM sample 141081
Figure 4: Sum of selected activations of the last layer of the panel encoder CNN, that encoded predominantly for shapes (maps 14, 24, 26, 29, 31) and for lines (maps 1, 3, 4, 12). Matrices show complete sets of context panels and the correct choice panel highlighted in red.
4.1 SAMPLE EFFICIENCY
In order to evaluate the sample efficiency of ARNe, we trained the model subsequently with growing fractions of the training dataset. The validation and test sets were kept unchanged during the entire experiment. Prior to splitting of the training set, the dataset was shuffled and the random seed was kept fixed throughout all experiments. The results are presented in Figure 3 ARNe achieved higher test accuracy across all dataset ratios 200% Using split size of 35%, the accuracy reads 87.64%, already close to the value when trained on the full training set and 8.65 ppt better than the WReN model 100% of the training dataset. The progression of the loss shows similar behavior (Figure 3D).
4.2 SELF-ATTENTION FOR VISUAL REASONING
In order to gain insights into how the self-attention mechanism of ARNe works internally we visu- alize the activations of the panel encoder for different encodings of the meta-targets in Figure It is important to note that these features captures all relevant information for reasoning about relational rules between objects. Therefore, recognizing objects of specific types is an important sub-task for deciding which objects are relevant for a particular relational rule. Figure Asowrs shows two examples for the object type meta-tare weta-targets shape and line. For each object type, output channels of the panel en- coder CNN were selected that strongly responded. The eight choices and the corresponding correct context panel in the bottom right, are shown.
We found that, after training, many panel encoder CNN output channels significantly encoded acti- vation maps for single meta-target type. The activations in Figure 4 most saliently select regions within the input panels that are part of line or shapes objects.
4.3 ABLATION
experimented with various configurations ARNe. This involves dropout to reduce over-fitting, different learning rate, the inclusion of the delimiter token when assembling sequences and the weighting of meta-targets 3. Table5]eppice depicts the results of the subsequent and more detailed ablation studies of ARNe. Further values a substantial performance boost, while this is not the case when the full dataset used. This is likely due to the additional samples preventing the network from over-fitting. Similarly, the addition
listed in the Appendix A For small dataset dropout gives
7

---

Under review as a conference paper at ICLR 2020
Table 5: Comparison of different model configurations. The highlighted lines refer the best performing model of the given dataset ratio. Values in the training, validation and test columns characterize the model's accuracy, (Ir learning rate).
Dropout [%]
Irx10
Delimiter
B
Ratio [%]
Train [%]
Val. [%]
Test [ [%]
0.5
10
35
81.83
78.67
78.32
10
0,5
10
35
82.21
79.39
78.69
17
0.5
10
35
93.74
85.10
84.35
10
35
93.83
87.65
87.64
10
0.5
100
12.50
12.65
12.55
100
86.27
87.72
87.11
10
100
88.86
88.66
87.95
17
0.5
10
100
88.23
88.42
88.04
10
0.5
10
100
89.06
88.77
88.18
Foed-forward net instead of Encoder
32.02
35.87
35.06
Feed-forward net instead of Self-attention
38.79
45.12
44.56
of deliminiter tends improve performance for small datasets, thus it might act regularizer. Furthermore, we find that the method robust to a changed initial learning rate.
An intriguing insight is that ARNe does not converge when meta-targets provided (3 0). This indicates the relevance of the auxiliary signals which connect the panel choice with the underlying implicit relational rules during training (at no meta-targets were presented). Beyond the findings of Table 5] we investigated improvements of components of WReN as well as the addition of FiLM et al. (2018) layers but did not notice a performance improvement. However, by reducing the input image size from 160 160 to 80 x 80 pixels, accuracy decreased by about 15 ppt. This suggests that relevant small structures exist and are used by the model to solve the PGM.
4.4 RAVEN DATASET
For comparison, we also evaluated our model on the recently published RAVEN dataset Zhang et al. 2019). We found it achieve performance of 92.23% for 50,000 samples for each figure configu- ration. However, using Raven- i.e. using only 10,000 samples, the performance dropped to 19.67%.
CONCLUSION
In this work we introduced ARNe, a new deep learning model that combines features from the Wild Relation Network (WReN) and the Transformer network to discover patterns in progressive matrix panels using aspects of fluid intelligence. More precisely, ARNe builds upon the WReN and extends with the attention mechanism of the Transformer. which originates from language modelling. making it to our knowledge the first deep learning approach that uses self-attention specifically for abstract visual reasoning. The learning is driven by an auxiliary loss that gives hints about the underlying patterns of the progressive matrix panels. In an extensive experimental comparison we find that ARNe outperforms state-of-the the-art abstract reasoning methods on the PGM dataset by large margin. Moreover, the analysis shows that our model is substantially more sample-efficient than competing approaches.
Our experiments also including the first application of MAC to the PGM dataset. But the MAC did not seem to be well suited for this task, and also showed significantly worse performance compared to baseline, when it was augmented with WReN-based panel encoder. Additional experiments are needed to gain deeper understanding on why MAC appears to fail here. First insights into the inner workings of ARNe are given in Figure We show that the panel encoder CNN generates meaningful features that aid the learning goal. This suggests that the WReN and Transformer network parts of ARNe learn to efficiently cooperate for abstract reasoning.
8

---

Under review as a conference paper at ICLR 2020
Altogether, these results suggests that the self-attention mechanism be helpful in domains be- yond text processing. Future work involves modifying our method such that can be used on visual reasoning datasets which require different structure, e.g. VisualQA datasets which involve parsing an explicit question.
REFERENCES
Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. A neural probabilistic language model. Journal of Machine Learning Research (JMLR), 3, 2003.
Warren B Bilker, John Hansen, Colleen M Brensinger, Jan Richard, Raquel E Gur, and Ruben C Gur. Development of abbreviated nine-item forms of the raven's standard progressive matrices test. Assessment, 19(3):55-369, 2012.
DeepMind. Procedurally generated matrices (PGM).
https://github:
com/ http://deppnnnnabbtttttteeeaaaae Suhhar446t22550ffoaa222222222133
2017.
commit
Jacob Devlin, Ming-Wer Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: 1810::0810004805, 2018.
Christopher D. Manning Drew A. Hudson. Compostional attention networks for machine
reasoning. http:://ww...aaaooooscss/ataatas network 2018. commit 77000dlloo77722500222aa444424413
lan Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, Cambridge, England, 2016. http://w/e h/wp:////www. www.deeeeeeeeppooooooe org
Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. CoRR, (bs//410,,401, 2014.
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska- Barwińska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, et al. Hybrid computing using neural network with dynamic external memory. Nature, 538 (7626):471, 2016.
D. Hagemann, M. Amelang, G. Stemmler, F. Spinath, and D. Bartussek. Differentielle Psychologie und Persintichkeissforcchnng Kohlhammer Standards Psychologie. Kohlhammer Verlag, 2016.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pp. 328-339, Melbourne, Australia, July 2018. Association for Computational Linguistics. URL
https: www
http://wwwwww.....cc/// wwwweb. orq/ wwtp///aaaao//////e/oee/e.11
Drew Hudson and Christopher Manning. Compositional attention networks for machine rea- soning. In International Conference Learning Representations (ICLR), 2018.
Susanne M. Jaeggi, Martin Buschkuehl, John Jonides, and Walter J. Perrig. Improving fluid intel- ligence with training working memory. Proceedings of the National Academy of Sciences, 2008.
Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2901-2910, 2017.
Samira Ebrahimi Kahou, Adam Atkinson, Vincent Michalski, Ákos Kádár, Adam Trischler, and Yoshua Bengio. Figureqa: An annotated figure dataset for visual reasoning. CoRR, abs/1710.07300, 2017.

---

Under review as a conference paper at ICLR 2020
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai- Chang. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv: (e0i:::990.03557,, 2019.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (NIPS), pp. 1111- 111119, 2013.
Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. In "Pirty-Seconu AAAI Conference on Artificial Intelligence, 2018.
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In Conference of the North American Chapter of the ACL (NAACL). 2018.
Alec Radford, Jeffrey Wu, Dario Amodei, Daniela Amodei, Jack Clark, Miles Brundage, and Ilya Sutskever. Better language models and their implications. OpenAI, 2018b. URL https:/lopenai. www.bloogbbettr--lnguuggee mooels,, 2019.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint wxXv:1600.002500 2016.
Xuezhu Ren, Frank Goldhammer, Helfried Moosbrugger, and Karl Schweizer. How does attention relate to the ability-specific and position-specifii components of reasoning measured by apm? Learning and Individual Differences, 22(1):1-7, 2012.
Xuezhu Ren, Karl Schweizer, and Fen Xu. The sources of the relationship between sustained atten- tion and reasoning. Intelligence, 41(1)551-58, 2013.
Adam Santoro, David Raposo, David G Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Tim Lillicrap. A simple neural network module for relational reasoning. In Ad- vances in Neural Information Processing Systems 30, pp. 4967-4976. Curran Associates, Inc., 2017.
Adam Santoro, Felix Hill, David Barrett, Ari Morcos, and Timothy Lillicrap. Measuring abstract reasoning in neural networks. In International Conference on Machine Learning, pp. 4477-4486, 2018.
Karl Schweizer. The relationship of attention and intelligence. In Handbook of individual differences in cognition, pp. 247-262. Springer, 2010.
Lazar Stankov. Attention and intelligence. Journal of educational psychology, 75(4):471, 1983.
Xander Steenbrugge, Sam Leroux, Tim Verbelen, and Bart Dhoedt. Improving generaliza- tion for abstract reasoning tasks using disentangled feature representations. arXiv preprint 1.4.01811104784. 2018
Alane Suhr, Mike Lewis, James Yeh, and Yoav Artzi. A corpus of natural language for visual reasoning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume pp. 217-223, 2017.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merriënboer, Armand Joulin, and Tomas Mikolov. Towards ai-complete question answering: A set of prerequisite toy 2015.
tasks. arXiv preprint arXiv:
Caiming Xiong, Stephen Merity, and Richard Socher. Dynamic memory networks for visual and textual question answering. In International conference on machine learning. pp. 2397-2406, 2016.
10

---

Under review as a conference paper at ICLR 2020
Vinicius Zambaldi, David Raposo, Adam Santoro, Victor Bapst, Yujia Li, Igor Babuschkin, Karl Tuyls, David Reichert, Timothy Lillicrap, Edward Lockhart, et al. Deep reinforcement learning with relational inductive biases.
Chi Zhang, Feng Gao, Baoxiong Jia, Yixin Zhu, and Song- Cong-Chun&20 Zhu Raven: A dataset for relational and analogical visual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5317-5327, 2019.

---

Under review as a conference paper at ICLR 2020
APPENDIX
A DETAILS TO ARNE IMPLEMENTATION
The model was trained using a learning rate of 0.5 10-1, A heuristic learning rate scheduler was employed which is based on the official MAC implementation (Drew A. Hudson 2018). It triggers below a loss of 0.6 and utilizes a decay parameter of 0.75. Early stopping (Goodfellow et al.) 2016) was used which stopped training after three subsequent epochs of no improvements of the model's 's validation loss. For optimization, Adam (Kingma & Ba (014) was used. Parameters of ARNe's Transformer encoder are listed in Table We used our own Py Torch implementation the Transformer since at the beginning of this project the reference implementation" was not available.
Table 6: Model parameters of the Transformer encoder used in the conducted experiments.
Dropout Layer
Parop
dropattenn
Pttection
10
drop position
Parameter
value
dmodel
512
dy dg de
64
64
64
h
10
Mayers
dnidden
2056
Linear Layer
value
linear, linear, linear, fc
Immondess d,y dmoodel, dy
Amondel, dg. [h dk, dmodel)
Convolutional Layers
(dinputs cloutputs kernel, stride, padding)
possition
[dmodel, chidden 1, 1,0) ReLU [daidden dmodets 1, 1,
B DETAILS TO WREN IMPLEMENTATION
The WReN model was re-implemented in PyTorch described in (Santoro et al. 2018 and tested
on the neutral PGM dataset (DeepMind) 2017). The model was trained with , 10 only. A deviating batch size of 64 instead of 32 used. This yields accuracy rates of 78.49% 80.02% and
79.00% for training, validation and testing respectively. This implies an improvement of 2.82 ppt for validation and 2.1 ppt for testing compared to the WReN baseline model (Santoro et al. 2018). Figure 5 displays the progression of accuracy and loss during training, validation and testing.
C DETAILS TO MAC IMPLEMENTATION
The MAC network was implemented as reported in (Hudson & Manning 2018 using PyTorch. The implementation was verified on the CLEVR dataset. Loss and accuracies of our implementation
.2.
12

---

Under review as a conference paper at ICLR 2020
100
Accuracy [%6]
80
60
40
3
Loss
2
1
Epoch
Training Choices
Validation Choices
Test Choices
Training Auxillary
Validation Auxillary
Test Auxillary
Training Total
Validation Total
Test Total
Figure 5: Accuracy and Loss of the self implemented WReN (Santoro et al. 2018 model for 3 10 on the neural PGM dataset. The development of both metrics include the progression of the panel choices, additional auxiliary data which describes the setup of the PG-Matrices. In addition, the loss shows the overall sum.
Best value for validation regarding the accuracy is 97.89%...... 83%. The training, validation demanded computation effort of about half a week on GeForce GTX TITAN X graphics card.
of MAC on the CLEVER dataset can be seen in Figure 6
D DETAILS TO WREN-MAS IMPLEMENTATION
The WReN-MAC model uses the same encoding mechanism the WReN baseline model and includes 12 MAC-cells. In order to interface MAC-cell, additional convolutional layers were appended to the adapted WReN convolutional network. The model analogously sequentially aligned like WReN WReN-Transformer whereas during each pass one knowledge base which encodes one distinct choice panel is used. The question vector q computed in analogy to WReN where Se and the subsequently applied sums were used. The computation of the model's loss also respected auxiliary structure set data of the corresponding PGMs.
We were not able to train WReN-MAC successfully the PGM dataset. Early stopping finished learning after 33 epochs. The final accuracies and losses are 49.26% and 3.04, 46.97% and 3.20, 46.89% and 3.23 for training. validation and testing respectively. Both accuracy and loss showed marginal improvements throughout all epochs. Due the recurrent nature of MAC-cells, we used gra- dient clipping (Goodfellow al.) 2016). The learning rate scheduler was set but didn't diminished the the learning rate which was initially set to 1 10-4, Additional hyperparameter tuning did not show significant performance increase.
13

---

Under review as a conference paper at ICLR 2020
100
90
Accuracy [%]
80
70
60
50
1.0
0.8
0.6
Loss
0.2
10
15
Epoch
Training
Validation
Figure 6: Learning curves during training and validation our implementation of MAC on CLEVR dataset. Accuracy: Final learning rate values after 25 epochs are 97.83% and 99.11% for the validation and training set respectively. Loss: Calculated losses the implemented neural network with respect epochs. Final loss values after 25 epochs are 92.30 10 and 22.90 10 for the validation and training set respectively.
14

---

Under review as a conference paper at ICLR 2020
8 Context-Choice Pairs
Generate Pairs
Po5. Encoding
WReN CNN
Embedding
MLP90
Context
Choices
WReN CNN
KB CNN
Generator
8 Choice Knowledge Bases
Choices
Output Uniit
Compute Scores
MAC Cell
MAC Cell
Auxiliary
Cell 1
Cell
TTansformer-Encoder Network
8 Sequential Passes
Scores
Figure 7: WReN-MAC model. First, eight sequences of embeddings and go activations gener- ated analogously to WReN. To interface a MAC-cell properly, the knowledge base is required. A knowledge base generated by the WReN CNN plus additional layers to match MAC's required dimensions. Every sequence passes 12 recurrent MAC-cells and the output unit sequentially. The computation of scores equals the method incorporated in the WReN model.
15


Reviews:
[{"review_id": "SJlIFDe6KH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper describes a somewhat novel approach to abstract visual reasoning using transformers in the so-called \"Attention Relation Network\" (ARNe), which the authors show to improve on the \"Wild Relation Network\" (WReN). The Transformer is motivated by the role that attention may play in Human information processing - which sounds plausible, but the paper does not expand on this theme.\n\nThe paper is well written and makes an interesting contribution, but I feel the results are not quite yet ready for publication. The authors are writing that they are still working on baseline results on the full dataset, which would provide interesting comparisons, and some details on the implementation (number of parameters, etc) are missing - or maybe I missed them.\n\nThe learning curve in Figure 3 (sample efficiency, test accuracy) suggests that the ARNe training is not fully stable - why would the model deteriorate when going from ~40% of the training data to ~60%? Is the model potentially overfitting, and how does the size of the proposed model compare to the size of the baseline model(s)? It seems that the field is also moving towards the RAVEN dataset, which presents a more complex structure; it would be more convincing to present results on both datasets, to show that attention can indeed also improve results on more complex setups.\n\nThe text in the \"Acknowledgments\" section should be removed for the camera ready version!\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I did not assess the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "HkgWMFWhKr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work introduced an attention-based model to solve the RPM cognitive tasks. The model is based on the transformer network, which performs relational reasoning through its self-attention mechanisms.\n\nTechnical novelty:\nThe method seems to be a straightforward application of the transformer network to the PGM task. The technical novelty of the proposed approach is unclear. I’d love to hear what the authors have to say about the technical contributions of the proposed ARNe model in comparison to prior work.\n\nSupervision with meta-targets:\nIt also seems that the meta-targets are crucial for attaining a good performance with the ARNe model. According to Table 4, the model without meta-target training (beta=0) only achieved 12% accuracy in train/val/test sets. However, prior work [Santoro* et al. 2018] has demonstrated that even without training on meta-targets, WReN still achieves a performance of over 60% accuracy (Table 1). This result suggests that the proposed ARNe model does not work well when training with weaker supervision without meta-targets. The results could be a lot stronger if the authors show ARNe outperforms the prior work when beta is set to 0.\n\nAblation studies:\nThis model is only tested in the neutral PGM dataset. The evaluation would be strengthed with the generalization results of this model in different generalization regimes (see Table 1, Santoro* et al. 2018) and comparing its performance with prior works.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "BygwFWrJcr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work proposes a new architecture for abstract visual reasoning, based on Transformer-style soft attention and relation networks. The authors test their network on the PGM dataset, and demonstrate a non-trivial improvement over previously reported baselines. \n\nIn general, abstract reasoning is an important field of current study in neural network-based machine learning, as it is an area that has notoriously eluded these types of models historically. The paper is reasonably well put together, and I have no reason to question the various technical aspects of the work.\n\nUnfortunately, I think there are significant shortcomings. Firstly, the PGM dataset was designed to stress out-of-distribution generalization, and performance on the Neutral split was not proposed as a particularly interesting challenge on its own. This is because, as the name implies, abstract reasoning requires the ability to identify abstract conceptual features of the data and compose them in novel ways at test time, which is *not* a feature of the neutral split.  The authors are encouraged to run their model on these other generalization splits. \n\nSecond, there seems to be little value to the field overall for research involving minor architectural improvements for single datasets. If the authors believe in this method, they are encouraged to demonstrate its effectiveness on a wide variety of data types. On this note, I should add that the authors are incorrect to state that this is the first work to use self-attention for abstract reasoning (please see Zambaldi, 2018 for one example of many papers that have incorporated self-attention into convolutional architectures). \n\nSo to sum up, while this work broaches an interesting subject and is technically fine, it does not surpass the threshold for acceptance because it fails to demonstrate the usefulness of the method on the task at hand, as well as broad utility of the proposed method.\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work proposes a new architecture for abstract visual reasoning called "Attention Relation Network" (ARNe), based on Transformer-style soft attention and relation networks, which the authors show to improve on the "Wild Relation Network" (WReN). The authors test their network on the PGM dataset, and demonstrate a non-trivial improvement over previously reported baselines. 

The paper is well written and makes an interesting contribution, but the reviewers expressed some criticisms, including technical novelty, unfinished experiments (and lack of experimental details), and somewhat weak experimental results, which suggest that the proposed ARNe model does not work well when training with weaker supervision without meta-targets. Even though the authors addressed some concerns in their revised version (namely, they added new experiments in the extrapolation split of PGM and experiments on the new RAVEN dataset), I feel the paper is not yet ready for publication at ICLR. 

Decision: Reject