Adaptive recurrent vision performs zero-shot computation scaling to unseen difficulty levels
Authors: Vijay Veerabadran
Srinivas Ravishankar
Yuan Tang
Ritik Raina
Virginia R. de Sa
Venue: NeurIPS 2023 poster (2023)
Keywords: cognitive science
recurrent neural networks
adaptive computation time
visual reasoning
TL;DR: Convolutional RNNs equipped with a learnable halting mechanism adaptively scale computational resources conditional on input to allow for zero-shot generalization to novel difficulty levels of visual reasoning.

Abstract: Humans solving algorithmic (or) reasoning problems typically exhibit solution times that grow as a function of problem difficulty. 
Adaptive recurrent neural networks have been shown to exhibit this property for various language-processing tasks. However, little work has been performed to assess whether such adaptive computation can also enable vision models to extrapolate solutions beyond their training distribution's difficulty level, with prior work focusing on very simple tasks. In this study, we investigate a critical functional role of such adaptive processing using recurrent neural networks: to dynamically scale computational resources conditional on input requirements that allow for zero-shot generalization to novel difficulty levels not seen during training using two challenging visual reasoning tasks: PathFinder and Mazes. We combine convolutional recurrent neural networks (ConvRNNs) with a learnable halting mechanism based on Graves (2016). We explore various implementations of such adaptive ConvRNNs (AdRNNs) ranging from tying weights across layers to more sophisticated biologically inspired recurrent networks that possess lateral connections and gating. We show that 1) AdRNNs learn to dynamically halt processing early (or late) to solve easier (or harder) problems, 2) these RNNs zero-shot generalize to more difficult problem settings not shown during training by dynamically increasing the number of recurrent iterations at test time. Our study provides modeling evidence supporting the hypothesis that recurrent processing enables the functional advantage of adaptively allocating compute resources conditional on input requirements and hence allowing generalization to harder difficulty levels of a visual reasoning problem without training.

Adaptive recurrent vision performs zero-shot computation scaling to unseen difficulty levels
Vijay Veerabadran Srinivas Ravishankar"*
Yuan Tang I Ritik Raina Virginia R. de Sal.2
Department of Cognitive Science, Hahccoogu Data Science Institute, University of California, San Diego, La Jolla, CA 92093 vveerabalucsd. edu
Abstract
Humans solving algorithmic (or) reasoning problems typically exhibit solution times that grow function of problem difficulty. Adaptive recurrent neural networks have been shown to exhibit this property for various language-proces tasks. However, little work has been performed assess whether such adaptive computation can also enable vision models to extrapolate solutions beyond their training distribution's difficulty level, with prior work focusing on very simple tasks. In this study, we investigate critical functional role of such adaptive processing using recurrent neural networks: dynamically scale computational resources conditional on input requirements that allow for zero-shot generalization to novel difficulty levels not seen during training using two challenging visual reasoning tasks: PathFinder and Mazes. We combine convolutional recurrent neural networks (ConvRNNs) with learnable halting mechanism based on (Graves) 2016). We explore various implementations of such adaptive ConvRNNs (AdRNNs) rang- ing from tying weights across layers to more sophisticated biologically inspired recurrent networks that possess lateral connections and gating. We show that 1) AdRNNs learn to dynamically halt processing early (or late) to solve casier (or harder) problems, these RNNs zero-shot generalize to more difficult problem settings not shown during training by dynamically increasing the number of recur- rent iterations at test time. Our study provides modeling evidence supporting the hypothesis that recurrent processing enables the functional advantage of adaptively allocating compute resources conditional on input requirements and hence allowing generalization to harder difficulty levels of a visual reasoning problem without training.
Introduction
Recurrent Neural Networks (RNNs) have emerged as a powerful tool for solving machine reasoning tasks, demonstrating impressive performance across variety of tasks that require processing of sequential inputs. While recurrent processing valuable for time-varying inputs, can also useful for static inputs as recurrent networks have the ability to scale computation to varying difficulty levels by varying the number of recurrent iterations. This can be helpful when different problems from the same task family exhibit significant variations in complexity (consider mazes, for example), However, conventional RNNs typically struggle to automatically generalize across different difficulty levels, requiring retraining, fine-tuning, human intervention to pick the number of recurrent iterations needed. This limitation hampers their practical utility weal-world scenarios, since the span of difficulty levels seen during training is finite.
"Equal contribution
Work done while UC San Diego
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Human reasoning, in contrast, is characterized by highly adaptible use of computation arbitrarily more computation can be used for more difficult tasks. Despite the evidence for such scaling in human computing there is limited work in the literature analyzing Adaptive RNNs (AdRNNs) that, on a sample-by-sampe
basis, automatically decide when to stop. Graves introduced Adaptive Computation Time (ACT), mechanism to automatically stop computation, by learning to generate a scalar halting probability (Graves 2016). However, similar to subsequent work (Banino et al. 2021), it was evaluated either simple tasks such as parity checking, or language tasks. Few studies examine the training and evaluation of these AdRNNs on visual reasoning tasks, which pose unique challenges due to the redundant and high- high-dimensional nature of visual data. Furthermore, extrapolating to harder instances within-task has not been sufficiently studied, with Banino et al. (202I) evaluating such zero-shot performance only the simple parity task.
In this paper, we study the problem of computation scaling in recurrent vision RNNs, with an emphasis on zero-shot extrapolation to harder/larger problems within the same task. The ability to handle unseen difficulty levels without fine-tuning, or human intervention, enables more robust and adaptable computer vision systems, which are crucial for real-world applications. We investigate the effectiveness of AdRNNs on two challenging, publically available, visual reasoning tasks based on curve tracing and route segmentation, namely PathFinder (introduced by Linsley et al. (2018) and Mazes (introduced by Schwarzschild et al. (202)).
The contributions of this work are as follows:
We combine Convolutional RNNs with an adaptive computation method of Graves (0116) producing Adaptive ConvRNNS (AdRNNs) that are capable of learning a downstream task simultaneously while also learning to scale their computational steps as per input image/task requirements.
We introduce LocRNN, a high performing recurrent architecture inspired by prior computa- tional models of recurrence in biological vision.
We show that AdRNNs learn to dynamically halt processing early (or late) to solve easier (or harder) problems when the train- and test-difficulty levels are matched on complex visual reasoning problems inspired by stimuli used in prior cognitive science research.
During test time we introduce a previously unseen harder difficulty level. We evaluate AdRNNs on these new difficulty levels and show that they zero-shot generalize to more difficult problem settings not shown during training by dynamically increasing the number of recurrent iterations at test time well beyond the number of recurrent steps used during training.
2
Related Work
Our work is relevant to the visual routines literature introduced by U1lman (1984) and further reviewed claborately by [RoeIfsema et al. (2000). The core idea of visual routines that make it relevant to our studied question of task extrapolation is the flexible sequencing of elemental operations resulting in dynamic computational graph, which make RNNs a natural approach to solve such tasks.
Prior research has proposed mechanisms for RNNs that learn the amount of recurrent computational steps required. Following are two particularly relevant attempts prior literature in this area: Graves (2016) developed the Adaptive Computation Time (ACT) method to train RNNs Natural Language Processing tasks with a sigmoidal halting unit that determines when to halt recurrent processing, and [aanioo et al. (202T) extend upon this pioneering work by taking a probabilistic approach to halting and introducing a geometric neither of these studies applied adaptive computation to vision RNNs (that are significantly trickier to optimize). Also the prior studies did not attempt to examine computation scaling under zero-shot generalization to harder difficulty levels. Additionally, Eyzaguirre and Soto (2020) developed version of ACT applied to visual reasoning problems, however, their observations on the CLEVR dataset (in which the issues related to object diversity are highlighted by Kim al. (2018) do not explore generalization to new difficulty levels which is central to our results.
based specification of computational budget. However,
In terms of generalization from easy to hard vision problems, our work is relevant to (Schwarzschild et al. 202T) and (Bansal et al. 2022) where the authors evaluate the application of recurrent neural networks to generalize from easier to harder problems. Our work extends and differs from their

---

intriguing study of recurennce in task extrapolation in the following ways: 1) For recurrent networks used in their study, human intervention is required to specify the number of recurrent computational steps during the testing phase. 2) Their work explores sequential task extrapolation in general with abstract problems such as Prefix Sum and solving Chess Puzzles while our work extends it to particularly focus on extrapolation in visual task learning. 3) We present evaluation on the Pathfinder challenge (Linsley et al. (018)) a relatively more large-scale visual reasoning task shown to be very challenging (Tay et al. 2020), the design of which dates back to (Jolicoeur et al. 1986). 4) Schwarzschild et al. (2021 D and Bansal et al. (2022) implement only the most straight form of recurrence realized by weight-tying. L.e., they only evaluate ResNets with weight sharing across residual blocks (albeit with a novel training scheme in the follow up study (Bansal et al., 2022)). In addition to such recurrent ResNets, we present analyses with highly sophisticated recurrent architectures specialized for recurrent image processing. 5) We introduce LocRNN, a high performing recurrent architecture based prior computational models of cortical recurrence.
On the design of recurrent architectures, our work loosely related to (Eigen et al. 2013), (Pinheiro and Collobert 2014). (Veerabadran and de Sa 2020) and (Liao and Poggio (016) which discuss the role of weight sharing in feedforward networks to produce recurrent processing. We are interested, however, in designing new specialized recurrent architectures that play a role both in human and machine vision. While there exist more such recurrent architectures informed by neuroscience such as Linsley et al. 2); Nayebi et al. (2022). find these architectures to be quite difficult to interpret and unstable to train (based on in-diffiullty performance evaluation included in the Supplementary). Our proposed LocRNN architecture in comparison is elegant and easy-to-interpy
implementation of recurrence that is stable to train. Our work is also relevant to prior work on modeling speed- accuracy tradeoffs observed during visual perception (Spoerer al. 2020
Datasets
For evaluating the ability of various models in exhibiting task extrapolation, curate two challenging visual reasoning tasks, Mazes and PathFinder, with instances at multiple parametric difficulty levels. Both tasks involve the visual routines of marking and curve tracing (Ullman 1984). These datasets are inspired by prior visual psychophysics research where such tasks were used abundantly to estimate the cognitive and neural underpinnings of sequential visual processing, such as incremental grouping. structure and preference of lateral connections, etc. (Jolicoeur and Ingleton, [991; Ullman 1984 Li al. 2006 Roelfsema 2006). the following subsections we describe the specifics of our tasks.
3.1
PathFinder challenge - curve tracing
(a) PathFinder-14
(b) 1athFinder-18
(c) PathFinder-24
(6) 9x0 Maze
(b) 15x15 Maze
(c) Maze
Maze
F
Figure I: Representative examples from PathFinder and Mazes datasets. Left: (a) Positive PathFinder- 9 (b) Negative PathFinder-15 18 (c) Positive PathFinder-244 Right: (a) 9x9 mazes, (b) 15 15 mazes, (a) 19x 19 mazes, (a) 25 x25 mazes
Task description: In the PathFinder task introduced by Linsley et al. (2018). models are trained to identify whether two circular disks in an input stimulus form the two ends of locally connected path made up of small "segments". Each image consists of two main long connected paths Po and P made up of locally aligned segments well as shorter distractor paths. Each image also contains two circular disks which are placed at two of the 4 possible endpoints of P. and Pt. Images that contain a disk on both ends of the same path are classified positive, and those containing disk endpoints of different paths are classified as negative. Examples are shown in Figure] II
Difficulty levels: Pathfinder is designed at different difficulty levels parameterized by the length (number of segments) of the paths Po and P1 mentioned above. The easiest version uses paths that are 9 segments long (PathFinder-9) while the medium and hard versions contain paths that are
3

---

14 (PathFinder- 14) and 18 (PathFinder- 18) segments long respectively (see example images in the Appendix). This dataset consists of total of 800,000 RGB images at each difficulty level, each with a spatial resolution of 160 x 160 pixels. There are an equal number of positive and negative instances at each difficulty level. We use 700,000 images for training and 100,000 images as the test set from each difficulty level. We combined these datasets to create more challenging dataset with varying levels of difficulty, which we refer to as PathFinder-Minn
To evaluate the zero-shot difficulty extrapolation on PathFinder in Sec. 5.2 we generated 100,000 images each with contour lengths 21 and 24 respectively; we call these Path Finder-21 and PathFinder-24.
Since this is classification challenge, we use accuracy, i.e. % correct on test- iast-images as the evaluation metric rank model performance on PathFinder. Model architectures receive an input image and process it via stack of standard convolution/recurrnnt watpr//rrccureetooooouttttoooooi layers followed by a classification two-class readout. Since this a binary classification challenge with balanced classes, chance performance is 50% for random predictions.
Evaluation criteria:
3.2 Mazes challenge route segmentation
Task description:
Human beings are adept at solving mazes, a task that requires application of similar serial grouping operation like PathFinder in order to discover connectivity from a starting point to the final destination of the maze amidst blocking walls. For evaluating model performance on solving mazes of varying difficulty. we use the publicly available version of the Mazes challenge developed by [Schwarzschild segmentation problem where models take N x N images of square-shaped mazes as input with three channels (RGB) with the start position, end position, permissible regions and impermissible regions marked in the image. The output produced by models is a binary segmentation of the route discovered by the model from the start the end position.
al. (20TD) They implemented this Mazes challenge as binary
Difficulty levels:
The Mazes challenge has been designed at several difficulty levels, each difficulty level is parameterized by the size of the square grid that the maze is designed into. We use maze datasets of grid sizes 9x9 and 15x 15 for training. Each dataset consists of 50,000 training images and 10,000 test images that are guaranteed to not overlap. The spatial resolution of 9x9 maze images is 24 x 24 pixels and that of 15x 15 maze images is 36 x 36 pixels. Similar to PathFinder-Minn we combine 9x9 and 15 15 grid sizes form Mazes-Mixed. We also constructed larger mazes with grid sizes 19 X 19 (with spatial resolution 44 x 44 pixels) and 25 x 25 (with spatial resolution 56 x 56 pixels) to evaluate a model's s ability to extrapolate to difficulties not seen during training. As above with PathFinder's ' extrapolation evaluation, these novel difficulty mazes were only used during testing and were not in any form used in the training or hyperparameter optimization process.
Evaluation criteria: Mazes is a segmentation challenge and hence, one could potentially consider partially correct routes during evaluation (for example with average of per-pixel accuracy). However this is less strict than giving each image a single binary score reflecting if all pixels are labeled correctly. Evaluation criteria for mazes is hence the total % of test-set mazes completely accurately solved at given difficulty level.
4
Model architectures and training
4.1
Implementations of adaptive computations evaluated on task extrapolation
In this section, we describe our choice of recurrent architecture designs that we use to study the behavior of adaptive computation and task extrapolation in deep learning. We process images (denoted as X E ReA.w) in three stages that are common to all models we evaluate in this study. These three stages are as follows: (1) an input convolution layer (denoted as input(.)) that operates on the image directly.
bo = ReLU (nnuut(n))))))
The output from this stage ho E IRd.h. is fed as input to the following recurrent block in (2).
h, r(b 1, bo), / E (mann)
where r(.) the recurrent block. Arain is a training hyperparameter indicating the maximum number of timesteps for unrolling r(.) during training. This block consists of sequence of convolution
4

---

layers applied in an iterative manner for any arbitrary number of timesteps. This is the sub-part of our model that capable performing adaptive computations. (The specific architecture of these convolution blocks constitute different implementations of recurrent operations described further below.) (3) a readout layer containing a block of convolution and pooling operations that produce the desired output from our network.
Rt - rea)out(h:), E [mmin]
While we keep the imput and readout layer architectures the same for all models evaluate, their intermediate recurrent blocks have different architectures (corresponding to their respective recurrent cells). We explore three different implementations of recurrent computations which used in the second, recurrent block of models. Our first choice as the intermediate feature processing block consists of residual network with weight tying across all layers; refer to this model R-ResNet- 30. Next, we study the performance of the following specialized convolutional recurrent units from prior work: horizontal convolutional GRU (hConvGRU) and its stable variant (Linsley et al.] 2018 2020) and a convolutional Gated Recurrent Unit (ConvGRU) (Ballas et al. 2015) with LayerNone (ConvGRU does not converge in the absence of LayerNorm). Third, we design novel recurrent cell based on prior computational models of cortical recurrent processing (Li et al.) (006);; this model equips the biologically inspired design choices long- range lateral interactions, gating, and separate population of interneurons. This model is referred to as LocRNN and is described in the following Sec. 4.3 Importantly all models are matched for trainable parameters.
4.2
Combining ConvRNNs with Adaptive Computation Time (ACT)
The central theme of our work is to show that RNNs can flexibly adapt (or scale) their computation according to input requirements. We achieve this ability by combining ConvRNNs with adaptive computation mechanism based on Graves (0166) called Adaptive Computation Time (ACT). A difference between our work and ACT that visual reasoning task involves static inputs (i.e. sequence of length I) whereas Graves (0166) deals with variable-length sequences and learns adaptive processing of each token. Owing this difference, our halting mechanism is similar to ACT applied to I-token input sequence.
The key idea of ACT is to introduce a separate "halting mechanism" that learns control the number of recurrent computation steps dynamically, conditioned on each input example's processing. In addition producing the next recurrent state, an RNN equipped with ACT also produces scalar value called the "halting score" for each computation step. In addition to the next hidden state computed using Equation we generate scalar halting score pt each step using learnt convolution layer (halt_conv)) that is shared across timesteps:
[
Pt
The RNN treats the cumulative sum of the halting scores up to timestep t (P, described below) as quantity used to determine whether processing is terminated at that timestep.
P.
Pe
The RNN keeps track of the accumulated halting scores and checks if the cumulative sum of the halting scores at each step reaches predefined threshold (1 1 E). the threshold reached, the computation stops (i.e., Pt = 0 Vt > taalr) where the cumulative halting score threshold is reached at timestep fhai = minft: P. >= (1 - c)}- The adaptive hidden state is then computed as a weighted average of the hidden states up to Enatt- scaled by the halting scores at each time step. The readout is then applied to this final adaptive hidden state to produce the ACT task prediction, Pazi
foat
Daet
Pt h
8 ac reaaoou(hhac)
(6)
The abovementioned ConvRNN combined with ACT is trained to optimize both the downstream task loss (lly Facrlpp) and an auxiliary 'ponder cost' corresponding to the halting mechanism
5

---

that encourages models to use the fewest number of recurrent step to process an input example. Ponder cost is computed as the cumulative halting score until timestep taub 1. maximizing this term encourages (or minimizing its negative shown below) completing the task as quickly (with as few recurrent steps) as possible. The following final objective function optimized train the full model containing ConvRNNs with ACT where is a hyperparameter.
A= D
C
lly' -Pacll
P
t=1
4.3 Formulation of LocRNN
We note that prior work has explored the development of recurrent architectures tailored for vision.
Here, we introduce similar but highly expressive recurrent architecture designed based on computational model of iterative contour processing in primate vision from [i (1998). This model
is an ODE-based computational model of interactions between cortical columns in primate area V1 mediated by "lateral connections" local to a cortical area.
hg
LocRNN architecture
To next layer
1y activations showing contour tracing in PF-14
L,
Input
5-1
t-3
t=7
hg
Conv@RU architecture To next layer
S
L
S
Figure 2: Contrasting the architectures of LocRNN (left) and ConvGRU (bottom-right). ho is shown as magenta arrow and does not change across timesteps. As illustrated also this figure, LocRNN's internerron activations (S₂) are not passed to the next layer. ConvGRU the other hand uses single uniform neural population. (Top-right) shows visualization of LocRNN's Lt activations as they perform contour tracing solve input image from PathFinder- (PF-14) (visualization of all timesteps available in Supplementary Fig. 8).
By discretizing the continuous-foo an interpretable and powerful set of dynamics for "LocRNN". As in ConvGRU, the effective receptive field of LocRNN output neurons increases linearly with recurrent timesteps.
ODE dynamics of processing units from Li (1988) arrived at
Unlike ConvGRU, the LocRNN's hidden state is composed of two neural populations. L and S: the activity in these populations are referred to as Lt and S, timestep & respectively. These two populations are motivated by the z (excitatory) and (inhibitory) neurons in Li (1998). However we found that restricting the signs of their weights (to reflect exclusively excitatory and inhibitory connections) led to less stable behavior. On the other hand, retaining the interneuron property (local computation not projecting using a single uniform population. The following equations illustrate the working of ACT and how the readout is applied in LocRNN.
for downstream processing) of the S population performed better than
Pt (inax_pool(aalt_conn....
(8)
Fral
$ Jazi readout
Pt Lt
Initially, both Lo and So populations set to a tensor of zeros with the same shape as ho. 4d tensor shape (batch_sse size channels xhei theight/i xwidth).
L. and S update gates G and G. (same shape L and S) computed functions of the input and current hidden states - and St-1 using 1xl convolutions U..
G o(LN(UL ho) + LN Lt-1))
(10)

---

G = - * ho) + LN(Us Sc-1))
(11)
Each of the types of lateral connections are modeled by convolution kernels WI.L,, WT.-s. Ws-s, and Ws-L respectively of shape d x d x k x k where d is the dimensionality of the hidden state and k represents the kernel spatial size.
L (WL ho + WL-L L + Ws- St-1)
(12)
S, (Ws ho WL-S Lt-1 Ws-s St-1)
(13)
Once the long-range lateral influences are computed and stored in L, and St, these are mixed with the previous hidden states using the gates computed in Eq. 10 and These hidden states are then passed on subsequent recurrent iterations where even longer-range interactions occur (as time increases).
Lt - <(LNGG) @ Ln + (1- @ Lt-1))
(14)
S. = <[LN + St-1))
(15)
In the above equations, LN() stands for Layer Normalization (Ba et al. 2016), and the nonlinearities and are both set to ReLU. One of the differences between LocRNN and ConvGRU (which we also evaluate) is the presence of the interneuron population in the former.
Results
5.1
Adaptive RNNs scale their computation as function of input difficulty
In this section, we report our observations from training AdRNNs on a mixture of difficulty levels from PathFinder and Mazes & evaluating them on held-out set of images from the same mixture of difficulty levels of PathFinder and Mazes. We created one training dataset for each of these challenges by combining input samples from multiple "easy" difficulty levels. Combining input samples from multiple difficulty levels increases the diversity of computational requirements during training and helps in developing models that do not degenerate to using the same computational steps for all input samples.
As described in Section 3 we created a PathFinder training set of images and corresponding labels by sampling an equal number of images from three difficulty levels: PathFinder-9, PathFinder- 14, and PathFinder-Is 18. Similarly, we created Mazes training set of images and ground-truth segmentation labels by sampling an equal number of images from two difficulty levels: 9x9 mazes and 15x15 mazes.
On held-out test sets that matched the difficulty level of the above-described training data, we tested whether models were able to scale their recurrent computational steps as a function of the input difficulty level. We show the results from this evaluation in Table. [ We observed that both the sophisticated AdRNNs tested (ConyGRU and LocRNN) were able to generalize to the held-out set. We also observed that variants of horizontal GRU (Linsley al., 2018) and Linsley et al. (2020) models generalized to the (within-difffcull held-out set without using ACT. The simpler recurrent network without gating, weight-tied R- R00 was unable to learn on PathFinder and on Mazes highlighting the importance of specialized operations such as gating and backpropagation through time. The presence of skip connections between R- Ress recurrent blocks could still not match the expressivity of the specialized RNNs.
If ACT is working expected, we must observe that AdRNNs that learn the task dynamically use less compute to solve easy examples and more compute (more recurrent iterations) to solve harder examples. To check for this trend, we analyzed the number of steps chosen by the model before halting for each example from the validation sets of PathFinder and Mazes; with varying contour lengths and maze sizes respectively. These results are shown by the cool colors in Figure 3 for PathFinder and in the Supplementary for Mazes. As is clearly observable from the trend in these Figures, examples that we consider as harder (longer contours in PathFinder and larger mazes) are assigned a higher number of recurrent computation steps by ACT than easy examples. Hence we show that AdRNNs obtained by combining ConvRNNs with ACT training are capable of learning both PathFinder and Mazes in addition to learning to adapt their recurrent computational steps as a function of input example difficulty. To the best of our knowledge, we are the first to show the above result for visual tasks inspired by stimuli used in prior cognitive science research.
7

---

PathFinder-Mixed (%)
Mazes-Mixed (%)
ResNet-30
50.41
0.0
R-ResNet-30 (ACT)
49.37
0.0
Linsley et (2020)
50.0
78.33
hConyGRU Lngleencce irain
89.66
99.69
AConvGRU (stable halting)
89.65
99.54
ConvGRU (ACT)
95.26
98.4
LocRNN (ACT) (ours)
97.13
98.4
Table 1: Accuracies (%) 1 of models on the two visual reasoning tasks. Chance performance is 50% for pathfinder and 0% for Mazes.
Distribution of halting steps: LoCRNN
Distribution of halting steps: ConvGRU
100000
PF9
5000
PP9
80000
PF18
PF18
Frequeacy
PF21
Frequracy
60000
PF21
60000
40000
40000
20000
20000
0
0
9
10
3
4
Halting step
Halting step
Figure 3: Distributions of halting steps across samples each validation set PathFinder. Compu- tation appears to scale to match difficulties of the datasets for both LocRNN (left) and ConvGRU (right) models. 9-1ength contours typically halt after 4-6 steps, while 24-1ength contours can take up to steps. The red bars show the distribution in the extrapolation datasets.
Perfrrunes 27-14 27-18 LowaMted)
Perfernanes Pabifatee
Perirmanc beld Puidituee3
LadNN
TEC
Lockbb
Lockdt
Inee-30
Roees 22
m5 800 HimeEee
8NN 80Nilemin
Figure 4: Vanilla RNNs (solid lines) and AdRNNs (dashed) trained on 12 iterations (a) within- difficulty (b) extrapolating to PathFinder-21 (c) extrapolating to PathFinder-24
5.2 Adaptive RNNs generalize to novel difficulty levels by scaling their computation
In typical circumstances where RNNs are expected to solve instances that are more difficult, human intervention in the form of specification of the number of recurrent iterations is commonplace (Schwarzschild et al. 2022). This approach is both laborious and expensive to perform as requires training and/or evaluation Additionally, this process needs to be repeated at every occasion of new incoming test data that potentially of increased difficulty. In the previous section, we showed the ability of AdRNNs in learning to solve PathFinder and Mazes while also demonstrating that they assign computation function of input difficulty on test-set with matched task difficulty level as the training set. Here we are interested in testing whether AdRNNs are able to extrapolate (by using more recurrent computational steps compared training) complex reasoning tasks. We constructed two additional datasets each in the PathFinder and Mazes families to test these adaptive models' ability to extrapolate, as mentioned in Section 3 While AdRNNs are trained on the training difficulty levels for a maximum of leain iterations, we evaluated them on these new (more difficult) datasets at inference with maximum of Finference 2 Imain recurrent iterations where we refer to AdRNNs as operating in their extrapolation phase. We report the results from this evaluation in Table. 2 and in Fig. 4
test with various settings of the number of recurrent timesteps.

---

PathFinder-21
PathFinder-24 (%)
Mazes-19 19 (%)
Mazes-25 (%)
(%)
ResNet-30
50.0
50.0
R-ResNect300 30 (ACT)
50.0
50.0
0.
Linsley et al.
2020
50.0
50.0
2.93
0.01
GConNGRU Liefernnce
Amoin
64.21
58.35
16.2 4: 2.66
5.29 1: 0.26
hConvGRU (stable halting)
50.0
50.0
50.26 + 6.04
21.36 =:: 2.82
ConvGRU (ACT)
82.63 + 4.84
74.14 + 6.52
75.1 11.96
46.93 = 4.2
LocRNN (ACT) (ours)
92.89 = 0.9
85.81 + 5.57
86.83 + 2.94
49.99 = 4.48
Table 2: Accuracies (Mean % = SEM) f on extrapolation datasets. Chance performance is 50% for pathfinder and 0% for Mazes.
"hConvGRU training converged only on 1 out of 3 seeds (which were chosen randomly for all models here) on PathFinder, corresponding results above show this converged model's performance.
Adaptive RNNs generalize to novel difficulty levels by scaling their computation Our evaluation of AdRNNs trained with ACT on novel harder difficulty levels shows that as expected, AdRNNs have learned to use the optimal number of recurrent computational steps required for achieving strong generalization to the novel difficulty levels on a per-image level. On PathFinder, as seen in Figure most instances from PathFinder-21 and PathFinder-24 take 9 steps in LocRNN and up to 7 steps ConvGRU, higher than the number of steps used for easy training difficulty examples. In Fig 5 we visualize the relationship between mazes' difficulty and the number of recurrent iterations used by ACT (on per-instance level). We observe that maze difficulty (length of the ground-truth route in pixels) and the number of ACT iterations are strongly positively correlated. Notably, AdRNNs trained with ACT choose to make thair during inference on longer mazes greater than halr used on the shorter training mazes.
100
30-
80
step
Halting step
60-
10-
LocRNN (19x19 muccss
X ConvCiRU (19x19maess) mazcs)
LocRNN muzes)
ConvORU (25x25 mazes)
200
40C
600
500
1000
200
400
san
1000
Moue solution length
bength
Figure 5: Relationship between halting step and difficulty level of Mazes for the extrapolation evaluation. Here we visualize the relationship between the halting step and solution length of mazes per-instance level. We note a strong positive correlation between the difficulty level (length of maze solution segmentation) and halting step used by LocRNN (left) and COnvGRU (right) AdRNNs.
5.3 AdRNNs outperform halting in hConvGRU based on stability of hidden-state dynamics
Using stability in hidden-state space as an alternative way to perform halting during extrapolation, we evaluate hConvGRU on extrapolation. For this method, we halted processing in hConvGRU when the mean absolute difference between two subsequent states (l - h--1ll) reduced to less than a tenth of the difference between the first two states (heuristic for stability). This method referred as hConvGRU (stable halting) in Tables [ and shows differences in generalization across our datasets. AdRNNs perform better than hCony (stable halting) both datasets; especially, stable halting completely fails to show generalization to PathFinder-22 and -24. We hypothesize this method work suboptimally for the following reasons:
Stability-based halting requires defining hand-engineered heuristic on how much change in the output considered small enough halt. typical segmentation tasks like Mazes, the network output for the initial few timesteps is highly stable in making nonsensical

---

predictions (predicting all pixels as the negative class) and thus, heuristics need to identify an inflection point in the output trajectory where meaningful predictions start emerge and stabilize. In the absence of ground truth information, one cannot pick a heuristic that generalizes to unseen data.
Learnable halting makes less assumptions about the hidden state's properties, and hence doesn't enforce hard constraints such as stability to be satisfied by training. Some, but not all RNNs, have stable hidden states wherein the network response stops changing after reaching an attractor. Linsley et al. (2020) argue that RNNs that are expressive have an intrinsic inability to learn stable hidden states. For RNNs to be stable, their hidden state transformation needs to model contractive mapping (Miller and Hardt [018:] Pascanu et al. 2013). That is, the recurrent transition function F satisfies the following inequality: [[ F(h--))llp < A11h; - h4-111p- RNNS with stable hidden states that satisfy the above inequality are quite difficult to train on challenging problems in practice. Even when stable models perform comparably to unstable models as in (Miller and Hardt [2018), the authors show unstable models' advantages such as performance improvements in the short-time horizon and lesser vanishing gradient issues.
6 Conclusion
The advantage of using recurrent networks for processing static inputs adaptively, particularly in order to zero-shot generalize to new difficulty levels is understudied. In this work, we show that deep convolutional networks with intermediate recurrent blocks (ConvRNNs operating image features) can be combined with an adaptive computation technique to learn to dynamically process different input examples based on per-instance difficulty level. We combine ConvRNNS with a learnable halting mechanism that is based on Graves (201)) to produce AdRNNs. We evaluated diverse implementations of recurrence in increasing level of sophistication/compleri weight-tying, ConvGRU and hConvGRU with gating, and LocRNN with two separate populations of horizontally connected units (one of which are interneurons) and gating on two challenging visual reasoning problems, PathFinder and Mazes, which are generated at various levels of difficulty. First, we showed that only the specialized RNNs (hConvGRU) and AdRNNs (LocRNN and ConVGRU) are capable of learning PathFinder and Mazes. Second, these AdRNNs trained with ACT are learning to dynamically use less (or more) recurrent computational steps for easy (or hard) PathFinder and Maze problems respectively as discussed in Section. 5.1 More interestingly when the difficulty level of the test set was increased relative to training difficulty levels, AdRNNs generalized to these harder instances in a zero-shot manner by allocating more recurrent computation than was ever used during training. They also outperform stability heuristics-based adaptive recurrent networks. Our work empirically shows this hypothesized advantage of using adaptive recurrent processing static-image tasks for the first time to the best of our knowledge.
R-ResNet-30 with
In addition to adaptively scaling computation according to the need, the AdRNNs can also solve the problems more efficiently, choosing to stop at earlier times than the adaapiity trained RNNs (compare halting times in Fig 3 with performance curves in Fig 4 with human arbitrarily chosen training iterations. Thus, just as computation can be scaled to unseen problem difficulties without human intervention, the training iterations required can be (better) discovered automatically.
Limitations and Future Work
The current work has been only applied to static images, but recurrent networks are able to process time-varying input; future work will explore models on video input. In addition, there appears to be a benefit of the LocRNN model over ConvGRU that deserves further study to determine the mechanism behind the benefit of the interneuron population.
For future work, we would like to further explore the benefits of training only on very small problem instances with the ACT mechanism automatically allowing for harder instances. This could greatly reduce latency and energy consumption both during training and inference, an increasingly relevant concern as the size, memory footprint, training time and inference latency of models grow drastically. Finally, we wish to explore how this approach might link to the human developmental literature, similar to carlier work by Elman (1993).
10

---

8 Acknowledgments & Funding disclosure
We thank Garrison W. Cottrell, Michael C. Mozer, Michael L. Iuzzolino, Pradeep Shenoy and Saurabh Singh for their feedback and insights on this work. This work is funded by: (1) NSF CRCNS Award No: 2208362, (2) Sony Faculty Innovation Award, and (3) Kavli Symposium Inspired Proposal and support from Social Sciences Computing Facility and Department of Cognitive Science at UC San Diego
References
J. L. Ba, R. Kiros, and G. E. Hinton. Layer normalization. arXiv preprint arXiv:1607.06450 2016.
N. Ballas, I.. Yao, C. Pal, and A. Courville. Delving deeper into convolutional networks for learning video representations. arXiv preprint 15 1eee:1111106333 1.06432, 2015.
A. Banino, J. Balaguer, and C. Blundell. arXiv:2107.05407, 2021.
Pondernet: Learning to ponder. arXiv preprint
A. Bansal, A. Schwarzschild, E. Borgnia, Z. Emam, F. Huang, M. Goldblum, and T. Goldstein. End- to-end algorithm synthesis with recurrent networks: Logical extrapolation without overthinking. arXiv preprini 222:v:2002.05826, 2022.
D. Eigen, J. Rolfe, R. Fergus, and Y. LeCun. Understanding deep architectures using recursive convolutional network. arXiv preprint 1rXiv:1312.1847, 2013.
J. L. Elman. Learning and development neural networks: The importance of starting small. Cognition, 48(1):71-99, 1993.
C. Eyzaguirre and A. Soto. Differentiable adaptive computation time for visual reasoning. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 12817-12825, 2020.
arXiv preprint A. Graves. Adaptive computation time for recurrent neural networks. 08800866 2016.
M. Iuzzolino, M. C. Mozer, and S. Bengio. Improving anytime prediction with parallel cascaded networks and remporal-differeme loss. Advances in Neural Information Processing Systems, 34: 27631-27644, 2021.
P. Jolicoeur and M. Ingleton. Size invariance in curve tracing. Memory & Cognition, 19(1):21-36. 1991.
Jolicoeur, S. Ullman, and M. Mackay. Curve tracing: possible basic operation in the perception of spatial relations. Memory & Cognition, 14(2):129-144) 1986.
J. Kim, M. Ricci, and T. Serre. Not-so-clevr: learning same same-diffene different relations strains feedforward neural networks. Interface focus, 8(4):2018001, 2018.
W. Li. V. Piech, and C. D. Gilbert. Contour saliency in primary visual cortex. Neuron, 5066)9951-96. 2006.
Z. Li. A neural model of contour integration in the primary visual cortex. Neural computation, 10(4): 903-90 0209 1998.
Q. Liao and T. Poggio. Bridging the gaps between residual learning, recurrent neural networks and visual cortex. arXiv preprint arXiv:/604 1604.03640, 2016.
D. Linsley, J. Kim, V. Veerabadran, C. Windolf, and T. Serre. Learning long-range spatial dependen- cies with horizontal gated recurrent units. In Advances Neural Information Processing Systems, pages 162,164,70 2018. 8.
D. Linsley, A. Karkada Ashok, L. N. Govindarajan, R. Liu, and T. Serre. Stable and expressive recurrent vision models. Advances in Neural Information Processing Systems, 13:10456110467, 2020.

---

J. Miller and M. Hardt. Stable recurrent models. arXiv preprint 1805 2018.
A. Nayebi, J. Sagastuy-Brena, D. M. Bear, K. Kar, J. Kubilius, S. Ganguli, D. Sussillo, J. J. DiCarlo, and D. L. Yamins. Recurrent connections the primate ventral visual stream mediate a tradeoff between task performance and network size during core object recognition. bioRxiv, pages 3021-02, 2022.
R. Pascanu, T. Mikolov, and Bengio. On the difficulty of training recurrent neural networks. In International conference machine learning, pages 1310-1318. Pmir, 2013.
A. Paszke, S. Gross, S. Chintala, G. Chanan, Yang. Vitioo, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in pytorch. 2017.
P. Pinheiro and R. Collobert. Recurrent convolutional neural networks for scene labeling. In International conference on machine learning, pages 82-90. PMIR, 2014.
R. Roelfsema. CORTICAL ALGORITHMS FOR PERCEPTUAL GROUPING. Annual Re- view of Neuroscience, 29(1):203-227, 2006. ISSN 0147-006X. doi: 10.1 146/amnurev.meure 29.05 1605 112939. URL httt:://ww annualreviews org/ /oi/10.i110011///// 1146/ annurav..v.... neuro 29. 00160516655..00 112939
R. Roelfsema, A. Lamme, and H. Spekreijse. The implementation of visual routines. Vision research, 40(10-12): 12:00122)2:18--111,, 2000.
A. Schwarzschild, E. Borgnia, A. Gupta, F. Huang, U. Vishkin, M. Goldblum, and T. Goldstein. Can you learn an algorithm? generalizing from easy hard problems with recurrent networks. Advances in Neural Information Processing Systems, 34:6695-6700 2021.
A. Schwarzschild, A. Gupta, M. Goldblum, and T. Goldstein. The uncanny similarity of recurrence and depth. 10th International Conference on Learning Representations, ICLR 2022, 2022.
C. Spoerer, T. C. Kietzmann, J. Mehrer, I. Charest, and N. Kriegeskorte. Recurrent neural networks can explain flexible trading of speed and accuracy in biological vision. PLoS computational biology, 16080): 2020.
Y. Tay, M. Dehghani, S. Abnar, Y. Shen, D. Bahri, P. Pham, J. Rao, L. Yang, S. Ruder, and D. Metzler. Long range arena: benchmark for efficient transformers. arXiv preprint arXiv:201 1.04006, 2020.
S. Ullman. Visual routines. Cognition, 18(1-3):97-159, 12 1984. ISSN 00100277. doi: 10.1016/ 0010-027784499022344.
V. Veerabadran and V. R. de Sa. Learning compact generalizable neural representations supporting perceptual grouping. arXiv preprint arXiv:2006.17711 11716, 2020.
12

---

Supplementary
A Training and implementation details
All architectures evaluated within task matched in terms of the number of parameters. The input convolutional layer's kernel size is 7 x 7. Number of channels used by the model remains unchanged across layers and is determined per-model for matching overall number of trainable parameters across models. For PathFinder, we fix the number of channels to be 32 for LocRNN, 21 for hConvGRU and ConvGRU, and 64 for ResNet-30 with filter size of 9x9 in the intermediate recurrent layers. For Mazes, we fix the number of kernels (d) to be 128 for LocRNN, ConvGRU, hConvGRU, and 100 for ResNet-30 R-ResNet-30 and the kernel size fixed be 5x5. Our readout layers for classification (PathFinder) contain a global average pooling layer followed by fully-connected layer with output dimensionality of producing the classification logit. We apply binary cross-entropy loss on the logit to train models on PathFinder. On Mazes, we use x convolution with output channel produce binary segmentation map. We use pixel-wise binary cross-entropy to train models on Mazes. These readout layers are used uniformly for all architectures evaluated.
On Mazes training minibatch size is set to 64 images (and inference batch size of 50 images) and learning rate schedule starting with warmup followed by step learning rate decay as indicated in Schwarzschild et al. (2021) for 50 total epochs of training. On PathFinder, we set the training minibatch size 256 images and constant learning rate of le-4 for all models for a total of 20 epochs of training. All models were trained on NVIDIA RTX A6000 GPUs and implemented using PyTorch. (Paszke (Pasze et al. 2017).
B Instability of other baseline ConvRNNs
ConvRNN training is often faced with instability issues that lead to sensitivity with respect to random seeds or lack of convergence of models on downstream tasks. We tested suite of ConvRNNs previously introduced that are similar to LocRNN on three difficulty levels of PathFinder (in-difficulty evaluation, i.e., training and testing each difficulty level independently). This evaluation high- lighted the above issue especially on the difficult levels of PathFinder where LocRNN was the only model which could converge to stable solutions across different random seeds unlike the other networks which performed at chance as shown below in Fig. 6
PathFinder-9 9
PuthFinder-14
MuthFinder-18
70
25
LocRNN
ACUGGRR
CORNe6-5
Con-RNNN
Figure 6: Performance of various ConvRNN models on PathFinder-9, PathFinder-14, and PathFinder- 18.
C Input and output format for PathFinder and Mazes
Each example maze is an x RGB matrix, with colored squares indicating the start (green) and end (red) positions in the maze. The output a binary matrix of size n x n with the segmented path indicating the maze solution. An example is shown in Figure (top).
Fach PathFinder example an n binary matrix as shown Figure (bottom). The output for one sample is pair of probabilities denoting which class the sample belongs to (negative, meaning
13

---

the disks are at the end of disconnected paths, or positive, meaning the disks are connected through the contour).
Model
Inputt image Suarge mazzn)
(Maze solution aegmentation)
Input Images
Modell output (Binary clessificaioo scorea)
[0.82, 0.18]]
Figure 7: (Top) Example images from 11x11 Mazes processed by a model to produce the solution segmentation prediction. (Bottom) Example input images from PathFinder-16 14 processed by classifier to produce binary classification output.
Figure 8: State activations L, of LocRNN for PathFinner-14 14 example, clearly displaying the contour integration strategy used by LocRNN. The activation maps suggest that the contours are integrated from both endpoints, and decision is taken based on whether they meet or not.
14


Reviews:
[{"review_id": "Mi79xPvLku", "reviewer": "Reviewer_T9g1", "summary": "Authors show that using recurrent networks for adaptively processing static inputs for a variable number of iterations allows for zero-shot generalization to more difficult problems by simply unrolling the model predictions for more time steps at inference. Authors propose LocRNN and show model performance on the pathfinder and maze datasets, highlighting that their model uses a greater number of iterations for harder problems and can generalize to harder problems than those seen during training. ", "questions": "Intermediate Predictions. How do we know that the iteration that the model stops at is optimal? Can we evaluate the model prediction at intermediate steps and evaluate the model prediction beyond when it naturally stops?\nArchitecture Diagram (Sec. 4.3). Providing an architecture diagram for LocRNN would help improve the clarity of Section 4.3. Please add this to the appendix. ", "limitations": "Authors highlight that their work explore this problem of adaptive compute on static images and will work on videos in future work. However, I would encourage authors to explore more complex problems within static image domains (e.g. classification, detection, segmentation of natural images instead of toy-like environments). ", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "Clarity of Writing. This was by far my favorite paper to read of my reviewing batch because of the clarity of writing. I was able to clearly understand the premise and motivation and follow along with the experiment setup and evaluation.  The insight that recurrent networks can be used to scale up or down the amount of compute depending on the difficulty of the problem is well articulated.\nCompelling Results. Authors show that LocRNN beats competitive baselines on two standardized benchmarks. Impressively, Table 2 shows that LocRNN beats other baselines on extrapolation datasets by a wide margin.\n", "weaknesses": "Motivation. Although I agree with the premise of the paper, I want to push back on using the biological inspiration of variable compute for neural networks. Although humans naturally can scale to harder and easier problems, state-of-the-art vision networks do not do this. Although authors cite efficiency as a reason for favoring RNNs for variable compute problems, no experimental evidence is provided.\n\nLimited Baselines. Similarly, although the focus of this paper is on adaptive compute, two strong baselines would be LocRNN trained with only one computational step and LocRNN trained with N computational steps (where N is some fixed number of maximum steps allowed). Given the significant engineering that went into the network architecture of LocRNN, the ResNet-30 baseline does not seem convincing to show the necessity of adaptive compute. \n\nDatasets. Although authors evaluate on two standard benchmarks, I would argue that these toy datasets do not represent real tasks. It would be more convincing to compete LocRNN against a state-of-the-art model on a more traditional computer vision benchmark (e.g. ImageNet, COCO, etc.) One place I think this approach could shine is in object detection. Given an image with a single near-field object, we should not need to spend a lot of compute. Similarly, given an image with many far-field small objects, we should likely spend more compute.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. Our experiments show that RNNs which scale computation are the only networks which are able to generalize to more challenging test samples by using more recurrent iterations at inference. We hypothesize that the vast levels of compute employed by SOTA networks today is thus driven by the long tail of the most difficult examples encountered during training and may be entirely unnecessary for the majority of the distribution. In a KDD23 plenary talk  (8/9/23) , Eric Horvitz  (CSO Microsoft) showed a plot of accuracy vs compute and showed that the last 1% accuracy of GPT-4 required 80% of the computational cost (through longer prompts).  Given the very real negative impact these massive computational systems have on the environment, there is strong motivation to reduce computation when possible. We have shown experimentally that with LocRNN and convGRU we can both reduce computation and increase accuracy because we’re not constrained to a one-size-fits-all level of compute like SOTA architectures.\n\n2. While we agree with the reviewer that the end-goal is to inspect the performance of these systems in a naturalistic task, we intentionally conducted this study using tasks in which difficulty could be experimentally manipulated to study the generalization of different architectures in complexities beyond training. This is a lot more difficult in a dataset like ImageNet, in which organizing the examples by complexity is a non-trivial endeavor. We intend to pursue this direction in future work, but believe the chosen task represents an important middle ground between the simplistic tasks in prior literature and the end-goal of naturalistic images.\n"}, {"review_id": "aZHOeQxnvx", "reviewer": "Reviewer_ikdz", "summary": "The primary subject of this paper is fostering computational efficiency in RNNs solving vision tasks by flexibly adapting the number of computing steps depending on the difficulty of the input. This is achieved through the addition of ACT (Adaptive Computation Time introduced by Graves, 2016) to recurrent vision models. The paper evaluates 4 models (3 with recurrence) on 2 visual reasoning tasks popular in the literature for studying RNN behavior (PathFinder and Mazes). One model is newly introduced here and given the name LocRNN. The key result put forward in the paper is that ACT-trained RNNs will indeed adaptively carry out more time steps before halting computation for difficult inputs (longer paths or larger mazes) than for easy inputs. Finally, another finding is that ACT-trained RNNs generalize to more difficult task conditions than those encountered during training.", "questions": "Please refer to the weaknesses section above.", "limitations": "Please refer to the weaknesses section above.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper touches upon a significant and topical challenge in deep learning: how to avoid ‘wasting’ computations on easy inputs while still solving the hard ones to achieve computational efficiency. Efforts in that direction are valuable from an ecological perspective, among other perspectives.\n\nAlthough the work doesn’t address this directly, mapping out which inputs are hard versus easy for a model can help acquire insights into the strategies a model has learned to solve a (visual) task. It is also of relevance for cognitive computational neuroscience and human-modal comparisons.\n\nThe paper reads very smoothly. The questions are formulated clearly, and so are the methods. It is overall well written.", "weaknesses": "It is good to see the prediction confirmed. Still, the finding that ACT will flexibly halt computation early for easier inputs does not tremendously advance the field, considering it’s what Graves (2016) already reported. With an ACT-like method (“DACT”) Eyzaguirre and Soto (2020) have already found this in visual reasoning tasks too. It’s worth a citation.\n\nSimilarly, there have been other thorough demonstrations that vision RNNs can generalize to more difficult task conditions, even on the very same Pathfinder task. I encourage the authors to check out and cite Linsley et al. (2020) \"Stable and expressive recurrent vision models\". In particular, Fig. 3 in this current paper is explored in detail and analyzed in the Linsley (2020) article. \n\nThe introduction of LocRNN then becomes the most novel aspect of the work, but there is not a lot of in-depth discussion in the main text about how it differs from prior models informed by neuroscience. At the very least, one would require a thorough comparison between the LocRNN and the model presented in Linsley et al. (2018) regarding their performance metrics and architectural similarities. \n\nFinally, can the authors comment on whether this approach would work in other scenarios of visual difficulty apart from simply spatial extent? The notion of visual complexity involves a host of other factors which a non-hierarchical model would fail to account for, and which needs discussion. \n\nTypos and minor fixes:\n\nSome of the citations appear to be in the wrong format (e.g., L40)\n\nL. 322: Fig.4 → Fig. 3?\n\nFig. 3: in a revised version, probably good to discuss these results in the main text more, specifically with regards to the non-ACT controls.\n\nL 328: Fig. 4 has no panel c.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "\"It is good to see the prediction confirmed. Still, the finding that ACT will flexibly halt computation early for easier inputs does not tremendously advance the field, considering it’s what Graves (2016) already reported. With an ACT-like method (“DACT”) Eyzaguirre and Soto (2020) have already found this in visual reasoning tasks too. It’s worth a citation.\"\n\n1. While we agree with the reviewer that Graves (2016) has shown that ACT will flexibly halt computation early for easier inputs, we would like to highlight that our unique contribution here is showing that it can be combined effectively with convolutional RNNs for improving performance on visual reasoning tasks. We thank the author for the relevant work by Eyzaguirre and Soto (2020). We shall add a citation to this work, but a key difference between this work and our submission here is that our models and evaluation are centered around extrapolation to novel difficulties on challenging visual reasoning problems. While we refer to [Kim et al 2018, Not-so-CLEVR] in support of the task used in Eyzaguirre and Soto (2020) as one known to not be challenging, it is certain from their experiments that they do not explore the scenario of generalizing to novel harder difficulty levels in their paper.\n\n\"The introduction of LocRNN then becomes the most novel aspect of the work, but there is not a lot of in-depth discussion in the main text about how it differs from prior models informed by neuroscience. At the very least, one would require a thorough comparison between the LocRNN and the model presented in Linsley et al. (2018) regarding their performance metrics and architectural similarities.\"\n\n2. Regarding comparison to Linsley et al. (2018) and Linsley et al. (2020), please refer to our comment addressing all reviewers where we discuss the difference between these works and our proposed work, as well as demonstrate our (empirical) high performance compared to models in both the above works.\n\n\"Finally, can the authors comment on whether this approach would work in other scenarios of visual difficulty apart from simply spatial extent? The notion of visual complexity involves a host of other factors which a non-hierarchical model would fail to account for, and which needs discussion.\"\n\n3. We also find it very interesting to test whether our proposed approach would scale to other scenarios of visual difficulty apart from spatial extent. Our work in progress and future scope will pursue this direction by studying AdRNNs on computer vision benchmarks based on natural images.\n4. Thank you for highlighting typos in the paper, we shall promptly fix these issues in the submission.\n"}, {"review_id": "KZo5P6ZARp", "reviewer": "Reviewer_1Kd6", "summary": "The manuscript presents adaptive recurrent networks for processing of static images. The proposed approach augments convolutional recurrent neural networks with the adaptive computation time mechanism, in which the RNN at each step computes an additional halting unit, the value of which is used to determine when to stop the iterative computation. This is shown to yield recurrent vision models that adapt their computational budget to scale with the difficulty level of tasks that require serial grouping operations, such as determining whether two points are located on opposite ends of the same path or segmenting the solution route of a maze. The experimental section also shows that adaptive recurrent networks with gating mechanisms successfully scale to solve the tasks at higher difficulty levels than seen during training.", "questions": "Overall, I think the proposed method is novel and has potential for impact, but given the issues mentioned in the weaknesses section, some more work is necessary for publication. I will consider to increase the score significantly should the issues be addressed during the author response period.\n\nI have some minor comments regarding the write-up:\nLine 63: starts upper-case after \"and\" in the previous item\nLine 100 and 105: Shouldn't it be \"4)\" and \"5)\" instead of \"2)\" and \"3)\"? Either way, it is a bit confusing.\nEquation 2: I'd explicitly state that $\\mathbf{r}$ is the recurrent block and how $h_{-1}$ is defined. Also, I think it'd be good to explicitly state what $t$ and $t_\\text{train}$ correspond to.\nLine 197: \"in Supp. Info\" should maybe be \"in the Appendix\" or \"in the supplementary material\"\nLine 266: \"While ... whereas\" (\"while\" or \"whereas\" should be removed)\nLine 340-343: I'd avoid nesting parentheses.\n\n## Acknowledgement of rebuttals\nI have read the rebuttal and provided follow-up questions. The rebuttal addressed my concerns, except the addition of a strong non-recurrent baseline. I have accordingly increased the score slightly.", "limitations": "The authors state some limitations of the approach and propose directions for future research.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "The proposed approach is novel and well-motivated. The experimental results confirm that the proposed models are able to extrapolate the learned knowledge beyond the training distribution to solve more difficult versions of the two tasks. This is quite impressive and I'm not aware of similar studies.", "weaknesses": "The write-up is a bit difficult to follow:\n\n1. It seems to me that some important details of the original ACT [1] halting mechanism are omitted in this manuscript or the proposed mechanism is a modified version, which should be clearly stated. I might be mistaken and only the formulation or notation is different. If so, please let me know.\n2. Both $p_t$ and $P_{t'}$ are not really probabilities, as I understand it. You could maybe simply call $P_{t'}$ a quantity that, if it reaches a predefined threshold, halts computation. \n3. $\\phi$ in Equation 4 is not defined\n4. It should be stated right after the first occurrence that LN stands for layer normalization.\n\nI think some details for reproducibility are missing or unclear:\n\n5. The sentence in line 188 states that *four* choices are explored, then in 191 it is stated that *three* different recurrent implementations are explored. As far as I understood, the first of the four choices is a non-recurrent ResNet-30. It is not clearly explained how this model is implemented as Equation 2 only covers recurrent architectures.\n6. The exact architectures including hyperparameters (e.g. number of kernels $d$ and their sizes) of the recurrent models are not provided.\n7. Why is R-ResNet-30 equipped with ACT in Table 1, but not in Table2?\n\nI think the experimental setup should take the following into consideration:\n\n8. For completeness, I think is would make sense to add a few more non-recurrent baselines. It seems intuitive that the considered tasks require multi-step reasoning, but it'd be important to show this empirically, e.g. by using a small vision transformer. I would be surprised if a transformer would completely fail on the task, given the fairly large number of training samples. I can, however, imagine that the proposed architecture performs much better on unseen difficulty levels. Showing this empirically would help in establishing AdRNNs as an important alternatives to current vision models.\n9. Accuracies are not reported as mean and standard deviation over multiple runs with different random seeds. This is very problematic, since with some tasks and architectures, single runs may underperform or not converge to any meaningful solution at all, especially with binary success measures (such as exactly correct maze segmentation). Aggregating metrics over several runs (e.g. 5) would help the reader interpret how significant a different in performance really is (e.g. is it more than 1 standard deviation?)\n\nRegarding references:\n\n10. I think the correct citation for *convolutional* GRUs would be [2]\n\n## References\n1. Graves, Alex. \"Adaptive computation time for recurrent neural networks.\" arXiv preprint arXiv:1603.08983 (2016).\n2. Ballas, Nicolas, et al. \"Delving deeper into convolutional networks for learning video representations.\" arXiv preprint arXiv:1511.06432 (2015).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. Our work is for the most part similar to the original ACT work, however, a key difference between our work and ACT is that our visual reasoning task involves static inputs whereas Graves (2016) can deal with variable-length sequences. Owing to this difference, our halting mechanism is the same as ACT applied to a 1-token input sequence, we will make this more explicit.\n2. Line 188 is a typo in our initial submission, we did perform evaluation only on 3 implementations of recurrent computations (R-ResNet-30, Convolutional GRU, and LocRNN). Based on the newly added baselines we will update this statement to reflect the 5 total RNN cells used (including Linsley et al. (2018) and Linsley et al. (2020)).\n3. All architectures were matched in terms of the number of parameters on PathFinder and Mazes evaluations respectively. For PathFinder, fix the number of kernels (d) to be 32 for LocRNN, 21 for ConvGRU and 64 for ResNet-30 with a filter size of 9x9 in the intermediate layers. For Mazes, we fix the number of kernels (d) to be 128 for LocRNN, ConvGRU, ResNet-30 and R-ResNet-30 and the kernel size is fixed to be 5x5.\n4. In Table 2, R-ResNet-30 was equipped with ACT and it is a typo that we missed to mention ACT here. We have fixed this typo.\n5. It would be very interesting to evaluate how well Vision Transformers perform on the two tasks we evaluate in our submission. Prior work (Tay et al. (2020)) showed that Transformers do not perform well on PathFinder. Given this and computational limitations, we were not able to test Vision Transformers on our versions of these tasks for the rebuttal.\n\n"}, {"review_id": "UUsPHwFqA4", "reviewer": "Reviewer_cX6r", "summary": "Authors combine Adaptive computation time (ACT) with convolutional recurrent neural networks to solve two tasks with which their generalization properties can be studied. These adaptive-timestep RNNs were found to halt quicker for easier problems while taking more steps for harder ones. This also leads to generalization to an unseen difficulty level.", "questions": "1. I am curious to hear why the authors say RNNs used in Bansal et al., (2022) \"are not adaptive, human intervention is required to specify the number of recurrent computational steps by brute force during the testing phase\" since the RNN can be run till response stops changing. \n\n2. What properties of the images makes the RNNs stop early vs late? For example, in Fig 4 some 19x19 mazes seem to need the timesteps as 25x25 mazes. Have you seen what causes this? Same for the other task. Can this lead to some interpretation/understanding of the RNNs and the solution they are implementing?", "limitations": "NA", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "The paper is well written and the results are convincing for the claim. \n* Pathfinder task is known to be challenging (Tay et al., 2020) so high accuracy and generalization on that task is very interesting. \n* Adaptive choosing of timesteps within a dataset is important & valuable for neuroscience and cognitive science research since it allows for identification of harder and easier stimuli. \n* Inference-time timesteps being more than training is a strong indicator that the RNNs are generalizing.", "weaknesses": "While I found the work interesting, I found the paper to be severely lacking in terms of novelty. The paper also misses some related works ([1], [2]). \n \n*  As mentioned in the paper, Bansal et al., (2022) have studied this with the maze task. Authors say \"recurrent networks used in their study are not adaptive, human intervention is required to specify the number of recurrent computational steps by brute force during the testing phase\" which I do not agree with - since their RNN stops changing its response after a while, that point can be used as to decide when to stop. \n* [2] has studied this with Pathfinder task - even finding generalization to difficult levels. \n\nBetween Graves et al. (2016), Bansal et al. (2022) and [2] I do not see the novelty of this work. \n\n\n\n\n\n\n1. Spoerer CJ, Kietzmann TC, Mehrer J, Charest I, Kriegeskorte N (2020) Recurrent neural networks can explain flexible trading of speed and accuracy in biological vision. PLOS Computational Biology 16(10): e1008215. https://doi.org/10.1371/journal.pcbi.1008215\n2. Drew Linsley, Alekh Karkada Ashok, Lakshmi Narasimhan Govindarajan, Rex Liu, Thomas Serre (2020) Stable and expressive recurrent vision models. NeurIPS 2020", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "\"I am curious to hear why the authors say RNNs used in Bansal et al., (2022) \"are not adaptive, human intervention is required to specify the number of recurrent computational steps by brute force during the testing phase\" since the RNN can be run till response stops changing\"\n\n1. As this is an important issue relevant to several reviews, we have addressed this general issue  in our common response. We also tried the  Linsley et al 2020 algorithm with a threshold on state change and it performed very poorly on Mazes (see Table 3 in the Rebuttal).\n\n\"What properties of the images makes the RNNs stop early vs late? For example, in Fig 4 some 19x19 mazes seem to need the timesteps as 25x25 mazes. Have you seen what causes this? Same for the other task. Can this lead to some interpretation/understanding of the RNNs and the solution they are implementing?\"\n\n2, The plot in Fig 4 plots halting step as a function of maze solution path length.  There is a clear linear relationship between solution path length (not maze size) for the shorter path lengths. For longer maze path lengths, the required computation saturates.\n"}, {"review_id": "QSjqxDPAxZ", "reviewer": "Reviewer_qD61", "summary": "The paper introduces a combination of Convolutional Recurrent Neural Networks (ConvRNNs) with a learnable termination mechanism from Adaptive Recurrent Neural Networks (AdRNNs), with the purpose of solving complex, variable-difficulty vision tasks. The paper adapts to the purpose a ConvGRU architecture, or alternatively a “LocRNN” architecture based on a computational model of iterative contour processing in primate vision. Both architectures, together with a ResNet baseline, are tested on the “Pathfinder” and “Mazes” tasks, with a variety of task difficulties. Importantly, the authors show that the AdRNNs architecture are able to generalize to task difficulties greater than those seen during training, requiring a number of computation steps beyond those used in training.", "questions": "What is the computational model of cortical processing relevant to the introduction of LocRNN? What is the intuition behind LocRNN’s improved performance and stability over other ConvRNNs?", "limitations": "The authors seem to have fairly acknowledged the limitations of their work in their “Limitations” section.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "The paper consists of an interesting and timely investigation into the hot topic of adaptive computation for problem solving.\n\nWhile the authors are not the first to work on this, they appear to be the first to have demonstrated that RNN models can be effectively applied for vision tasks with adaptive difficulty, and generalize to unseen difficulty levels.\n\nThe work is clearly motivated, and the overall story and contribution is immediately apparent.", "weaknesses": "I believe that the following points need to be addressed for my assessment of the paper to be changed more positively:\n* **[a]** Overall, the main contribution of the paper is the introduction of LocRNN, as all results, including the one on generalization to unseen difficulty levels for tasks, are downstream of LocRNN being a better architecture than alternatives. However, Section 4.3 appears crowded and hard to parse. I believe that the inclusion of figures illustrating the architectures and describing the differences between ConvRNN and LocRNN would greatly improve the presentation. Moreover, not much effort is spent intuitively justifying the reasons behind the improved performance of LocRNN over other ConvRNNs.\n* **[b]** In Line 197, it is mentioned that the computational models of cortical recurrent processing are described in detail in Supp. Info, but there is no such section in the supplementary material. Please include this section, to again better explain the intuition behind the introduction of LocRNN.\n* **[c]** There are small formatting issues. In Line 40, citation names are used without brackets even when they don’t naturally flow within the sentence.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We take the PDEs proposed in (Li, 1998), which model the connections between V1 neurons in continuous time. We take these PDEs, apply Euler integration to them in order to convert them into discrete-time difference equations and implement them using convolutional Recurrent Neural Networks. The final RNN obtained from Li 1998 after Euler integration is what we refer to as LocRNN in our submission. We have contrasted the architectures between LocRNN and GRU in Fig 8. \n\nWe will include the original PDEs and the derivations which we missed in the Supplementary as well as fix the formatting issues. \n\nReferences:\nTay et al. (2020), Tay, Y., Dehghani, M., Abnar, S., Shen, Y., Bahri, D., Pham, P., ... & Metzler, D. (2020). Long range arena: A benchmark for efficient transformers. arXiv preprint arXiv:2011.04006.\n\n"}]

Author rebuttal:
Thank you all for taking the time to carefully review our paper. Here, we address common points based on overlapping comments in the reviews. We provide our responses to the constructive comments common to all reviewers in the following sections of our response:

### Differences with respect to Linsley et. al, 2020, Linsley et. al, 2018
CC: ikdz, cX6r
We thank the reviewers for bringing up this very relevant related work that we have by mistake not cited. We will certainly add a citation to Linsley et al, 2020, we agree that it is indeed relevant to our submission. We went one step further in including this model (hConvGRU trained with C-RBP) as well as the one in (Linsley et al, 2018) as part of our baselines using code made available by the authors (please find the updated results in Table 3 in the rebuttal). We find strong evidence in support of our model, despite tuning the above-mentioned models considerably, it seems very difficult to make them converge to a stable solution that better generalizes out of distribution on the tasks we used (PathFinder and Mazes) in comparison to LocRNN.

Differences between our papers:
1) @Reviewers ikdz, cX6r: It is important to note that  Linsley et al, 2020 uses a simplified version of PathFinder (the stimuli used look similar but the tasks have different rules) – The 2020 task involves densely supervised segmentation of the path with a circle on one of its ends whereas our task involves binary supervision of whether two circles are on the same path (which is the original Linsley et al 2018 task). The main difference is the information content of the supervision - the 2020 task provides pixel-by-pixel labeling, which is very informative and simple to learn as it enables learning to trace paths pixel-by-pixel; our task (following their prior work from 2018) provides only binary (sparse) classification information which is harder to learn.
Hence, the OOD generalization shown by Linsley et al, 2020 and what we show here (and used in Linsley et al, 2018) are different and not numerically comparable.

2) Linsley et al., 2018 & Linsley et al., 2020 use the hConvGRU RNN (in 2018 they train it with BPTT whereas in 2020, they train it with C-RBP). We add both these models to our evaluation baselines and in our experiments they don’t match the performance of LocRNN (see last three rows of the updated Table 3 in the Rebuttal). Despite increasing the number of recurrent iterations and using network output stability as the stopping criterion (as reviewer cX6r suggested), we find the network to not work well on Mazes 19x19 or Mazes 25x25 (both ConvGRU and LocRNN fare reasonably well on both these challenges). Considering that the same hConvGRU architecture learns these tasks while using BPTT supports the hypothesis that (a) hConvGRU+C-RBP is quite complex a model with many free parameters and hyperparameters making the network difficult to train (and also interpret), or (b) it is sensitive to particular random initializations where it works well.
3) Overall, with the above experiments we find clear and striking evidence of the novelty and improved performance of LocRNN with respect to hConvGRU (Linsley et al, 2018), and hConvGRU + C-RBP (Linsley et al, 2020).

### Using stability as a halting criterion as opposed to learning halting
We considered using network stability as a criterion for halting.  
We found this to work suboptimally for a number of reasons. 
1) Avoiding hand-engineered heuristics: Most importantly, it requires defining a hand-engineered heuristic on how much change in the output is considered small enough to halt. In typical segmentation tasks, the network output for the initial few timesteps is highly stable in making nonsensical predictions (predicting all pixels as the negative class) and thus, heuristics need to identify an inflection point in the output trajectory where meaningful predictions start to emerge and stabilize. In the absence of ground truth information, one cannot pick a heuristic that generalizes to unseen data.
2) Avoiding hard assumptions / constraints on stable hidden states: Learnable halting makes less assumptions about the hidden state’s properties, and hence doesn’t enforce hard constraints such as stability to be satisfied by training. Some, but not all RNNs, have stable hidden states wherein the network response stops changing after reaching an attractor. In fact, Linsley et al., 2020 that reviewer cX6r refers to argue that RNNs that are expressive have an intrinsic inability to learn stable hidden states.

For RNNs to be stable, their hidden state transformation needs to model a contractive mapping (Miller & Hardt, 2019; Pascanu et al, 2013). That is, the recurrent transition function F satisfies the following inequality:
			||F(h_t) - F(h_{t-1})||_p < \lambda ||h_t - h_t-1||_p
RNNs with stable hidden states that satisfy the above inequality are quite difficult to train on challenging problems (these kinds of networks are similar in nature to Deep Equilibrium Models [1] and Neural ODEs [2] which are not as successful yet as RNNs in their ability to solve a wide variety of tasks) as there is an inherent tradeoff between stability of the hidden state and expressivity (Linsley et al., 2020). Even when stable models perform comparably to unstable models as in Miller & Hardt, 2019, the authors show unstable models’ advantages such as performance improvements in the short-time horizon and lesser vanishing gradient issues.
3) Finally models operating on dynamic input (as in the real world) may not be expected (or desired) to reach a stable hidden state.
4) We also tried the  Linsley et al 2020 algorithm with a threshold on state change and it performed very poorly on Mazes (see Table 3 in the Rebuttal).


Meta-review: This paper explores the problem of generalization to unseen visual reasoning  tasks, through learning to allocate computation time that is proportional to the task's difficulty. The authors also proposed a novel architecture, LocRNN.
The reviewers initially had concerns about the impact of the architecture and the statistical significance of the results. The authors provided in-depth complimentary experiments which shed more light on the mechanism, thus making the submission stronger.
Decision: Accept (poster)