The Uncanny Similarity of Recurrence and Depth
Authors: Avi Schwarzschild
Arjun Gupta
Amin Ghiasi
Micah Goldblum
Tom Goldstein
Venue: ICLR 2022 Poster (2022)
Keywords: Deep learning
recurrent networks
depth

Abstract: It is widely believed that deep neural networks contain layer specialization, wherein networks extract hierarchical features representing edges and patterns in shallow layers and complete objects in deeper layers. Unlike common feed-forward models that have distinct filters at each layer, recurrent networks reuse the same parameters at various depths. In this work, we observe that recurrent models exhibit the same hierarchical behaviors and the same performance benefits as depth despite reusing the same filters at every recurrence. By training models of various feed-forward and recurrent architectures on several datasets for image classification as well as maze solving, we show that recurrent networks have the ability to closely emulate the behavior of non-recurrent deep models, often doing so with far fewer parameters.

Published as a conference paper at ICLR 2022
THE UNCANNY SIMILARITY OF RECURRENCE AND DEPTH
Avi Schwarzschild" Department of Mathematics University of Maryland College Park, MD, USA avil @udd edu
Arjun Gupta"
Amin Ghiasi
Department of Robotics University of Maryland College Park, MD, USA ar jrign1 5 5 edu
Department of Computer Science University of Maryland College Park, MD, USA
Micah Goldblum
Tom Goldstein
Department of Computer Science University of Maryland College Park, MD, USA
Department of Computer Science University of Maryland College Park, MD, USA
ABSTRACT
is widely believed that deep neural networks contain layer specialization, wherein neural networks extract hierarchical features representing edges and patterns in shallow layers and complete objects in deeper layers. Unlike common feed-forward models that have distinct filters at each layer, recurrent networks reuse the same parameters at various depths. In this work, we observe that recurrent models exhibit the same hierarchical behaviors and the same performance benefits with depth feed-forward networks despite reusing the same filters every recurrence. By training models of various feed-forward and recurrent architectures datasets for image classification as well as maze solving, we show that recurrent networks have the ability closely emulate the behavior of non-recurrent deep models, often doing so with far fewer parameters.
several
INTRODUCTION
It is well-known that adding depth to neural architectures often enhance performance on hard problems (He et al.) 2016 Huang et al. (017). State-of-the-ar models contain hundreds of distinct layers (Tan Le, 2019. Brock et al. 2021). However, it is not obvious that recurrent networks can experience performance boosts by conducting additional iterations, since this does not add any parameters. On the other hand, increasing the number of iterations allows the network to engage in additional processing. In this work, we refer to the number of sequential layers (possible not distinct) as effective depth. While it might seem that the addition of new parameters a key factor in the behavior of deep architectures, we show that recurrent networks can exhibit improved behavior, closely mirroring that of deep feed-forward models, simply by adding recurrence iterations (to increase the effective depth) and no new parameters at all.
In addition to the success of very deep networks, number of works propose plausible concrete benefits of deep models. In the generalization literature, the sharp-flat hypothesis submits that networks with more parameters, often those which are very wide or deep, are biased towards flat minima of the loss landscape which are thought to coincide with better generalization (Keskar et al. 2016, Huang et al. 2020; Foret et al., 2020). Feed-forward networks are also widely believed to have layer specializatton. That is, the layers in feed-forward network are thought to have distinct convolutional filters that build on top of one another to sequentially extract hierarchical features (Olah)
al. 2017 Geirhos al. 2018). For example, the filters in shallow layers of image classifiers may be finely tuned to detect edges and textures while later filters may be precisely tailored for detecting semantic structures such as noses and eyes (Yosinski al. 2015). In contrast, the filters early and deep iterations of recurrent networks have the very same parameters. Despite their lack of layer-wise
"qqual contribution.

---

Published as a conference paper at ICLR 2022
specialization and despite the fact that increasing the number of recurrences does not increase the parameter count, we find that recurrent networks can emulate both the generalization performance and the hierarchical structure of deep feed-forward networks. and the relationship between depth and performance is similar regardless of whether recurrence or distinct layers are used.
Our contributions can be summarized as follows:
We show that image classification accuracy changes with depth in remarkably similar ways for both recurrent and feed-forward networks.
We introduce datasets of mazes, a sequential reasoning task, on which we show the same boosts in performance by adding depth to recurrent and to feed-forwart models.
With optimization based feature visualizations, we show that recurrent and feed-forward networks extract the very same types of features at different depths.
1.1
RELATED WORKS
Recurrent networks are typically used to handle sequential inputs of arbitrary length, for example in text classification, stock price prediction, and protein structure prediction (Lai et al. 2015 Borovkova & Tsiamas 2019 Goldblum et al.) 2020) Baldi & Pollastri (003). We instead study the use of recurrent layers place of sequential layers convolutional networks and on non-sequential inputs, such as images. Prior work on recurrent layers, or equivalently depth-wise weight sharing, also includes studies on image segmentation and classification as well as sequence-based tasks. For example, Pinheiro & Collobert (2014) develop fast and parameter-efficient methods for segmentation using recurrent convolutional layers. Alom et al. 2018) use similar techniques for medical imaging. Also, recurrent layers have been used to improve performance image classification benchmarks (Liang & Hu 2013). Recent work developing transformer models for image classification shows that weight sharing can reduce the size of otherwise large models without incurring large performance decrease (Taegle et al. (02T).
Additionally, the architectures we study are related to weight tying. Eigen et al. (2013) explore very
similar dynamics in CNNs, but they do not address MLPs or ResNets, Boulch 2017) ) study weight
sharing only in ResNets. More importantly, neither of those works have analysis beyond accuracy metrics, whereas we delve into what exactly is happening at various depths with feature visiualization, linear separability probes, and feature space similarity metries. |Bai et al. (2018) propose similar weight tying for sequences, and they further extend this work to include equilibrium models, where infinite depth networks are found with root-finding techniques (Bai et al., 2018 2019). Equilibrium models and neural ODEs make use of the fact that hidden states in recurrent network can converge to fixed point, training models with differential equation solvers or root finding algorithms (Chen) al. 2018 Bai et al. 2019).
Prior work towards understanding the relationship between residual and highway networks and recurrent architectures reveals phenomena related to study. Greff et al. (2016) present view of highway networks wherein they iteratively refine representations. The recurrent structures we study could easily be thought of in the same way, however they are not addressed in that work. Also, Jastrzebski et al. (2017) primarily focus on deepening the understanding of iterative refinement in ResNets. While our results may strengthen the argument that ResNets are naturally doing something iterative, this is tangential to our broader claims that for several families of models depth can be achieved with distinct layers or with recurrent ones. Similarly, [iao & Poggio (2016) show that deep RNNs can be rephrased as ResNets with weight sharing and even include study on batch normalization in recurrent layers. Our work builds on theirs further elucidate the similarity of depth and recurrence, specifically, we carry out quantitative and qualitative comparisons between the deep features, as well as performance, of recurrent and analogous feed-forward architectures as their depth scales. We analyze variety of additional models, including multi-layer perceptrons and convolutional architectures which do not include residual connections.
Our work expands the scope of the aforementioned studies in several key ways. First, we do not focus on model compression, but we instead analyze the relationship between recurrent networks and models whose layers contain distinct parameters. Second, we standard neural network training process, and we study a variety of architectures. Finally, we conduct our experiments on image data, well reasoning task, rather than sequences, that recurrent models can be viewed
2

---

Published as a conference paper at ICLR 2022
standard cully-co residual networks but with additional weight-sharing constraints. For each family of architectures and each dataset we study, we observe various trends as depth, or equivalently the number of recurrence iterations, is increased that are consistent from recurrent to feed-forward models. These relationships are often not simple, for example classification accuracy does not vary monotonically with depth. However, the trends are consistent whether depth is added via distinct layers or with iterations. We are the first to our knowledge to make qualitative and quantitative observations that recurrence mimics depth.
RECURRENT ARCHITECTURES
Widely used architectures, such as ResNet and DenseNet, scale up depth in order to boost performance on benchmark image classification tasks (He al. 2016 Huang et al. 2017). We show that this trend also holds true in networks where recurrence replaces depth. Typically, models are made deeper by adding layers with new parameters, but recurrent networks can be made deeper by recurring through modules more times without adding any parameters.
To discuss this further, is useful define effective depth. Let a layer of neural network be denoted by Then, p-layer feed-forward network can be defined by a function f that maps an input x to label y where
02 f(z) 0.. lxx).
In this general formulation, each member of the set (e))- can be different function, including compositions of linear and nonlinear functions. The recurrent networks in this study are of the following form.
f(x) = -p o... lx+1 my o m, o... oo o.... (1(x)
(1)
We let (x) = (())) .... e(T) (x) and call it q-layer recurrent module, which is applied
successively between the first and last layers. The effective depth of a network (feed-forward or recurrent) is the number of layers, not necessarily unique, composed with one another to form f. Thus, the network in Equation equation Taas an effective depth of p + nq, where n the number of iterations of the recurrent module. A more concrete example is feed-forward network with seven layers, which has the same effective depth as a recurrent model with two layers before the recurrent module, two layers after the recurrent module, and a single-layer recurrent module that runs for three iterations. See Figure [ for graphical representation the effective depth measurement.
We train and test models of varying effec- tive depths on ImageNet, CIFAR-10, EMNIST, and SVHN to study the relationship between depth and recurrence (Russakovsky et al. 2015)
(CS)
Krizhevsky 2009 Cohen al., 2017; 7 Netzer et al.] 2011 For every architecture style and every dataset, we see that depth can be emulated by recurrence. The details of the particular recur- rent and feed-forward models we use in these experiments are outlined below. For specific training hyperparameters for every experiment, see Appendix A.2
Coer 2D
Figure 1: Effective depth measures the depth of the unrolled network. For example, network with iterations of its single-layer recurrent module and 4 additional layers has effective depth of 7.
The specific architectures we study in the context of image classification include families of multi- layer perceptrons (MLPs), ConvNets, and residual networks. Each recurrent model contains an initial set of layers, which project into feature space, followed by a module whose inputs and outputs have the same shape. As result of the latter property, this module can be iteratively applied in recurrent fashion an arbitrary number of times. The feed-forward analogues of these recurrent models instead stack several unique instances of the internal module on top of each other. In other words, counterparts but with depth-wise
recurrent models can be considered the same as their weight sharing between each instance of the internal module.
'Code for reproducing the experiments in this paper is available in the code repository https//ghhhhhhmmm///mmm////////n
3

---

Published as a conference paper at ICLR 2022
2.1 MULTI-LAYER PERCEPTRONS
The MLPs we study are defined by the width of their internal modules which is specified per dataset in Appendix [A. We examine feed-forward and recurrent MLPs with effective depths from 3 to 10. The recurrent models have non-recurrent layers at the beginning and end, and they have one layer in between that can be recurred. For example, if a recurrent MLP has effective depth five, then the model is trained and tested with three iterations of this single recurrent layer. A feed-forward MLP of depth five will have the exact same architecture, but instead with three distinct fully-connected layers between the first and last layers. All MLPs full-connected layers.
have ReLU activation functions between
2.2 CONVOLUTIONAL NETWORKS
Similarly, we study group of ConvNets which have two convolutional layers before the internal module and one convolutional and one fully connected layer afterward. The internal module has a single convolutional layer with ReL.U activation, which can be repeated any number of times to achieve the desired effective depth. Here too, the output dimension of the second layer defines the width of the model, and this is indicated in Appendix A.I for specific models. Each convolutional layer is followed by a ReLU activation, and there are maximum pooling layers before and after the last convolutional layer. We train and test feed-forward and recurrent ConyNets with effective depths from 4 to 8.
2.3 RESIDUAL NETWORKS
We also employ networks with residual blocks like those introduced by He et al. (2016). The internal modules are residual blocks with four convolutional layers, so recurrent models have single recurrent residual block with four convolutional layers. There one convolutional layer before the residual block and one after, which is followed by linear layer. The number of channels in the output of the first convolutional layer defines the width of these models. Each convolutional layer is followed by a ReLU activation. As is common with residual networks, we use average pooling before the linear classifier. The residual networks we study have effective depths from 7 to 19.
3 MATCHING IMAGE CLASSIFICATION ACCURACIES
Across all model families, we observe that adding iterations to recurrent models mimics adding layers to their feed-forward counterparts. This result is not intuitive for three reasons. First, the trend in performance as function of effective depth is not always monotonic and yet the effects of adding iterations to recurrent models closely match the trends when adding unique layers to feed-forward networks. Second, adding layers to feed-forward networks greatly increases the number of parameters, while recurrent models experience these same trends without adding any parameters at all. On the CIFAR- 10 dataset, for example, our recurrent residual networks each contain 12M parameters, and yet models with effective depth of 1.5 perform almost exactly well as 1.5 feed-forward residual network with 31M parameters. Third, recurrent models do not have distinct filters that are used in early and deep layers. Therefore, previous intuition that early filters tuned for edge detection while later filters are tailored to pick up higher order features cannot apply to the recurrent models (Olah al. 2017 Yosinski et al. 2015).
In Figure 2 the striking result is how closely the blue and orange markers follow each other. For example, on SVHN, we see from the triangular markers that residual networks generally perform better as they get deeper and that recurrent residual models with multiple iterations perform as well as their non-recurrent counterparts. On CIFAR-10, we see that MLP performance as a function of depth is not monotonic, yet we also see the same pattern here as we add iterations to recurrent models as we do when we add distinct layers to non-recurrent networks. The increasing trend in the ImageNet accuracies, which are matched by the recurrent and feed-forward models, suggests that even deeper models might perform even better. In fact, we can train a ResNet-50 and achieve 76.48% accuracy on the testset. While training is dificult, we can training a 50 modified so that layers within residual blocks that permit inputs/outputs of the same shape share weights. This model is 75.13% accurate the testset. These
recurrent models, which do not have striding or pooling, at that depth
4

---

Published as a conference paper at ICLR 2022
CIFAR-10 Performannce
EMNIST Perfermance
16-
CanANet
CanrNa
C.Cavv
B Coaviee
Rediisal Ner
MiL
MLL
36-
R.MLP
R-MPP
Effecthre
Effecttree
SVII Performannce
ImageNet Performance
94-
Redited
66-
86-
Recitial
44
R-Radtad
$2-
15
17
19
13
17
Edffective depth
Edffective depth
Figure 2: Average accuracy of each model. The x-axis represents effective depth and the y-axis measures accuracy. The marker shapes indicate architectures, and the blue and orange markers indicate feed-forward and recurrent models, respectively. The horizontal bars are one standard error from the averages over several trials. Across all three families of neural networks, recurrence can imitate depth. Note, the y-axes are scaled per dataset to best show each of the trends.
experiments show that the effect added depth these settings is consistent whether this depth is achieved by iterating recurrent module adding distinct layers.
3.1 SEPARATE BATCH NORMALIZATION STATISTICS FOR EACH RECURRENCE
The models described above comprise range of architectures for which recurrence can mimic depth. However, those models do not employ the modern architectural feature batch normalization (loffe & Szegedy 2015). One might wonder if the performance recurrent networks boosted even further by the tricks that help state-of-the residual network the well-known ResNet-1 18 architecture (He et al. 2016). There is one complexity that arises when adding batch normalization recurrent models; despite sharing the same filters, the feature statistics after each iteration the recurrent module may vary, so we must use a distinct batch normalization layer for each iteration. Although it is not our goal further the state of the art, Table showss that these recurrent models perform nearly as well as ResNet-18, despite containing only a third of the parameters. This recurrent architecture not identical to the models used above, rather it designed to be as similar to ResNet- 18 as possible; see Appendix A.1 for more details.
models achieve such high performance. We compare recurrent
Table 1: Models with batch normalization. Performance shown for three different classification datasets. We report averages (+: one standard error) from three trials.
Dataset
Model
Params
Acc (%)
CIFAR-10
Ours
3.56M
93.99 0.10
ResNet-18
11.2M
94.69 + 0.03
SVHN
Ours
3.56M
96.04 +: 0.08
ResNet-18
11.2M
95.48 0.11
EMNIST
Ours
3.57M
87.42 0.19
ResNet-18
11.2M
87.71 0.09
5

---

Published as a conference paper at ICLR 2022
4 THE MAZE PROBLEM
To delve deeper into the capabilities of recurrent networks, we shift our attention from image classification and introduce the task of solving mazes. this new dataset, the mazes are represented by three-channel images where the permissible regions are white, the walls are black, and the start and end points are green and red squares, respectively. We generate 180,000 mazes in total, 60,000 examples each of small (9x9), medium (11x11), and large mazes (13x13). We split each set into 50,000 training samples and 10,000 testing samples. The solutions are represented as a single-channel output of the same spatial dimensions the input maze, with I's on the optimal path from the green square to the red square and O's elsewhere. For model to "solve" a maze, it must output binary segmentation map each pixel in the input is either on the optimal path or not. We only count solution as correct if it matches labels every pixel correctly. See Figure 3 for an example maze.
4.1 MAZE GENERATION
The mazes are generated using a depth-firs: first search based algorithm (Hill, 2017). We initialize a grid of cells with walls blocking the path from each cell to its neighbors. Then, starting at an arbitrary cell in the maze, we randomly choose one of its unvisited neighbors and remove the walls between the two and move into the neighboring cell. This process continues until we arrive at a cell with no unvisited neighbors, at which point we trace back to the last cell that has unvisited neighbors. This generation algorithm provides mazes with a path from each cell to every other cell in the initialized grid. The images are made by plotting all initial cells and all removed walls as white squares, leaving cell boundaries that were not deleted in black. We also denote the "start" and "end" points of the maze with green and red squares, respectively. After the mazes are generated. they are solved with Dijkstra's 's algorithm, and the solutions are saved as described above. Figure 4 shows the distribution of optimal path lengths in each of the sets of mazes. There is a non-zero chance of repeating a maze, however we observe that fewer than 0.5% of the mazes are duplicated within each of the training sets, and the training and testing sets do not overlap.
Figure 3: A sample maze and its solution. This an example of an input/target pair used for training and testing neural maze solvers.
4.2 MAZE SOLVING NETWORKS
Small
We extend our inquiry into recurrence and depth to neu- ral networks that solve mazes. Specifically, we look at residual networks' performance on the maze dataset. The networks take a maze image as input and output binary classification for each pixel in the image. These models have one convolutional layer before the internal module and three convolutional layers after. Similar to classifi- cation networks, the internal modules are residual blocks and single block is repeated in recurrent models while feed-forward networks have a stack of distinct blocks. We employ residual networks with effective depths from 8 to 44 layers.
Medion
Earge
Chnnt
Path Lengthe
Figure 4: The distribution of optimal path lengths in each dataset (small, medium, and large mazes).
4.3 SOLVING MAZES REQUIRES DEPTH (OR RECURRENCE)
We train several recurrent and non-recurrent models of effective depths ranging from 8 to 44 on each of the three maze datasets, and we see that deeper models perform better in general. Figure Shoos these results. More importantly, we observe that recurrent models perform almost exactly as well as feed-forwart models with equal effective depth. Another crucial finding confirmed by these plots is that larger mazes are indeed harder to solve. We see that with a fixed effective depth, models perform the best on small mazes, then medium mazes, and the worst large mazes. This trend matches the

---

Published as a conference paper at ICLR 2022
Small
Medium
Large
Accuracy (%6
40-
20
Recurrent
16
20 24 28
28
Effective depth
Figure 5: Recurrent models perform similarly to feed-forward models of the same effective depth. All models here are residual models as described Section
intuition that larger mazes require more computation solve, and it confirms that these datasets do, in fact, constitute problems of varying complexity.
One candidate explanation for these trends is the receptive field, or the range of pixels in the input that determine the output at given pixel. Intuitively, solving mazes requires global information rather than just local information which suggests that the receptive field successful models must be large. While the receptive field certainly grows with depth, this does not fully account for the behavior we see. By computing the receptive field for each effective depth, we can determine whether the accuracy improves with even more depth beyond what is needed for complete receptive field. For the small mazes, the receptive field covers the entire maze after eight x 3 convolutions, and for performance rise in all three
the large mazes, this takes 12 such convolutions. In Figure 5 plots even after effective depth grows beyond 20. This indicates that receptive field does not fully explain the benefits of effective depth there is further improvement brought about by adding depth, whether by adding unique parameters or reusing filters in the recurrent models.
We further test this by training identical architectures those described above, except that the x convolutional filters are dilated (this increases the receptive field without adding parameters or depth). fact, these models are better at solving mazes, but most importantly, we also see exactly the same relationship between recurrence and depth. This confirms that recurrence has strong benefit beyond merely increasing the receptive field. See Table for numerical results.
Table 2: Dilated Filters. The average accuracy of models with dilated filters trained and tested on small mazes.
Effective Depth
Model
12
16
20
Recurrent
75.07
90.41
94.70
95.19
75.59
90.58
93.84
95.00
RECURRENT MODELS REUSE FEATURES
Test accuracy is only one point of comparison, and the remarkable similarities in Figure raise the question: What are the recurrent models doing? Are they, in fact, recycling filters, or are some filters only active on a specific iteration? In order answer these questions, we look the number of positive activations in each channel of the feature map after each recurrence iteration for a batch of 1,000 randomly selected CIFAR-10 test set images. For each image- filter pair, we divide the number of activations at each iteration by the number of activations at the most active iteration (for that pair). this measurement is positive on iterations other than the maximum, then the filter is being reused on multiple iterations, In Figure it is clear that large portion of the filters have activation patterns on their least active iteration with at least 20% of the activity of their most active iteration. This observation leads to conclude that these recurrent networks supporting the idea that the very same filters can be used in shallow layers as deep layers without a loss in performance. See Appendix A.4 for similar figures from different models and datasets.
indeed reusing filters, further
7

---

Published as a conference paper at ICLR 2022
5.1 RECURRENT AND FEED-FORWARD MODELS SIMILARLY DISENTANGLE CLASSES
Table 3: Classifiability of Features. CIFAR-10 10 classification accuracy of linear classifiers trained on intermediate features output by each residual block (or each iteration) networks with 19-layer effective depths. Fach cell reports the percentage of training/testing images correctly labeled.
Block #1
Block #2
Block #3
Block #4
Feed Forward
84.7 / 81,6
94.1 / 86.9
99.3 / 89.8
100.0 90.6
Recurrent
86.3 / 82.3
95,9 87.5
99.5 / 89.4
99.9/ / 90.3
In order to compare the features that image classifiers extract, we train linear classifiers on intermediate features from various layers of the network. In trying to classify feature maps with linear models, we can analyze network's ' progress towards disentangling the features of different classes. For this experiment, we examine the features of CIFAR-10 images output by both feed-forward and recurrent residual networks. Similar to the method used by Kaya et al. (2019), we use the features output by each residual block train a linear model predict the labels based on these feature vectors.
Using residual networks with effective depth of 19 (as this is the best performing forardorward depth), we show that the deeper features output by each residual block, or iteration in the recur- rent model, are increasingly separable. More im- portantly, as shown in Table 3, the feed-forward and recurrent models separating features very similarly at corresponding points in the networks. Details the training of these linear classifiers, in addition to results with pooling are in Appendix A..5
16000
14000
12000
10000
8000
6818
4800
2010
Another tool for quantifying relationships be- tween deep neural networks is feature space similarity metrics. We employ the techniques proposed by Kornblith al. (2019) to compare models trained on CIFAR- 10 and obtain a sim- ilarity score between zero and one. When we compare feed-forver forward ConvNets with their re- current counterparts, the results indicate that models with the same effective depths are in- deed very simlar. Specifically, the features extracted by an eight-layer feed-forward ConvNet are more similar to those extracted by its recurrent counterpart with the same effective depth (CKA 0.843) than they are to those of feed-forwart models different depths (e.g. CKA = 0.819 for a ConvNet with 4 layers). That is, feed-forward and recurrent networks of the same effective depth more similar to each other than they are to models of different depths.
Artivations (Compared to Most Active Brenstion)
Must Active 2nd Active
Figure 6: The number of active neurons after each iteration of recurrent ConvNet with 5 iterations. Note that the most active iteration is left off this plot since vations at that iteration.
normalizing by the number acti-
5.2 VISUALIZING THE ROLES OF RECURRENT AND FEED-FORWARD LAYERS
In previous sections, we performed quantitative analysis of the classification performance of recurrent and feed-forward models. In this section, we examine the qualitative visual behavior that transpires inside these networks. It is widely believed that deep networks contain hicrarchical structure in which early layers specialize in detecting simple structures such as edges and textures, while late layers are tuned for detecting semantically complex structures such as flowers or faces (Yosinski) cet al. 2015). In contrast, early and late layers of recurrent models share exactly the same parameters and therefore are not tuned specifically for extracting individual simple or complex visual features. Despite recurrent models applying the same filters over and over again, we find that these same filters detect simple features such as edges and textures in early layers and more abstract features in later layers. To this end, we visualize the roles of individual filters by constructing images which maximize filter's activations particular layer equivalently at a particular recurrence iteration.
8

---

Published as a conference paper at ICLR 2022
Depth
Recurrent
Feed Forward
Depth
Recurrent
Feed Forward
2
2
6
4
11
8
16
Figure 7: ImageNet filter visualization. Filters from recurrent model iterations alongside corresponding feed- forward layers.
Figure CIFAR-10 filter visualization. Filters from recurrent model iterations and corresponding feed-forward layers.
We follow similar approach to Yosinski et al. (2015). We also employ regularization technique involving batch-norm statistics for fine-tuning the color distribution our visualizations (Yin et al.) 2020). We study recurrent and feed-forward networks trained on ImageNet and CIFAR-TO. More details on the networks, hyperparameters and regularization terms can be found in Appendix [A66]
Figures and 8 show the results for ImageNet and CIFAR-10 datasets, respectively. Note that in these figures, the column of visualizations from recurrent models show the input-space image that activates output for the same filter but at different iterations of the recurrent module. These experiments show qualitatively that there are no inherent differences between the filters in the shallow layers and deep layers. In other words, the same filter is capable of detecting both simple and complex patterns when used in shallow and deep layers. For more visualizations and larger images, see Appendix A.7
DISCUSSION
With the qualitative and quantitative results presented in our work, it is clear that the behaviour of deep networks is not the direct result of having distinct filters at each layer. Nor is it simply related the number of parameters. If the specific job of each layer were encoded in the filters themselves, recurrent networks would not be able to mimic their feed-forward counterparts. Likewise, the trends we see as we scale common network architectures were due to the expressiveness and veerpraamtterization, recurrent networks should not experience these very same trends This work calls into question the role of expressiveness in deep learning and whether such theories present an incomplete picture of the ability of neural networks to understand complex data.
In addition to comparing performance metrics, we show with visualizations that features extracted at early and late iterations by recurrent models are extremely similar those extracted at shallow and deep layers by feed-forward networks. This observation confirms that the hierarchical features used process images be extracted, not only by unique specialized filters, but also by applying the same filters over and over again.
The limitations our work come from our choice of models and datasets. First, we cannot necessarily generalize our findings to any dataset. Even given the variety of image data used in this project, the conclusions we make may not transfer other image domains, like medical images, or even to other tasks, like detection. With the range of models we study, we are able to show that our findings about recurrence hold for some families of network architectures, however there are regimes where this may not be the case. We are limited in making claims about architectures that are very different in nature from the set that we study, and even possibly in addressing the same models with widths and depths other than those we use in our experiments.
Our findings together with the above limitations motive future exploration into recurrence other settings. Analysis of the expressivity and the generalization of networks with weight sharing would deepen our grasp on when and how these techniques work. More empirical investigations covering new datasets and additional tasks would also help to broaden conclusion, we show that recurrence can mimic depth in several ways within the bounds of our work, i.e. the range of settings in which we test, and ultimately confirm our hypothesis.
understanding of recurrence. In

---

Published as a conference paper at ICLR 2022
7 ETHICS AND REPRODUCIBILITY
We do not foresee any ethical issues with the scientific inquiries in this paper. In order to reproduce any of the experiments, see the code in the supplementary material and Appendix A.2 These materials are freely available and include all the details on training and testing, including specific hyperparamters, that one could need run our experiments.
LCKNOWTES
This work supported by the AFOSR MURI program, and ONR MURI program, DARPA GARD, and the National Science Foundation DMS program.
REFERENCES
Md Zahangir Alom, Mahmudul Hasan, Chris Yakopeic, Tarek M Taha, and Vijayan K Asari. Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv preprint arXiv: 8802506955 2018.
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. Trellis networks for sequence modeling. In International Conference on Learning Representations, 2018.
Shaojie Bai, Zico Kolter, and Vladlen Koltun. Deep equilibrium models. Advances in Neural Information Processing Systems, 02:690-701, 2019.
Pierre Baldi and Gianluca Pollastri.
The principled design of large-scale recursive neural network architectures-dag-mms and the protein structure prediction problem. The Journal of Machine Learning Research, 4:575-602, 2003.
Svetlana Borovkova and Ioannis Tsiamas. An ensemble of lstm neural networks for high-frequency stock market classification. Journal of Forecasting, 38(6):600-619, 2019.
Alexandre Boulch. Sharesnet: reducing residual network parameter number by sharing weights. arXiv preprint arXiv:1702.08773 2017.
Andrew Brock, Soham De, Samuel L. Smith, and Karen Simonyan. High-performance large-scale image recognition without normalization. arXiv preprint arXiv:2102.06171 2021.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 6572-6583, 2018
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and André van Schaik, Emnist: an extension of mnist to handwritten letters, 2017.
David Figen, Jason Rolfe, Rob Fergus, and Yann LeCunn Understanding deep architectures using recursive convolutional network. arXiv preprint 1rX1v 2013.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.014/2. 2020.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Mathhis Bethge, Felix Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv: 12231. 2018.
Micah Goldblum, Avi Schwarzschild, Ankit B. Patel, and Tom Goldstein. Adversarial attacks on machine learning systems for high-frequency trading, 2020.
Klaus Greff, Rupesh K Srivastava, and Jürgen Schmidhuber. Highway and residual networks learn unrolled iterative estimation. arXiv preprint arXiv:1612.07771. 2016.
K. He, X. Zhang. S. Ren, and Sun. Deep residual learning for image recognition. In 2016 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
10

---

Published as a conference paper at ICLR 2022
Christian Hill. Making maze, Apr 2017. URL https://oc.pyttoo.ccmmm///
making-aamazer/
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Weinberger. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708, 2017.
W Ronny Huang, Zeyad Emam, Micah Goldblum, Liam Fowl, Justin K Terry, Furong Huang, and Tom Goldstein. Understanding generalization through visualizations. 2020.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp. 448-456. PMLR, 2015.
Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol Vinyals, and Joao Carreira. Perceiver: General perception with iterative attention. arXiv preprini 1:Xiv:2103.0322: 2021.
Stanislaw Jastrzebski, Devansh Arpit, Nicolas Ballas, Vikas Verma, Tong Che, and Yoshua Bengio. Residual connections encourage iterative inference. arXiv preprint 1rXiv:1710.04773, 2017.
Yigitcan Kaya, Sanghyun Hong, and Tudor Dumitras. Shallow-ders networks: Understanding and mitigating network overthinking. In International Conference on Machine Learning, pp. 3301-3310 10. PMLR, 2019.
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint 1609. 2016.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited, 2019.
Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009.
Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao Recurrent convolutional neural networks for text classification. Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015.
Ming Liang and Xiaolin Hu. Recurrent convolutional neural network for object recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3367-3375, 2015.
Qianli Liao and Tomaso Poggio. Bridging the gaps between residual learning, recurrent neural networks and visual cortex. arXiv preprint 16Xv::100.003400 2016.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2(11):e7, 2017.
Pedro Pinheiro and Ronan Collobert. Recurrent convolutional neural networks for scene labeling. In International conference on machine learning, pp. 82-90. PMLR, 2014.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115 (3):211-252, 2015. doi: 10.1007/s1 1263-015-0816.y.
Mingxing Tan and Quoe Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning, pp. 6105-61 14. PMLR, 2019.
Hongxu Yin, Pavlo Molchanov, Jose M Alvarez, Zhizhong Li, Arun Mallya, Derek Hoiem, Niraj Jha, and Jan Kautz. Dreaming to distill: Data-free knowledge transfer via deepinversion. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2020.
Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. Understanding neural networks through deep visualization. arXiv preprint arXiv: 065.9669606699 2015.
11

---

Published as a conference paper at ICLR 2022
A
APPENDIX
A.1 MODEL ARCHITECTURES
We train and test MIPs, ConvNets, and residual networks on three image classification datasets. but the specific widths used for each dataset are detailed
These models are described in Section 2 below.
Multi Layer Perceptrons (MLPs). All of the MLPs have a single fully connected layer before the internal module, and single fully connected layer afterward. Each layer in an MLP can be defined by matrix dimension. See Table 4
Table MLPs: The size of each fully connected layer the MLPs we use for image classification.
Dataset
First Layer
Internal Layers
Last Layer
CIFAR-I0
3072 x 200
200 x 200
200 x 10
SVHN
3072 500
500 500
500 10
EMNIST
3072 x 500
500 x 500
500 X 47
ConyNets. The ConvNets have two convolutional layers before the internal module and one convo- lutional and one fully connected layer afterward. Each convolutional layer is followed by ReLU activation, and there are pooling layers before and after the last convolutional layer. For all convo- lutional layers outside of the internal module, we use In order to preserve the exact shape of the inputs, the internal module has convolutional layers with 3 x kernels, a stride of one pixel, and padding of one pixel in each direction.
3 filters with stride of and no padding.
Table 5: ConvNets: The number of channels the output of each layer.
Dataset
First Layer
Second Layer
Internal Layers
Last Conv Layer
Linear Layer
CIFAR-I0
32
64
64
128
10
SVHN
32
64
64
128
10
EMNIST
32
64
64
128
47
Residual Networks. The residual networks employ convolutional layers with x 3 kernels. The first layer has stride of two pixels and padding of one pixel in every direction. Each layer in the internal modules has striding by one pixel, while the final convolutional layer strides by two pixels. There ReL. activations after every convolution, and average pooling before the linear layer. All the convolutional layers output 512 channels.
We also train and test residual networks with BatchNorm in order to compare the performance with ResNet-18 18 models. These residual networks have a unique batch norm layer after every convolutional layer followed by ReLU activation function. Since the input the internal module at iteration the output at iteration Thus, we use separate batch norm layer for each iteration.
1 (for = 2,3, 4, ...,, the batch statistics for every iteration is different.
Maze Solvers. For solving mazes, we train and test residual networks with slightly different architectures. These models are fully convolutional and every convolutional layer has 3 kernels that stride by one pixel with padding of one pixel in each direction. They have one layer before the internal module, residual blocks of four layers the internal module, and three convolutional layers afterward. All of the layers before the third to last one have 128 output channels, and in the final three layers the numbers of output channels are 32, 8, and 2.
A.2 HYPERPARAMETERS
All the models are trained to convergence using suitable hyperparameters. For each model and each dataset, we fine tune the number of epochs, along with the learning rate and the decay schedule. For specifics, see the launch files in the code repository <Suppressed for anonymi ty>. tyy..
12

---

Published as a conference paper at ICLR 2022
A.3 COMPUTE RESOURCES
All of our experiments are done on Nvidia GeForee RTX 2080Ti GPUs. All training, except for ImageNet models, done on a single GPU. We train models on ImageNet data using four GPUs at time. Each image classification model can be trained in several hours on single GPU, and maze solving models require eight hours.
A.4 ADDITIONAL. VISUALIZATIONS
The discussion about activation patterns in Section 5 continues here with depictions of the activity in ConvNet trained SVHN and residual network trained on CIFAR-10. See Figures] 10 and [1 Although these plots show distinct behavior from Figure 6 the body of this paper, the conclusions consistent. In recurrent models, features being reused. This clear since every filter being used on more than one iteration, and for many filters there is significant activity in most iterations.
3500
3000
2500
Count
2000
1500
1000
0
0.0
0.2
Activations (Compared to Most Active Iteration)
5th Most Active
3rd Most Active
4th Most Active
2nd Most Active
Figure 9: Relative activations at each layer in the recurrent module of an untrained CNN. This plot confirms that the distributions in the other similar plots from trained models significant.
A.5 CLASSIFIABILITY OF FEATURES
Compare the accuracy when a linear classifier is trained on features form different layers. As Kaya et al. (2019) do in their study of overthinking, features in this experiment.
average pooling to reduce the size of the
Table 6: Classifiability of Features. CIFAR-10 classification accuracy of linear classifiers trained on intermediate features output by each residual block (or each iteration) networks with 19-layer effective depths. Average pooling is used change the dimension of feature maps in this experiment. This is consistent with (Kaya et al. 2019). Each cell reports the percentage training/testing images correctly labeled.
Block
Block #2
Block #3
Block
Feod Forward
85.7 / 83,4
94.1 / 88.0
90.2 99.4 90.2
99.8 / 90.8
Recurrent
88.1 / 84.1
96.9 / 88.4
99.7 89.8
99.9 90.4
Additional experiments on the linear classifiability of features recurrent models shows that other architectures also separating features with each iteration.
13

---

Published as a conference paper at ICLR 2022
30000
20000
Count
15000
10000
5000
0.0
0.2
0.8
Activations (Compared to Most Active Iteration)
5th Most Active
3rd Most Active
4th Most Active
2nd Most Aclive
Figure 10: Relative activations at each layer in the recurrent module of CNN trained on SVHN.
300090
1869)
15K00
10900
3000
Actiration (Compuree Most Acthve Sirratioo)
Activatona (Campared Mart Acdin licraton)
Active
Murt Autiv
Actiee Red Mait Arthe
Trl Moad Antie
20000
35199
15K00
Cate
6a
43
64
stien (Conpard Astive lncraiio)
Activatian (Campared to Htrration)
Acthvs
Mart Autive
Actie
Figure 11: Relative activations each layer in the recurrent module of a residual network trained on CIFAR-10.
A.6 FEATURE VISUALIZATOIN
We discuss the feature visualization technique used in 5.2 in further detail. For both ImageNet and CIFAR-10 datasets, we use networks with residual connections.
Similar to Yin et al. 2020), pixel jitter moves the input image both horizontally and vertically by selecting two random number between (-32, +-22) for each axis. The portion of image that moves out of the viewpoint fill up
pixel jitter augmentation and total variation as regularizer. The
14

---

Published as a conference paper at ICLR 2022
Table Classifiability of Features. CIFAR-10 classification accuracy of linear classifiers trained on intermediate features output by each iteration. Each cell reports the percentage of training/testing images correctly labeled.
Layer
Layer #2
Layer
Layer
Layer #5
Layer #66
R-MLP 46.52/44.28
50.67/47.22
57.54/52.07 65.39/70.74
59.43/54.08
61.43/55.19
61.54/55.39
R-CNN 58.74/68.67
59.68/66.99
67.73/73.84
the empty space on the opposite side. For the total variation, we use the anisotropic version in four directions: horizontal, vertical, and two diagonal directions. We use the f2-norm for computing the total variation all directions.
We find that the batch statistics from the first convolutional layer provides strong enough signal for improving the quality through regularization.
For CIFAR-10, optimize mini-batch of size = 64 of 32 X 32 images for 1000 iterations. For ImageNet we use mini-batch size n = 32 of 224 x 224 images for 1000 iterations. Both experiments are done single Nvidia GeForce RTX 2080Ti GPU.
As the main loss for CIFAR-10 experiments, we maximize the l2- fo-norm of activations. More precisely, assuming that (I;) represents the activation for filter in layer on input 2is then solve:
(2;)1(
For ImageNet experiments, use contrastive version of the loss to promote diversity:
max n
(zx)ll2
We use the ADAM optimizer and cosine annealing based learning rate schedule. The initial learning rate is 0.01.
Assuming that Amain Ato. Aun represent the coefficients for the main loss, total variation and batch normalization regularizer, respectively, we set λ. = 0 and Aon = O for the visualizations in the first layer, and linearly increase them as we go deeper. For the last layer, in CIFAR- experiments. Amain 0.64, Ato 1.0, and Atm. 10. For ImageNet experiments, in the deepest layer we set Amain 0.01, Anp = 5.0, Abn 50.0.
A.7 EXTENSIVE QUALITATIVE VISUALIZATION
Similar to Section 5.2 here show the results of our visualizations in Figures 12 and 13 that the first and final layers are not part of the recurrent modules.
We note
15

---

Published as a conference paper at ICLR 2022
Depth
Recurrent
Feed Forward
Figure 12: ImageNet filter visualization. Filters from recurrent model iterations alongside corre- sponding feed-forwart layers.
16

---

Published as a conference paper at ICLR 2022
Depth
Recurrent
Feed Forward
2
3
5
7
10
12
13
14
15
17
Figure 13: CIFAR-10 filter visualization. Filters from recurrent model iterations alongside corre- sponding feed-forwan layers.
17


Reviews:
[{"review_id": "pE4wZBLmHz", "reviewer": "Reviewer_UD8X", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "#############\n\nUPDATED REVIEW:\n\nThank you answering all my questions and apologies, for the delayed response.\n\nUpon reading the rebuttal and revised draft, I believe the authors have satisfactorily addressed my concerns and hence, I am increasing my score by 1 point. I would be interested, however, to see how the hierarchical representation plays out in the future with different architectures and more complex tasks!\n\n#############\n\nStrengths: The paper asks an interesting question on whether depth in artificial neural networks can be replaced with recurrence that would use much fewer parameters. The analyses on qualitatively visualizing the learned filters was exciting and insightful in how these models represent information/learn hierarchical structure even with recurrence.\n\nWeaknesses: My comments and questions are listed below.\n\nImage classification task:\nI am not sure I find this result particularly surprising. In both MLP and R-MLP, there are additional non-linear transformation on the model inputs and this leads to some patterns in accuracy. Granted that the “inputs” to the middle layer are recurrent in one case and not the other, which has implications on the number of model parameters. But in several cases:\n\n1. Are the differences between the effective depth across a model architecture statistically signifiant to begin with? If not, I am not sure it is fair to claim that the recurrent versions mimic the non-monotonic patterns of depth. Instead, they just achieve similar accuracy to the original model \n\nThe above comment is in reference to:\n> First, the trend in performance as a function of effective depth is not always monotonic and yet the effects of adding iterations to recurrent models closely match the trends when adding unique layers to feed-forward networks. \n\n> In Figure 2, the striking result is how closely the blue and orange markers follow each other. \n\nFor example, in the MLP based models, it is not clear if the differences across depth are significantly different from each other.\n\n2. The biggest differences in accuracy seem to arise from different architectures themselves and not the absence or presence of recurrence. This further suggests that the types of functions approximated by the model (either as stacked non-linearities or recurrent ones) is more important.\n\n3. For ImageNet, we observe a monotonically increasing accuracy curve as the number of non-linearities in the model increase (irrespective of how they increased). What conclusions do the authors draw from this aside from the following?\n> These experiments show that the effect of added depth in these settings is consistent whether this depth is achieved by iterating a recurrent module or adding distinct layers. \n\nWhile the results are demonstrative that using recurrence (and hence, fewer parameters) one can achieve similar task performance on image classification, without a theoretical proof, I do not think they indicate that the models learn similar functions (I understand the authors never claim this but perhaps I'm missing the bigger/less obv picture painted by the generalization performance result).\n\nMaze task: The dataset itself is very interesting! As a minor point, the authors mention that fewer than 0.5% of the mazes are duplicated within each of the training sets. What is this statistic for the train-test overlap? I also found the linear probing analyses in section 5.1 to be demonstrative of the increasingly separable features learned as (pseudo-) depth increases.\n\nMy comments and questions in the image classification task apply here as well.\n\nTo me the most exciting part of the paper was the analysis in section 5.2. While it seems obvious to me that accuracy improves as depth increases (whether or not the parameters are reused), the finding that qualitatively these models learn similar features despite using different mechanisms of weight sharing is insightful. It suggests that the information captured by neural networks is possibly not in the parameters themselves but the states of the model. However, I would be interested in learning the authors' thoughts on whether this is expected given the increasingly non-linear nature of the input to the model. Further, can the authors comment on the lack of alignment in the recurrent and feed-forward columns visualized in Fig. 12?\n\nThe paper would greatly benefit from adding theoretical proofs to suggest that these models are perhaps approximating similar functions or adding a wider suite of complex tasks & architectures. See for example this paper that theoretically and empirically proves recurrence and depth are equivalent in RNNs: https://arxiv.org/pdf/1909.00021.pdf\n\nIn its current form, I am not convinced the generalization results are surprising/insightful given that both version of the models have access to stacked non-linearities but I encourage the authors to explore and emphasize on the hierarchical structure emulated by recurrence in the future.\n> we find that recurrent networks can emulate both the generalization performance and the hierarchical structure of deep feed-forward networks\n\nOther questions:\n- What are the results for classifiability of features with other architectures? (table 3)\n- How was hyper-parameter tuning done? What are the train-val-test splits for different datasets? How much time did each model take to train and test? Was there signifiant differences in optimizing one over the other?", "summary_of_the_paper": "In this work, the authors compare and contrast models wherein layer depth is replaced with equivalent number of recurrent time steps. In particular, they explore feed-forward, CNN and residual deep network wherein the intermediate layers are replaced by an equivalent recurrent block. For each architecture, the authors compare accuracy of the stacked and recurrent variants for image classification on 4 datasets- CIFAR-10, EMNIST, SVHN and ImageNet and a novel Maze task wherein the network has to assign binary labels to each image pixel (0= not a valid point in path, 1=valid point in path to solve the maze). Overall, the authors observe that across all models and tasks, the recurrent and stacked networks achieve similar performance. They also verify this with batch normalization and dilated CNNs in some tasks. Next, they investigate the number of positive activations at each recurrent step to find that filters are active/recycled across recurrent depth. They also analyze the linear classification accuracy at different depths of the stacked and recurrent CIFAR-10 feedforward model, finding comparable accuracy at each depth. Finally, they visualize filter activations at different depths for both CIFAR-10 & ImageNet, finding comparable feature maps in both architectures.", "summary_of_the_review": "The paper has interesting results on how hierarchical representations emerge in recurrent networks akin to stacked networks, despite the former sharing parameters across its \"depth\". However, the similarity in task accuracy between recurrent and stacked networks with similar number of non-linearities is not terribly surprising. It also doesn't shed light on whether these models approximate similar functions on whether this finding can be generalized to other task paradigms. I believe this is a promising research direction and encourage the authors to analyze the similarities and differences in the information learned by the two paradigms in more depth.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ibEFxHl9h1T", "reviewer": "Reviewer_Ej1w", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n- I believe that the authors are studying an interesting, although not new, question about the similarity between recurrence and depth in information processing in neural networks. There has been immense progress in the field of AI (particularly computer vision) with feedforward models and it is puzzling why recurrent models haven't caught up (despite being computationally sophisticated and present in abundance in the cortex). This work is sparking further interest in studying the relationship between depth and recurrence.\n- The authors conduct extensive quantitative and qualitative analyses with several model families and diverse effective depth ranges to test their hypothesis that recurrence mimics depth. \n- The newly proposed maze challenge is very interesting and connects well to prior work in Vision science and psychophysics that study the influence of global cues in visual processing. \n- I find Section 5 on recurrent models reusing features to be interesting, but the clarity of writing in this section could be improved for better understanding on the readers' end.\n\nCons:\nI have the following concerns about this work:\n1. How are the findings in this work different from Liao and Poggio 2016 [1]? For e.g. Figure 3 in their paper conducts the same analysis that is conducted by Section 3 (albeit with a wider range of datasets). While the observed results confirm that deep architectures and recurrent architectures achieve similar classification accuracy, I didn't find this section particularly novel.\n2. The studied recurrent models are very simplistic in nature (stack of convolutions with weight sharing) and do not incorporate several crucial components that make recurrence interesting (both in ANNs and in the brain) such as gating, bypassing, temporal decay, etc. \n3. There isn't enough details about the recurrent architectures leading to following unresolved questions: Exactly how many layers are recurrent in each of these architectures? How many steps of recurrence is implemented in these layers?\n4. The maze solving task is interesting, however, could one explain the results merely based on the effective receptive field size growth with recurrence and depth? The authors say that RF size cannot explain the full effect as the accuracy increases beyond the point where models achieve enough RF size to capture the full maze, however this is confounded with increase in # parameters and capacity as the effective depth is increased. The authors themselves report results from a dilated convolution experiment showing superior performance with larger receptive fields at shallow network depths. Hence, I strongly feel that the similarity observed can be largely reduced to both recurrent and feedforward networks increasing their effective receptive field size with more depth / iterations.\n5. I feel that the paper makes quite a few very strong claims in the Discussion section that aren't fully tested by the Methods. I feel that this section needs to be toned down further to promote more exploration into these areas through future experiments.\n\nReferences\n[1] Liao, Q., & Poggio, T. (2016). Bridging the gaps between residual learning, recurrent neural networks and visual cortex. arXiv preprint arXiv:1604.03640.", "summary_of_the_paper": "The authors explore the similarity between neural networks that accrue information processing via depth and via recurrent iterations. They have conducted several interesting qualitative and quantitative analyses to study the above question and conclude that recurrent processing mimics depth in the experiments they have conducted and draw links to relevant work in computational and experimental neuroscience that also study the role of recurrence in information processing in the cortex.", "summary_of_the_review": "I find the question approached by the authors to be an interesting (although not novel) one and requires studying from various angles in order to shed more light on how recurrence and depth are related. The authors have made a nice effort to perform various analyses (both quantitative and qualitative) to study their hypothesis that recurrence mimics depth of processing. However, I find part of this study (Section 3) to be quite similar to prior work in this area although scaled up with more datasets. In Section 4, I find contradictory claims such as the authors claiming that performance rise not fully explained by receptive field size, and contradicting this statement with models containing dilated convolutions (exponential receptive field growth with depth) that produce large performance gains. The recurrent models studied are quite simplistic and lack several interesting components that have contributed to the expressivity of recurrent processing in the past (such as gating, bypassing, nonlinearities, temporal decay etc.). \n\nAt the current stage, I feel that the readers do not gain much new information about the relationship between depth and recurrence beyond what has been established by prior work. I suggest the authors to experiment with more sophisticated recurrent networks and explore more novel analyses (such as the maze challenge, which I find to be really interesting but needs to be scaled up to explain further similarities beyond receptive field size growth) to understand the relationship between depth and recurrence. This is what I feel is required to truly expand our understanding of (whether, and under what conditions) recurrence mimics depth, and under what conditions / tasks the former is more expressive than the latter and vice versa.\n\n======= Update after author response =======\n\nI have read the authors' response and updated manuscript, after which I am increasing my score to 6. I believe that the updated manuscript's contributions are clear and builds on the interesting work from Liao and Poggio, 2016, adding further information with novel analyses and a new dataset (Maze challenge). I believe this work will spark further discussion and exploration of the similarity between recurrence and depth.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "a6AwoDXHRgr", "reviewer": "Reviewer_opFT", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Strong points\n\n-   The paper is clear and easy to understand\n-   The authors conclude multiple experiments to support their claims\n-   They use multiple datasets, including ImageNet\n-   They use multiple network families, including MLPs, CNNs and ResNets\n-   They also use linear probing of layer-wise representation power, which support their claims\n-   Feature visualisation also supports their finding\n-   The correspondence of feature visualisations, linear predictive power and general trends is very surprising.\n\n# Weaknesses\n\n-   The minimal effect of weight sharing was already shown in some cases [1,2], although this paper analyses it much more thoroughly and shows a broader picture of the effect    \n-   There are some indications that the similarity would break down with deeper networks.\n-   The analysis of feature reuse can be because of different effects (Section 5, first part).\n-   For filter visualisations, it is unclear whether the same neurons are responsible for high and low-level features\n\nPlease find more details below.\n\n# Questions/suggestions\n-   For the maze problem, it is unclear how the performance is measured. Is it pixel-wise, or the whole maze should be correct for the sample to be considered correct?\n-   For the ResNet architectures, the parameters of the batch norm are not shared between the layer. But multiple papers are suggesting that these parameters can have a significant effect on the behaviour of the network [3,4,5]. Therefore, it would be worth considering making a partially shared variant of the batch norm that shares the affine parameters but not the statistics.\n-   The analysis of the activation strengths (Section 5, first part) might not indicate what one might think. If there are biases in the network, the outputs of specific neurons can be high even when “they are not doing anything”. Also, it is questionable whether having high activation most of the time means that they are actually doing computation most of the time. The actual information might be carried on top of a high bias. That said, I don’t have a good suggestion on how this can be adequately measured. Maybe measuring variance between the different samples might be a better metric. Probably visualisation of the same unit in different depths would be informative. Maybe soften the claim in a bit when discussing.\n-   For feature visualisations, it is unclear whether the low-level and high-level features shown in the figures come from the same or different neurons for feature visualisations. How does the presence of the high and low-level features change through the layers? Figure 12 shows high-level features together with low-level ones already at depth 11. It would be nice to visualise the same neuron over “depth”.    \n-   It would be nice to cite [6] when discussing feature visualisations in Section 5.2. It is an outstanding analysis of how networks work.\n    \n# Additional suggestions that could make the paper better\n-   In Figure 2, the gap starts growing noticeably with deeper networks (19 layers on CIFAR-10, EMNIST, ImageNet). It seems like the RNNs overfit sooner than the CNNs (CIFAR-10, EMNIST), which is the opposite that I would expect. It would be worth investigating this effect more.\n-   The ImageNet experiment uses a very small ResNet and with 19 layers. Together with the previous point, I would be curious about the gap with a more realistically-sized ResNet.\n\n[1] Liao et al: Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex (2016)  \n[2] Alexandre Boulch: ShaResNet: reducing residual network parameter number by sharing weights (2017)  \n[3] Revisiting Batch Normalization For Practical Domain Adaptation (2017)\n[4] Frankle at al.: Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs (2020)  \n[5] Kanavati et al.: Partial transfusion: on the expressive influence of trainable batch norm parameters for transfer learning (2021)  \n[6] Cammarata et al.: Thread: Circuits (2020)  ", "summary_of_the_paper": "The paper shows that sharing the weights between inner layers of image classification networks has minimal effect on their performance: they achieve similar accuracy with significantly fewer parameters. This questions the commonly accepted view that later layers of CNNs function as more and more specialised filters. It also suggests that instead of the number of weights, the amount of computation might be the main reason behind the good performance of the deep models. Interestingly the authors use feature visualisation to show that although the weights are shared, the network still learns hierarchical features.", "summary_of_the_review": "I recommend accepting the paper because it highlights the importance of computation depth instead of the number of parameters (which is commonly assumed to be a good indicator of the predictive power). This might motivate further research in models with increased computation depth. Despite the weaknesses, the authors presented enough evidence for supporting the main messages of the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "GWsJDgtyQ1v", "reviewer": "Reviewer_9nUr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n- **Intriguing Topic**. Deep learning papers, lectures, and even books at this point all underline the importance of parameters in the success of deep networks as compositions of learned and nonlinear functions. Scrutinizing this more closely, and questioning if its the sheer number of parameters that matters, is of definite interest. Existing work provides evidence that recurrence across depth/shared weights can suffice, but a paper that thoroughly studies shared/unshared layers could draw more attention to the topic. This is true whether the results reinforce or erode the existing intuitions!\n- **Variety of Measures**. While task accuracy is studied first, and should be, further experiments examine processing through other measurements. For example, depth is contrasted with receptive field size (for mazes, Sec. 4.2), activation magnitudes are compared across layers (Figure 6), and discriminability is checked layer-by-layer (Table 3). Visualization is also used to qualitatively compare filter responses across recurrent and feedforward models (Figures 7 & 8).\n- **Clear Writing**. The introduction is direct and concise in justifying the research question of whether and how models with shared weights differ from models with distinct weights. The setup (Sec. 2) defines its terms and goals, with sufficient detail on the architectures studied to reproduce the experiments. The figures and tables are legible and appropriately captioned.\n\nWeaknesses\n- **Missing Related Work**. The related work is incomplete and does not sufficiently credit existing demonstrations of networks that are recurrent in depth. [A] analyses the effects of sharing weights across depth for convolution and contrasts it with other design choices. [B] defines and experiments on partially recurrent residual networks, and addresses the same point about normalization as Section 3.1 in this work. [C] explores aggressive sharing of weights across depths in residual networks and compares with standard architectures at common scales of model depths and parameters. More afield, but not unrelated, are methods that unroll an optimization process and show that sharing weights can be effective, such as [D]. This work could provide more in-depth analysis, or more up-to-date analysis w.r.t. the models considered, but prior work must be credited, above all in case the earlier work can inspire more analysis.\n- **Narrow Experimental Scope**. The experiments are restricted to relatively shallow nets (with depth of 19 or less) in the visual domain. The chosen architectural constraints, like no intermediate pooling or change in channel dimensions, do conflict with standard networks. Experiments that control for these are valuable, and are included in this work, but the experiments should also examine standard architectures. At the current scope, I am concerned this work runs the risk of confusing matters, while a broader look would be more clarifying for the community. To start, experiments could cover deeper nets, and then go on to look at other tasks like sequence modeling (as done by implicit nets, which also share weights).\n- **Technical Redundancy**. Recurrent nets of this type have already been defined and experimented on, as done by [A, C] and partially [B]. The prior existence of such networks leaves this work without technical novelty. To be clear this is not a fatal flaw, but it does squarely place the value of this work in the breadth and depth of its empirical results.\n- **Task Choice**. The maze task is non-standard, and therefore less informative than any other established task, although it does make some sense. The analysis does support its soundness by showing that there is a distribution of difficulties, that depth helps across them, and that receptive field size does not fully explain away the results. However, all of this could be spared by adopting a task like semantic segmentation on PASCAL VOC or Cityscapes, which are both well-known, and even image-to-image tasks not unlike the maze task.\n\nFor Rebuttal\n- **Related Work**. Please position this work relative to the papers raised in this review, especially [A, B]. Please also situate the contributions of this work w.r.t. the implicit models in work on DEQs and NODEs that show such models can rival explicit models, and even include purely recurrent models (as ablations of their inference optimization) as baselines as in Bai et al. 2020.\n- **More Depth**. Please experiment at depth=50 for the results reported in Figure 2. Please likewise experiment with a standard ResNet-50 architecture and compare it to a recurrent analogue, such as a net that shares the parameters across blocks in each stage (but not the normalization layers, as higlighted by Section 3.1).\n- **Reuse Analysis**. Please discuss other options for measuring reuse, and furthermore interventions to check the analysis. For instance, what if channels are masked out (that is, multiplied by zero) at their least active layer? Severe loss in accuracy from this masking would provide stronger evidence that the reuse is happening and necessary. In the same vein, what does Figure 6 look like for an untrained network? If it differs, this would support the use of the proposed analysis by counting.\n\nMiscellaneous Feedback\n- [clarity] Please define \"depth\" up front, in the introduction. The definition at the beginning of Section 2 is good, but a one sentence summary in the first paragraph or at least on the first page would orient the reader more immediately.\n- [format] Please consider sharing the y axes across the plots in Figure 2 to consistently convey the size of the differences.\n- [method] For normalization (Table 1), consider sharing the parameters—but not the statistics—of the batch norm layers. Sharing the affine parameters would make the model even more recurrent, while still allowing for different statistics across recurrent iterations. In the same vein, consider a comparison with group norm, which is known to work with ResNets, but unlike batch norm has purely local state.\n- [clarity] The neuroscience reference in the discussion comes out of nowhere, and would be more illuminating if its content was concretely connected to the architectural choices and experimental results in this paper.\n- [related work] As a possible counterpoint to theory of layer specialization, you may be interested in reading https://arxiv.org/abs/1605.06431 for its experiments that question the importance of any one layer in a residual network.\n\nReferences\n- A. Understanding Deep Architectures using a Recursive Convolutional Network. Eigen et al. arxiv'13 and ICLRW'14.\n- B. Residual Connections Encourage Iterative Inference. Jastrzębski et al. ICLR'18\n- C. ShaResNet: reducing residual network parameter number by sharing weights. Boulch. arxiv'17 and PRL'18\n- D. Conditional random fields as recurrent neural networks. Zheng et al. CVPR'15", "summary_of_the_paper": "This work examines what constitutes depth for deep networks by examining inference for networks with and without distinct parameters at each layer. Sharing weights across layers defines networks that are recurrent in depth, and still compose the same number of nonlinearities, but reduce the number of parameters. Recurrent architectures of this kind challenge a common intuition about hierarchy in deep representations and so deserve analysis. The experiments in this work carry out controlled comparisons between pairs of networks with shared/unshared parameters, and measure: (1) task accuracy, (2) depth vs. receptive field, (3) activation statistics across layers, (4) discriminability of features across layers, and (5) visualization of filter responses. The tasks studied are image classification, with standard datasets like ImageNet and CIFAR-10, and a custom task concerning images of 2D mazes. The architectures control for differences between the feedforward/unshared and recurrent/shared models, but exclude standard architectures as references. The networks in these experiments are relatively less deep (at 19 or fewer layers), and differ from standard networks (like a ResNet-50) in their lack of pooling and fixed channel dimensions. As suggested by the title, the results for a given depth are indeed quite similar with or without recurrence, at least in most cases for the networks and tasks studied. In this scope, the work convincingly shows that distinct parameters are not necessary, but that does not necessarily generalize to more standard deep networks for vision, nor to other modalities of data, nor to other prediction tasks.", "summary_of_the_review": "It is nice to see this work take a closer look at what exactly differs or not across layers with distinct or shared parameters. This is an important topic to scrutinize since deep models are now so common while sharing weights across depth is still rather rare. The intuitive or \"folkloric\" understanding of deep networks as strictly hierarchical is not beyond empirical analysis, and so in general I welcome this effort. Nevertheless there is more to be done for this work to be sufficiently thorough and informative. Putting aside the missing related work, and considering only the content in the paper, the analysis done here is quite narrow. The only modality considered is vision, the only mainstream task is classification, and standard/unshared models are only studied where they most closely align to recurrent models. Inference is studied in several aspects, like accuracy, feature visualization, and layer-wise prediction, but optimization is not addressed at all. That is, does a recurrent model train differently than its paired feedforward model, even if they ultimately converge to similar accuracy? In summary, I encourage the authors to continue in this direction but with a wider lens, and re-submit with more coverage of different data and tasks, and most of all broaden the bounds of which shared vs. unshared architectures they consider (for instance, deeper nets like ResNet-50 due to its popularity, and perhaps hybrid architectures that still have pooling but share all weights within a \"stage\" at a given resolution, and so on.).\n\n**Final Review** The author response addressed the points requested for rebuttal: inclusion of missing related work, an experiment at greater depth with greater resemblance to a standard ImageNet classifier, and more justification of the layer-wise activation analysis. While I still find the technical and empirical novelty of this work to be lacking, as much or all of it can be reconstructed from the related work, I can see value in the unified experimental setup and the juxtaposition of different results. As this paper could bring more attention to recurrence and depth, as earlier work seems not to have done, it could inform the community. I have therefore raised my score to 6.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper show that in several different neural network  architectures, recurrent  
networks that share parameters over iterations have comparable 
performance and similar features to feed-forward networks of the same
"effective depth".  

Reviewers initially had some reservations about novelty and 
generalizability to deeper SOTA networks.  These were successfully
addressed by the authors and all reviewers feel the paper is above the
bar due to the importance of the area, and that this paper brings
together many important insights that, while many may have been known 
before, had not previously been all brought together before.  The maze 
task was also considered a useful task for the field.  I agree that
the paper makes a worthwhile contribution and am in favor of 
acceptance.
Decision: Accept (Poster)