Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: ## Paper Summary

This paper studies the correspondence between residual networks and iterative algorithms that repeat computations and converge to a solution. The authors suggest that residual networks can in principle implement such iterative algorithms and experimentally show that networks trained in practice do not naturally learn them. They also define three indices to quantify the degree to which a ResNet shows properties of iterative algorithms. Finally, they show that while soft gradient coupling across layers within stages can ensure that learned ResNets behave more like iterative algorithms, this does not appear to provide a useful inductive bias for image classification tasks.

## Strengths

Studying the nature of programs that are learned by neural networks of various architectures is an interesting and important research problem. This paper makes a contribution to it by examining the extent to which ResNets implement algorithms similar to iterative solvers.

The authors define numerical indices to formalize the criteria for "iterative-ness" that they are looking for, which are useful for comparisons.

The paper contains a negative result about the utility of forcing iterative behavior on ResNets using the proposed gradient coupling trick. This negative result may be useful to researchers interested in similar ideas in the future.

## Weaknesses

The motivation for this paper is somewhat weak, or at least weakly justified. The authors define an iterative method/algorithm as one that uses repeated iterations and convergences to a solution. It is then hypothesized that such behavior might be a good inductive bias for neural networks, but it is not discussed why this might be expected. After all, iterative algorithms are designed to converge after a (non-fixed) number of steps, and neural nets are not. I think that either the motivation should be justified better, or it is simply a question without a strong motivation (this doesn't necessarily make it unimportant, just less important).

Moreover, if we'd like the outputs of the neural network to be "stable" for reasons other than metrics like accuracy, there are certain methods and lines of research on this subject, such as Ciccone et al. cited by the authors. Those works already claim that computations learned by ResNets are not stable, and suggest methods to make them so. Doesn't that make the question investigated (whether ResNets learn iterative convergent behavior) in this paper somewhat redundant?

Finally, the negative experimental results are interesting but I think they need to be stronger to be convince a reader that this is a result that can be expected to generalize. Due to the simplicity of the datasets (and no confidence intervals on the numbers in Table 1), evidence for the negative impact of encouraging iterative convergent behavior on performance is still preliminary. More datasets and tasks of higher complexity will certainly help here.

## Review Summary

Since the paper lacks strong motivations, clear significance and highly convincing results, I am currently unable to recommend an acceptance. If the authors can elaborate on their motivations and discuss them in light of related work as mentioned above, I am willing to reconsider my score. 

## After Author Response

I think the changes to the paper have improved it. In particular, the reference to Spoerer et al. gives more weight to the motivations of the paper. I'm increasing my score slightly as a result.
However, the relation to prior work remains hazy. The model of Ciccone et al., for example, has shared weights, stability over increased depth, and still performs as well as ResNets generally. Doesn't this go against the conclusions of this paper?

review_title: Concerns about motivations and results

final_score: rating: 5: Marginally below acceptance threshold
review_content: 
Summary:

This paper investigates the extent to which the computations implemented by an optimized ResNet resemble those of a recurrent network. They develop new tools to study this question, and find evidence that ResNet performance is *hurt* when it is forced to act like a specific kind of RNN.

Strengths:

I am excited that the authors are re-examining the assumptions of now classic work likening RNNs to ResNets. They develop elegant tools for continuously transforming between the two, and compare performance on several image classification tasks.

Weaknesses:

I'm not totally convinced by the author's pitch for inverse problems. This idea of the visual system acting as a generative model is still far from worked out, and I'd prefer the authors hedge their language on it. There's also other tasks that are more closely linked to recurrent processing than the ones studied here. For example, I recommend the authors experiment with Pathfinder or cABC of [1], which are solved in fewer samples by recurrent networks vs. feedforward models. This would allow you to plot the amount of gradient coupling on one axis, and the number of samples need to solve (e.g.) Pathfinder on the other axis, which I think would be very elegant.

I appreciate the authors laying out their hypotheses like they did, but I found the language to be indirect. Is there a simpler way to motivate these hypotheses?

Figure 1 is difficult to understand. Are you plotting activities against each other? What do the different dots represent for the feedforward model? Is the interaction in (d) meaningful or is this just by happenstance, and the divergence from x is the meaningful quality. 

Why ResNet-104?

When dropping ResNet blocks, how do you deal with the subsampling between layers? When you drop out the first layer do you also drop out the max pooling at that layer? Is there any chance that these distinctions in computations and resolutions between blocks that you're dropping could bias your observed results?

Regarding the Divergence index, the authors should review [2]. I don't think a high divergence index necessarily means that the ResNet isn't learning the function of an RNN — only that the learned function is not stable, which makes sense given ResNet hyperparams. This paper suggests that if you change the model nonlinearities to globally contractive ones like tanh or sigmoid (or use their algorithm) you'll control this problem.

The gradient coupling is forcing a fixed combination between the gradients of successive layers. But gated RNNs are standard for recurrent vision models, and these do not have such a constraint. Is it possible that the ResNet without shared weights is learning a the function of a gated RNN rather than the vanilla RNNs that you're comparing to here?

"Our findings also suggest, however, that deep feedforward computations may not be characterized as iterative refinement on a latent representation, but, at most, as non-iterative refinement on this representation." How do you refine non-iteratively/incrementally? More generally, I felt like the authors overloaded "iterative" and the manuscript would benefit from a more careful treatment of the exact computations they're /referring to. Give concrete examples.

Are you comparing ResNets to RNNs or iterative algorithms (as are alluded to in the intro)? I am confused by the motivation here, which changes from paragraph to paragraph.

[1] Kim et al. Disentangling neural mechanisms for perceptual grouping. ICLR 2020.
[2] Linsley et al. Stable and expressive recurrent vision models. NeurIPS 2020.
review_title: Review

final_score: rating: 5: Marginally below acceptance threshold
review_content: > Summary: This paper investigates the relationship between deep ResNets and (implicitly) iterative computations. The authors introduce two main hypotheses that are at the core of the investigation: 1) whether the iterative inductive bias improves ResNet performance; and 2) whether recurrent ResNets are more parameter-efficient. The paper also proposes three metrics for studying the convergence and divergence behaviors of these networks in order to investigate this matter.

----------------

Post-rebuttal thoughts:

I would like to thank the authors for their detailed response and the revisions made to the paper. I'm updating my score to 5 as part of my concerns are satisfactorily addressed, and I wished I could have more opportunities to discuss with the authors on their response. In general, my opinion is that the authors have introduced too many "artificial" components to the study (e.g., soft gradient coupling, the convergence/divergence indices) that make me slightly dubious of how generalizable this characterization is. For example, as the authors indicated, spectral normalization creates a different phenomenon (at a cost of worse performance), but with no change to the structure itself (so unlike the soft gradient coupling), a different phenomenon could be challenging the conclusion of the paper.

My suggestion would be that the authors delve deeper into the observations here and better integrate the revisions with their original approach (e.g., the high-dimensional discussion; the spectral normalization discussion, etc.)

----------------
  
- I feel that in the rebuttal phase the authors made certain important new edits to the paper (e.g., 

My general opinion is that this paper investigates an interesting direction on the learning behavior of ResNets, but is still not quite ready for publication in a venue like ICLR. There is an obvious gap in related work (see my detailed comment below) on implicit deep networks; moreover, the definition of the various indices (e.g., convergence index) is also rather confusing to me. The empirical results are not strong enough evidence, in my view, to make most of the claims conclusively. I also have some doubts on the motivation for the methodology that the authors are using. 

Pros:

1. Interesting direction; as the author shows, the ResNet architecture itself is expressive enough for implementing iterative computations/algorithms. So it is worthwhile to study its behavior along this trail.
2. The paper is overall written in a clear manner and the author explained their methodology well.

Cons:

1. Many arguments are too hand-wavy and I don't particularly find the metrics the authors define to analyze convergence/divergence particularly convincing. (See my comment below)
2. The experimental setup is mostly on small scales.
3. Even with the small scale setups, the experimental results don't seem conclusive enough (at least to me) to draw the conclusion that the authors were trying to claim. The verification of hypothesis 2 is especially hasty.
4. The motivation behind the soft gradient coupling is not clear to me.
5. There is a clear missing gap in the related work that I think the authors should pay attention to.

--------------------------------------

I will expand on some of the Cons above, and provide the following detailed comments/questions:

1. Again, I think it is interesting to investigate the relationship between ResNets and iterative computations. But besides the canonical, plain unrolling of the layers that the authors have looked at, *implicit models* (i.e., models that study the continuous dynamics of a layer $f$) like Neural ODEs [1] and Deep Equilibrium Models [2] (there's a ResNet version of it) are both also looking at compact recurrent networks. In particular, the deep equilibrium models especially targets the convergence (i.e., the "fixed point" of the layer), and seems to demonstrate state-of-the-art level performances. In contrast to what the authors provided in the last paragraph of Section 1, I would therefore argue (based on Neural ODEs and deep equilibrium nets) that recurrence does offer some notable advantages like constant memory cost and analytical gradients. The other related thread of work is simply the classical recurrent backprop (RBP) theories, which study the convergence of recurrent networks and how one can leverage such property for the backward pass of these networks. I found the current version of the paper did not discuss either aspect of this, which I believe is important literature that actually is on the opposite side (partially) of what the authors are trying to claim. 

2. There are actually many ways that I can think of to make recurrent residual blocks converge when you infinitely repeat it. For example, with spectral normalization [3], we can simply make the Jacobian of the block have an operator norm $<1$. Then Banach fixed point theorem will guarantee convergence. Other methods are also possible (e.g., via a provably convergent optimization perspective). These are not discussed in the paper (nor are they the main focus, I guess), but this doesn't mean that ResNets do not converge in general. The authors argue that "some balance between feedforward and iterative computations might have been learned by the ResNets", but there is actually a lot of noise in the analysis... for example, the networks could be overfitting, etc. The point is, as long as you regularize the model in that direction, the ResNets could still converge.

3. One main problem that I found about this paper is its definition of the convergence/divergence indices. The "convergence" concept in this paper is constrained to look at the accuracy convergence, by which the authors look at the inverse of the AUC of the classification rate curve. But given the nature of softmax and classification task itself, I don't think a convergence in accuracy is a good "index" for measuring convergence of an architecture, which Section 3.1 looks at (for $\hat{z}_i^{(t)}$). For example, softmax is constant up to a shift of constant. And for classification of, let's say 10 objects $(x_1, \dots, x_{10})$, getting $x_1, \dots, x_5$ correct is still different from getting $x_6, \dots, x_{10}$ correct, even though they both have "50% accuracy". The paper investigates CIFAR-10, where one can achieve >94% accuracy, but in cases like ImageNet where 70% accuracy is normal, these two 50% are certainly non-convergent to me. Also, I'm assuming the entire Figure 1 is on the simple 2-dimensional linear task? Does the phenomenon in Figure 1d repeat in high dimensionality? If so, what does it look like? (My experience with this suggests that if you keep stacking the same block, the activations will eventually oscillate, if not converge, but it could differ by initialization.)

4. Some arguments are also a bit handwavy to me and I'd appreciate if the authors can expand on them. For example, in Section 3.2, the paper claims "in contrast, the skip connections encourage a ResNet to use the same representational format across blocks... [and] are therefore better aligned with the final decoder". As another example, the paper claims ResNets learn a balance between "feedforward and iterative computations". These are all intuitively reasonable arguments indeed, but considering that this is an empirical study paper, I think actually verifying these would make the paper stronger.

5. About the soft gradient coupling, doesn't this simply mix the gradients and inject more stochasticity to them? In general, would you expect (when $0 < \lambda < 1$) that just like in typical SGD, this stochasticity will be averaged out by the optimization procedure of deep networks? Since the $\tilde{\Delta}_t$ no longer fully reflect the mini-batch gradient descent direction, have you checked how the block parameters within the same stage gradually deviate from one another as you optimize the network (e.g., how does the standard deviation of $\mathbf{W}_l^{(s)}$ over all layer $l$'s in the same $s$ change over training iterations? Do they deviate or stay around? If these weights are eventually still different, why can one still consider them to be "similar" (other than the RI metric, which I find to be a debatable metric given the #3 above...)? 

6. For the EPC, have you computed the EPC of an ordinary ResNet and a purely recurrent ResNet? How do their EPC look like when compared to the soft gradient coupled ResNets (e.g., $\lambda=0.5$)?

7. In Section 5.3, the paper claims that "if this is the case, we would expect soft gradient coupling to find such a solution." Why? And isn't a soft gradient coupled ResNet still a non-recurrent ResNet (in the sense that you can't simply unroll a single layer to get the output; you still need to store all parameters of the network, rather than only a single layer of it)?

--------------------------------------------

I look forward to the authors' response on my questions/comments above. I'm happy to consider adjusting my score accordingly.


[1] https://arxiv.org/abs/1806.07366
[2] https://arxiv.org/abs/1909.01377
[3] https://arxiv.org/abs/1802.05957
review_title: Interesting investigation but I found the metrics and conclusion not quite convincing

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper presents an empirical study that characterizes and quantifies the implicitly recurrent nature of residual networks (ResNets). A ResNet can be construed as a general formulation of a recurrent neural network (RNN) unfolded for a fixed number of time steps. In particular, the authors propose "soft gradient coupling," a novel way to control the degree of weight-sharing between the different residual blocks. This gives them the ability to smoothly interpolate between a "no weight sharing" scenario to a "full weight sharing scenario." Soft gradient coupling imparts the ability to share similar "computations" without necessarily sharing the same weights. They introduce metrics such as convergence, recurrence, divergence indices, and an effective parameter count to quantify "iterative" behavior numerically. Finally, they also test the impact of "iterative" computations in a ResNet trained for non-trivial visual recognition tasks.

Pros:
The problem that the authors tackle is undoubtedly interesting and useful. This is particularly true in light of a growing literature analyzing ResNets as discretized dynamical systems. The demonstration that ResNets can express iterative algorithms but do not learn such algorithms by default is intuitive and powerful. The authors use a simple toy example (a linear function) to articulate the desiderata and then work with larger datasets.

Though the manipulations to test iterative computations and implicit recurrence (early read out to determine convergence; residual block drop outs to determine recurrence; and repeated application of residual blocks to determine divergence) are not entirely novel, they are applied quite aptly. The most salient observation is that of divergence and is reminiscent of stability analysis of RNNs for vision. The notion that ResNets learn to tradeoff (and balance) feedforward vs. iterative computations is an interesting proposal. 

The proposed "soft gradient coupling" scheme allows for different residual blocks to implement similar "computations" without necessarily sharing weights or changing the primary optimization problem. This is an interesting suggestion. 

This paper also presents fairly extensive numerical experiments.

Cons:
The strong conclusion that ResNets do not benefit from recurrence regularization is premature, given the current set of experiments presented in this manuscript. As the authors themselves point out, "iterative computation" is an inductive bias. However, there is little reason to believe that this inductive bias is the right one for a classification problem. Have the authors tried to consider problems other than image classification? For instance, there has been recent literature on the relevance of iterative computation (visual routines) for contour detection and segmentation problems. Moreover, "accuracy" is not the only way to quantify the benefit. Have the authors tried to measure sample efficiency? i.e., can a ResNet employing iterative computations learn from fewer training samples than a non-iterative ResNet?

How does the performance benchmarking of a "fully recurrent" ResNet compare to a comparable-sized LSTM/GRU trained on this task? Or even a weight-shared ResNet? These comparisons seem to be necessary to discern if the soft gradient coupling is introducing other artificial biases.

It is unclear why the authors believed that a high degree of soft-gradient coupling would help with the divergence issue in the first place. Implementing the same computation repeatedly only converges (and stays there) given certain other properties of the transformation function applied (for instance, the spectral radius of each Residual block's Jacobian). There is quite a bit of theoretical/empirical work in the literature in this regard.

The manuscript would benefit from some discussion on theoretical results from the RNN literature that outline necessary and sufficient conditions for the forward pass of RNNs to behave like convergent dynamical systems. Given this paper's focus on iterations, convergence, and divergence, this body of work seems relevant.

Minor:
(Fig. 3a) The recurrence index was normalized, yet there are points with a recurrence index greater than 1. Is there any explanation for this?

The recurrence index measure also does not seem to add much value. (Fig 2a,b; second to left panel)

Are the values reported in Table 1., for example, point estimates? Did the authors estimate some confidence intervals on these values by running a few repeats of the experiments?

Clarity: (Pg. 2) "Encouraging iterative behavior in this way therefore does not improve the inductive bias": Is not iterative behavior *the* inductive bias?

(Fig. 3a) There must be a discussion on the non-monotonicity of these curves (especially convergence/divergence).

The paper can do with a through reformatting of the reference list to make all entries consistent in citation style (for ex: including URLs, DOIs, proper and consistent journal/conference abbreviations, etc.)
review_title: Review for "Evidence against implicitly recurrent computations in residual neural networks"

Meta-review: This work provides evidence against the hypothesis that ResNets implement iterative inference, or that iterative convergent computation is a good inductive bias to have in these models. The reviewers indicate that they think this hypothesis is interesting and relevant to the ICLR community, but they do not find the current work sufficiently convincing. Both theoretically and experimentally the paper does not fully demonstrate the claim that iterative inference is not useful in ResNets, and the reviewers are unanimous in their recommendation to reject the paper until the evidence for this claim is strengthened.

Decision: Reject