Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Post rebuttal
====

Thank the authors for the responses. Since the responses addressed my original concerns, I raised my score. However, I agree with reviewer i4TZ and qY8f that the theoretical and empirical studies are still somewhat simple, and could still be improved.



This paper considers how to obtain a parametric approximation to the empirical function space distance (FSD) of two networks, i.e., $\mathbb E_{p_{emp}}[\rho(f(x, \theta_0), f(x, \theta_1))]$. The key assumption here is we cannot store the training data $p_{emp}$, as it is a  infinite stream. On the other hand, the overhead of storing the model is acceptable.

The paper takes a linearization approach, BGLN, which approximates the activation of the second network $\theta_1$ with a first-order Taylor's expansion at the activation of the first network $\theta_0$. By the linearity of expectation, the FSD can be written with the activation gradient's moment: $\mathbb E_{p_{emp}}[\phi^\prime(a_0)]$. By computing and storing such moments (i.e., sufficient statistics) in advance, the FSD can be computed without accessing the data distribution $p_{emp}$.

The proposed BGLN is empirically shown to be more accurate than competitive approaches, such as second-order approximation to the FSD on the parameter space or NTK-based approximation, which again linearizes on the parameter space. The effectiveness of the proposed approach is shown on continual learning and influence function estimation tasks.
strength_and_weaknesses: Strength:
- The proposed approach is quite simple and accurate.

Weaknesses:
- The presentation needs to be involved. The current paper has some logical discontinuities. For example, at the end of Sec. 3.1, there misses a part on how to compute the FSD with the activation moments. There should be some derivations and pseudocode for that. Otherwise, the discussions in Sec. 3.2 and Sec. 3.3 seem somewhat random.

Sec. 2 might also be somewhat disruptive for the readers might not understand what is parametric and nonparametric approaches here. It might be better to put the concrete formulation of continual learning in Sec. 5.1 forward to Sec. 2 to act as a motivating example.

There is some typos: for example, in Eq. (3), some (l) should be in fact (l-1).

- The advantage of linearizing the activation over competitive approaches should be discussed in more detail. Currently, there is only empirical evidences of the accuracy of BGLN, which is not strong enough. The paper would be more solid if there is some error / bias analysis and comparison. 
clarity,_quality,_novelty_and_reproducibility: Clarity: the clarity needs to be improved, as discussed in weaknesses.

Quality: the quality of the BGLN algorithm is fine. However, the advantage of competitive approaches should be make more clearly.

Novelty: as far as I know, the idea of approximating FSD with moments is novel.

Reproducibility: code is not submitted. There is a pseudocode in the supplementary material. I believe the reproducibility is generally ok.
summary_of_the_review: The paper proposes a reasonable solution to a practical problem of function space distance estimation. However, presentation and relationships with existing works should be improved. The significance of the paper is not clear in its current form.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a method to estimate the function space distance (FSD) of two networks. This method relies on transforming the ReLU layers of the network into Bernoulli stochastic gates, and measuring the distance between the networks while sampling from these gating variables. Using this method, the authors are able to estimate the function distance between two networks, and can use this technique for tasks where this function distance is required, such as in continual learning.
strength_and_weaknesses: Strengths:

- The method proposed is very easy to understand and can be easily implemented. The authors measure function distance by essentially substituting the ReLU functions of the network with stochastic gating variables. Implicitly, this process samples from the various linear regions of the network, making the calculation of distance between them straightforward.

- The authors examine an interesting setting for the application of their function space distance approximation in continual learning. In this setting, the authors use their FSD approximation to train models that sequentially learn new tasks, while aiming to retain accuracy in the previous ones.

Weaknesses:

- One major weakness of the proposed FSD approximation technique is that there is not sufficient evidence that it is a good approximation of the distance between two networks. In order to prove this theoretically, there would need to be proofs of bounds for the FSD estimated via the proposed method compared to the true FSD, or at the very least a simple toy example demonstrating the FSD estimated on a given network, on a toy dataset. To prove this experimentally, the authors would need to include further comparisons like those included in Figure 3a. Currently, this comparison between estimated and true FSD is only done on an MNIST variant, so doing so over more complicated datasets would be helpful to properly evaluate how well the proposed method performs. Including either of the two would greatly improve the paper.

- In a similar vein, the experimental results on continual learning are not convincing enough for the capabilities of the proposed method. More specifically, the proposed methods only show performance benefits in the case of MNIST variants, and not on the CIFAR100 continual learning benchmark. This, combined with the previous result, makes me wonder whether the approximation derived from the proposed method is useful for the desired continual learning task. I believe the authors would need to further analyze why their method underperforms in the CIFAR100 variant.

- Finally, the authors state that their method can approximate the FSD between two networks without expensive storage requirements, compared to other nonparametric methods. I believe that these storage comparisons should be included (for example, in Table 1), in order to fully understand the benefits of the proposed method, from a memory requirement perspective.

Update post rebuttal: The authors have included extra experiments that somewhat alleviate my concerns regarding the quality of the FSD approximation. The proposed method seems to outperform NTK in Split-CIFAR100, as far as FSD approximation is concerned. Moreover, the authors have included explicit memory requirements for their proposed methods.

However, the experimental part is still inconclusive, in the sense that while there is an experimental indication that the proposed method performs well with respect to estimating the FSD, it lags behind NTK on the same problem. Furthermore, while there is theoretical motivation with respect to the gating units used, it hinges on these being independent (as pointed out by another reviewer), which seems to be a very limiting assumption. Including a simple toy example where this choice of gating is the best one would be a good addition to the paper.

clarity,_quality,_novelty_and_reproducibility: Overall, the paper is clear. I would suggest that the authors improve upon the following two points:

- In Figure 3a, a different shape should be used for each method, as otherwise it is very difficult to read.

- I would be grateful if more details could be given in Table 1 about each of the proposed methods, and especially about the class-conditioned (CW) variants.

Regarding the novelty, I think that the idea behind the work is interesting, but the work itself does not make a convincing enough argument about its usefulness.

Regarding reproducibility, the authors plan to release their code, and provide further training details in the Appendix.
summary_of_the_review: I think this work presents an interesting idea, but given that there is not sufficient evidence that the FSD approximation proposed performs well in general, and the main results do not make a convincing argument for the usefulness of the method (other than memory requirements, which are not fully elaborated on), I believe that it needs to be improved.

Update post-rebuttal: I have raised my score following the authors' responses, as some of my concerns have been alleviated. I believe that the paper can still be improved, by further examining why the proposed method lags behind NTK in performance (despite experimentally seeming to better approximate the FSD) and including further theoretical justification on why the gating units chosen are a good choice.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: **Post rebuttal summary**

I thank the authors for their thoughtful responses. This well-written paper tackles the important problem of FSD estimation. The methodology introduced is not accompanied by a sufficiently convincing theoretical or empirical analysis. The updated empirical analysis hints that this method might be worth further exploring more thoroughly beyond isolated examples. I encourage the authors to submit an improved version of the paper in future.

======

Summarising the properties of a dataset (perhaps with respect to a model) finds useful application in problems such as continual learning, domain adaptation, semi-supervised learning, influence-function estimation and even regularisation. In particular, it may be used to avoid catastrophic forgetting. The authors use function space distance (FSD) for this purpose, applied to ReLU neural networks. Using a certain linear approximation and stochastic Bernoulli approximation of the FSD, they derive an efficient algorithm for approximating the FSD. They find their algorithm to be competitive with SOTA on some toy examples.
strength_and_weaknesses: **Strengths:**
- The problem considered is a worthy one.
- The paper is very clearly presented, well-written, and conveys and elegant message.

**Weaknesses:**
- Section 3.1. What does linearisation (5-10) represent, and where does it come from? It is not a first-order Taylor in the inputs, since the Taylor series of the ReLU does not converge to the ReLU. 
- "The upshot of our approximation is that, in the ReLU case, it only depends on the training data through the signs of all the activations (since passing layer inputs through $\phi$ and $\phi'$ both result in a multiplication by 0 or 1)". I am not understanding something here. I thought it would depend on the sign of the *preactivations* $s$. Additionally, passing through $\phi$ involves multiplication by zero or one, but passing through $\phi'$ involves taking the sign of the input.
- Beyond computational advantage, why is the step function approximated using an independent Bernoulli random variable? This changes the dynamic. Does this modification preserve the original formulation in any sense?
- Given that this paper contains no mathematical theory, I would have hoped to see more realistic datasets for baselines beyond MNIST, CIFAR, Wine, Housing, etc. 
clarity,_quality,_novelty_and_reproducibility: The paper is very well written and easy to follow. I am not convinced there is enough mathematical novelty in the solution provided, since it makes two seemingly arbitrary choices of linearisation and independent Bernoullis without any justification or analysis. The experiments appear to be adequately reproducible. 
summary_of_the_review: While well-written, I am not convinced that this paper contains enough insight to be published. 

It would be okay for the benchmark datasets to be somewhat toy *if* the mathematical formulation, motivation and analysis were complete. At the moment, I can see two unprincipled approximations: (1) The "linearisation" in input space, and (2) the replacement of a step function with an independent(!) Bernoulli random variable. Alternatively, if the benchmarks were more convincing, the mathematical analysis could be acceptably less complete.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors propose a way to approximate the function space distance for certain broad classes of neural networks. This methods is less computationally expensive and more precise than alternatives based on the Taylor decomposition of the FSD, and are particularly suited for continual learning.
strength_and_weaknesses: STRENGTHS:
* The paper studies an important problem. Catastrophic forgetting is a well-known problem in the field, and the method has applications to other areas as well.
* The algorithmic ideas proposed in the work are novel, technically sound, and non-trivial. The depth of the technical results and their derivations is significant.
* The experiments are extensive and look at a wide range of interesting settings: influence functions, continual learning, etc.

WEAKNESSES:
* I think some of the tasks used some experiments can be made more realistic. The influence functions experiments look at a few toy UCI datasets. I'm not an expert in continual learning, but the main datasets seem versions of MNIST as well.
* I think the presentation can be further improved by better explaining the limitations of the method relative to existing approaches. Additionally, the authors discuss extensions to non-fully-connected architectures in the paper, but I feel like most of the details are in the appendix. A clear explanation of the class of models to which this does or does not apply would help.
clarity,_quality,_novelty_and_reproducibility: The paper is well written. However, a more clear and centralized definition of the method, its assumptions, and its limitations would be useful. The results seem to be novel, high quality, and reproducible.
summary_of_the_review: I can see this paper being a useful addition to this line of literature.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes a new method to estimate the functional distance between two networks, a problem of great interest in the transfer / few-shot learning literature.

All reviewers appreciated the idea and its computational qualities. They had some doubts, however, on the empirical performance and the approximations made. We had further interactions with the reviewers as the consensus was that, even though the paper shows some promise, it could still be improved.

Given the total volume of papers published, I am favoring works that address these questions completely, given that the partial work can be advertised anyway and that a complete analysis will make it easier for researchers to fully understand the landscape.

As such, I vote for rejection.
justification_for_why_not_higher_score: There are unjustified approximations and the empirical performance improvement is not clear compared to existing methods.
justification_for_why_not_lower_score: This is a new approach for an important problem.
Decision: Reject