Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The author proposes modulating learning using masking some learning updates. The value of the mask is generated using a simple LTU unit with zero threshold. The mask is learned using the first-order approximation to MAML (FOMAML). The authors use the straight-through estimator because LTU is not differentiable. 

The paper claims that (1) update sparsity emerges through this meta-learning process, (2) different layers have different level of sparsity, and the level of sparsity depends on the problem, (3) binary sparsity performs better than simply meta-learning the step-sizes, (4) sparsity improves the meta-testing performance on unseen task distributions and (5) sparsity improves continual learning performance for online meta-learning. 
main_review: The paper is clearly written, and empirically tests the proposed approach on multiple environments and settings. The central idea --- meta-learning to modulate learning in different parts of the network --- is sound, and generalizes both MAML and ANIL.

It's not clear to me why a binary mask learned using a biased method (Straight-through estimator) would be better than simple meta-learning step-sizes. I would expect both to do well, and it's unclear to me how much of this difference would actually generalize to a bigger set of problems. 

One issue I have with the paper is that it ignores all prior work done on meta-learning step-sizes that wasn't done in the past 5 years. Two relevant pieces of work (more thoroughly discussed in Limitations sections) are IDBD (Sutton, 1992) and SMD [2]. There has been follow-up work to SMD as well that the authors should include in discussion. I'm happy to recommend this paper for acceptance once a discussion on prior work and an exponentiated version of step-size has been added as a baseline. 


Sutton, R. S. (1992, July). Adapting bias by gradient descent: An incremental version of delta-bar-delta. In AAAI (pp. 171-176).
[2] Local Gain Adaptation in Stochastic Gradient Descent∗ Nicol N. Schraudolph
limitations_and_societal_impact: Limitations: 
1. The paper misses prior work on meta-learning for step-sizes. Two seminar works that should be discussion are IDBD (Sutton, 1992) and SMD [2].
1. Additionally, Instead of using ReLU to make step-sizes positive in Meta-Sgd, the paper should use an exponential form for the step-size as used by IDBD (Sutton, 1992)  and SMD[2]. This assures that the step-sizes are always positive, and differentiable. I would expect the exponential form to fix most of the issues related to dead ReLU masks and result in a more robust system. 


Sutton, R. S. (1992, July). Adapting bias by gradient descent: An incremental version of delta-bar-delta. In AAAI (pp. 171-176).
[2] Local Gain Adaptation in Stochastic Gradient Descent∗ Nicol N. Schraudolph
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper proposes a strategy called Sparse-MAML, which extends the MAML algorithm to learn sparse updates.

The Sparse-MAML algorithm is applied in two settings: few-shot learning with MiniImageNet, and continual learning with MNIST rotations and permutations. The results show a mixed performance – some gains, some losses, compared to different baselines. The authors also provide many ablation studies and analysis which show that Sparse-MAML does exhibit the sparsity in the gradient updates of MAML.

main_review: Disclaimer: I am not familiar with the benchmarks presented in the paper, namely miniImageNet and MNIST rotations/permutations. Therefore, I cannot appreciate the experimental results in the paper. I can only provide my assessment of the paper based on comparing the numbers provided therein, without much further insight.

On the methodology side, sparsity is a preferred property in many deep learning settings, so the motivation of learning sparse updates in this paper is reasonable. The formulation of Sparse-MAML, which uses a ReLU vector to shut off the gradient updates to various parameters is simple and not computationally intensive. On top of this, the paper is clearly written and easy to follow.

On the experimental side, the authors do provide analysis which confirms that Sparse-MAML indeed leads to sparser updates to the model parameters. My main concern is the seemingly weak performance of the Sparse-MAML in terms of its resulting model’s qualities. For instance, in Table 1, Sparse-MAML loses to MT-Net and Meta-SGD, and in Table 2, Sparse-MAML sometimes loses to ANIL and BOIL. While the performance of Sparse-MAML+ is sometimes better, Sparse-MAML+ seems to be less sparse than Sparse-MAML (as the authors explain that “gradient masks are generated from a low-dimensional Gaussian vector”). This raises the question whether sparsity should be enforced to the extent of Sparse-MAML.

limitations_and_societal_impact: I cannot comment since I have almost no insight about the experiments in the paper.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes a sparse meta gradient update method during MAML by adopting binary masks that modulate the gradient during the inner update. The masks depend on the angle-similarity between the averaged inner loss gradients and outer loss gradients, where two gradients are opposite directions, then mask to be zero, otherwise one. The method shows better performance compared to MAML and existing meta-learning baselines using the modulated learning rate strategies. Furthermore, the method can be combined with meta continual learning methods, such as La-MAML, and shows superior performances.
main_review: Overall, the paper is well written and easy to follow. 
The methodology is simple and reasonable, several ablation studies and analyses are helpful for a better understanding of the proposed method. Although the performance gain compared to recent baselines are not much impressive and sometimes looks marginal, I think this is good work. 

Questions and comments are as follows:

- In Figure 1, performance is also robust across the different sparsity initializations?

- Recent Continual Learning literature deploy CL model on modern deep neural networks, such as ResNet 18 or MobileNet v2. Even La-MAML validates their method on simple CNN, I strongly recommend validating the proposed method on modern nets.

- In Figure 4, why does the sparsity decrease when layer sizes increase? As the network has additional neurons, it looks natural that there are more redundant/unnecessary updates (sparsity can be $\uparrow$).

- Please describe the merits/drawbacks of meta-learning-based continual learning, compared to other CL directions (e.g., regularization-based, rehearsal-based, architecture-based).
limitations_and_societal_impact: The authors addressed.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors introduce a new version of the MAML algorithm which has learnable sparsity on the weights. Using a linear, first order approximation to the masking gradients, they present a tractable algorithm for meta-learning the sparsification of the gradients along with the rest of the parameters. They show small improvements on relevant fewshot learning and continual learning tasks.
main_review: I appreciated the exposition which described the sparse MAML algorithm; it gave a clear path to converting understanding of MAML to an understanding of sparse MAML, complete with an interpretation of the update steps. In fact, this interpretation helped me understand why the gradient approximation may be reasonable, and under what conditions it might fail.

In the fewshot and continual learning settings, the sparse MAML algorithm seems to generally perform better than vanilla MAML, and is competitive with other SOTA aproaches. In addition, the analysis of the learning dynamics of sparsification as well as the layer-wise analysis of the sparsification is interesting.

One concern is the fact that the final sparsification factors are not large (2x - 4x gradient reduction) - which to me suggests that the sparse MAML may be a useful final step after all other optimizations have been done, but will never provide qualitatively different results. Researchers may be better served by tuning architecture hyperparameters over tuning sparsity. A related concern is how the sparsifying method fares when pushed into a larger input dimension/dataset/model size regime. This concern was noted by the authors in their conclusion, but may limit the overall impact of the work.

One interesting thing to know would be the rate of reversion of the masks - that is, how often is a gradient turned on after it has been turned off? If reversion is rare, then it may have implications for the efficiency of the algorithm, as well as the general learning dynamics it induces.

Update: The authors addressed many of my concerns. They showed that the sparsification has to be learned simultaneously with the overall training. They also provided an interesting analysis of the rate of reversion of masks (most don't revert, but a non-trivial fraction do). Given these strong additions, I am changing my review from a 5 to a 6.
limitations_and_societal_impact: Adequately addressed.
review_title: null

Meta-review: This paper proposes an extension to MAML that also meta-learns weight update masks, and analyses the resulting learning outcomes under a variety of settings. Both conventional few-shot learning and continual learning benchmarks are considered. Although some work has been done in this direction such as MT-Net, reviewers are generally positive  after rebuttal phase and find this to be worthwhile contribution. Overall I recommend acceptance.
Decision: Accept (Poster)