Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes to use meta-learning to learn an optimizer that automatically seeks wide local minima without the need to explicitly compute the sharpness measure on the fly. When training the optimizer, the parameters of the optimizer are updated in such a way that the training loss and the sharpness measure at the end of the updated trajectory are minimized. Theoretical results were provided for the generalization guarantees of the learned optimizer under some technical conditions. Finally, the experimental results demonstrate that the sharpness-aware optimizer outperforms baselines on simple models.
main_review: **Pros**

- The proposed method demonstrates large performance over the baseline L2O approaches.
- The presentation of the method is clear and the paper is well-written.

**Cons**

- The method seems difficult to scale to larger models, considering the complexity of training the learned optimizer. While this may be ok when compared to other L2O (which are equally inefficient), it makes less sense when there already exists a much better regularizer [1] which not only applies to truly state-of-the-art models but also does not require an expensive meta training phase for the optimizer.
- In a similar vein, it’s unclear how or can the proposed model generalize to larger and more realistic models. “How” as in whether the optimizer learned on the smaller model will find wide minima for a larger model and “can” as in whether there exists reasonable hardware that allows for this.
- While the method doesn’t require explicit sharpness computation, there is extra complexity when using the learned optimizer. I would like to see a wall clock comparison between the proposed method and [1,2]
- In `Conv-Large-MNIST` of figure 5, the training loss is completely gone from the plot while the test accuracy is really good. This doesn’t really feel like an “exception” but rather a bug. I would like to see some analysis of this phenomenon.

**Question / Comments**

- I am confused by the introduction of $L_M$ in eq 2. If I understand correctly, unlike regular meta-learning, the method is actually not optimizing for the validation loss but rather just the geometry on the training data. $L_M$ is not used at all after this introduction.
- [1] should be cited and discussed.
- The paper does have an interesting idea which is that you can specify properties of desirable optima at meta training time and learn an optimizer that implicitly looks for solutions with the desired properties; however, I feel like sharpness may not be the most convincing application for this technique due to aforementioned reasons and I believe there could be more compelling use cases.

**Reference**

[1] Sharpness-aware Minimization for Efficiently Improving Generalization. Foret et al.

[2] Entropy-SGD: Biasing Gradient Descent Into Wide Valleys. Chaudhari et al.

------------------------------

**Update**

Thank you for the response and additional experiments.

I am now convinced that you can train L2O on larger model but the performance of these models are much worse than standard SGD (i.e., < 80% test accuracy) so I am not sure if the proposed method is significant for real models. The claim about L2O doing better than SGD and Adam is also somewhat questionable since it's only done in the small convnet setting but the performance of this convnet is pretty bad on cifar10 and mnist. In particular, it achieves 80% test accuracy on mnist. In my opinion, this number is unusually bad since one can easily write an MLP that does > 90% accuracy on mnist by just following a tutorial. Is there no momentum? If so, why not?

Overall, the new experiments and rebuttal have not changed my opinion of the paper and therefore I am keeping my current evaluation.
summary_of_the_review: While the paper proposes an interesting idea with solid improvement over the baseline, I do not see the practicality of using a learned optimizer that seeks a wide solution when there exist much better alternatives that don't require meta-learning. I believe that a similar idea could be useful for something other than sharpness but in the current form, I don’t feel the paper is good enough for publication.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work proposes to apply flatness-aware regularizations to learning to optimize methods. Specifically, the approximate hessian spectrum and entropy based regularizers are added to the L2O objective. They performed a theoretical analysis of the regularized L2O method and evaluated different variants of the regularizer on simple CNN and MLP models trained on CIFAR and MNIST. Similar to the effect of flatness-aware regularization on the ordinary training, the regularizer improved the learned optimizers to find minima that generalizes better. 

main_review: Strength:

1. This work applies the flatness-aware regularizer to the learning to optimize objective to improve the generalization of the learned optimizers, which is a novel application of the existing technique. 

2. The authors performed both theoretical and empirical analysis of the hessian and entropy based regularizers, and demonstrated its advantages. The improvement over the baseline (without regularizer) seems quite large. 

Weakness:

1. Similar to existing L2O works, the evaluation is only performed on simple CNN and MLP models and CIFAR/MNIST. Given the sizable improvement over the other L2O method, I wonder if this method can be applied to more realistic models such as Wide ResNet or ResNet-50 for ImageNet since the learned optimizer seems to generalize over different architectures and datasets as is discussed in section 5. If not, it would help to add some discussion of the limitations and bottlenecks so that future works can be better informed. 

2. Given the large variance, it would be helpful to add some confidence interval to the curves in the figures. Or it helps to add some tables with mean and standard deviation for different methods for comparisons, which can complement the figures. 

3. It would be helpful to get more insight of the learned optimizer. One suggestion is to add a comparison with a baseline that applies flatness-aware regularizer to the analytical optimizer, which could help understand where the gain over the analytical optimizer is from, i.e., whether the learned optimizer learns to act like a flatness-aware regularized optimizer or there is more than that. 

The paper claims that the learned optimizer "favors generalization and requires no more time calculating Hessian or Entropy information while in use". It would require some evidence to support this claim. For example, it would help to compare the Hessian spectrum or entropy of the minima found by learned optimizer with regularizations with the ones found by the baseline learned optimizer to show that it is indeed finding a flatter minima. 
summary_of_the_review: This paper introduces flatness-aware regularizer to L2O objective to improve the learned optimizers, which is a novel contribution. The empirical result looks promising and it could be improved by adding more insights of the learned optimizer, some discussions of the limitations (or some results applying to a more realistic model) and confidence intervals to the results.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper aims to solve the poor generalization performance of Learning to Optimize (L2O) methods by introducing flatness-aware regularizers. The idea is to add a regularization term to the meta-training objective that encourages the final iterate obtained by applying the learned optimizer, to lie in a flat region (measured by the spectral norm of the Hessian, or the local entropy function). The authors prove a bound for the generalization error, and show empirically that adding flatness-aware regularizers to existing L2O methods, can improve generalization or both the learned optimizer (across unseen tasks) and the solution (across dataset splits).
main_review: Strengths:
To the best of my knowledge, this is the first paper to add such regularizers to the L2O objective, and prove a bound in the generalization error. The paper is clear and easy to follow.

Weaknesses:
- This paper doesn’t show enough evidence of the practicality of the proposed method. The experiments are done on small networks and on datasets (MNIST and CIFAR) that are more or less “solved”. Instead of spending extra resources doing meta-learning to train an optimizer, one could tune an off-the-shelf optimizer with a more expressive model. We do not know whether this idea scales and generalizes to more realistic settings.
- The performance of the baselines (SGD and Adam) are hard to believe. On MNIST, SGD and Adam barely reach 90% accuracy. On CIFAR, they both reach below 50%. These are with 10k steps. I’m suspicious that the hyperparameters for SGD and Adam are chosen poorly, because given the same architecture, the learned optimizer is able to achieve much better performance.
- The relationship between “flatness” and the generalization ability of a solution are tenuous-- sharpness measures can be varied arbitrarily without changing the output of the neural network (Dinh et al., 2017). The generalization bounds do take into account flatness, but I’m not sure how tight this bound is, or how realistic the assumptions are.

Dinh, Laurent, et al. "Sharp minima can generalize for deep nets." International Conference on Machine Learning. PMLR, 2017.

------
Update:
I have read all the reviews and the author rebuttals. I am increasing my score (3-> 5) because idea is novel and it improves over L2O methods. However, I'm still not convinced that the paper should be accepted given that it wasn't tested on settings that are able to achieve competitive results on the chosen datasets, and because the experiments section could be polished and more organized.

summary_of_the_review: Given this paper’s lack of practicality and questionable experiment results, I recommend rejection.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper aims at improving the generalization of the learned optimizers. Some regularizers to induce the flatness of local minimas including Hessian spectrum and the entropy function are proposed. Both theoretical analysis and empirical results are presented to demonstrate that flatness-aware regularizers can enhance the generalization ability of optimizees.
main_review: Strengths:
- The generalization issue in L2O has been a significant problem preventing the applications of learned optimizers in the real-world tasks, and it is good to develop an algorithm to improve the generalization.
- Theoretical analysis and comprehensive experiments are conducted to support the proposed flatness-aware regularizers.

Weaknesses:
- The experimental part lacks some important information. As the authors claimed in the checklist, no running time and variance is reported. Since training the neural optimizer with those two proposed regularizers~(Hessian and Entropy) is expensive and time-consuming, reporting the running time is important for the audience to evaluate the algorithm. Besides, in each independent run, the final result might vary significantly and it is necessary to give the statistics accounting for the variance.
- It seems weird to keep both Figure 3 and Figure 4 in the main paper, since Figure 4 only adds the plot for L2O-DM-CL + Entropy. Why not just use Figure 4 for illustration?
- In the abstract, the authors mentioned two types of generalization, optimizer generalization and optimizee generalization. However, in the whole paper, I think the authors mainly focused on improving the optimizee generalization using two flatness-aware regularizers. On the other hand, in Section 5.2, it was also claimed that entropy regularizer "boosts both optimizee and optimizer generalizations of L2O in most cases, as shown in Figure 4". I did not see any explanations why these regularizers can improve the optimizer generalization. 
- Why did the authors proposed two different regularizers? There is no clear description or conclusion that under certain scenarios, one of them will be preferred, except for some observations in four empirical settings. I think this is an important problem to be investigated, otherwise it seems that the authors just finds two irrelevant regularizers for the flatness and puts them together in one paper without further thoughts. Another interesting attempt could be combining the two regularizers in one training objective to see whether there will be further improvement.
- Instead of Hessian spectrum, directly using spectral normalization on parameters of the optimizee can also benefit the generalization, as demonstrated in [1]. Did the authors try this simpler and efficient method compared with Hessian spectrum?
- It is worth carrying out some experiments to incorporate the flatness-aware regularizers in the meta-testing stage for both learned optimizers and traditional analytical ones. It would be good if the performance trained with the learned optimizer (trained with flatness-aware regularizers) by the vanilla classification loss can outperform the performance trained with analytical ones by the regularized training objective.

[1] Yoshida, Yuichi, and Takeru Miyato. "Spectral norm regularization for improving the generalizability of deep learning." arXiv preprint arXiv:1705.10941 (2017).
summary_of_the_review: The current version needs some modifications on the experimental part to include more important information like running time and variance, as well as more experiments comparing the proposed algorithm with traditional optimizers. Also, it lacks the description of connections between two different regularizers. Thus, I think the paper is slightly below the acceptance criterion.
review_title: null

Meta-review: This paper proposes a learning-to-optimize approach that is "flatness-aware", i.e. it tries to find flatter minima in the loss landscape. The idea is accompanied by both theoretical and empirical verification. However, in the current state, the paper did not convince the reviewers about its potential impact. In particular, reviewer ZY6B points out that comparison with "sharpness-aware" minimization (SAM), which is a non-learned optimizer for seeking flat minima, is an important comparison that is currently missing. Another issue mentioned by the reviewers  ZY6B and wp2U relates to the lower performance of the learned optimizer compared to standard SGD in large models (lower than 80% test accuracy). These reviewers are interested in the question of whether this method can still produce good results in the competitive setting (e.g. > 90% accuracy on CIFAR). Considering the performance/baseline issue even on small datasets such as MNIST and CIFAR, the impact of the proposed method is unclear. I encourage authors to adopt more conventional baselines to better indicate the potential impact of their method, and do consider comparison with optimizers that directly aim to improve flatness, such as SAM.
Decision: Reject