Venue: ML_Reproducibility_Challenge 2020

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This manuscript provides an pytorch implementation and reproducibility report of "SAdam: A Variant of Adam for Strongly Convex Functions". Most of the claims are consistent with the original paper. Experimental results include L2 regularized softmax regression, 4 layer CNN, ResNet 18 on MNIST, CIFAR10 and CIFAR100 datasets with exhaustive hyperparameter tuning. It would be interesting to see the results on ImageNet.
review_title: Good reproducibility report

final_score: rating: 5: Marginally below acceptance threshold
review_content: The report clearly presents its scope and reimplements SAdam on pytorch where they compare their implementation with the version obtained from the authors which is in tensorflow. The report has multiple positive aspects:
- communication with the authors
- reimplementation of the code
- clear reproduction of the original paper
- clear description of the problem and the presentation of the report 
- additional experiments on the resnet model

Major shortcomings of the paper include:
- lack of a range of hyperparameter search for the robustness of results
- testing the algorithm in contexts other than simple vision problems

The area of algorithmic improvements is a tricky subject where it is hard to confirm with strong confidence that the results are robust across wide variety of hyperparameters, and confirming the validity of the results in a way that translates to different structures of problems. I think one of the strongest points a reproducibility can make is to show that this is the case. That said, the present report does a great job at reproducing the core components of the investigated paper; I encourage the authors to extend their hyperparameter search for robustness and comparisons as well as applying the algorithm to wider (different) sets of optimization problems.
review_title: A decent reimplementation and confirmation of SAdam on simple frameworks

final_score: rating: 7: Good paper, accept
review_content: Reproducibility Summary : The report has this summary that contains major findings.
Scope of reproducibility: The report states the scope clearly, and follows it.
Code: The authors reproduced the code in PyTorch.
Communication with original authors: Fair communications are mentioned, but I am not sure whether the original authors have evaluated the results in this report.
Hyperparameter Search: It is mentioned in the report that hyperparameters were varied, but not all results are included in the report.
Ablation Study: The ablation study is not comprehensive.
Discussion on results: The report discusses the state of reproducibility of the original paper, and mentions the easy parts and difficult parts. The numerical results of this report are consistent with those in the original paper. Only the results on ResNet-18 deviate.
Recommendations for reproducibility: It seems the report does not discuss a lot on how the original authors can improve reproducibility.
Results beyond the paper: The experiments in the report are almost the same as those in the original paper.
Overall organization and clarity: The overall quality is good.

review_title: nice report

Meta-review: There is a lack of a range of hyperparameter search for the robustness of results and in testing the algorithm in contexts other than simple vision problems.
Decision: Reject