reviews: 
Comments: In this paper, the author revisited the methodology of Meta-Continual Learning (Meta-CL) and, for the first time, provides a formal connection between meta-continual learning and seminal regularization-based methods (like Elastic Weight Consolidation (EWC)) which mainly exploits the empirical Hessian matrix to provide the regularization to counter forgetting. The main finding is that Meta-CL methods implicitly utilize the second-order Hessian information through the hypergradient obtained by bi-level optimization for meta-learning. From this new perspective, the author further points out the issue existing in the methodology of Meta-CL, i.e., the presence of erroneous information in the Hessian information due to insufficient memory data. To resolve the problem, the author correspondingly proposes a momentum-based Variance-Reduced Meta-CL (VR-MCL) method and provides extensive theoretical analysis to demonstrate how the proposed method can impose a penalty on the online estimated Hessian such that the model can be updated with caution to preserve crucial parameters. Extensive experiments are conducted on standard continual learning benchmarks, and the proposed theoretical method outperform both representative and state-of-the-art (SOTA) continual learning methods.
Strengths: 1. The reviewer really enjoys reading this paper. This should be the first paper that formally and clearly dissects the relationship between seminal regularization-based methods and the methodology of meta-continual learning. The key message and insights are conveyed smoothly in the whole paper, and the author does a really good job of presenting them in a decent way. Table 1 provides a very precise and clear summary and comparison of the seminar and the state-of-the-art regularization-based method for the reader to get their main idea in common, making it easier for the reader to comprehend the novelty and contribution made by the present paper. Figures 1 and 2 are also compact and reduce the difficulties for the reader to understand the technical details of the iterative update process, which also highlights the difference made in this paper.

As the Hessian information is widely used not only in continual learning but also in many different areas of deep learning (e.g., meta-learning and flatness-aware optimization), the reviewer believes that the theoretical findings provided by this paper may not only motivate novel methods on Meta-CL but may also motivate novel methods for other areas in general.

2. The unification of the Meta-CL and Regularization-based method is sound. Although there exist papers that try to unify different regularization-based CL methods in a unified framework, the CL methods they considered are mainly for CL in a fully-supervised setting, to the best of my knowledge, this paper should be the first one to connect the regularization-based CL methods with the methodology of Meta-CL, which may stand as a new research direction in the future.

3. The reviewer also appreciates the understanding provided by the author in Section 4.2 after Proposition 3. It is refreshing to see that the variance-reduce method can ensure cautious updates such that the model can prevent excessive updates triggered by the wrongly estimated low-curvature direction of the Hessian, which may mitigate the partiality and erroneousness in the insufficient memory data, which should also be a desideratum about the kind of model update we should purse for. The insight may also motivate future work in continual learning and may also in areas like parameter-efficient finetuning.

4. The extensive comparison with state-of-the-art methods in both CL and Meta-CL further demonstrates the significance and effectiveness of the proposed method. The questions listed in each subsection of the Experiments section provide good guidance for the reviewer to focus on and reason about the results. It is also great to see that the author also conducts many empirical analyses in both the main paper and supplementary to validate the correctness of the proposed theorem.
Weaknesses: 1. In Proposition 3, the author assumes that the batch size for inner step adaptation is sufficiently large. How do we quantify the term "sufficiently large" in reality? Is there any principle we can obtain from the proposed theorem to guide us in choosing the batch size?
Recommendation: 10: strong accept, should be highlighted at the conference
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 4 excellent
Clarity: 4 excellent