Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposed a meta-learning approach to learn piecewise-Lipschitz functions in the multi-task setting. Simply applying a traditional single-task exponential forecasting method multiple times cannot make the regret bound diminishes as the number of tasks $T$ increases. To tackle this issue, they first attempt to obtain a bound that involves the task similarity term, and then the authors propose two meta-learning algorithms, based on the observation that the regret bound depends on two things: learning rate and initialization probability choosing. Hence, they proposed two meta-learning algorithms, to efficiently learn to choose better initialization and learning rate. The theoretical results show that this algorithm can provably improve the baseline of applying a single-task algorithm $T$ times.

main_review: This paper proposes a novel meta-learning approach for online learning of piecewise-Lipschitz functions   in a multi-task environment. To my best knowledge, this paper is the first to generalize learning piecewise-Lipschitz functions problem to multi-task setting. It shows that the trivially applying single-task algorithm $T$ times will not make regret bound vanish, and then naturally introduce a quantity $V$ that captures the task similarity. Based on the bound, the proposal of two meta algorithms to learn to choose hyperparameters is also novel to me. The theoretical results look sound as far as I checked. The paper is well-organized and the motivations are very clear. I do not have much experience in this field but I learnt from this paper and enjoy reading it. Hyperparameter choosing is a longstanding difficulty in the machine learning community, and this paper sheds some light on this aspect, and the present paper has some merits. 

limitations_and_societal_impact: I notice that in Line 214:  with positive volume $vol(C_t) > 0$ that is revealed after choosing $w_t$. I think $C_t$ is the ball center at optimum at $\rho_t^*$, so does it mean we need some oracle knowledge to reveal $vol(C_t)$?
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors studied the initialization-based meta-learning of piecewise-Lipschitz functions, demonstrating how online convex optimization over an adaptive discretization can find an initialization that improves the performance of the exponential forecaster across tasks, assuming the tasks have related optima. They then applied this result in two settings: online configuration of clustering algorithms and adversarial robustness in online learning. 
main_review: Many existing works of  meta-learning an initialization are largely restricted to the convex Lipschitz setting. This pape relaxes this assumption to study the meta-learning of online algorithms over piecewise-Lipschitz  functions, which can be nonconvex and highly discontinuous. This is the first theoretical study of meta-learning in these application settings.
limitations_and_societal_impact: I think this paper is theoretically sound. One suggestion is to incorporate more experiments to demonstrate the efficiency of the proposed algorithms.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper study the initialization-based meta-learning focus on the piecewise-Lipschitz functions whose discontinuities are dispersed. The authors designed an initialization method that implicitly run on online convex optimization procedure over an adaptive discretization to improve the performance of the exponential forecaster across multi-task. To demonstrate the usefulness of their proposed method, the authors instantiate the method in two settings: online configuration of clustering algorithms and adversarial robustness in online learning.
main_review: A good initialization is critical to the success of a meta-learning method. The paper designed the meta-learning of the initialization and step-size of online learning algorithms for piecewise-Lipschitz functions in a multi-task environment. The method optimizes a sequence of data-dependent upper-bounds on the within-task regret. I consider this work interesting, However, I have some concerns too.
1.	To demonstrate the effectiveness of the proposed new meta-initialization algorithm, the authors design experiments for clustering on the real Omniglot dataset and a simulated Gaussian mixture binary classification dataset. As the result showed in the Omniglot dataset, the meta-initialized method has little improvement over single task.. The authors should do more ablation experiments to demonstrate the effective of the proposed meta-initialization method.
2.	In the original experiments, the authors chose the same hyperparameters of meta-initialization method at different experiment situations. The authors should try different hyperparameters to obtain best performance in various tasks.
3.	The authors need to add more experiments to compare other initialization and step-size search methods. 
4.	More experiments and datasets are needed to further investigate the robustness and limitations of this approach.
5.	Note the addition of punctuation in some sentences！For example, In Line 44,48-49.


limitations_and_societal_impact: Yes
review_title: null

Meta-review: The paper gives an online algorithm for learning convex piecewise functions under certain assumptions regarding dispersion of the points of discontinuity. This follows recent work on algorithm meta-design and online learning.
Decision: Accept (Poster)