SlowGuess/NeurIPS_2023_OCR huggingface dataset:
1: for  $t = 1, \dots, T$  do
2: Sample the training datasets  $\mathcal{D}_t^{tr}$  from the distributions  $\mathcal{D}_t$  for task  $\mathcal{T}_t$
3: Update and deploy the task-specific parameter  $\theta_t = \mathcal{A}lg(\lambda, \phi_t, \mathcal{D}_t^{tr})$  defined in (3) for task  $\mathcal{T}_t$
4: Sample the evaluation dataset  $\mathcal{D}_{0,t}^{val}$  from the distributions  $\mathcal{D}_{0,t}$
5: Generate the random perturbed vector  $\sigma_t \in \mathbb{R}^d$  by i.i.d sampling:  $\{\sigma_{t,j}\}_{j=1}^d \sim \mathrm{Exp}(\eta)$
6: Update the meta parameter  $\phi_{t + 1}$  by solving (4)
7: end for
8: Return  $\{\theta_1, \dots, \theta_T\}$  for tasks  $\{\mathcal{T}_1, \dots, \mathcal{T}_T\}$
3.4 Algorithm statement.
The online constrained meta-learning algorithm is formally stated in Algorithm 1. Note that line 3 is a deterministic convex optimization problem, which has been widely studied and solved by [10, 6]. The optimization problem (4) in line 6 is a deterministic constrained bilevel optimization problem, which can be solved by the algorithm in [55]. The implementation details are included in our proposed practical algorithm (Algorithm 2 in Appendix B).
In this section, we derive the upper bounds of the TAOG and the TACV produced by Algorithm 1, which characterize the regrets of the generalization errors of the models deployed for the sequential tasks. First, we study the generalization error bound produced by our within-task algorithm (3) for a single constrained learning task (Section 4.1). Next, we show the upper bounds of the TAOG and the TACV when the meta-parameter can be arbitrarily selected (Section 4.1). Finally, we obtain the theoretical bounds of the TAOG and the TACV, when the meta-parameter is not arbitrarily selected but produced by Algorithm 1 (Section 4.2).
Before the result statements, we introduce some definitions and the required assumptions about the constraint qualifications for the optimization problems in (1) and (3). The constraint qualification assumptions are usually used in constraint optimization analyses [7, 19].
We consider an optimization problem with inequality constraints:  $\min_x g(x)$  s.t.  $h_i(x) \leq 0$ ,  $i \in I \triangleq \{1, \ldots, m\}$  and denote it as (P). Denote the feasible set of (P) as  $\mathcal{K} \triangleq \{x \mid h_i(x) \leq 0, \forall i \in I\}$  and  $I(x) \triangleq \{i \in I \mid h_i(x) = 0\}$  for  $x \in \mathcal{K}$ .
Definition 1. The Linear Independence Constraint Qualifications (LICQ) holds for  $(P)$ , if vectors  $\{\nabla h_i(x) \mid i \in I(x)\}$  are linearly independent for any  $x \in \mathcal{K}$ .
Definition 2. Slater's condition (SC) holds for  $(P)$  with the margin  $\mathcal{C} > 0$ , if there exists  $\bar{x}$  such that  $h_i(\bar{x}) - \mathcal{C} \leq 0$ ,  $\forall i \in I$ .
Assumption 1 (Constraint qualifications). (i) There exists a compact set with diameter  $\mathcal{B} > 0$  such that the following properties hold for each  $t$ : for the given data distribution  $\mathcal{D}_t$ , the feasible set  $\mathcal{K}_t$  of the optimization problem in (1) is included in the compact set; with probability 1, for  $\mathcal{D}_t^{tr}$  sampled from  $\mathcal{D}_t$ , the feasible set  $\mathcal{K}_t^{tr}$  of the optimization problem in (3) is included in the compact set.
(ii) There exists  $\mathcal{C} > 0$  such that the following properties hold: for the given data distribution  $\mathcal{D}_t$ , the SC holds for the optimization problem in (1) with the margin  $\mathcal{C}$ ; with probability 1, for  $\mathcal{D}_t^{tr}$  sampled from  $\mathcal{D}_t$ , the SC holds for the optimization problem in (3) with the margin  $\mathcal{C}$ .
(iii) For the given data distribution  $\mathcal{D}_t$ , the LICQ holds for the optimization problem in (1).
Remark 1. A sufficient condition for part (i) of Assumption 1 is that  $\Theta$  is a compact set.
We also require the following assumptions on the loss function and the constraint functions, which are standard in the analysis of meta-learning problems [4, 15, 24, 20].
Assumption 2 (Function properties). (i) For any  $z \in \mathcal{Z}$ , the loss function  $\ell_0(\cdot, z)$  and the constraint functions  $\ell_1(\cdot, z), \dots, \ell_m(\cdot, z)$  are twice continuously differentiable. (ii) For any  $z \in \mathcal{Z}$ ,  $\ell_0(\cdot, z)$  is  $L_0$ -Lipschitz, i.e.,  $\| \ell_0(w, z) - \ell_0(u, z) \| \leq L_0 \| w - u \|$  for any  $w, u \in \Theta$ . (iii) For any  $z \in \mathcal{Z}$ ,  $\ell_0(\cdot, z)$  is  $\rho$ -smooth, i.e.,  $\| \nabla \ell_0(w, z) - \nabla \ell_0(u, z) \| \leq \rho \| w - u \|$  for any  $w, u \in \Theta$ . (iv) For any  $z \in \mathcal{Z}$  and  $1 \leq i \leq m$ ,  $\ell_i(\cdot, z)$  is  $L_c$ -Lipschitz, i.e.,  $\| \ell_i(w, z) - \ell_i(u, z) \| \leq L_c \| w - u \|$  for any  $w, u \in \Theta$ . (v) For any  $z \in \mathcal{Z}$  and  $0 \leq i \leq m$ ,  $\ell_i(\cdot, z)$  is convex. (vi) For any  $t \in \{1, \dots, T\}$ ,  $z \in \mathcal{Z}$ ,  $w \in \Theta$ , and  $1 \leq i \leq m$ ,  $\ell_i(w, z) - c_{i,t}$  are bounded by  $M$ .
4.1 Generalization of constrained learning with biased regularization
We begin with the generalization error bound on a single constrained learning task  $\mathcal{T}_t$ , which is produced by the within-task algorithm (3) with an arbitrarily given meta-parameter  $\phi_t$ . As stated in (3), the algorithm replaces all the expectations in (1) by the sample averages over the given datasets and adds a regularization term with the bias  $\phi_t$  to the objective function.
Proposition 1. Suppose that Assumptions 1 and 2 are satisfied. For any given metaparameter  $\phi_t$  and regularization weight  $\lambda >0$ , the following bounds hold for task
$$
\begin{array}{l} \mathcal {T} _ {t}: \quad \mathbb {E} _ {\mathcal {D} _ {t} ^ {t r} \sim \mathcal {D} _ {t}} \left[ R _ {0, t} \left(\mathcal {A} l g \left(\lambda , \phi_ {t}, \mathcal {D} _ {t} ^ {t r}\right)\right) \right] \leq \frac {\lambda}{2} \left\| \theta_ {t} ^ {*} - \phi_ {t} \right\| ^ {2} + \mathcal {O} \left(\frac {\ln | \mathcal {D} _ {0} ^ {t r} |}{\lambda | \mathcal {D} _ {0} ^ {t r} |} + \sqrt {\frac {\ln | \mathcal {D} _ {+} ^ {t r} |}{| \mathcal {D} _ {+} ^ {t r} |}}\right), a n d \\ \mathbb {E} _ {\mathcal {D} _ {t} ^ {t r} \sim \mathcal {D} _ {t}} \left[ R _ {i, t} (\mathcal {A} l g (\lambda , \phi_ {t}, \mathcal {D} _ {t} ^ {t r})) \right] \leq \mathcal {O} \left(\sqrt {\frac {\ln | \mathcal {D} _ {+} ^ {t r} |}{| \mathcal {D} _ {+} ^ {t r} |}}\right), \forall i = 1, \dots , m. \\ \end{array}
$$
Proposition 1 shows the generalization error bounds of the loss function  $\ell_0$  and the constraint violation for each  $\ell_i$ . The coefficients of the notations  $\mathcal{O}$  and the proofs are shown in Propositions 3 and 4 of Appendix E. As the objective function in (3) includes the biased regularization term  $\frac{\lambda}{2} \| \theta - \phi_t \|^2$ , where the bias serves as the prior estimation for the solution, the generalization error bound includes a term of  $\frac{\lambda}{2} \| \theta_t^* - \phi_t \|^2$ , which is decreasing as the estimation  $\phi_t$  is more accurate. The objective function and the constraint functions include the sample averages over the training dataset  $\mathcal{D}_t^{tr}$ . As the numbers of data points  $(|\mathcal{D}_0^{tr}|)$  and  $|\mathcal{D}_{+}^{tr}|$  are larger, the sample average approximation is more accurate, and then the generalization error is smaller.
The generalization error bounds shown in Proposition 1 motivate learning a good meta-parameter  $\phi_t$  by meta-learning. Next, we study the generalization error bounds for the task sequence  $\{\mathcal{T}_1,\dots ,\mathcal{T}_T\}$ , when the meta-parameter  $\phi$  is arbitrary and fixed. Proposition 2 shows the upper bounds of the TAOG and the TACV for any given meta-parameter  $\phi$ , i.e., the task-specific parameter  $\theta_t$  in (2) is computed by  $\theta_t = \mathcal{A}lg(\lambda ,\phi ,\mathcal{D}_t^{tr})$  for all  $t$ .
Proposition 2. Suppose that Assumptions 1 and 2 are satisfied. Consider the task sequence  $\{\mathcal{T}_1,\dots ,\mathcal{T}_T\}$ , for any given meta-parameter  $\phi$ , choose the regularization parameter  $\lambda = \frac{2\sqrt{d}(\rho\mathcal{B} + L_0\sqrt{\ln|\mathcal{D}_0^{tr}|})}{\mathcal{D}ist(\phi,\mathcal{T}_{1:T})\sqrt{|\mathcal{D}_0^{tr}|}}$ . Then, the following bounds hold:
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} _ {\mathcal {D} _ {t} ^ {t r} \sim \mathcal {D} _ {t}} \left[ R _ {0, t} (\mathcal {A} l g (\lambda , \phi , \mathcal {D} _ {t} ^ {t r})) \right] \leq \mathcal {O} \left(\mathcal {D} i s t (\phi , \mathcal {T} _ {1: T}) \sqrt {\frac {\ln | \mathcal {D} _ {0} ^ {t r} |}{| \mathcal {D} _ {0} ^ {t r} |}} + \sqrt {\frac {\ln | \mathcal {D} _ {+} ^ {t r} |}{| \mathcal {D} _ {+} ^ {t r} |}}\right), \quad a n d \\ \frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} _ {\mathcal {D} _ {t} ^ {t r} \sim \mathcal {D} _ {t}} \left[ R _ {i, t} (\mathcal {A} l g (\lambda , \phi , \mathcal {D} _ {t} ^ {t r})) \right] \leq \mathcal {O} \left(\sqrt {\frac {\ln | \mathcal {D} _ {+} ^ {t r} |}{| \mathcal {D} _ {+} ^ {t r} |}}\right), \forall i = 1, \dots , m. \\ \end{array}
$$
The coefficients of the notations  $\mathcal{O}$  and the proofs are shown in Proposition 5 of Appendix E. Following the result in Proposition 2, as the average distance between the meta-parameter  $\phi$  and the optimal task-specific parameters  $\{\theta_1^*,\dots ,\theta_T^*\}$ , i.e.,  $\mathcal{D}ist(\phi ,\mathcal{T}_{1:T})$ , becomes small, the upper bound of the TAOG is small. By minimizing the distance  $\mathcal{D}ist(\phi ,\mathcal{T}_{1:T})$  over  $\phi$ , we can get the optimal upper bound of the TAOG:
$$
\frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} _ {\mathcal {D} _ {t} ^ {t r} \sim \mathcal {D} _ {t}} \left[ R _ {0, t} \left(\mathcal {A} l g \left(\lambda , \phi^ {*}, \mathcal {D} _ {t} ^ {t r}\right)\right) \right] \leq \mathcal {O} \left(\mathcal {S} ^ {*} \left(\mathcal {T} _ {1: T}\right) \sqrt {\frac {\ln | \mathcal {D} _ {0} ^ {t r} |}{| \mathcal {D} _ {0} ^ {t r} |}} + \sqrt {\frac {\ln | \mathcal {D} _ {+} ^ {t r} |}{| \mathcal {D} _ {+} ^ {t r} |}}\right), \tag {5}
$$
when the meta-parameter is selected by  $\phi^{*} = \arg \min_{\phi}\mathcal{D}ist(\phi ,\mathcal{T}_{1:T})$ . Note that the optimal upper bound is not achievable since the optimal task-specific solution  $\theta_t^*$  is not achievable for each  $t$ . In the next section, we show that the sequence of meta-parameters  $\phi_t$  produced by Algorithm 1 produces a comparable performance as the optimal meta-parameter  $\phi^{*}$ .
4.2 Generalization of the online constrained meta-learning algorithm
Consider the meta-parameter is updated by (4) in Algorithm 1. As we apply the FTPL algorithm to the meta-objective function for the online constrained meta-learning problem, the upper bounds of the TAOG and the TACV produced by Algorithm 1 are shown in Theorem 1 and Corollary 1. As the algorithm to compute  $\theta_{t}$  is stochastic, which depends on the task sequence and the sampling of the training data  $\mathcal{D}_t^{tr}$  and evaluation data  $\mathcal{D}_{0,t}^{val}$ , we denote the expectation of the optimality gap and the constraint violations  $R_{i,t}(\theta_t)$  as  $\mathbb{E}[R_{i,t}(\theta_t)]$ , and the expectations of the TAOG and the TACV are denoted as  $\mathbb{E}[\bar{R}_{i,[1:T]}] = \mathbb{E}\left[\frac{1}{T}\sum_{t=1}^{T}R_{i,t}(\theta_t)\right]$  for all  $i = 0,\dots,m$ .
Theorem 1. Suppose that Assumptions 1 and 2 are satisfied. Suppose that  $\Theta$  is included in a compact cube with edge of length  $D_{l}$ , i.e.  $\| \phi \|_{\infty} \leq D_{l}$  for any  $\phi \in \Theta$ . Choose the regularization parameter  $\lambda = \frac{2\sqrt{d}(\rho\mathcal{B} + L_0\sqrt{\ln|\mathcal{D}_0^{tr}|})}{S^*(\mathcal{T}_{1:T})\sqrt{|\mathcal{D}_0^{tr}|}}$  and the perturbed parameter  $\eta = \frac{1}{L_0\sqrt{dT}}$ . For the task
sequence  $\{\mathcal{T}_1,\dots ,\mathcal{T}_T\}$ , the following bounds hold for the TAOG and the TACV of Algorithm
$$
\begin{array}{l} I: \mathbb {E} [ \bar {R} _ {0, [ 1: T ]} ] \leq \mathcal {O} (\mathcal {S} ^ {*} (\mathcal {T} _ {1: T}) \sqrt {\frac {\ln | \mathcal {D} _ {0} ^ {t r} |}{| \mathcal {D} _ {0} ^ {t r} |}} + \sqrt {\frac {\ln | \mathcal {D} _ {+} ^ {t r} |}{| \mathcal {D} _ {+} ^ {t r} |}} + \sqrt {\frac {\ln | \mathcal {D} _ {0} ^ {v a l} |}{| \mathcal {D} _ {0} ^ {v a l} |}} + \frac {1}{\sqrt {T}}), a n d \mathbb {E} [ \bar {R} _ {i, [ 1: T ]} ] \leq \\ \mathcal {O} \left(\sqrt {\frac {\ln | \mathcal {D} _ {+} ^ {t r} |}{| \mathcal {D} _ {+} ^ {t r} |}}\right),   \forall i = 1, \ldots , m. \\ \end{array}
$$
Corollary 1. If the task  $\mathcal{T}_t$  at each  $t$  is sampled from a fixed distribution  $p(\mathcal{T})$ , we choose  $\lambda = \frac{2\sqrt{d}(\rho\mathcal{B} + L_0\sqrt{\ln|\mathcal{D}_0^{tr}|})}{\mathcal{S}^*(p(\mathcal{T}))\sqrt{|\mathcal{D}_0^{tr}|}}$ , then  $\mathcal{S}^*(\mathcal{T}_{1:T})$  in the upper bound of  $\mathbb{E}[\bar{R}_{0,[1:T]}$ ] shown in Theorem 1 can be replaced by the fixed constant  $\mathcal{S}^*(p(\mathcal{T}))$ .
The coefficients of the notations \(\mathcal{O}\) and the proofs are shown in Appendices E and F. In the online constrained meta-learning problem, as stated in Algorithm 1, after the task-specific parameter \(\theta_{t}\) for task \(\mathcal{T}_t\) is deployed, the agent is usually able to sample the dataset \(\mathcal{D}_0^{val}\) and \(|\mathcal{D}_0^{val}| \gg |\mathcal{D}_0^{tr}|\). In this case, we can ignore the term \(\sqrt{\frac{\ln|\mathcal{D}_0^{val}|}{|\mathcal{D}_0^{val}|}}\) in the upper bound of \(\mathbb{E}[\bar{R}_{0,[1:T]}\]\(T]\) in Theorem 1. We compare the upper bound of the TAOG by Algorithm 1 with the unachievable optimal upper bound shown in (5), where the term \(\sqrt{\frac{\ln|\mathcal{D}_0^{val}|}{|\mathcal{D}_0^{val}|}}\) is ignored. The only additional term included in Theorem 1 is \(\mathcal{O}(\frac{1}{\sqrt{T}})\). It implies that the regret of the optimality gap between the meta-parameter \(\phi_t\) updated by Algorithm 1 and the optimal meta-parameter \(\phi^*\) used in (5) is sublinear with respect to the task number \(T\).
In particular, when the constraint is ignored from Problem (1), the upper bound of the TAOG reduces to  $\mathbb{E}[\bar{R}_{0,[1:T]}\leq \mathcal{O}(\mathcal{S}^{*}(p(\mathcal{T}))\sqrt{\frac{\ln|\mathcal{D}_0^{tr}|}{|\mathcal{D}_0^{tr}|}} +\frac{1}{\sqrt{T}})$ , which has the same order as the upper bound shown in [18] with respect to the task dissimilarity  $\mathcal{S}^{*}(p(\mathcal{T}))$ , the data number  $|\mathcal{D}_0^{tr}|$ , and task number  $T$ .
Corollary 1 considers that the revealed tasks are sampled from a static task distribution. In this case, when  $T$  is sufficiently large, Algorithm 1 degenerates into an offline constrained meta-learning algorithm. If we further ignore the constraints in Problem (1), the bound in Corollary 1 of the TAOG holds the same order as the upper bound shown in [15].
5.1 Meta imitation learning
Meta-imitation Learning [25, 40] has been developed as a powerful tool for a learner to finish a new task by observing the demonstrations of an expert. However, existing methods do not consider any constraint, e.g., collision avoidance, for the new task. In this experiment, we apply the constrained meta-learning algorithm to a constrained meta-imitation learning problem where the learner learns to write letters in a cluttered environment [30, 29]. At each round  $t$ , the expert writes a different letter in a free space. The learner can observe the locations and velocities of a few points of the letter and is asked to write the same letter in a cluttered environment where the obstacle is known. The learner learns the policy by using constrained learning, where the loss function is defined by the tracking error of the learned policy, and the constraint is defined by a collision metric. Note that the expert performs in the free space, and the learner performs in the cluttered environment. The additional obstacle makes the problem more difficult. To the best of our knowledge, it is the first time to study
Figure 1: Results of few-shot imitation learning. Left: Loss on test data; Middle: Constrained violation metric on test data. Right: Collision probability.
the problem. We provide the details of the problem formulation and the implementation setting in Appendix C.1.
We compare the proposed method with two benchmarks: (a) starting from scratch; (b) online MAML [24] with constraint penalty. In (a), the optimization problem in (3) is solved from a random initial parameter. In (b), we add a weighted penalty term for constraint violation to the loss function of the online MAML. For each task, the three methods share the data points.
Few-shot imitation learning. Consider that the learner collects the locations and velocities on a few points (few-shot data) from the expert for each task. Fig. 1 compares the proposed constrained meta-learning with the selected benchmarks. Fig. 1 Left shows that our approach has the fastest learning rate, i.e., its test error is smaller than 20 at around  $T = 10$ , while the online MAML achieves that at around  $T = 30$ , and the test error of starting from scratch is always above 20. Moreover, our approach has the best learning accuracy, i.e., the steady-state test error of our approach is much smaller than those of the other two methods. In terms of constraint violation, our approach achieves the lowest collision probability in Fig. 1 Right and the lowest constraint violation metric in Fig. 1 Middle.
Improve full-shot imitation learning by meta-learning. We consider that the learner can collect the locations and velocities on sufficient points (full-shot data) from the expert for each task, and test the proposed method and the selected benchmarks. In this experiment, we show that meta-learning can improve the performance of constrained learning, even if suf
Figure 2: Learned trajectories for full-shot imitation learning. Left: Starting from scratch; Middle: MAML with constraint penalty; Right: Constrained meta-learning. The blue line is the demonstration trajectory, the red line is the learned trajectory, the red circle represents the obstacle.
ficient data can be accessed by the learner. As shown in Fig. 2, our method can achieve superior performance than that in the two benchmark approaches (only our method can completely draw the shape of the letter). Fig. 3 shows that our method achieves a comparable adaptation time to online-MAML while outperforming online-MAML in terms of test error and collision avoidance. Both our method and online-MAML much outperform starting from scratch in terms of test error, collision avoidance, and adaptation time.
Figure 3: Results of full-shot imitation learning. Left: Loss on test data; Middle left: Constrained violation metric on test data; Middle right: Collision probability; Right: Adaptation time.
5.2 Few-shot image classification with robustness
Meta-learning has been widely used for few-shot learning. Few-shot learning aims to train a model with only a few data samples, which is widely solved from the perspective of meta-learning by learning prior knowledge from multiple tasks. Consider a problem of few-shot image classification with robust learning [57]. In particular, after it is updated by a few data points, we require that the task-specific model has high accuracy on clean test data and is also robust to adversarial attacks on the test data. Robust learning for a single task can be formulated by a constrained optimization problem [12]. Correspondingly, in this experiment, the robust few-shot image classification is formulated as a constrained meta-learning problem and is solved by the proposed algorithm. The details of the problem formulation and the implementation setting are shown in Appendix C.2.
To evaluate the performance of the proposed algorithm, we test the task-specific model on two sets of each dataset: (a) the clean test dataset; (b) the corrupted test dataset which is obtained by adding perturbations on the clean test dataset through the Projected Gradient Descent (PGD) method [37]. Correspondingly, we show the test accuracy on (a) as clean accuracy; the test accuracy on (b) as PGD accuracy. Moreover, we use Balance Score (B-score) [57] as a metric to evaluate the comprehensive performance of the model, which is defined as B-score  $= 2 \times (\mathrm{CA} \times \mathrm{PA}) / (\mathrm{CA} + \mathrm{PA})$  with CA and PA denoting clean accuracy and PGD
accuracy, respectively. We compare the proposed constrained meta-learning (CML) with several benchmarks: (i) MAML [23] with constraint penalty (CP); (ii) ProtoNet [49] with CP; (iii) BOIL [44] with CP; (iv) MAML with MOML [57]; (v) ProtoNet with MOML; (vi) BOIL with MOML. As the baseline methods for few-shot learning, MAML, ProtoNet, and BOIL, can only deal with unconstrained meta-learning, at (i)(ii)(iii), we add a weighted penalty term of the robustness constraint violation to their loss functions. At (iv)(v)(vi), MAML, ProtoNet and BOIL fit into a multi-objective meta-learning (MOML) framework [57], which regards the clean accuracy and the PDG accuracy as two competing objectives. Since all baseline methods are offline meta-learning, for comparisons, we use the similar way shown in Algorithm 2 of Appendix B to adapt them to the online setting. Specifically, in each round, we sample a batch of tasks from the revealed tasks, and use the gradient-based method on their meta-objective functions over the data of sampled tasks. The modification from the offline baseline methods to their online versions are exactly following our approach in terms of optimization of the online meta-objective.
Table 2: Clean accuracy (abbreviated as "Clean Acc.") and PGD accuracy (abbreviated as "PGD Acc.") on the mini-ImageNet dataset for 5-way 5-shot and 5-way 1-shot learning.
Method	Clean Acc.	PGD Acc.	B-score
1-shot	MAML + CP	40.78 ± 0.75	23.91 ± 0.67	29.83 ± 0.43
MAML + MOML	39.23 ± 0.76	25.80 ± 0.67	31.12 ± 0.70
ProtoNet + CP	38.65 ± 0.72	23.10 ± 0.65	28.67 ± 0.67
ProtoNet + MOML	35.06 ± 0.70	27.24 ± 0.65	30.51 ± 0.66
BOIL + CP	40.44 ± 0.79	25.94 ± 0.69	31.29 ± 0.75
BOIL + MOML	41.22 ± 0.83	27.77 ± 0.75	32.98 ± 0.79
CML (ours)	39.52 ± 0.80	33.11 ± 0.79	36.03 ± 0.79
5-shot	MAML + CP	56.16 ± 0.72	34.85 ± 0.72	42.91 ± 0.71
MAML + MOML	55.66 ± 0.78	39.38 ± 0.77	45.89 ± 0.77
ProtoNet + CP	59.11 ± 0.71	39.41 ± 0.73	46.93 ± 0.71
ProtoNet + MOML	58.72 ± 0.74	41.59 ± 0.75	48.59 ± 0.74
BOIL + CP	58.54 ± 0.76	34.28 ± 0.75	42.94 ± 0.78
BOIL + MOML	60.21 ± 0.79	35.47 ± 0.78	44.37 ± 0.78
CML (ours)	59.74 ± 0.75	49.48 ± 0.76	54.01 ± 0.74
Figure 4: Test accuracy v.s. training task index on dataset mini-ImageNet. Left: Clean accuracy; Middle: PGD accuracy; Right: B-score.
We test the algorithms on two few-shot learning datasets, CUB [53] and mini-ImageNet [52]. Due to the page limit, the results on the CUB are shown in Appendix C.2. Table 2 shows that our method significantly improves the PGD accuracy and the B-score than the benchmarks and keeps the clean accuracy comparable. Fig. 4 shows that our method outperforms the benchmarks in terms of both test accuracy and learning speed, i.e., its test accuracy is larger than 0.5 at  $T = 30$ , while the benchmarks take at least 50 tasks to achieve so.
In this paper, we develop an online constrained meta-learning framework, which learns the meta-prior from learning tasks with constraints. We theoretically quantify the optimality gaps and constraint violations produced by the proposed framework. Both two metrics consider the generalization ability of the task-specific models to unseen data. Moreover, we propose a practical algorithm for the constrained meta-learning framework. Our experiments on meta-imitation learning and few-shot image classification demonstrate the superior effectiveness of the algorithm.
This work is partially supported by the National Science Foundation through grants ECCS 1846706 and ECCS 2140175. We would like to thank the reviewers for their constructive and insightful suggestions.