SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yunzhe Qi *
yunzheq2@illinois.edu
Yikun Ban*
yikunb2@illinois.edu
Tianxin Wei
twei10@illinois.edu
Jiaru Zou
jiaruz2@illinois.edu
Huaxiu Yao
Chapel Hill, NC
huaxiu@cs.unc.edu
Jingrui He
jingrui@illinois.edu
Meta-learning has been proven an effective learning paradigm for training machine learning models with good generalization ability. Apart from the common practice of uniformly sampling the meta-training tasks, existing methods working on task scheduling strategies are mainly based on pre-defined sampling protocols or the assumed task-model correlations, and greedily make scheduling decisions, which can lead to sub-optimal performance bottlenecks of the meta-model. In this paper, we propose a novel task scheduling framework under Contextual Bandits settings, named BASS, which directly optimizes the task scheduling strategy based on the status of the meta-model. By balancing the exploitation and exploration in meta-learning task scheduling, BASS can help tackle the challenge of limited knowledge about the task distribution during the early stage of meta-training, while simultaneously exploring potential benefits for forthcoming meta-training iterations through an adaptive exploration strategy. Theoretical analysis and extensive experiments are presented to show the effectiveness of our proposed framework.
Meta-learning algorithms [19] have been receiving increased attention due to their strong generalization performance across a wide range of tasks [32, 20]. Most existing meta-learning methods often assume a uniform distribution when drawing meta-training tasks, treating each task as equally important [19]. However, this assumption can possibly fail in real-world scenarios. For example, during data collection, candidate training tasks can be subject to noise perturbation, leading to performance bottlenecks in the meta-model if noisy and clean tasks are treated on an equal footing [43, 45]. In addition, some tasks can pose greater challenges for the meta-model to adapt to, necessitating a more flexible allocation of computational resources during the meta-training process. Furthermore, the task distribution may be skewed, with "tail" tasks receiving inadequate attention under uniform sampling. As a result, the task scheduling methods [14, 51, 31, 24, 29] have been proposed for a refined meta-training strategy.
Existing scheduling approaches mainly aim to improve meta-training strategies based on various pre-defined criteria and assumptions, including both non-adaptive and adaptive methods. Among them, non-adaptive methods working on the gradient that updates trainable parameters [31, 13] have proven their superiority over meta-training methods with uniform sampling. But they are unable to adjust the task scheduling strategy adaptively based on the status of the meta-model. On the other hand, adaptive methods aim to schedule the tasks based on Curriculum Learning [14, 52, 50] or adaptively sample the tasks based on the task adaptation difficulty (loss) [51]. However, the existing approaches are all greedy algorithms, which means that they tend to make locally optimal decisions based on the current knowledge (i.e., exploitation only). As the learner only has limited knowledge regarding the task and data distribution at the early stage of meta-training, the greedy
Figure 1: Exploitation and exploration in meta-training iteration  $k \in [K]$ . E.g., tasks 1, 2, 3 are "frequent" tasks in a skewed task distribution, and tasks 4, 5, 6 are from task distribution "tail". Exploring the "tail" tasks can help improve the meta-model generalization performance (Subsec. 6.2).
strategy can lead to the sub-optimal meta-model due to multiple reasons, such as being misled by the noisy tasks or affected by the skewness of the task distribution. Here, we use Figure 2 to illustrate an example where the greedy approach (only exploitation) may be trapped in sub-optimal solutions.
One intuitive solution for the aforementioned problems is tackling the exploitation-exploration dilemma [4] in meta-training process: balancing the exploitation of current knowledge of selected tasks, and the exploration of under-explored tasks for potential long-term benefits. Therefore, unlike existing approaches with greedy strategies, we propose a novel task scheduling framework under the Contextual Bandits settings [15, 28, 4, 1, 54, 53, 2], named BA ndit TaSk Scheduler (BASS). In each meta-training iteration, we formulate each candidate meta-training task as an arm under the contextual bandit settings, and the corresponding arm reward will naturally be the meta-model generalization ability score after including this candidate task (arm) into the meta-training process. To achieve this objective, BASS directly learns the mapping from the meta-parameters to the meta-model generalization ability score, instead of depending on the hand-crafted criteria or assumptions. This design also enables us to update meta-parameters and BASS simultaneously within one round of meta-optimization. In particular, instead of greedily scheduling the meta-training tasks solely based on the current knowledge (i.e., exploitation), BASS leverages an additional adaptive exploration module with two different exploration objectives to explore for unrevealed benefits. Our main contributions can be summarized as follows:
- [Problem and Proposed Framework] We are the first to formulate the meta-learning task scheduling problem under the Contextual Bandits settings, where we optimize the meta-model w.r.t. chosen task batches in each meta-training iteration. To tackle this problem, we propose a novel bandit-based meta-learning task scheduling framework named BASS, which is model-agnostic and can be applied to various meta-learning frameworks. Different from existing works, BASS directly learns the relationship between the meta-model parameters and the meta-model generalization ability. In addition, instead of greedily exploiting the current knowledge as in existing works, BASS utilizes a novel exploration module to adaptively plan for the future meta-training iterations.
- [Theoretical Analysis] Under the general meta-learning settings as well as standard assumptions of over-parameterized neural networks and neural bandits, we derive the theoretical performance guarantee for the proposed BASS framework in terms of regret bound.
- [Experiments] To demonstrate the effectiveness of BASS, we compare our method against seven strong baselines on three real data sets, with different specifications. In addition, complementary experiments as well as a case study on Ensemble Inference are also provided to better understand the property and behavior of BASS.
In this section, we briefly review the related works of our proposed BASS framework from the aspects of task scheduling in meta-learning and contextual Bandits.
Task Scheduling in Meta-Learning. By considering each meta-training task to be equally important, many existing works sample the training tasks uniformly from the given task distribution [19, 40]. Then, with fixed sampling strategies, [24, 35] propose to assign the task sampling probability based on the quantity of task information, and [31] utilizes a probabilistic sampling method based on class-pairs. Meanwhile, there are also works try to improve task-specific gradients for the randomly sampled tasks [13, 34]. On the other hand, Curriculum-based approaches [14, 52, 50] schedule the tasks based on the difficulty of task adaptation, and [51] propose to adaptively schedule the tasks based on the assumed correlation between the difficulty of meta-training tasks and the meta-model generalization ability. However, the existing works are all greedy approaches that solely focus on the instant benefits, which can lead to sub-optimal meta-models due to the potential performance bottleneck. Compared with existing works, our proposed BASS directly learns the mapping from the meta-parameters to the generalization loss with no pre-defined criteria. With the adaptive exploration strategy, our proposed BASS helps tackle the insufficient knowledge regarding the task distribution by balancing the exploitation and exploration, as well as focusing on the long-term effects.
Contextual Bandits. Contextual bandits algorithms aim to solve the sequential decision making problem under the online learning settings, such as online recommendation [28, 49, 6]. Assuming that the mapping from arm contexts to rewards is linear, linear upper confidence bound (UCB) algorithms [15, 28, 4] are proposed to solve the exploitation-exploration dilemma. After kernel-based methods [46, 17] being applied to deal with the non-linear setting where the reward mapping is assumed to be a function in Reproducing Kernel Hilbert Space (RKHS), neural bandit algorithms [54, 53, 7, 5, 39, 8] are proposed to utilize neural networks for the reward and confidence bound estimation. In particular, neural bandit algorithms have demonstrated their superior performance over linear and kernel-based algorithms [54], thanks to the representation power of neural networks. Moreover, instead of using non-negative UCB, EE-Net [9, 10] achieves adaptive exploration by adopting an additional neural network to estimate the potential gain of reward estimations. In this paper, we model the task scheduling problem under the Contextual Bandit settings to balance the exploitation and exploration dilemma regarding the meta-task scheduling.
3 Problem Definition and Learning Objective
Under the settings of few-shot supervised meta-learning [19], our goal is to train a meta-model  $\mathcal{F}(\cdot;\Theta)$  with parameters  $\Theta \in \mathbb{R}^p$  that can generalize well to meta-testing tasks, where  $p$  is the number of trainable meta-parameters. The meta-model parameters are initialized as  $\Theta^{(0)}$ . Note that in this work, the trainable parameters are all represented by column vectors, for the simplicity of notation. Following similar settings in [19, 51], in each training iteration  $k \in [K]$ , we will receive a pool of candidate training tasks  $\Omega_{\mathrm{task}}^{(k)} = \{\mathcal{T}_{k,i}\}_{i \in \mathcal{N}}$  where its cardinality  $|\Omega_{\mathrm{task}}^{(k)}| = N_{\mathrm{task}}$  and the index set  $\mathcal{N} = \{1,\dots,N_{\mathrm{task}}\}$ . Given a task distribution  $\mathcal{P}(\mathcal{T})$ , we draw each candidate task  $\mathcal{T}_{k,i}$  from  $\mathcal{P}(\mathcal{T})$  to form the candidate pool, i.e.,  $\mathcal{T}_{k,i} \sim \mathcal{P}(\mathcal{T})$ . Each task  $\mathcal{T}_{k,i}$  is also associated with a support data set  $D_{k,i}^{s}$  and a query data set  $D_{k,i}^{q}$ , where the samples (including their labels) of  $D_{k,i}^{s}, D_{k,i}^{q}$  are drawn from the corresponding task data distribution  $\mathcal{D}_{\mathcal{T}_{k,i}}$  of task  $\mathcal{T}_{k,i}$ .
Meta-training Process. Then, we need to choose a batch of  $B$  tasks  $\Omega_{k} = \{\mathcal{T}_{k,\widehat{i}}\}_{\widehat{i}\in \widehat{\mathcal{N}}_{k}}\subset \Omega_{\mathrm{task}}^{(k)}$  for each meta-training iteration  $k\in [K]$ , where  $\widehat{\mathcal{N}_k}\subset \mathcal{N}$  refers to the indices of chosen tasks. With  $\Theta^{(k - 1)}$  being the meta-model parameters after completing the  $(k - 1)$ -th meta-training iteration, for each chosen meta-training task  $\mathcal{T}_{k,\widehat{i}},\widehat{i}\in \widehat{\mathcal{N}}_k$ , we run Gradient Descent (GD) for  $J$  steps on its support set  $D_{k,\widehat{i}}^s$  to obtain the task-specific parameters  $\Theta_{k,\widehat{i}}^{(J)}$ . This refers to the inner-loop optimization. In this work, we consider a loss function  $\mathcal{L}(\cdot ;\cdot)$  that maps the meta-model parameters and the input sample (or sample set) to the loss value, with the range [0, 1] (e.g., the MSE loss on normalized user ratings [27]). Denoting  $\Theta_{k,\widehat{i}}^{(0)} = \Theta^{(k - 1)}$  in meta-training iteration  $k$ , each GD step is represented as
$$
\boldsymbol {\Theta} _ {k, \widehat {i}} ^ {(j)} = \boldsymbol {\Theta} _ {k, \widehat {i}} ^ {(j - 1)} - \eta_ {1} \cdot \nabla_ {\boldsymbol {\Theta}} \mathcal {L} \left(D _ {k, \widehat {i}} ^ {s}; \boldsymbol {\Theta} _ {k, \widehat {i}} ^ {(j - 1)}\right), \quad \forall j \in [ J ] \tag {1}
$$
where  $\eta_1 \in \mathbb{R}^+$  is the learning rate of GD. For the simplicity of notation, we formulate the above  $J$ -iteration inner-loop optimization as an operator  $\mathcal{I}(\cdot, \cdot): \mathcal{T} \times \Theta \mapsto \Theta$ , such that
$$
\boldsymbol {\Theta} _ {k, \widehat {i}} ^ {(J)} = \mathcal {I} \left(\mathcal {T} _ {k, \widehat {i}}, \boldsymbol {\Theta} ^ {(k - 1)}\right). \tag {2}
$$
Figure 2: In meta-training iteration  $k \in [K]$ , the BASS framework overview. We only need one round of the optimization process (LHS of the figure) to update the meta-model and BASS.
Then, we optimize the meta-parameters through the outer-loop optimization with query sets
$$
\boldsymbol {\Theta} ^ {(k)} = \boldsymbol {\Theta} ^ {(k - 1)} [ \Omega_ {k} ] = \boldsymbol {\Theta} ^ {(k - 1)} - \eta_ {2} \cdot \nabla_ {\boldsymbol {\Theta}} \left(\frac {1}{| \Omega_ {k} |} \sum_ {\mathcal {T} _ {k, \widehat {i}} \in \Omega_ {k}} \mathcal {L} \left(D _ {k, \widehat {i}} ^ {q}; \boldsymbol {\Theta} _ {k, \widehat {i}} ^ {(J)}\right)\right) \tag {3}
$$
where  $\eta_{2} \in \mathbb{R}^{+}$  is the learning rate. Here, the trained meta-model parameters  $\Theta^{(K)}$  is expected to minimize the generalization loss  $\mathbb{E}_{\mathcal{T} \sim \mathcal{P}(\mathcal{T}), x \sim \mathcal{D}_{\mathcal{T}}} \left[ \mathcal{L}(x; \mathcal{I}(\mathcal{T}, \Theta^{(K)})) \right]$ , where we let  $\mathcal{D}_{\mathcal{T}}$  being the associated data distribution of task  $\mathcal{T}$ .
Evaluating Task Batch Benefits. Recall that in meta-training iteration  $k \in [K]$ , the meta-model will be updated based on the chosen batch of  $B$  tasks (Eq. 3), and we will need to evaluate the benefit in terms of the whole task batch. By the task scheduling problem definition, the learner will select one task batch  $\Omega_k \subset \Omega_{\text{task}}^{(k)}$  based on its strategy, and the chosen task batch  $\Omega_k$  will be used for meta-training in this iteration  $k$ . Here, we define the reward of the corresponding task batch  $\Omega_k$  as
$$
\left. h \left(\Theta^ {(k - 1)} [ \Omega_ {k} ]\right) = 1 - \mathbb {E} _ {\mathcal {T} \sim \mathcal {P} (\mathcal {T}), \boldsymbol {x} \sim \mathcal {D} _ {\mathcal {T}}} \left[ \mathcal {L} \left(\boldsymbol {x}; \mathcal {I} \left(\mathcal {T}, \Theta^ {(k - 1)} [ \Omega_ {k} ]\right)\right) \right] \right. \tag {4}
$$
where  $\Theta^{(k - 1)}[\Omega_k]$  refer to the meta-parameters after adapting  $\Theta^{(k - 1)}$  to task batch  $\Omega_{k}$  based on Eq. 3. For simplicity, we let  $h:\mathbb{R}^p\mapsto \mathbb{R}$  be the mapping function conditioned on distribution  $\mathcal{P}(\mathcal{T})$  which maps the trained meta-model parameters  $\Theta^{(k - 1)}[\Omega_k]$  to task batch rewards.
Learning Objective. Up to meta-training iteration  $k$ , we have  $\Omega^{*}(K) = \{\Omega_{1}^{*},\dots,\Omega_{K}^{*}\}$  being a series of unknown optimal task batches that minimizes the generalization loss. Each optimal batch is  $\Omega_k^* = \{\mathcal{T}_{k,i^*}\}_{i^*\in \mathcal{N}_k^*}\subset \Omega_{\mathrm{task}}^{(k)},k\in [K]$  including a total of  $|\Omega_k^* | = B$  tasks, with the indices  $\mathcal{N}_k^*\subset \mathcal{N}$ . The corresponding optimal meta-parameters are defined as  $\Theta^{(K),*} = \arg \inf_{\Theta \in \mathcal{W}}\left[\mathbb{E}_{\mathcal{T}\sim \mathcal{P}(\mathcal{T}),\boldsymbol {x}\sim \mathcal{D}_{\mathcal{T}}}\bigl [\mathcal{L}\bigl (\boldsymbol {x};\mathcal{I}(\mathcal{T},\Theta)\bigr)\bigr ]\right]$ , where  $\mathcal{W}\subset \mathbb{R}^p$  refers to the reachable parameter space, which contains the possible trained meta-parameter values, after observing the available candidate tasks  $\{\Omega_{\mathrm{task}}^{(k)}\} ,k\in [K]$  and the randomly initialized meta-parameters  $\Theta^{(0)}$ . Meanwhile, denoting our selection of the meta-training task batches as  $\Omega (K) = \{\Omega_1,\ldots ,\Omega_K\}$ , we will have the corresponding trained meta-parameters  $\Theta^{(K)}$ . Here, we can define the  $K$ -iteration regret as
$$
R (K) = \mathbb {E} _ {\mathcal {T} \sim \mathcal {P} (\mathcal {T}), \boldsymbol {x} \sim \mathcal {D} \mathcal {T}} \left[ \mathcal {L} \left(\boldsymbol {x}; \mathcal {I} \left(\mathcal {T}, \boldsymbol {\Theta} ^ {(K)}\right)\right) - \mathcal {L} \left(\boldsymbol {x}; \mathcal {I} \left(\mathcal {T}, \boldsymbol {\Theta} ^ {(K), *}\right)\right) \right] \tag {5}
$$
which measures the difference of the generalization ability between the trained meta-parameters  $\Theta^{(K)}$  and the optimal meta-parameters  $\Theta^{(K),*}$ , after  $K$  meta-training iterations.
4 Proposed Framework: BASS
In this section, we introduce our proposed BASS framework, which simultaneously optimizes the meta-model and the task scheduling strategy on the fly. The pseudo-code is presented in Algo. 1, and the illustration for each meta-training iteration  $k \in [K]$  is shown in Figure 2.
Algorithm 1 BA ndit TaSk Scheduler (BASS)
1: Input: Task distribution  $\mathcal{P}(\mathcal{T})$ . Iterations  $K$ . GD steps  $J$ . Number of chosen tasks  $B$ . Learning rates for meta-model  $\eta_{1}, \eta_{2}$ . Learning rates for BASS  $\eta_{1}^{\theta}, \eta_{2}^{\theta}$ . Exploration coefficient  $\alpha \in (0, 1]$ .  
2: Output: Trained meta-parameters  $\Theta^{(K)}$ .  
3: Initialization: Randomly initialized meta-model parameters  $\Theta^{(0)}$ , and BASS parameters  $\theta^{(0)}$ .  
4: for each meta-training iteration  $k \in [K]$  do  
5: Sample a pool of  $N_{\mathrm{task}}$  candidate meta-training tasks  $\Omega_{\mathrm{task}}^{(k)}$  from the task distribution  $\mathcal{P}(\mathcal{T})$ .  
6: -- Task Scheduling --  
7: Derive two arm context vectors  $\chi_{k,i}^{s}, \chi_{k,i}^{q}$  [Eq. 6].  
8: Calculate arm benefit score  $\widehat{s}_{k,i} = \alpha \cdot f_1(\cdot; \theta_1^{(k-1)}) + f_2(\cdot; \theta_2^{(k-1)})$  with BASS. [Eq. 7]  
9: end for  
10: From  $\Omega_{\mathrm{task}}^{(k)}$ , choose the top- $B$  tasks  $\Omega_k \subset \Omega_{\mathrm{task}}^{(k)}$  with the highest scores  $\{\widehat{s}_{k,i}\}_{i \in \mathcal{N}}$ .  
11: -- Parameter Updating --  
12: for each chosen training task  $\mathcal{T}_{k,\widehat{i}} \in \Omega_k$  do  
13: Derive the corresponding rewards  $\widetilde{r}_{k,i}$  and exploration score  $\widetilde{e}_{k,i}$ . [Eq. 8-9]  
14: end for  
15: Update meta-parameters to  $\Theta^{(k)}$  based on chosen arms (tasks)  $\mathcal{T}_{k,\widehat{i}} \in \Omega_k$ . [Remark 2, Eq. 3]  
16: Update BASS parameters to  $\theta^{(k)}$  with chosen arms. [Eq. 8-9, Subsec. 4.4]
Remark 1 (Task Scheduling with Contextual Bandits). By the problem definition, there are  $N_{\text{batch}} = \binom{N_{\text{task}}}{B}$  candidate task batches in each meta-training iteration, which can be a large number. In this case, enumerating all possible task batches and estimating their rewards will be time consuming. Therefore, under the Contextual Bandits settings, BASS alternatively considers each candidate task  $T_{k,i} \in \Omega_{\text{task}}^{(k)}$  as an arm, and directly chooses  $B$  arms as the meta-training tasks  $\Omega_k \subset \Omega_{\text{task}}^{(k)}$ . As a result, BASS can (1) reduce the arm space size from  $\mathcal{O}(N_{\text{batch}})$  to  $\mathcal{O}(N_{\text{task}})$ , while (2) enjoy the performance guarantee (Section 5) in terms of regret bound (Eq. 5).
Section Outline. Here, we will first present our definition of the arm contexts (Subsec. 4.1), whose formulation is challenging, because our settings are different from conventional Contextual Bandits where arm contexts are readily available from the environment. Then, applying two neural networks  $f_{1}, f_{2}$  for exploitation and exploration respectively, we formulate the arm benefit score (Subsec. 4.2), which measures the benefit if we include the corresponding arm (task) into the current meta-training process. Next, we define the arm rewards and exploration scores as the labels for training  $f_{1}, f_{2}$  respectively. In particular, to deal with the challenge of achieving exploration in task scheduling, we incorporate the information and the dynamics w.r.t. meta-optimizations, and formulate two separate exploration objectives for a refined exploration strategy (Subsec. 4.3). Finally, we update BASS with GD (Subsec. 4.4), and train the meta-model with chosen tasks.
4.1 Formulating Arm Contexts
To encode the information from both the task side and the meta-model side, for each candidate task (i.e., arm)  $\mathcal{T}_{k,i} \in \Omega_{\mathrm{task}}^{(k)}$  in meta-training iteration  $k \in [K]$ , we formulate its arm contexts as the meta-parameters after task adaptations, denoted by
$$
\chi_ {k, i} ^ {s} := \Theta_ {k, i} ^ {(J)} = \mathcal {I} \left(\mathcal {T} _ {k, i}, \Theta^ {(k - 1)}\right); \quad \chi_ {k, i} ^ {q} := \Theta^ {(k - 1)} \left[ \mathcal {T} _ {k, i} \right] = \Theta^ {(k - 1)} - \eta_ {2} \nabla_ {\Theta} \mathcal {L} \left(D _ {k, i} ^ {q}; \Theta_ {k, i} ^ {(J)}\right) \tag {6}
$$
where  $\Theta_{k,i}^{(J)}$  are the task-specific parameters after adapting meta-parameters  $\Theta^{(k - 1)}$  to task  $\mathcal{T}_{k,i}$  with inner-loop optimization (Eqs. 1-2); while  $\Theta^{(k - 1)}[\mathcal{T}_{k,i}]$  refer to the meta-parameters after adapting the current  $\Theta^{(k - 1)}$  to task  $\mathcal{T}_{k,i}$ , with both inner-loop and outer-loop optimization (as in Eq. 3). In particular, we assign each arm  $\mathcal{T}_{k,i}$  with two different arm contexts to model the dynamics of meta-parameters w.r.t. inner-loop optimization and outer-loop optimization respectively. For example, the variance of the corresponding data distribution  $\mathcal{D}_{\mathcal{T}_{k,i}}$  can be high. In this case, the support set  $D_{k,i}^{s}$  and the query set  $D_{k,i}^{q}$  will be considerably different, which tends to make the corresponding
arm contexts  $\chi_{k,i}^{s}$ ,  $\chi_{k,i}^{q}$  divergent. As a result, the gradient vectors  $\nabla_{\theta}f_{1}(\chi_{k,i}^{s})$ ,  $\nabla_{\theta}f_{1}(\chi_{k,i}^{q})$  will likely be distinct from each other. Alternatively, if the support set  $D_{k,i}^{s}$  and the query set  $D_{k,i}^{q}$  are not significantly distinct (the distance between  $\chi_{k,i}^{s}$  and  $\chi_{k,i}^{q}$  is also likely to be relatively small), these two gradient vectors tend to change dramatically when adapting to  $\mathcal{T}_{k,i}$ . The reason is possibly that the exploitation model  $f_{1}$  is not well adapted to this task  $\mathcal{T}_{k,i}$ . For both scenarios above, it can be beneficial to include more exploration for the task  $\mathcal{T}_{k,i}$ , and the target is helping  $f_{1}$  better learn the reward for this task by actively acquiring the knowledge of it. And the two formulated arm contexts can provide important reference for our exploration module.
Remark 2 (Recycling Arm Contexts). In order to derive the arm contexts (Eq. 6), the gradients for the outer-loop optimization  $\nabla_{\Theta}\mathcal{L}(D_{k,\hat{i}}^{q};\Theta_{k,\hat{i}}^{(J)})$ $\widehat{i}\in \widehat{\mathcal{N}}_k$  of the chosen arms  $\mathcal{T}_{k,\widehat{i}}\in \Omega_k$  are calculated. As a result, these gradients can be recycled to update the meta-model parameters based on Eq. 3 (line 15, Algo. 1), which helps reduce the computational cost when updating the meta-model.
4.2 Estimating Benefit Scores for Tasks