## 3 Methodological Framework for Meta-Learning
### Theoretical Models of Meta-Learning
#### 3.1.1 Model-Based Meta-Learning
The approach relies on a well-defined metric that quantifies the similarity or distance between these embeddings. Common choices include Euclidean distance, cosine similarity, and Mahalanobis distance. Metric-based meta-learning excels in few-shot learning tasks where the model must make accurate predictions after seeing only a few examples. By learning a robust similarity measure, the model can effectively compare new, unseen data points with a small number of labeled examples (support set) to make predictions. The pseudocode of metric-based meta-learning is given in Algorithm 2. Metric-based meta-learning is a powerful technique for quickly adapting to new tasks by learning a space in which data points can be effectively compared. Its strength lies in its ability to make the most out of limited data, making it an invaluable approach among various meta-learning techniques. The success of metric-based meta-learning heavily relies on the quality of the learned embeddings. The embedding function must be capable of capturing the nuances and variances across tasks. ```
1:Initialize the model \(M\) with parameters \(\theta\)
2:for each meta-training iteration do
3: Sample a batch of tasks \(k\) from the task distribution \(P_{K}\)
4:for each task \(k\) in \(K\)do
5: Split \(k\) into support set \(D_{k}^{tr}\) and query set \(D_{k}^{test}\)
6: Embed all points in \(D_{k}^{tr}\) and \(D_{k}^{test}\) using \(M\):
7:\(Embeded\_D_{k}^{tr}=\text{Embed}(M,D_{k}^{tr},\theta)\)
8:\(Embeded\_D_{k}^{test}=\text{Embed}(M,D_{k}^{test},\theta)\)
9: Compute pairwise distances between points in \(Embeded\_D_{k}^{test}\) and \(Embeded\_D_{k}^{tr}\)
10: Assign labels to query points based on nearest neighbors in the support set
11: Calculate the loss based on the accuracy of these assignments
12:endfor
13: Update the model parameters \(\theta\) based on the loss
14:endfor
15: The model \(M\) with updated parameters \(\theta\) is now ready for new tasks Function Embed(Model \(M\), DataSet \(D\), Parameters \(\theta\)):
16: Return the set of embedded points for each data point in \(D\) using \(M\) with parameters \(\theta\)
```

**Algorithm 2** Metric-based meta-learning

#### 3.1.3 Optimization-Based Meta-Learning

Optimization-based meta-learning is a prominent approach within meta-learning, focusing on optimizing the learning process itself. This approach aims to determine optimal initial parameters or learning strategies that enable rapid adaptation to new tasks. The central idea of optimization-based meta-learning is to learn optimal initial conditions (like model parameters or learning rates) that can be quickly fine-tuned for various tasks. This approach is grounded in the hypothesis that there exists a set of initial conditions from which a model can rapidly adapt to a new task with minimal additional training. In an optimization-based ML algorithm, the model learns a set of initial parameters that are not task-specific but provide a good starting point for a variety of tasks. The model can quickly adapt these parameters to new tasks using only a few training examples. It involves two nested learning processes: an inner loop that performs task-specific learning and an outer loop that optimizes the initial parameters across tasks. The pseudocode for optimization-based ML is given in Algorithm 3. ```
1:Initialize model parameters \(\theta\)
2:for each meta-training iteration do
3: Sample a batch of tasks \(k\) from the task distribution \(P_{K}\)
4:for each task \(k\) in \(K\)do
5: Adapt \(\theta\) to task \(k\) to get \(\theta_{k}^{\prime}\):
6:\(\theta_{i}^{\prime}=\theta-\alpha\
abla_{\theta}L(k;\theta)\)
7: Evaluate adapted model on \(k\)
8:endfor
9: Update initial parameters \(\theta\) based on performance across tasks:
10:\(\theta=\theta-\beta\
abla_{\theta}\sum_{k}L(k;\theta_{k}^{\prime})\)
11:endfor
12: The model with updated initial parameters \(\theta\) is now ready for rapid adaptation to new tasks