## 3 Methodological Framework for Meta-Learning
### Theoretical Models of Meta-Learning
#### 3.1.1 Model-Based Meta-Learning  
Model-based meta-learning is one of the primary approaches in meta-learning. In this approach, the learning model itself incorporates mechanisms to remember or internalize information from tasks it has previously encountered. This approach often employs memory-augmented neural networks or models with dynamic components that can alter their behavior based on the data they see. The model-based meta-learning is equipped with an internal memory or state that stores information about the tasks it has learned. This memory aids in the quick adaptation to new tasks. The model's internal structure can change dynamically in response to new data, allowing it to apply previously learned knowledge to new situations. Model-based meta-learning excels in scenarios where tasks share underlying structures or patterns. The internal memory or state of the model allows for leveraging past experiences, making it particularly effective in few-shot learning and tasks with limited data. However, one of the main challenges in model-based ML is the design of the memory mechanism. It needs to be capable of storing relevant information without becoming overwhelmed by less useful data. Additionally, ensuring that the model generalizes well to truly novel tasks, not just variations of previously seen ones, can be complex. To allow a better understanding of the algorithm, the pseudo-code of model-based meta-learning is given in Algorithm 1:```
1:Initialize the model \(M\) with parameters \(\theta\) and initial state \(s\)
2:for each meta-training iteration do
3: Sample a batch of tasks \(k\) from the task distribution \(P_{k}\)
4:for each task \(k\) in \(K\)do
5: Split \(K\) into support set \(D_{k}^{tr}\) and query set \(D_{k}^{test}\)
6: Update the model's state \(s_{k}\) based on \(D_{k}^{tr}\):
7:\(s_{k}^{\prime}=\text{UpdateState}(M,D_{k}^{tr},\theta,s)\)
8: Evaluate the model on \(D_{k}^{test}\) using the updated state \(s_{k}^{\prime}\):
9:\(loss_{k}=\text{ComputeLoss}(M,D_{k}^{test},\theta,s_{k}^{\prime})\)
10:endfor
11: Update the model parameters \(\theta\) based on the aggregated loss across tasks:
12:\(\theta=\text{UpdateParameters}(\theta,\{loss_{k}\})\)
13:endfor
14: The model \(M\) with updated parameters \(\theta\) is now ready for rapid adaptation to new tasks Function UpdateState(Model \(M\), SupportSet \(D\_tr\), Parameters \(\theta\), State \(s\)):
15: Process \(D_{tr}\) through \(M\) with parameters \(\theta\) and state \(s\)
16: Update and return the new state \(s^{\prime}\)
17: Function ComputeLoss(Model \(M\), QuerySet \(D\_test\), Parameters \(\theta\), State \(s^{\prime}\)):
18: Make predictions on \(D_{test}\) using \(M\) with \(\theta\) and updated state \(s^{\prime}\)
19: Calculate and return the loss based on the predictions and true labels Function UpdateParameters(Parameters \(\theta\), Losses \(\{loss_{k}\}\)):
20: Perform an optimization step (e.g., gradient descent) to minimize the aggregated losses Return the updated parameters \(\theta\)
```

**Algorithm 1** Model-based meta-learning

#### 3.1.2 Metric-Based Meta-Learning

Metric-based meta-learning is an approach within meta-learning that focuses on learning a measure of similarity (or a metric) between data points. It is particularly effective in few-shot learning scenarios, where the goal is to classify new data points based on a small number of examples. In metric-based meta-learning, the learning model is trained to embed input data points into a feature space where similar points are close together, and dissimilar points are far apart. The key idea is to learn a metric or distance function that can be used to compare new data points with a small set of labeled examples (support set) to make predictions. At the heart of metric-based meta-learning is an embedding function that maps input data into a feature space. This function is represented by a neural network with parameters \(\theta\) that transforms raw data into embeddings where similar instances are closer together, and dissimilar ones are further apart.