# Few-Shot Federated Learning: A Federated Learning Model for Small-Sample Scenarios
## 4 Model System  
### Model Learning Process  
Figure 2 illustrates the overall workflow of the model, which is as follows:  
1. Initialization. The global model is initialized on the server side and distributed to all clients participating in the training. 2. Local Model Training by Clients. Each client trains the received global model on its local dataset. Specifically, clients calculate the prototypes for each class in their local dataset and based on these prototypes, complete the training of the model. 3. Client Parameters Upload. After the local model training is completed in the current round, each client uploads the model update parameters to the server. 4. Server-Side Aggregation. Upon receiving the model update parameters from the clients, the server averages and aggregates these parameters to obtain the global model parameters. After aggregation, the global model is evaluated for performance, specifically testing its classification accuracy on a test set. 5. Iterative Training. The server distributes the updated global model to all clients, who then commence the next round of model training. Figure 1: Schematic diagram of a small sample. Personalized Knowledge Distillation. When clients receive a global model that is not in an initialized state, they use their local model as the student model and the received global model as the teacher model. Through a personalized knowledge distillation algorithm, the student model is guided to learn knowledge relevant to its local classification task from the teacher model. Algorithm 1 shows the specific workflow. ```
1:Client Collection{}
```
2:Global model with completed training
3:Server side sends initial global model to client side
4:forClient{} ```
5:Clients use local data for model few-shot learning
6:Upload model training parameters to the server
7:endfor
8:forClient{}
```
9:Average aggregation of model parameters uploaded by clients
10:endfor
11:while Global model does not converge
12:forClient{} ```
13:\(w_{g}\gets PersonalizedKD\big{(}M_{c},M_{g}\big{)}\)
14:\(w_{c}^{t+1}=(w_{c}^{t}+w_{g})/2\)
15:endfor
16:Repeat the process until the model converges

Figure 2: Schematic of the FsFL model. ### FsFl

This section describes the training process of the FsFL model in detail. Initially, in each round, clients randomly sample their local dataset to obtain a meta-training task set \(D_{mt}=\{D_{support},D_{query}\}\) and conduct preliminary training on the meta-learner for this dataset. \[ \begin{array}{ll}w^{\prime}_{t+1}&=w^{\prime}_{t}-\alpha\
abla L\big{(}w^{ \prime}_{t};D_{support}\big{)}\\ &=w^{\prime}_{t}-\alpha\frac{1}{|D_{support}|}\sum\limits_{x_{i}\in D_{support }}\
abla L(w^{\prime}_{t};x_{i})\end{array}  \tag{6}\]

The cross-entropy loss between the global model parameters of the current round t and the data samples in the support set is calculated to determine the direction of model updates for the next round, with the learning rate controlling the pace of the model updates. Subsequently, the model's learning effectiveness is evaluated on \(D_{query}\), and the model is adjusted based on its classification performance on \(D_{support}\). Specifically, the training process on the client side proceeds as follows: Firstly, feature extraction \(f_{\theta}\) is performed on the samples in the \(D_{support}\) set using a neural network to obtain feature vectors. Then, for each category \(c\), the mean feature vector \(p_{c}\) of that category is calculated. \[p_{c}=\frac{1}{|S_{c}|}\sum\limits_{(x_{i},y_{i})\in S_{c}}f_{\theta}(x_{i}) \tag{7}\]

The image data \(x\) in \(D_{query}\) is classified based on the calculation of the L2 distance \(d\) for \(x\). \[p(y=c|x)=\frac{\exp(-d(f_{\theta}(x),p_{c}))}{\sum_{c^{\prime}}\exp(-d(f_{ \theta}(x),p_{c^{\prime}}))} \tag{8}\]

The model parameters \(\theta\) are optimized to achieve the minimum loss on \(D_{query}\).