SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \operatorname {K L} \left(q \left(w ^ {m}, \gamma^ {m} \mid \phi\right) \| p \left(w ^ {m}, \gamma^ {m} \mid \mathcal {D} ^ {m}\right)\right) = \int q \left(w ^ {m}, \gamma^ {m} \mid \phi\right) \log \frac {q \left(w ^ {m} , \gamma^ {m} \mid \phi\right)}{p \left(w ^ {m} , \gamma^ {m} \mid x ^ {m} , y ^ {m}\right)} \partial w ^ {m} \partial \gamma^ {m} \\ = \int q \left(w ^ {m}, \gamma^ {m} \mid \phi\right) \log \frac {q \left(w ^ {m} , \gamma^ {m} \mid \phi\right) p \left(y ^ {m} \mid x ^ {m}\right)}{p \left(y ^ {m} \mid x ^ {m} , w ^ {m}\right) p \left(w ^ {m} , \gamma^ {m}\right)} \partial w ^ {m} \partial \gamma^ {m} (7) \\ = \int q (w ^ {m}, \gamma^ {m} | \phi) \left\{\log \frac {q (w ^ {m} , \gamma^ {m} | \phi)}{p (w ^ {m} , \gamma^ {m})} + \log p (y ^ {m} | x ^ {m}) - \log p (y ^ {m} | x ^ {m}, w ^ {m}) \right\} \partial w ^ {m} \partial \gamma^ {m} \\ = \operatorname {K L} \left(q \left(w ^ {m}, \gamma^ {m} | \phi\right) \| p \left(w ^ {m}, \gamma^ {m}\right)\right) + \log p \left(y ^ {m} \mid x ^ {m}\right) - \mathbb {E} _ {q \left(w ^ {m}, \gamma^ {m} | \phi\right)} \left[ \log p \left(y ^ {m} \mid x ^ {m}, w ^ {m}\right) \right]. (8) \\ \end{array}
$$
Eq. (2) is derived from Bayes' rule:  $p(w^m, \gamma^m | \mathcal{D}^m) = \frac{p(y^m | x^m, w^m)p(w^m, \gamma^m)}{p(y^m | x^m)}$ . By reordering Eq.(3), we get
$$
\begin{array}{l} \log p \left(y ^ {m} \mid x ^ {m}\right) \geq \mathbb {E} _ {q \left(w ^ {m}, \gamma^ {m} \mid \phi\right)} \left[ \log p \left(y ^ {m} \mid x ^ {m}, w ^ {m}\right) \right] - \mathrm {K L} \left(q \left(w ^ {m}, \gamma^ {m} \mid \phi\right) \| p \left(w ^ {m}, \gamma^ {m}\right)\right) (9) \\ = \mathbb {E} _ {q (w ^ {m} | \phi)} \mathbb {E} _ {q (\gamma^ {m})} [ \log p (y ^ {m} | x ^ {m}, w ^ {m}) ] - \operatorname {K L} (q (w ^ {m}, \gamma^ {m} | \phi) | | p (w ^ {m}, \gamma^ {m})) \\ = \mathbb {E} _ {q \left(w ^ {m} \mid \phi\right)} \left[ \log p \left(y ^ {m} \mid x ^ {m}, w ^ {m}\right) \right] - \mathrm {K L} \left(q \left(w ^ {m}, \gamma^ {m} \mid \phi\right) \mid p \left(w ^ {m}, \gamma^ {m}\right)\right) (10) \\ \end{array}
$$
The lower-bound in Eq.(4) is due to the positivity of the  $\mathrm{KL}(q(w^m,\gamma^m|\phi)||p(w^m,\gamma^m|\mathcal{D}^m))$ . Here, Eq. (5) corresponds to the ELBO objective of Eq. (3) in the manuscript.
KL term. If we further decompose the KL divergence term in Eq. (5),
$$
\begin{array}{l} \mathrm {K L} \left(q \left(w ^ {m}, \gamma^ {m} | \phi\right) | | p \left(w ^ {m}, \gamma^ {m}\right)\right) = \mathrm {K L} \left(q \left(w ^ {m} | \phi\right) | | p \left(w ^ {m} | \gamma^ {m}\right)\right) + \mathrm {K L} \left(q \left(\gamma^ {m}\right) | | p (\gamma^ {m})\right) \\ = \sum_ {k = 1} ^ {K} \left\{0. 5 \log \left(1 + \left(\alpha_ {k} ^ {m}\right) ^ {- 1}\right) \right\} + \sum_ {k = 1} ^ {K} \left\{\log (b - a) \right\} \tag {11} \\ \end{array}
$$
The  $\log (b - a)$  is independent of the unknown variables  $\alpha^m$ ,  $\theta$ , and  $\gamma^m$ . Thus, we do not need to specify the value of hyperparameters  $a$  and  $b$  and can neglect them in practice. Eq. (6) provides the rationale behind the
3For a more detailed proof of this, please see the appendix section of [47]
KL divergence term that we mentioned in Section 3.2 of the manuscript. In fact, the independence between the KL term and the parameter  $\theta$  ensures compatibility with other optimization meta-learning algorithms. Also, the two-level structure in a hierarchical system can generate a much more complex distribution, expanding the potential solution spaces for selecting feasible prior. Thus, the hierarchical prior is a suitable prior for interpreting the variety of different clients' models in the FL environment. The same hierarchical prior is uniformly applied across all  $1\dots M$  clients to ensure the global posterior decomposition assumption in Eq.(1).
C Additional Related Works
Bayesian Neural Networks and Variational Dropout. To address the issue of overfitting caused by limited data, Bayesian neural networks (BNN) [93, 56, 94, 57] were proposed, which impose a prior distribution on each parameter. Various inference approaches have been proposed to model the posterior distribution of BNNs [58, 46, 59]. Variational dropout (VD) [60, 46, 61, 62, 47] encompasses a set of techniques that model the variational posterior distribution based on the dropout regularization method. Dropout regularization [95, 96] randomly deactivates some of the model parameters during training by multiplying them with discrete Bernoulli random noise. This method was initially popularized to prevent overfitting in neural network models. Subsequently, fast dropout [63, 97] demonstrated that multiplying the parameters with continuous noise sampled from Gaussian distributions yields similar results to conventional Bernoulli dropout [95]. VD approaches often employ sparse priors for regularization [46, 61, 62, 47], which facilitate the learning of independent dropout rates on the neural network parameters as variational parameters. This property distinguishes VD approaches from conventional dropout, which uses a single fixed rate for all parameters.
D Dataset and Methods
We follow the datasets and the evaluation protocol of pFL-Bench [18], which is a recently proposed benchmark for federated learning.
Datasets. Here, we present descriptions of the dataset used in our experiment.
- The CIFAR-10 and CIFAR-100 datasets [98] are popular for 10-class and 100-class image classification respectively. Each dataset contains 50,000 training and 10,000 test images with a resolution of  $32 \times 32$  pixels. Following the heterogeneous partition manners used in [18], we use Dirichlet allocation to split this dataset into 130 clients with different Dirichlet parameters as  $\dot{\alpha} = [5, 0.5, 0.1]$  (a smaller  $\dot{\alpha}$  indicates a higher heterogeneous degree).
- The Federated Extended MNIST (FEMNIST) is a widely used FL dataset for 62-class handwritten character recognition [72]. The original FEMNIST dataset contains 3,550 clients and each client corresponds to a character writer from EMNIST [99]. Following [18], we adopt the sub-sampled version, which contains 400 clients and a total of 85350 training and 21536 test images with a resolution of 28x28 pixels.
- The CelebA is a FL dataset based on [100] for 2-class image classification; Smiling or Not. Following [18], we adopt the sub-sampled version, which contains 500 clients and a total of 8752 training and 2347 test images with a resolution of  $84 \times 84$  pixels. Each client is assigned images of a single celebrity.
We randomly select 30 clients for CIFAR-10 and CIFAR-100 and 40 clients for FEMNIST as OOD clients who do not participate in the FL processes. For all of the datasets, we follow the same heterogeneous patterns exhibited in the pFL-Bench, which covers a wide range of scales, partition manners, and non-IID degrees. This enables comprehensive comparisons and analysis among different methods in the non-IID data environment.
Baselines. We present an overview of the baseline models used in our experiment, covering various popular and state-of-the-art approaches across three categories: Non-PFL, meta-learning-based PFL, and Bayesian FL methods.
The following Non-PFL methods are considered in our experiments:
- FedAvg [1] is a standard FL algorithm that averages gradients weighted by the data size of clients in each FL round.
- FedProx [73] employs a proximal term to encourage updated local models for clients not to deviate too much from the global model.
The following (meta-learning-based) PFL methods are considered in our experiment:
- Reptile [22] inserts a meta-learning fine-tuning phase of [70, 55] after the federated averaging algorithm stage to provide a reliable personalized model.
- MAML [21] enhances federated learning by integrating a MAML-based meta-learner [55], dividing the dataset into support and query sets for more robust local training.
- PerFedAvg [23] method focuses on the convergence analysis of the HF-MAML algorithm [71] in the FL scenario, offering a provably convergent method based on MAML to tackle non-convex functions
The following Bayesian FL algorithm are also compared in our experiment:
- FedBE [31] enhances robust aggregation by adopting a Bayesian inference approach, sampling high-quality global models, and combining them through Bayesian model ensemble using Gaussian or Dirichlet distributions fitted to local models.
- pFedGP [34] employs Gaussian processes for personalized federated learning, leveraging shared kernel functions parameterized by a neural network and a novel inducing point approach for the generalization in the low data regime.
The following Pruning algorithm is also compared in our compression experiment in the appendix:
- SNIP [101, 102] algorithm is a deep learning pruning technique that identifies and removes less important connections in neural networks before the training begins using a small subset of the dataset. It is compared with our algorithm in the model compression experiment in the FL environment.
Additionally, Fine-tuning (FT) in the baseline's name indicates fine-tuning the local models with a few steps before evaluation within the FL processes, which is similar to the adaptation step of the optimization-based meta-learning.
E Implementation Details.
Models. To maintain consistency with previous research, we employ the widely adopted CNN model for all algorithms and baselines [25, 74, 3]. Specifically, the global model comprises three convolutional layers with 64 filters and 3x3 kernels, followed by three fully-connected layers of 256, 128, and 64 hidden units.
The hypernetwork architecture of MetaVD consists of an embedding layer, followed by two consecutive blocks containing a linear layer and a LeakyReLU activation function, and one block containing a linear layer with exponential activation to output the dropout logit parameter  $\alpha$ . The dimension of client embedding  $e^{m}$  is proportional to the number of clients  $M$  and is calculated as  $(1 + M / 4)$ . The hidden units' size in the hypernetwork was set to 200. The predicted dropout logit parameter is then applied to each weight of the MetaVD layer within the global model. In our study, we selectively apply MetaVD to just one fully-connected layer right before the output layer of the global model. This simple adaptation of MetaVD only in one fully-connected layer yielded significant performance improvements across all experiments.
Hyperparameters. For all datasets, we set  $T$  to 1000 to ensure sufficient convergence following conventions [18]. The batch size was set to 64, and local steps was set to 5. Personalization was executed with a batch size of 64 and a 1-step update. In order to ensure a fair comparison between the algorithms, the results presented in all of our experiments are obtained with the optimal hyperparameters for each model. To do so, we conducted an extensive parameter optimization using an optimization tool called Optuna<sup>4</sup> [103]. We employed both the Tree-structured Parzen Estimator algorithm and Random Sampler as hyperparameter samplers in Optuna.
For all methods, we investigated the server learning rate and local SGD learning rate within identical ranges. The server learning rate  $\eta$  was explored within the range of [0.6, 0.7, 0.8, 0.9, 1.0]. The local SGD learning rate was investigated within the range of [0.005, 0.01, 0.015, 0.02, 0.025, 0.03]. In MAML and PerFedAvg, an additional client learning rate  $\gamma$  is required, for which we searched within the range of [0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1]. For MetaVD, an additional KL divergence parameter  $\beta$  is needed, and we sought its optimal value within the range of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]. We follow the hyperparameter setting outlined in pFedGP, except for adjusting the batch size to 64 or 320 and investigating the learning rate within the range of [0.03 to 0.1]. To ensure the reproducibility of the experiments, we will release all code, including baselines, on our GitHub repository.
F Experiment Outline
To evaluate the effectiveness and robustness of the proposed PFL methods, we tested the algorithms under various FL scenarios, including:
<sup>4</sup>https://optuna.org/
- Degree of Data Heterogeneity: we assessed the performance of each algorithm in scenarios where data from different clients are heterogeneous, considering factors such as variations in data distributions, label imbalance, and situations while some clients have limited training data available.
- Ablation Study: we conducted an ablation study to verify the advantages of employing a hypernetwork in MetaVD compared to naive Variational Dropout (VD) or Ensemble VD approaches in Federated Learning.
- Client Participation: we evaluated the performance of each algorithm under different levels of client participation rates in each FL round.
- Uncertainty Calibration: we used Expected Calibration Error (ECE) measures to evaluate the accuracy of probabilistic predictions made by the predictive models. These measures help determine whether a model is overconfident or underconfident in its predictions and identify any biases in the model's predictions.
- Communication Efficiency: we measured the communication overhead of each algorithm in terms of the rates of model compression.
- Multi-domain Datasets: Multi-domain learning aims to leverage all available training data across different domains to enhance the performance of the model, but it typically results in a suboptimal global model. We tested our approach in FL with multi-domain learning.
- Model Compression: we compared MetaVD's compression capabilities with the existing pruning algorithm and evaluated the performance of MetaVD with and without compression.
G Additional Results with Non-IDD Settings
Hetrogenity
Method	CIFAR-100 dataset
α = 5.0	α = 0.5	α = 0.1
Test (%)	OOD (%)	Δ	Test (%)	OOD (%)	Δ	Test (%)	OOD (%)	Δ
FedAvg [1]	42.35	43.08	+0.73	41.92	41.96	+0.04	37.57	37.90	+0.33
FedAvg+FT [18]	41.49	42.45	+0.96	40.99	39.83	-1.16	36.79	37.00	+0.21
FedProx [73]	42.23	44.11	+1.88	42.03	40.51	-1.52	38.07	39.36	+1.29
FedBE [31]	45.17	45.43	+0.26	44.29	44.23	-0.06	40.89	41.56	+0.67
pFedGP [34]	42.69	43.07	+0.38	42.44	42.53	+0.09	37.65	38.09	+0.44
Reptile [22]	47.87	47.73	-0.14	46.13	45.94	-0.19	40.71	43.59	+2.88
MAML [21]	48.30	49.14	+0.84	46.33	46.65	+0.32	41.56	45.19	+3.63
PerFedAvg (HF-MAML) [23]	48.19	47.35	-0.84	46.22	46.36	+0.14	42.46	42.92	+0.46
FedAvg+MetaVD (ours)	47.82	50.26	+2.44	47.54	47.55	+0.01	42.34	42.65	+0.31
Reptile+MetaVD (ours)	53.71	54.50	+0.79	52.06	51.50	-0.56	44.56	44.62	+0.06
MAML+MetaVD (ours)	52.40	51.78	-0.62	50.21	49.75	-0.46	43.84	48.17	+4.33
PerFedAvg+MetaVD (ours)	51.67	51.70	+0.03	50.02	48.70	-1.32	43.31	46.19	+2.88
Table 7: Results of classification accuracies with different (non-IID) heterogeneity strengths of  $\dot{\alpha} = 5.0$ ,  $\dot{\alpha} = 0.5$ , and  $\dot{\alpha} = 0.1$  in the CIFAR-100 dataset. We report the results of participating clients during the training (Test) dataset and non-participating clients (OOD). The higher, the better.
To evaluate the generalization capabilities of the models under non-IID data conditions, we conducted tests on both CIFAR-10 and CIFAR-100 datasets with varying degrees of heterogeneous partitions. We randomly selected 30 out of 130 clients as non-participating out-of-distribution (OOD) clients, who were not involved during the training phase. Table 11 and Table 12 display the weighted average accuracy while adjusting the non-IID degrees with different Dirichlet parameters  $\dot{\alpha}$ . A smaller  $\dot{\alpha}$  corresponds to a higher degree of heterogeneity. Test(\%) denotes the weighted average accuracy of participating clients' test samples, OOD(\%) signifies the weighted average accuracy of the non-participants, and  $\Delta$  represents the participation generalization gap calculated as the difference between OOD accuracy and test accuracy.
In both datasets, PFL methods such as Reptile, MAML, and PerFedAvg generally outperform non-PFL methods like FedAvg and FedProx. The Bayesian ensemble approach on FedAvg (FedBE) offers a slight improvement in overall performance but still lags behind PFL methods in terms of OOD accuracy. Notably, when combined with MetaVD, all baselines experience significant performance enhancements, regardless of whether they employ FL or PFL approaches. Importantly, our method consistently improves OOD accuracy across all degrees of non-IID data, illustrating its versatility in effectively augmenting conventional FL algorithms without being limited by specific optimization techniques, addressing non-IID data and model overfitting issues. Additionally, our experimental results indicate that pFedGP performs relatively poorly when the data distribution among clients is heterogeneous.
Hetrogenity
Method	CIFAR-10 dataset
α = 5.0	α = 0.5	α = 0.1
Test (%)	OOD (%)	Δ	Test (%)	OOD (%)	Δ	Test (%)	OOD (%)	Δ
FedAvg [1]	80.73	79.98	-0.75	78.28	78.90	+0.62	71.65	71.57	-0.08
FedAvg+FT [18]	80.20	79.63	-0.57	76.96	75.83	-1.13	69.62	68.38	-1.24
FedProx [73]	80.79	80.30	-0.49	78.55	77.60	-0.95	72.27	73.75	+1.48
FedBE [31]	81.55	81.07	-0.48	79.44	79.46	+0.02	70.23	69.19	-1.04
pFedGP [34]	83.73	83.27	-0.46	79.56	79.37	-0.19	71.94	76.83	+4.99
Reptile [22]	83.48	82.88	-0.60	79.35	79.41	+0.06	73.93	76.36	+2.43
MAML [21]	82.61	81.69	-0.92	80.31	80.03	-0.28	76.06	74.89	-1.17
PerFedAvg (HF-MAML) [23]	82.57	81.99	-0.58	80.87	80.85	-0.02	75.42	79.56	+4.14
FedAvg+MetaVD (ours)	82.60	82.74	+0.14	79.08	80.33	+1.25	76.87	76.25	-0.62
Reptile+MetaVD (ours)	84.70	84.28	-0.42	83.20	83.40	+0.20	76.51	82.07	+5.56
MAML+MetaVD (ours)	83.93	84.95	+1.02	81.32	81.81	+0.49	77.27	79.05	+1.78
PerFedAvg+MetaVD (ours)	83.88	83.96	+0.08	81.06	81.47	+0.41	76.06	81.77	+5.71
In the original pFedGP paper [34], the data samples were partitioned across clients according to [25, 24], resulting in each client having 2 and 10 classes for the CIFAR-10 and CIFAR-100 datasets, respectively. In contrast, in our non-IID experiment, we do not predefine the number of classes each client has; instead, it is determined by the Dirichlet parameter  $\dot{\alpha}$ . In addition, we found some limitations of pFedGP, which could not classify data points when encountering unseen labels within a client (especially when the label shift [104, 105] occurs between the training and testing distributions of the clients). To ensure that pFedGP would run on the unseen labels, we circumvented this limitation by allowing the client to have access to 40 or 50 data points for each class while evaluating the model's performance.
Table 8: Results of classification accuracies with different (non-IID) heterogeneity strengths of  $\dot{\alpha} = 5.0$ ,  $\dot{\alpha} = 0.5$ , and  $\dot{\alpha} = 0.1$  in the CIFAR-10 dataset. We report the results of participating clients during the training (Test) dataset and non-participating clients (OOD). The higher the better.
Dataset	Dirichlet parameter α
α = 5.0	α = 0.5	α = 0.1
CIFAR-100	99.78 ± 0.5	73.18 ± 5.2	33.78 ± 7.33
CIFAR-10	10 ± 0	9.10 ± 0.94	4.65 ± 1.49
H Additional Ablation Results in FEMNIST Dataset
Table 9: Statistics of the number of classes assigned to each client.
FEMNIST dataset
Method	Test (%)	OOD (%)	Δ
Reptile	87.86	88.22	+0.36
Reptile+VD	87.93	85.88	-2.05
Reptile+EnsembleVD	87.99	87.97	-0.02
Reptile+MetaVD (ours)	89.43	88.71	-0.72
Table 10: MetaVD ablation study results in the FEMNIST dataset.
In this experiment, we further performed the ablation study using the FEMNIST dataset to evaluate the benefits of MetaVD's hypernetwork in FL. We compared MetaVD to naive VD [46] and Ensemble VD [79] approaches. The naive (global) VD model keeps one global dropout parameter shared with all clients; the dropout parameter is treated as a global model parameter as in FedAvg. In EnsembleV, we updated client-specific dropout rates in a manner analogous to the local adaptation step of MetaVD but maintained independent variational dropout rates for different clients on the server. In contrast, MetaVD utilized a hypernetwork to learn the dropout rates. All models employed Bayesian posterior aggregation rules based on dropout rates to update the global model parameter.
The ablation study's results on the FEMNIST dataset are outlined in Table 10 in the manuscript. It is evident from the table that MetaVD's conditional variational dropout-based hypernetwork surpasses all other baselines
in classification accuracy. Here too, Reptile+MetaVD consistently outperforms other methods. We noticed that in baselines like VD or EnsembleVD, client-specific dropout rates were not fully optimized due to restricted client participation. Consequently, the initial dropout rates and KL divergence loss in VD and EnsembleVD remained fairly static. Conversely, in MetaVD, both dropout rates and KL divergence loss converged in all tests. This observation underscores that MetaVD's hypernetwork presents a more data-efficient approach to learning the client-specific model uncertainty compared to other baselines.
I Additional Results on Uncertainty Calibration
Addressing uncertainty calibration issues is crucial for enhancing the accuracy and reliability of a model's predictions, particularly when the model is employed for making significant decisions. Uncertainty calibration metrics evaluate the accuracy of probabilistic predictions made by a predictive model. By identifying and addressing any calibration issues, it becomes possible to improve the model's prediction accuracy and reliability, which is of essential importance in decision-making processes. In a Federated Learning environment where clients have limited non-IID data, overfitting can easily occur, making the calibration of prediction uncertainty even more critical. We conducted experiments to evaluate the effectiveness of our proposed algorithm in improving uncertainty calibration, demonstrating its potential to strengthen model performance in various real applications.
Hetrogenity
Method	CIFAR-100 dataset
α = 5.0	α = 0.5	α = 0.1
ECE (%)	MCE (%)	ECE (%)	MCE (%)	ECE (%)	MCE (%)
FedAvg [1]	0.29	22.05	0.53	32.35	0.60	36.79
FedAvg+FT [18]	0.41	28.36	0.46	31.91	0.69	45.04
FedProx [73]	0.43	21.92	0.40	26.74	0.67	39.69
FedBE [31]	0.43	29.75	0.38	27.54	0.50	34.66
pFedGP [34]	0.15	22.99	0.14	16.57	0.12	17.40
Reptile [22]	0.73	46.53	0.77	46.19	0.77	50.52
MAML [21]	0.40	27.74	0.70	48.16	0.75	46.47
PerFedAvg (HF-MAML) [23]	0.65	45.44	0.40	23.74	0.69	45.27
FedAvg+MetaVD (ours)	0.26	19.99	0.32	26.60	0.39	25.27
Reptile+MetaVD (ours)	0.37	25.24	0.32	23.86	0.57	42.40
MAML+MetaVD (ours)	0.31	23.18	0.38	32.56	0.52	37.26
PerFedAvg+MetaVD (ours)	0.26	20.46	0.39	27.59	0.43	30.20
Table 11: Results of uncertainty calibration scores (ECE and MCE) in the CIFAR-100 dataset. The lower is the better.
Hetrogenity
Method	CIFAR-10 dataset
α = 5.0	α = 0.5	α = 0.1
ECE (%)	MCE (%)	ECE (%)	MCE (%)	ECE (%)	MCE (%)
FedAvg [1]	1.93	28.87	2.39	26.45	3.16	27.37
FedAvg+FT [18]	2.02	20.80	2.71	27.86	3.20	33.22
FedProx [73]	2.01	25.96	2.52	25.24	3.06	28.62
FedBE [31]	1.78	21.33	2.18	21.89	3.35	29.72
pFedGP [34]	1.23	8.34	0.91	6.23	0.84	6.08
Reptile [22]	2.40	30.53	3.06	38.89	2.75	27.76
MAML [21]	2.61	36.86	2.49	30.67	2.66	32.10
PerFedAvg (HF-MAML) [23]	2.19	26.78	2.39	27.13	2.58	24.47
FedAvg+MetaVD (ours)	1.52	18.02	1.76	21.17	1.40	13.92
Reptile+MetaVD (ours)	1.60	25.46	2.16	29.84	1.76	17.84
MAML+MetaVD (ours)	1.39	16.46	1.97	21.51	0.35	5.80
PerFedAvg+MetaVD (ours)	1.14	17.55	1.51	20.38	0.29	3.63
Table 12: Results of uncertainty calibration scores (ECE and MCE) in the CIFAR-10 dataset. The lower is the better.
Expected Calibration Error (ECE) and Maximum Calibration Error (MCE) are widely used metrics for measuring uncertainty calibration. ECE represents the average discrepancy between the model's confidence and its accuracy. To compute ECE, we group samples based on their confidence levels and calculate the average confidence and the percentage of correct samples for each group. ECE is then derived as the weighted average of the differences