# Hybrid Uncertainty Metrics-Based Privacy-Preserving Alternating Multimodal Representation Learning
## 3 Materials and Methods
### Dynamic Fusion Mechanism  
The dynamic fusion mechanism adjusts modality fusion weights based on a hybrid uncertainty metric \(u_{m}\). The computation proceeds as follows:  
Figure 1: The overview of our proposed PAMRL.  
1. Entropy Calculation \(e_{m}\). The uncertainty of predictions for modality \(m\) is measured using Shannon entropy: \[e_{m}=-\sum_{c=1}^{C}\ p_{m,c}logp_{m,c}\] (1) where \(p_{m,c}\) represents the predicted probability of modality \(m\) for class \(c\), and \(C\) is the total number of classes. 2. KL Divergence Calculation \(\mathbf{KL_{m}}\). In our multimodal learning framework, we utilize the Kullback-Leibler (KL) divergence to measure the difference between the predictive probability distributions of two modalities. Given two distributions, \(p_{1}\) and \(p_{2}\), corresponding to the predictions of the first and second modalities, respectively, the KL divergence is defined as follows: \[KL(\mathbf{p}_{i}\|\mathbf{q}_{i})=\sum_{c=1}^{C}\ p_{i,c}(logp_{i,c}-logq_{i,c})\] (2) where \(p_{1,c}\) and \(q_{2,c}\) are the probabilities assigned to class \(c\) by \(p_{1}\) and \(p_{2}\), respectively. 3. Normalized Entropy. Normalized entropy adjusts the original entropy value to a range of [0, 1], allowing us to measure uncertainty consistently, regardless of the number of classes involved. The formula is as follows: \[Normalized\ e_{m}=\frac{e_{m}-\mu_{c}}{\sigma_{c}+\epsilon}\] (3) \[\mu_{e}=\frac{1}{B}\sum_{j=1}^{B}\ e_{mj}\] (4) \[\sigma_{e}=\sqrt{\frac{1}{B}\sum_{j=1}^{B}\ \left(e_{mj}-\mu_{e}\right)^{2}}\] (5) where \(\epsilon\) is a predefined hyperparameter used to prevent the denominator from being zero, and B is batch size. 4. Normalized KL Divergence. KL divergence (Kullback-Leibler divergence) measures how much one probability distribution differs from another, but it does not have a fixed upper bound. Normalizing it helps us interpret the difference on a consistent scale. The formula is as follows: \[Normalized\ KL_{m}=\frac{KL(p_{i}\|\mathbf{q}_{i})-\mu_{KL}}{\sigma_{KL}+\epsilon}\] (6) \[\mu_{KL}=\frac{1}{B}\sum_{i=1}^{B}\ KL(\mathbf{p}_{i}\|\mathbf{q}_{i})\] (7) \[\sigma_{KL}=\sqrt{\frac{1}{B}\sum_{i=1}^{B}\ \left(KL(\mathbf{p}_{i}\|\mathbf{q}_{i })-\mu_{KL}\right)^{2}}\] (8) where \(\epsilon\) is a predefined hyperparameter used to prevent the denominator from being zero, and B is batch size.  
(5) Hybrid Uncertainty Metric \(\mathbf{u_{m}}\). The hybrid uncertainty combines entropy and KL divergence with a weighting factor \(\alpha\): \[u_{m}=\beta e_{m}+(1-\beta)D_{KL,m}\] (9) Here, \(\beta\in[0,1]\) is a hyperparameter set to 0.5 by default, balancing the contributions of entropy and divergence. (6) Fusion Weight \(\mathbf{w_{m}}\). Fusion weights are computed using a SoftMax-like normalization to favor modalities with lower uncertainty: \[w_{m_{i}}=\frac{exp(-\left(\text{ Normalized }e_{m_{i}}+\text{Normalized }KL_{m_{i}}\right))}{\sum_{m}exp(-\left(\text{ Normalized }e_{m_{i}}+\text{Normalized }KL_{m_{i}}\right))}\] (10) \[w_{m_{j}}=1-w_{m_{i}}\] (11) where \(m\) is the total number of modalities. (7) Final Prediction \(\mathbf{\hat{y}}_{\text{final}}\). The final prediction is a weighted sum of individual modality predictions: \[\hat{y}_{\text{final}}=\sum_{m=1}^{M}w_{m}\cdot f_{m}(x_{m})\] (12) where \(f_{m}(x_{m})\) denotes the prediction output of modality \(m\) given input \(x_{m}\).