SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} p (\mathbf {Y}, \boldsymbol {\lambda}, \mathbf {M}, \boldsymbol {\Omega}, \mathbf {F}) = \prod_ {n = 1} ^ {N} \prod_ {c = 1} ^ {C} 2 ^ {- (y _ {n} ^ {c} + m _ {n} ^ {c})} \exp \left(\frac {y _ {n} ^ {c} - m _ {n} ^ {c}}{2} \frac {f _ {n} ^ {c}}{\tau} - \frac {\omega_ {n} ^ {c}}{2} \left(\frac {f _ {n} ^ {c}}{\tau}\right) ^ {2}\right) \mathrm {P G} (\omega_ {n} ^ {c} | m _ {n} ^ {c} + y _ {n} ^ {c}, 0) \\ \cdot \frac {\lambda_ {n} ^ {m _ {n} ^ {c}}}{m _ {n} ^ {c} !} \exp (- \lambda_ {n}) \cdot \prod_ {c = 1} ^ {C} \mathcal {N} \left(\mathbf {f} ^ {c} \mid \mathbf {a} ^ {c}, \mathbf {K} ^ {c}\right), \tag {4.2} \\ \end{array}
$$
where  $\mathbf{f}^c = [f_1^c, \dots, f_N^c]^\top$  is the  $c$ -th column of  $\mathbf{F}$ ,  $\lambda_n$  is the augmented Gamma variable,  $m_n^c$  is the augmented Poisson variable,  $\omega_n^c$  is the augmented Pólya-Gamma variable,  $\mathbf{a}^c$  is the mean and  $\mathbf{K}^c$  is the kernel matrix for  $c$ -th class with  $N$  samples. The original model has been now transformed into a conditional conjugate one, which possesses excellent mathematical properties.
Mean-field Approximation Based on Eq. (4.2), we can obtain the closed-form conditional densities for all variables, which constitutes a Gibbs sampler if we iteratively draw a sample from each conditional distribution. However, the Gibbs sampler is not efficient because the sampling operation is time-consuming. In order to improve efficiency, we derive a mean-field variational inference that provides an approximate posterior but with better efficiency.
In the mean-field algorithm, we need to approximate the true posterior  $p(\lambda, \mathbf{M}, \Omega, \mathbf{F} \mid \mathbf{Y})$  by a variational distribution which is assumed to factorize over some partition of latent variables. Here, we assume the variational distribution  $q(\lambda, \mathbf{M}, \Omega, \mathbf{F}) = q_1(\mathbf{M}, \Omega)q_2(\lambda, \mathbf{F})$ . Following the standard derivation provided in Appendix V, we obtain the optimal density for each factor:
$$
\begin{array}{l} q _ {1} (\boldsymbol {\Omega} | \mathbf {M}) = \prod_ {n, c = 1} ^ {N, C} \operatorname {P G} \left(\omega_ {n} ^ {c} \mid m _ {n} ^ {c} + y _ {n} ^ {c}, \widetilde {f} _ {n} ^ {c}\right), \quad (4. 3 \mathrm {a}) \quad q _ {2} (\boldsymbol {\lambda}) = \prod_ {n = 1} ^ {N} \operatorname {G a} \left(\lambda_ {n} \mid \alpha_ {n}, C\right), (4.3c) \\ q _ {1} (\mathbf {M}) = \prod_ {n, c = 1} ^ {N, C} \operatorname {P o} \left(m _ {n} ^ {c} \mid \gamma_ {n} ^ {c}\right), \quad (4. 3 \mathrm {b}) \quad q _ {2} (\mathbf {F}) = \prod_ {c = 1} ^ {C} \mathcal {N} \left(\mathbf {f} ^ {c} \mid \tilde {\boldsymbol {\mu}} ^ {c}, \tilde {\boldsymbol {\Sigma}} ^ {c}\right), (4.3d) \\ \end{array}
$$
$$
\widetilde {f} _ {n} ^ {c} = \frac {1}{\tau} \sqrt {\mathbb {E} \left[ f _ {n} ^ {c ^ {2}} \right]} = \frac {1}{\tau} \sqrt {\widetilde {\mu} _ {n} ^ {c ^ {2}} + \widetilde {\sigma} _ {n n} ^ {c ^ {2}}}, \tag {4.4a}
$$
$$
\gamma_ {n} ^ {c} = \frac {\exp \left(\psi \left(\alpha_ {n}\right) - \widetilde {\mu} _ {n} ^ {c} / 2 \tau\right)}{2 C \cosh \left(\widetilde {f} _ {n} ^ {c} / 2\right)}, \tag {4.4b}
$$
$$
\alpha_ {n} = \sum_ {c = 1} ^ {C} \gamma_ {n} ^ {c} + 1, \tag {4.4c}
$$
$$
\tilde {\Sigma} ^ {c} = \left(\operatorname {d i a g} \left(\bar {\omega} _ {n} ^ {c} / \tau^ {2}\right) + \mathbf {K} ^ {c - 1}\right) ^ {- 1}, \tag {4.4d}
$$
$$
\widetilde {\boldsymbol {\mu}} ^ {c} = \frac {1}{2 \tau} \widetilde {\boldsymbol {\Sigma}} ^ {c} \left(\mathbf {y} ^ {c} - \boldsymbol {\gamma} ^ {c}\right) + \widetilde {\boldsymbol {\Sigma}} ^ {c} \mathbf {K} ^ {c ^ {- 1}} \mathbf {a} ^ {c}, \tag {4.4e}
$$
$$
\bar {\omega} _ {n} ^ {c} = \mathbb {E} [ \omega_ {n} ^ {c} ] = \frac {\gamma_ {n} ^ {c} + y _ {n} ^ {c}}{2 \widetilde {f} _ {n} ^ {c}} \tanh \frac {\widetilde {f} _ {n} ^ {c}}{2}. (4. 4 \mathrm {f})
$$
where  $\psi (\cdot)$  is the digamma function. Update the posterior of  $\Omega$ ,  $\mathbf{M}$ ,  $\lambda$  and  $\mathbf{F}$  iteratively by Eq. (4.3), we obtain an efficient mean-field algorithm to provide the approximate posterior for each task.
4.3 Meta-level Optimization
We learn a set of hyperparameters  $\Theta$  of deep kernels in the outer loop that maximizes the marginal likelihood across all tasks, which is also called the empirical Bayes [17]:
$$
\sum_ {t = 1} ^ {T} \log p (\mathbf {Y} _ {t} \mid \boldsymbol {\Theta}) = \sum_ {t = 1} ^ {T} \log \int p (\mathbf {Y} _ {t}, \boldsymbol {\lambda} _ {t}, \mathbf {M} _ {t}, \boldsymbol {\Omega} _ {t}, \mathbf {F} _ {t} \mid \boldsymbol {\Theta}) d \boldsymbol {\lambda} _ {t} d \mathbf {M} _ {t} d \boldsymbol {\Omega} _ {t} d \mathbf {F} _ {t}, \tag {4.5}
$$
where the subscript  $t$  indicates the  $t$ -th task. Unfortunately, the integral in Eq. (4.5) is intractable, so we utilize an approximate approach: given the variational parameters of each task, we try to maximize the evidence lower bound (ELBO)  $\mathcal{L}$  as a function of  $\Theta$  [16]. Thanks to the data augmentation, we have an analytical expression of the ELBO which is provided in Appendix VI, and the gradient  $\nabla_{\Theta} \mathcal{L}$  can be computed by the automatic differentiation [2]. In addition, Snell & Zemel [27] has found that the predictive likelihood (PL) can also serve as a loss of  $\Theta$ . We follow their derivation and use the approximate gradient estimator:
$$
\nabla_ {\boldsymbol {\theta}} \mathcal {L} _ {\mathrm {P L}} \approx \frac {1}{M} \sum_ {m = 1} ^ {M} \nabla_ {\boldsymbol {\theta}} \log p (y _ {*} = k \mid \mathbf {x} _ {*}, \mathbf {X}, \mathbf {Y}, \widehat {\boldsymbol {\Theta}}),
$$
where  $M$  denotes the number of samples drawn from Eq. (4.6b).
Alternating between the optimization steps w.r.t. the variational parameters of each task in the inner loop and the hyperparameters of deep kernels in the outer loop, we obtain an efficient Bayesian meta-learning method for classification. However, the coordination of the gradient flow between the inner loop and the outer loop remains an open question. Previous research detaches the task-level variables in the inner loop and updates the hyperparameters of the deep kernel using gradients produced by the outer loop [27]. Nevertheless, we have discovered that the task-level variables in the inner loop should not be detached when using mean-field approximation.
4.4 Prediction
Given a test task with support dataset containing the input  $\mathbf{X} = \{\mathbf{x}_l\}_{l=1}^L$  and label  $\mathbf{Y} = \{\mathbf{y}_l\}_{l=1}^L$  from which the model can learn about the new classes, and an unlabeled data point  $\mathbf{x}_*$  in the query dataset, the predictive probability of test label  $y_* = k$  is:
$$
p \left(y _ {*} = k \mid \mathbf {x} _ {*}, \mathbf {X}, \mathbf {Y}, \widehat {\boldsymbol {\Theta}}\right) = \int p \left(y _ {*} = k \mid \mathbf {f} _ {*}\right) \prod_ {c = 1} ^ {C} q \left(f _ {*} ^ {c} \mid \mathbf {X}, \mathbf {Y}, \widehat {\boldsymbol {\Theta}}\right) d \mathbf {f} _ {*}, \tag {4.6a}
$$
$$
q \left(f _ {*} ^ {c} \mid \mathbf {X}, \mathbf {Y}, \widehat {\boldsymbol {\Theta}}\right) = \int p \left(f _ {*} ^ {c} \mid \mathbf {f} ^ {c}\right) q \left(\mathbf {f} ^ {c} \mid \mathbf {X}, \mathbf {Y}, \widehat {\boldsymbol {\Theta}}\right) d \mathbf {f} ^ {c} = \mathcal {N} \left(f _ {*} ^ {c} \mid \mu_ {*} ^ {c}, \sigma_ {*} ^ {2 ^ {c}}\right), \tag {4.6b}
$$
where  $k\in \{1,\ldots ,C\}$ ,  $\widehat{\Theta}$  is the learned kernel hyperparameter in the training procedure,  $p(y_{*} = k\mid$ $\mathbf{f}_{*})$  is the logistic-softmax likelihood,  $\mathbf{f}_* = [f^1 (\mathbf{x}_*),\dots,f^C (\mathbf{x}_*)]^{\top}$ ,  $q(\mathbf{f}^{c}\mid \mathbf{X},\mathbf{Y},\widehat{\Theta})$  is Eq. (4.3d),  $\mu_{*}^{c} = \mathbf{k}_{*l}^{c}\mathbf{K}_{ll}^{c - 1}\widetilde{\boldsymbol{\mu}}^{c}$  and  $\sigma_{*}^{2c} = k_{**}^{c} - \mathbf{k}_{*l}^{c}\mathbf{K}_{ll}^{c - 1}\mathbf{k}_{l*}^{c} + \mathbf{k}_{*l}^{c}\mathbf{K}_{ll}^{c - 1}\widetilde{\boldsymbol{\Sigma}}^{c}\mathbf{K}_{ll}^{c - 1}\mathbf{k}_{l*}^{c}$ . The integral in Eq. (4.6a) is intractable, so we resort to Monte Carlo to compute it and classify the test point by the highest predictive probability. The complete training and test procedure is described in Alg. 1 of Appendix VII.
In this section, we present the results for few-shot classification tasks including accuracy and uncertainty quantification. We consider two challenging standard benchmark datasets: the Caltech-UCSD Birds [32] and mini-ImageNet [24].
5.1 Few-shot Classification and Domain Transfer
In this subsection, we describe the experimental setup and report the results of our few-shot classification tasks. While we notice that there are various settings for few-shot classification (e.g., the distribution calibration technique in Yang et al. [36]), we adopt the vanilla setting in Bayesian meta-learning to better compare the likelihoods. Following the procedure of prior work [20], we employ a standard Conv4 architecture [31] as the backbone and assess our models under six different settings, including 1-shot and 5-shot scenarios for both in-domain and cross-domain tasks. We benchmark our models against various baselines and state-of-the-art models, including Feature Transfer [5], Baseline++ [5], MatchingNet [31], ProtoNet [28], RelationNet [29], MAML [7], DKT [20], Bayesian MAML [38], ABML [23], LS [9], and OVE [27]. As for LS, we utilized the Gibbs sampling version implemented by Snell & Zemel [27], which is more computationally demanding than our mean-field method. We note that DKT, LS, and OVE are similar to our proposed method as they are all GP-based models but use different likelihood functions and inference methods. To ensure a fair comparison, we also apply the cosine kernel as in Patacchiola et al. [20] and Snell & Zemel [27].
We train and evaluate our models (denoted as CDKT) with the ELBO loss (denoted as ML) using the default number of epochs from Patacchiola et al. [20], and with the predictive likelihood loss (denoted as PL) using 800 epochs. However, we find that the ML ( $\tau < 1$ ) domain-transfer experiment requires 800 epochs to avoid underfitting. We set the temperature parameter to  $\tau = 0.5$  for the 5-shot PL experiment of mini-ImageNet and domain transfer, and  $\tau = 0.2$  for all other experiments. Drawing inspiration from Section 3, we observe that placing a negative mean prior improves our method by  $0.5\%$  to  $1\%$  across all experiments. To ensure numerical stability, we train our models with a zero mean prior and use a constant negative mean of  $-5$  at test time for all ML ( $\tau < 1$ ) experiments. We also use this technique for the PL ( $\tau < 1$ ) experiment of the domain-transfer task. We also note that the training of PL loss is generally less efficient than the ML loss, as it requires Monte Carlo sampling. Our mean field approximation converges at an extremely fast rate and we only need 2 steps for task-level updates. More experimental details are provided in Appendix VIII.
We report the average accuracy and standard deviation of our models evaluated on 5 batches of 600 episodes with different random seeds in Table 1. Our model achieves the highest accuracy in both 1-shot  $(65.21\%)$  and 5-shot  $(79.10\%)$  experiments on the CUB dataset and achieves comparable results on the mini-ImageNet dataset. As for the domain-transfer task, we also achieve the highest result for the 1-shot  $(40.43\%)$  scenario and a near-optimal result for the 5-shot  $(56.18\%)$  scenario. Overall, while our mean-field approximation method works well for the ELBO loss (ML), the predictive likelihood loss (PL) generally does not match the performance demonstrated by the Gibbs sampling version of LS proposed by Snell & Zemel [27]. However, we find the PL loss effective for the domain-transfer scenario, indicating its potential at dealing with this type of task. Moreover, we observe a significant improvement  $(3\% - 5\%)$  in accuracy by adding a small temperature scaling  $(\tau = 0.2$  or 0.5) to the default logistic-softmax function  $(\tau = 1)$  in all scenarios. This finding suggests that our approach to controlling the a priori confidence of logistic-softmax is highly effective.
5.2 Uncertainty Quantification
Uncertainty quantification is an important aspect of few-shot learning because classifiers trained on limited data may have high variance and uncertainty in their predictions. This uncertainty can arise from a variety of sources, such as the small amount of training data, the complexity of the model, and the variability of the input data. A robust meta-learning method should be able to properly deal with such uncertainty, especially in high-risk areas including medical diagnosis, judicial adjudication, and autonomous driving.
We use 2 widely-applied metrics for uncertainty quantification, namely expected calibration error (ECE) [11] and maximum calibration error (MCE) [19]. ECE measures the average difference between confidence (probability outputs) and accuracy within each bin. MCE is similar to ECE but it
Table 1: Average 1-shot and 5-shot accuracy and standard deviation on 5-way few-shot classification. Baseline results are from Patacchiola et al. [20] and Snell & Zemel [27]. Results are evaluated over 5 batches of 600 episodes with different random seeds. We highlight the best results in bold.
Method	CUB	mini-ImageNet	mini-ImageNet → CUB
1-shot	5-shot	1-shot	5-shot	1-shot	5-shot
Feature Transfer	46.19 ± 0.64	68.40 ± 0.79	39.51 ± 0.23	60.51 ± 0.55	32.77 ± 0.35	50.34 ± 0.27
Baseline++	61.75 ± 0.95	78.51 ± 0.59	47.15 ± 0.49	66.18 ± 0.18	39.19 ± 0.12	57.31 ± 0.11
MatchingNet	60.19 ± 1.02	75.11 ± 0.35	48.25 ± 0.65	62.71 ± 0.44	36.98 ± 0.06	50.72 ± 0.36
ProtoNet	52.52 ± 1.90	75.93 ± 0.46	44.19 ± 1.30	64.07 ± 0.65	33.27 ± 1.09	52.16 ± 0.17
RelationNet	62.52 ± 0.34	78.22 ± 0.07	48.76 ± 0.17	64.20 ± 0.28	37.13 ± 0.20	51.76 ± 1.48
MAML	56.11 ± 0.69	74.84 ± 0.62	45.39 ± 0.49	61.58 ± 0.53	34.01 ± 1.25	48.83 ± 0.62
DKT + Cosine	63.37 ± 0.19	77.73 ± 0.26	48.64 ± 0.45	62.85 ± 0.37	40.22 ± 0.54	55.65 ± 0.05
Bayesian MAML	55.93 ± 0.71	72.87 ± 0.26	44.46 ± 0.30	62.60 ± 0.25	33.52 ± 0.36	51.35 ± 0.16
Bayesian MAML (Chaser)	53.93 ± 0.72	71.16 ± 0.32	43.74 ± 0.46	59.23 ± 0.34	36.22 ± 0.50	51.53 ± 0.43
ABML	49.57 ± 0.42	68.94 ± 0.16	37.65 ± 0.22	56.08 ± 0.29	29.35 ± 0.26	45.74 ± 0.33
LS (Gibbs) + Cosine (ML)	60.23 ± 0.54	74.58 ± 0.25	46.75 ± 0.20	59.93 ± 0.31	36.41 ± 0.18	50.33 ± 0.13
LS (Gibbs) + Cosine (PL)	60.07 ± 0.29	78.14 ± 0.07	47.05 ± 0.20	66.01 ± 0.25	36.73 ± 0.26	56.70 ± 0.31
OVE PG GP + Cosine (ML)	63.98 ± 0.43	77.44 ± 0.18	50.02 ± 0.35	64.58 ± 0.31	39.66 ± 0.18	55.71 ± 0.31
OVE PG GP + Cosine (PL)	60.11 ± 0.26	79.07 ± 0.05	48.00 ± 0.24	67.14 ± 0.23	37.49 ± 0.11	57.23 ± 0.31
CDKT + Cosine (ML) (τ<1)	65.21 ± 0.45	79.10 ± 0.33	47.54 ± 0.21	63.79 ± 0.15	40.43 ± 0.43	55.72 ± 0.45
CDKT + Cosine (ML) (τ=1)	60.85 ± 0.38	75.98 ± 0.33	43.50 ± 0.17	59.69 ± 0.20	35.57 ± 0.30	52.42 ± 0.50
CDKT + Cosine (PL) (τ<1)	59.49 ± 0.35	76.95 ± 0.28	44.97 ± 0.25	60.87 ± 0.24	39.18 ± 0.34	56.18 ± 0.28
CDKT + Cosine (PL) (τ=1)	52.91 ± 0.29	73.34 ± 0.40	40.29 ± 0.14	60.23 ± 0.16	37.62 ± 0.32	54.32 ± 0.19
Table 2: Expected calibration error (ECE) and maximum calibration error (MCE) for 5-shot 5-way tasks on CUB, mini-ImageNet, and domain-transfer. Baseline results are from Snell & Zemel [27]. All metrics are computed on 3,000 random tasks from the test set.
Method	CUB	mini-ImageNet	mini-ImageNet→CUB
ECE	MCE	ECE	MCE	ECE	MCE
Feature Transfer	0.187	0.250	0.368	0.641	0.275	0.646
Baseline++	0.421	0.502	0.395	0.598	0.315	0.537
MatchingNet	0.023	0.031	0.019	0.043	0.030	0.079
ProtoNet	0.034	0.059	0.035	0.050	0.009	0.025
RelationNet	0.438	0.593	0.330	0.596	0.234	0.554
DKT + Cosine	0.187	0.250	0.287	0.446	0.236	0.426
Bayesian MAML	0.018	0.047	0.027	0.049	0.048	0.077
Bayesian MAML (Chaser)	0.047	0.104	0.010	0.071	0.066	0.260
LS (Gibbs) + Cosine (ML)	0.371	0.478	0.277	0.490	0.220	0.513
LS (Gibbs) + Cosine (PL)	0.024	0.038	0.026	0.041	0.022	0.042
OVE PG GP + Cosine (ML)	0.026	0.043	0.026	0.039	0.049	0.066
OVE PG GP + Cosine (PL)	0.005	0.023	0.008	0.016	0.020	0.032
CDKT + Cosine (ML)	0.005	0.036	0.009	0.015	0.007	0.020
CDKT + Cosine (PL)	0.018	0.223	0.025	0.140	0.010	0.029
measures the maximum difference. Following the protocol of Patacchiola et al. [20] for evaluation, we first tune the temperature parameter on the validation set for alignment and then compute the ECE and MCE on the test set. The results of 5-shot experiments are summarized in Table 2. We observe that our introduction of temperature significantly improves the calibration of logistic-softmax compared to the one implemented by Snell & Zemel [27]. Specifically, we obtain the lowest ECE on CUB (0.005) and domain transfer (0.007) and the lowest MCE on mini-[ImageNet (0.015) and domain transfer (0.020). Our model performs marginally worse than state-of-the-art in MCE on CUB and ECE on mini-[ImageNet. Overall, this result indicates that our model is highly reliable on uncertainty calibration, demonstrating promising robustness in few-shot scenarios.
Figure 3 shows the reliability diagrams of our models. The confidence barplot should match the diagonal line for a robust uncertainty quantification model. We observe that the models trained with the ELBO loss (ML) generally perform better than those trained with the predictive likelihood (PL). Specifically, ML models closely match the diagonal line while PL models tend to underestimate the accuracy of low-confidence outputs. In conclusion, our findings indicate that ML models are better at handling uncertainty in the context of mean-field approximation inference. This is an interesting result as it contrasts the findings in Snell & Zemel [27], where PL models with Gibbs sampling generally perform better. Nonetheless, we leave the analysis of this phenomenon to future work.
(a) CUB (ML)
(b) Mini (ML)
(c) DT (ML)
Figure 3: Reliability diagrams on 5-shot classification with expected calibration error (ECE) and maximum calibration error (MCE) metrics. Mini denotes the mini-ImageNet dataset, and DT denotes the domain transfer task of CUB  $\rightarrow$  mini-ImageNet. Results are computed on 3,000 test tasks.
(d) CUB (PL)
(e) Mini (PL)
(f) DT (PL)
GP classification has been extensively studied, and several approaches have been proposed to address its challenges, including label regression and Laplace approximation [25; 34]. Recently, Polson et al. [21] introduced Polya-Gamma augmentation, and Galy-Fajou et al. [9]; Snell & Zemel [27] utilized different likelihood functions and data augmentation to approximate the softmax function. These methods have advanced the field by providing intuitive frameworks and improving modeling capabilities for GP classifiers. Our work builds upon these foundations by focusing on a more generalized likelihood formulation with enhanced flexibility and modeling capabilities for GP classifiers.
Bayesian meta-learning has been explored through various approaches to leverage prior knowledge and adapt to new tasks. Finn et al. [8] introduced a Bayesian hierarchical modeling perspective, capturing uncertainty at different levels. Grant et al. [10] recast meta-learning as inference in a GP. Yoon et al. [38] introduced Bayesian MAML on the basis of Finn et al. [7]. Patacchiola et al. [20]; Snell & Zemel [27] utilized GPs with deep kernels for task-specific inference. These works made contributions to Bayesian meta-learning by addressing parameter updates, uncertainty modeling, and prior distributions. Our work contributes by providing an effective alternative for task-level updates and further provides insights into the coordination problem of bi-level optimization.
While the logistic-softmax with temperature has proven to be theoretically superior to softmax in data modeling, its performance and suitability are only verified in Bayesian meta-learning, leading to certain limitations. Further research is necessary to explore the logistic-softmax function's performance and adaptability to various domains and problem settings.
In this paper, we introduce the logistic-softmax function with temperature which is simple yet highly effective in improving classification accuracy and uncertainty calibration. Furthermore, we delve into the theoretical property of the redesigned logistic-softmax function including both limiting behavior and data modeling capability. Moreover, we apply mean-field approximation for deep kernel based GP meta-learning for the first time. We also shed some light on the coordination problem between the inner loop and the outer loop that appeared in bi-level optimization. In the future, it is an interesting track to apply redesigned logistic-softmax to other domains such as multi-label classification.
This work was supported by NSFC Project (No. 62106121), the MOE Project of Key Research Institute of Humanities and Social Sciences (22JJD110001), the fund for building world-class universities (disciplines) of Renmin University of China (No. KYGJC2023012), and the Public Computing Cloud, Renmin University of China.
[39] Zhang, S., Xu, R., Xiong, C., and Ramaiah, C. Use all the labels: A hierarchical multi-label contrastive learning framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16660-16669, 2022.
[40] Zhao, G., Dougherty, E., Yoon, B.-J., Alexander, F., and Qian, X. Efficient active learning for gaussian process classification by error reduction. Advances in Neural Information Processing Systems, 34:9734-9746, 2021.