## 4 Experimental Results
### Performance Verification Experiment  
#### 4.2.1 Neural Network Architectures  
In the model design of this paper, we first use an age estimation network pre-trained on an unoccluded dataset as the teacher model to guide the student model's learning. The teacher model selects three models that perform well in the age estimation task: CORAL, DLDL, and MWR:  
The CORAL (Cumulative Ordinal Regression) model effectively utilizes the ordinal relationship of age labels by converting the age estimation task into multiple sequential binary classification problems. This model divides the age labels into \(K-1\) binary classification tasks, where the \(k\)-th binary task determines whether the sample's age reaches or exceeds the threshold \(r_{k}\). For example, for \(K\) age levels, CORAL contains \(K-1\) binary classifiers, each sharing the same weight parameters but with independent bias terms, ensuring the consistency and monotonicity of the predictions. The loss function for CORAL is defined as  
\[L_{\text{CORAL}}=-\sum_{i=1}^{N}\sum_{k=1}^{K-1}\Bigg{[}y_{i}^{(k)}\log\Bigl{(} p_{i}^{(k)}\Bigr{)}+\Bigl{(}1-y_{i}^{(k)}\Bigr{)}\log\Bigl{(}1-p_{i}^{(k)} \Bigr{)}\Bigg{]},\]  
where \(y_{i}^{(k)}\) is the true label of sample \(i\) in the \(k\)-th binary classification task, indicating whether the sample's age reaches the threshold \(r_{k}\), and \(p_{i}^{(k)}\) is the probability predicted by the model for sample \(i\) in the \(k\)-th task, calculated by the Sigmoid function:  
\[p_{i}^{(k)}=\sigma(g(x_{i},W)+b_{k}),\]  
where \(g(x_{i},W)\) is the output of the shared weight layer and \(b_{k}\) is the bias for the \(k\)-th task. To ensure ordering consistency, the CORAL model requires  
\[b_{1}\geq b_{2}\geq\cdots\geq b_{K-1},\]  
ensuring that the predicted probabilities for each task satisfy monotonicity:  
\[p_{i}^{(1)}\geq p_{i}^{(2)}\geq\cdots\geq p_{i}^{(K-1)}.\]  
DLDL (Deep Label Distribution Learning) transforms the age estimation task into label distribution learning by constructing the probability distribution of age labels to capture the uncertainty of age. The output layer of DLDL contains the probability for each age label, representing the probability distribution over ages. By predicting a probability distribution instead of a single age value, DLDL reflects the ambiguity in age estimation, leading to higher robustness when processing real-world data, especially when age labels are vague or hard to annotate accurately. The label distribution is modeled using a Gaussian distribution, with the distribution center at the true age \(\mu\) and standard deviation \(\sigma\), defined as follows:  
\[y_{j}=\frac{p(l_{j}|\mu,\sigma)}{\sum_{k}p(l_{k}|\mu,\sigma)},\]  
where  
\[p(l_{j}|\mu,\sigma)=\frac{1}{\sqrt{2\pi}\sigma}\exp\left(-\frac{(l_{j}-\mu)^{ 2}}{2\sigma^{2}}\right).\]  
The optimization goal of DLDL is to minimize the Kullback-Leibler (KL) divergence between the predicted age distribution \(\hat{y}\) and the true label distribution \(y\), so that the predicted distribution is as close as possible to the true label distribution:\[L_{\text{DLDL}}=\sum_{k=1}^{C}y_{k}\log\left(\frac{y_{k}}{\hat{y}_{k}}\right),\]  
where \(y_{k}\) is the true label distribution probability at age label \(k\), and \(\hat{y}_{k}\) is the probability predicted by the model. By learning the probability distributions between different ages, DLDL better captures the ambiguity of age, thus improving robustness in the presence of sparse data or imprecise labels. MWR (Moving-Window Regression) is an innovative ordinal regression algorithm that achieves the precise estimation of ordered data by predicting relative ranks (\(\rho\)-rank). Traditional ordinal regression typically predicts the absolute value of the target, whereas MWR uses the concept of \(\rho\)-rank, transforming the prediction task into estimating the relative position with respect to reference points.