Adaptive Group Robust Ensemble Knowledge Distillation
Authors: Patrik Kenfack
Ulrich Aïvodji
Samira Ebrahimi Kahou
Venue: Accepted by TMLR ()

Abstract: Neural networks can learn spurious correlations in the data, often leading to performance degradation for underrepresented subgroups. Studies have demonstrated that the disparity is amplified when knowledge is distilled from a complex teacher model to a relatively ``simple'' student model. Prior work has shown that ensemble deep learning methods can improve the performance of the worst-case subgroups; however, it is unclear if this advantage carries over when distilling knowledge from an ensemble of teachers, especially when the teacher models are debiased. This study demonstrates that traditional ensemble knowledge distillation can significantly drop the performance of the worst-case subgroups in the distilled student model even when the teacher models are debiased. To overcome this, we propose Adaptive Group Robust Ensemble Knowledge Distillation (\AGREKD), a simple ensembling strategy to ensure that the student model receives knowledge beneficial for unknown underrepresented subgroups. Leveraging an additional biased model, our method selectively chooses teachers whose knowledge would better improve the worst-performing subgroups by upweighting the teachers with gradient directions deviating from the biased model. Our experiments on several datasets demonstrate the superiority of the proposed ensemble distillation technique and show that it can even outperform classic model ensembles based on majority voting. Our source code is available at https://github.com/patrikken/AGRE-KD.

Published in Transactions on Machine Learning Research (11/2025)
Adaptive Group Robust Ensemble Knowledge Distillation
Patrik Joslin Kenfack
Sutrik-jostin.kenffuce co...cl
ÉTS Montréal, Mila Quebec Al Institute
Ulrich Aïvodji
Ubrich.Aiioddj@ddsmmt....
ETS Montréal, Mila Quebec Al Institute
Samira Ebrahimi Kahou
Samira Ebrahimri. Kahor com com
University of Calgary, Mila Quebec Al Institute Canada CIFAR Al Chair
Reviewed OpenReview: https: // o/penrve//perevveeu. net/ per/fo id= G2BEBaKd8Y
Abstract
Neural networks can learn spurious correlations in the data, often leading to performance degradation for underrepresented subgroups. Studies have demonstrated that the disparity is amplified when knowledge is distilled from complex teacher model to relatively "simple" student model. Prior work has shown that ensemble deep learning methods can improve the performance of the worst-case subgroups; however, it is unclear if this advantage carries over when distilling knowledge from an ensemble of teachers, especially when the teacher models are debiased. This study demonstrates that traditional ensemble knowledge distillation can significantly drop the performance of the worst-case subgroups in the distilled student model even when the teacher models are debiased. To overcome this, we propose Adaptive Group Robust. Ensemble Knowledge Distillation (AGRE-KD), simple ensembling strategy to ensure that the student model receives knowledge beneficial for unknown underrepresented subgroups. Leveraging an additional biased model, our method selectively chooses teachers whose knowledge would better improve the worst-performing subgroups by upweighting the teachers with gradient directions deviating from the biased model. Our experiments on several datasets demonstrate the superiority the proposed ensemble distillation technique and show that can even outperform classic model ensembles based on majority voting. Our source code is available at https: httpps//github.ccm//aarrrkk////
1 Introduction
When trained with empirical risk minimization (ERM), neural networks susceptible to capturing spurious correlations in the data Tiwari & Shenoy 2023 which are features that correlate with but not are causally related to the class label (Qiu et al.) 2023 In particular, the class label might spuriously correlate with patterns in the data that are easier to learn than the intended pattern. For example, in the Waterbirds dataset (Sagawa et al.] (019), which contains images of landbirds and waterbirds, most landbirds are located in land background, and waterbirds in water background. Instead of predicting the actual bird species, the model trained with ERM can achieve high accuracy by relying the background to make predictions. This results in a significantly higher error for underrepresented subgroups that do not exhibit the spurious correlation (o.g., landbird on water background and waterbird on land background). Several works have shown that. model compression mots such as pruning (Hooker el al. 2020), and knowledge distillation (Lukasik et al. 2023 Lee & Lee 2023 Wang et al. 2023) can exacerbate the performance disparities between different subgroups. In knowlodge distillation (KD), ) a network with a smaller capacity (student)) is trained using the output of higher capacity network (teacher) (Hinton et al.) (2015). While KD can improve the student.
model's average performance, the gain is not uniform across subgroups (Lukasiik et al. 2023

---

Published in Transactions on Machine Learning Research (11/2025)
On the other hand, deep ensemble models have been shown to enhance generalization performance compared to individual models (Ganaie et al., 2022). and simple deep ensemble models with the same architecture, objective, and optimization settings can attenuate this shortcoming and improve the worst-case group performance (Ko et al. 2023 Kenfack ct al. 2021). However, evaluating several models at test time can be computationally expensive, making them less practical for deployment. on edge devices. To address this issuc, ensemble knowledge distillation involves distilling the knowledge of multiple teachers to a single student model O Youu et al., 2017 Radwan et al., 2024), and it remains unclear whether distilling from an ensemble of teachers improves the student's worst-group performance.
This paper studies how knowledge distillation from multiple teachers impacts underrepresented subgroups. We investigate whether the subgroup performance gains of deep ensemble models apply to ensemble knowledge distillation. Focusing on logit distillation, we consider ERM-trained (biased) and debiased teachers obtained by deep feature reweighting (DFR). DFR method proposed by Kirichenko et al. (2022), showing that simply retraining the last layer of the ERM-trained model using small held-out validation set of group-balances data can mitigale the spurious correlation. In this work, we also investigate whether the student model can learn debiased representations from the teacher's output, whose only the last layer retrained to mitigate bias.
The rationale behind focusing on logit distillation for investigating bias in ensemble knowledge is also motivated by the study by [zmailov et al. (2022) where the authors demonstrated that several debiasing methods do not learn better feature representation but correctly weight core features in the last classification layer, which reduce the reliance on spurious features for predictions. Our results reveal that underrepresented subgroups can be negatively impacted when training the student model using the aggregated outputs of multiple teachers. Other ensemble distillation approaches have been designed to boost the student's performance by modelling a better aggregation of the teachers' knowledge (Du et al. 2020 Zhang et al., 2022). In particular, the ensemble distillation method proposed by Du et al.] (2020) aims to find a better compromise when teachers have conflicting predictions, and the mothod by [Zhang et al. (2022) ensures distillation is donc only using teachers with confident predictions. Our results show that these ensemble distillation methods do not effectively fix the performance disparity across groups in the student model.
We therefore propose Adaptive Group Robust En- semble Knowledge Distillation (AGRE-KD) method to encourage the student to improve the performance of unknown worst-case subgroups. Specifically, our method relies on a model that has captured the spu- rious correlation (i.e., biased model) to guide the teachers' outputs aggregation process and ensure the student model does not capture biased knowledge from teachers. Prior work has relied on reference classifier to target and upweight samples from unknown worst- case, using the errors of the reference classifier (Liu lt al.) 2021 Nam et al. 2020) or its per-sample gra- dient magnitude (Ahn et al. 2022). In contrast, our proposal uses the gradient direction of the biased model to select and weigh teachers' outputs adaptively during the training.
T3
Biesed Direction
Average Direction
Teacher Direction
Adagtive Avarage
Direction
Illustration of our adaptive weighting method based on gradient direction. The bolder lines indicate the teacher's higher weight in the ag- gregated output.
Figure 1:
Intuitively, training the student model solely following the gradient direction that minimizes its KD loss with a biased teacher can result in local/global minima with a higher loss for the underrepresented groups (Lukasik let al. 2021). Our results suggest this behaviour can be exacerbated in ensemble knowledge distillation since the inherent consensus from the teachers' gradient direction can be dominated by a direction that minimizes the average error at the expense of the worst-group error. Our proposed methods compare the gradient direction of student loss with the biased model and each teacher in the ensemble, and upweight teachers whose gradient direction deviates the most from the biased model. Figure illustrates the gradient directions of the student loss with the biased model and three teachers and shows how classic averaging yields a direction that
2

---

Published in Transactions on Machine Learning Research (11/2025)
mostly aligns with the biased teachers, while our adaptive weight prioritizes the least biased models. As the gradient magnitude can be very noisy, we compare gradient directions by computing the dot product of their normalized vectors (i.e,, the cosine similarity), which removes the influence of the gradient magnitude. The contributions of this paper can be summarized as follows:
We demonstrate empirically that ensemble knowledge distillation amplifies the performance drop for the worst-case group, contrary to deep ensemble models. We attribute this the reduced capacity of the student network by showing that ensemble self-distillation using models with the same capacity improves robustness to spurious correlation and achieves comparable performance with the teachers.
We show that distilling knowledge from teacher models with only their last layer retrained to mitigate bias provides significantly more robust knowledge to the student model.
We propose novel gradient-based weighting scheme to ensure the distillation process minimizes the student's loss towards better worst-case group error. The proposed method utilizes model that has learned the spurious correlation (biased model) to orchestrate the distillation process in the gradient space.
We perform intensive experiments four well-known benchmarks, and the results demonstrate the superiority of the proposed method.
2 Background
We consider a multiclass classification task using given the training D = {(t;; yi, as)) where x; E X' is the imput feature, Yi E V the target variable with c= [ classes, and a; E A an unknown spurious feature. We aim to build a classifier is(-) that accurately predicts the class of the unlabelled test dataset. The classifier such that & is the output logits of uses mapping function f X RS, that assigns scores (211),.....((22))) exp(2y)
the given sample x and αg (z) the softmax function defined as (z(2) Excoo exp(xy)) Vy E [c].
The classifier derived by predicting the class label that maximizes the softmax, h(x) argmax (z(z). We evaluate the classifier's performance during training using a loss function, such as the softmax cross-entropy loss function, which measures how accurately samples are classified. When the attribute spuriously correlates with the target y, ERM-trained models strongly rely on the spurious feature and fail to generalize on the subgroups where the spurious correlation does not hold (Ye et al. 2024
Knowledge Distillation (KD). In KD, a student network f aims to achieve performance close to the higher-capacity network by mimicking the teacher model f (Hinton et al. 2015). In practice, we train the student model to mimic the teacher's output by minimizing the Kullback-Leiblee (KL) divergence between their oulputs, defined as follows:
=2 KL( ol
where 2" and zt are the student and the teacher logits, respectively. T is the temperature parameter controlling the smoothness of the probability distribution for more fine-grained information. The student loss can be combined with the classification loss on the ground truth label using the following double loss,
L= a LKD + a)) Lels
(2)
where Lets is the classification loss (e.g., cross-entropy loss) between the student's output and the ground
truth label (v), and is hyperparameter controlling the classification loss and knowledge distillation loss. In other KD techniques, the student network is enforced to mimic the teacher's internal representation instead of only outputs (Romero et al. 2014). Transferring knowledge from intermediary representation (feature-level) can provide more fine-grained information and boost the students' performance (Romero el al.] 2014 On the
other hand, recent studies have shown that models trained with ERM still learn core features while spurious
features are only amplified in the last classifier layer (Kirichenko et al. 2022 Qiu et al. 2023 Izmailov
et al. 2022). In this regard, we restrict ourselves to logit distillation and leave feature distillation for future exploration.
3

---

Published in Transactions on Machine Learning Research (11/2025)
Deep Ensemble. Ensemble learning is widely used technique to boost the generalization performance of model (Allen-Zhu & Li 2020p. Studies have shown that training several independent models and averaging their predictions at test time can yield a model that outperforms each individual model in the ensemble (Ganaic et al. 2022). However, evaluating multiple models for predictions at test time limits the practical use of deep ensembles due to computations overhead. Knowledge distillation can address this issue by enforcing the student network to mimic the ensemble's output.
Ensemble KD. Like in ensemble learning, distilling knowledge from multiple teachers instead of a single one is expected to improve the student model (You et al., 2017). In ensemble KD, train the student model using the averaged softened output or the averaged knowledge distilaaionn losses of M teachers as follows:
M
1
LeusKD =
(3)
M m=
This ensemble KD loss can be plugged in equation 2 for training the student with KD from an ensemble of M teachers. In contrast to these methods, our method is fully unsupervised, both based in terms of ground truth class label and group information. Related works more aligned with our approach are unsupervised methods that leverage teachers' diverse knowledge improve students' performance. (You et al.] 2017 Du
ct al., 2020 Fukuda ct al. 2017: Zhang et al. 2022 Kwon ct al., 2020). For example, Fukuda ct al. (2017 suggest that randomly selecting teacher during mini-batch training can allow the student model to caplure complementary knowledge of teachers. Other authors argue that simply averaging teachers' softmax outputs can mislead the student model, particularly when there is competition or contradictions between teachers. In this regard, Zhang et al.] (2022) proposes a sample-wise weighting for teacher loss based on the confidence of the teacher's prediction compared to the ground label. Other met consider label-free weighing schemes by comparing teachers in the gradient space. For instance, Du et al. (2020) use a multi-objective optimization approach in the gradient space to find teachers that agree the most in the gradient direction that minimizes their loss with the student model. Similarly, Zhou el al. (2024) sample-wise teachers selection by only averaging the majority of teachers with the same gradient directions. Our method also operates within gradient space to enhance students' resilience to spurious correlation. Our experiments demonstrate that adhering to the majority of teachers" opinions does not always benefit the underrepresented subgroups.
Spurious Correlation Let Der = ((),,;))}' be the training dataset, where each input x, E X belongs to one of K classes Y1 V. Along with the label ye, each sample also contains a spurious attribute a; E A, which is correlated with but not causally predictive of yı. A spurious correlation, denoted (y a), refers to A. In practice, an attribute may be
the association between class y E y and a spurious attribute linked to multiple classes, i.e., Φb : A + yk* for some < K" S K, depending on the training data Drr- To capture this structure, each sample can be assigned group label labels is 9 -= y x A. For example, within Der, the same attribute a might appear in two different classos, giving rise to correlations (y. a) and (y),a). A model trained naively on this data may incorrectly rely on a to predict y, which will lead to systematic errors when encountering samples with (3/,a) (Ye et al. (024)). As the average accuracy does not fully capture the robustness of the model to spurious correlation, we use the worst-group accuracy (WGA) to mcasure the model's reliance on spurious correlation for predictions. Furthermore, in the presence of spurious features, model ʃ parametrized by 0 is optimized to minimize the loss of the worst-performing subgroup, i.c.,
(y. a), and the set of all such group
1 lgl
arg min max 6 gEG
Σ C(f(x;)) g iE9
(4)
The WGA therefore quantifies the accuracy of the group experiencing the smallest test accuracy. Another line of related work aims to make the model robust to distribution shift (Yang et al. 2024), especially covariate shift. Unlike spurious correlation, covariate shift occurs when the training and test data follow different.
Du al. 2020: showed that the averaged softened output is equal to averaged KT. loss.
4

---

Published in Transactions on Machine Learning Research (11/2025)
distributions, while the conditional distribution of output values given imput points is unchanged. In other words, the marginal distribution of features changes, but the conditional relationship between labels and features remains unchanged. Techniques like importance weighting or domain adaptation aim to correct for this shift (Sugiyama et al. 2007), while we focus in this work on addressing model reliance on spurious features during training. Both notions could be related if the 'spurious feature' is the source of the covariate shifts, i.e., a feature that was predictive for the training data becomes spurious in the test set; however, such scenarios are less likely to occur since the model should not rely on spurious features.
3 Related Work
Bias mitigation without group label. We consider settings where samples in the dataset D are associated with unknown group labels that spuriously correlate with the class label. For example, in the CelebA dalaset, the class 'hair color' (blond, non-blond) correlates with the gender (male, female) since most images with blond hair belong to the female group. Neural networks can capture this correlation, resulting in poor performance for cerlain subgroups (e.g., blond males) (Sagawa et al., 2019). We aim lo ensure that the model does not capture spurious correlations the data and accurately classifies samples from all subgroups. In particular, we measure the model's bias using the worst-case group accuracy (WGA), ie., group of samples where the spurious correlation does not apply. Several methods have been proposed to mitigate these biases in individual models (Kenfack et al. 2024a; Ye et al.) 2024). When the group information is known, Sagawa ct al. (219)) propose Group Distributionally Robust Optimization (DRO) that minimizes the loss of the group experiencing the maximum loss. However, group information can be costly to collect or unavailable due to privacy restrictions (Kenfack et al.) 2024b)). Several methods have been proposed in this setting to improve the worst-case group performance without group labels (Ye et al., (0244). For instance, [iu et al. (2021) ) and Nam et al. (2020) rely on a reference classifier to target and upweight worst-performing subgroups. These methods use the mistakes of the reference classifier to improve group robustness by up- weighting misclassified samples (Liu et al. 2021). b. The reference classifier is generally trained to amplify the misclassification of samples from the unknown worst- porst-performig group (Nam et al.] 2020 While these methods do not use group labels during the training, they require access to a small va adliaaioon set with group labels for model selection and intensive hyperparameter tuning (Kenfack et al. 2024a). Kirichenko et al. (2022) proposed Deep Feature Reweighting (DRF) for training group robust model by retraining the last layer of ERM model with a small subset of a held-out group- balanced dataset, which achieves state-of-the-art worst-case group performance. However, all debiasing methods use neural networks of high capacity, and they are less effective models with smaller capacity Izmailov et al. (2022). Furthermore, the effect of distilling these debiased models on the student model's robustness remains underexplored. This work builds on Kirichenko et al.] (2022) for debiasing the teacher models by retraining their last layers using a held-out group balanced with fower group annotations, and we investigate whether distilling knowledge from an ensemble of (debiasod) teachers can lead to more robust student models.
group-balanced
Bias in Knowledge Distillation. teacher model (Lukasik et al. 2023 Lee & Lee 2023 Lukasik et al. 2023 Tiwari et al.) 2024 Bassi al. 2024). In particular, Lukasik et al. (023) demonstrates that teacher errors can be amplified by the student during distillation and proposed a mitigation strategy that distills only the confident predictions of the teacher. However, their study focuses on worst-class errors and KD with a single teacher, while we study worst-subgroup errors with multiple teachers. Lee & Lee (2023) propose an adapted version of Simple knowledge distillation (SimKD (Chen et al., (0222) that transplants the last layer of the teacher to student and only distills features. With the teacher trained with Group DRO, they show that transplanting the teacher's last layer to the student only improves the worst-case group performance if the feature distillation is performed by upweighting misclassified sample from reference classifier (Lee & Lee 2023). However, their method uses the reference classifier by Liu et al. (2021), where the efficiency of sample weighting requires intensive luning of the number of epochs to train the reference classifier. Similarly, Tiwari et al. (2024) uses the earlier layers of the neural network to train a reference classifier and shows that it improves the recall of worst group samples within the misclassification set, which are upweighted in the KD loss. However, their method also requires class labels to derive the misclassified samples. In Lukasik et al. (2023), the authors study where it is best to apply the debiasing mechanism (Group DRO) and conclude that applying the robust.
Several works have studied bias in knowledge distillation with a single
5

---

Published in Transactions on Machine Learning Research (11/2025)
KL Loss
Biased Network
Teacher Networks
Logits
Gradients
Gradient Dat Products
Adaptive weighting
Weights
KL Losses
Butth Samples
AVG
Student
Network
Weighted Average KL Losses
Figure 2: Overview of AGRE-KD.
loss to both the teacher and student model improves the average performance along with the wonst-case group performance. In contrast to these prior works, we study bias in knowledge distillation with multiple teachers without group information and class labels. We investigate whether the subgroup's performance gain observed in deep ensemble models also applies when the knowledge of the ensemble distilled to a single model. We aim achieve better worst-case performance across subgroups when aggrogating the outputs of multiple teachers in knowledge distillation. To the best of our knowledge, this represents the first study on bias in ensemble knowledge distillation.
4 Adaptive Group Robust Ensemble Knowledge Distillation (AGRE-KD)
In this work, we considered each teacher in the ensemble to have the same architecture and trained using different random initializations. Prior work has shown ensemble models with different random initializations are diverse enough to improve the performance (Ganaie et al., 2022). The first contribution of this work is showing that, unlike deep ensemble models that can improve the WGA (Ko et al. 2023), ensemble knowledge distillation tends to amplify bias in the distilled model.
To address this problem, we propose AGRE-KD, an adaptive ensembling knowledge distillation strategy that ensures the student model captures robust knowledge from the teachers. AGRE-KD relics model pretrained with ERM that captured the dataset's spurious correlation (biased model). Intuitively, suppose student model takes gradient steps toward the direction that minimizes its KD loss with the biased model. In that case, the resulting student model will likely capture and even amplify the reliance on the spurious correlation, i... the local/ /global minimum in that direction likely provides the worst performance for the underrepresented subgroups. Following this intuition, we derive weighting scheme that prioritizes teacher models whose gradient direction disagrees with the biased ERM model. Figure provides an overview of our proposed method.
Teacher training. We train each teacher model in the ensemble independently using standard ERM and cross-cntropy loss with the ground truth class labels. Teacher models have the same architecture and hyperparameters and only differ in random seeds used for weights initialization; prior studies have shown that independent training with different random weight initializations can provide sufficient ensemble diversity to improve the performance of the model ensemble dAllen-Zhu & Li 2020 Ganaic al.) (022). As
aforementioned, we obtain debiased teachers using the approach introduced by Kirichenko et al. (2022)
retrain the last. layer of the ERM model on small proportion of the held-out. group-balanced dataset. We

---

Published in Transactions on Machine Learning Research (11/2025)
perform the retraining step of the last layer using group-balanced batch sampling instead of the averaging models trained over group-balanced subsets of the data (LaBonte et al. 2024). This debiasing process is very simple and computationally inexpensive since it involves training a logistic regression model on a smaller dataset. Following Kirichenko et al.] [2022] LaBonte et al. (2024), we use half of the validation set cach benchmark to perform last-layer retraining with DFR; as we do not use the group and class labels during the KD training, we do not perform further hyperparameter tuning or model selection. After training, teacher models are frozen, and their output is aggregated to train student models as illustrated in figure
Biased model. For the biased model used by our method to compute teachers' weights, randomly select one teacher model trained with ERM. ERM-trained models are known to be sensitive to spurious correlation (Kirichenko et al. 2022 Ye ct al. 2024]. This provides us with intuition that the gradient direction updates of a model during ERM training are biased towards the majority, and leverage this intuition to upweight teacher models that deviate from the direction provided by an ERM model. Moreover, as we show via an ablation study in Supplementary D (Table 5), the performance of our gradient- based weighting scheme remains robust to different choices of ERM model architectures.
Gradient-based weighting scheme. For a given minibatch, we compute the student KD loss regarding the biased model b and each teacher t. Before aggregating the teachers' outputs, we compute sample weights based on the similarity between the gradient direction of each teacher and the biased model. Consider (0) the KD loss on sample regarding the (-teacher, and (0) KD loss regarding the biased model, with the parameters of the student model. The dot product ((:-)) between the normalized gradients of the student KD loss with the teacher and the biased model indicates which teachers align with the biased model in gradient space. In particular, when (V), (0)) > 0 both models have the same gradient directions and their gradients are in the opposite direction when (V)()), vE)(0)) < O. In extreme cases, when the dot product of the gradients is closer to 1, both models have precisely the same directions; if the dot product is close to 1, the models are in opposite directions. Thus, to penalize teachers with a gradient direction closer to the biased model, we downweight teachers' out for samples having dot products get closer to and upweight samples as their dot products get closer -. More formally, we compute the sample-wise teacher's weight as follows:
Wx:x:) =1 (VE(O), (((())
Adaptive knowledge distillation. The weighting scheme in Equation 5 suggests that leacher models who behave similarly to the biased model in the gradient space will have less influence on the aggregated outputs. Therefore, we aggregate the teacher's outputs using the sample-wise weighted average KD (wKD) loss defined as follows:
Wo(w;) LLD LwKD
(6)
E, We
And train the student model with the following final loss
C = aLwKD + (1 - Cets
(7)
As aforementioned, we focus on unsupervised knowledge distillation, i.e., we set = 1 to investigate the robustness the "dark knowledge" provided by an ensemble of eventually debiased teachers. Supplementary A provides a detailed algorithm of the training process of AGRE-KD. Note that the use of an additional biased model orchestrate the distillation process does not add extra complexity compared related work using a reference classifier to identify the worst-performing subgroups (Nam al. 2020 Liu et al.) 2021 Kenfack
et al. 2024a Furthermore, we will see in the experiments, our proposed method is robust to the choice of biased model architecture trained using ERM without.
Taable the Supplementary shows the importance ignoring the magnitude of the gradients.
7

---

Published in Transactions on Machine Learning Research (11/2025)
5 Experimental Results
In this section, we present the experimental setup and the empirical results. We experiment with several ensemble KD techniques and analyze their impact on the worst- group accuracy of the student model. We compare our adaptive ensemble knowledge distillation to these methods and demonstrate its effectiveness in improving the student model's worst-group performance.
5.1 Setup
We evaluate the worst-case performance of the proposed method on four classification tasks: one synthetic dataset (Colored MNIST), two real-world datasets from the vision domain (Waterbirds and CelebA datasets), and one worldworld dataset from the language domain (CivilComments).
Colored MNIST following Nam et al. (2000); Li & Vasconcelos (2019), we alter the MNIST dataset (LeCun et al. 2002), designed for digit classification, by artificially adding a Color attribute. To define the Color, we randomly select ten distinct RGB values and fix them across all experiments. Fach of these values serves as the mean of a 3-dimensional Gaussian distribution, forming ten distinct. Color distributions. To create correlations between digits and colors, we pair each digit with one of these distributions. A bias-aligned sample is generated by coloring digit with RGB value drawn from its paired distribution, while bias-conflict ing sample is produced by sampling from any of the remaining nine color distributions (Nam et al.) 2020). Under this setup, a model that captures the spurious correlation will predict color instead of the digits.
Waterbirds (Sagawa et al. 2019 Liu et al. 2021 dataset of birds derived from Caltoch-UCSD Birds (CUB) (Wah et al. 2011 by synthetically creating spurious correlation between bird species and the background. In particular, the class label is the type of bird appearing in the image (waterbirds, landbirds), and the background landscape (water, land) spuriously correlates with the bird type. Here, the minority subgroups represent images with the background landscape not aligned with the bird type, i.e., (waterbird, land background} and {landbird, water background).
CelebA Liu et al. 2015) dataset contains images of celebrities with 40 facial attributes. In this dataset, the attribute hair color is spuriously correlated with gender. We consider hair color [blond, non-blond) as the class label and gender (male, female} as group information.
CivilComments (Koh et al. 2021 is textual dataset collected from online comments. The task is to predict whether a comment is toxic or non-toxic. The label is spuriously correlated with comments related to some demographic subgroups such as gender (male, female), race (white, black), and sexual orientation (I.GBT). We consider a binary indicator of comments related to these demographic subgroups as spurious group information.
Network Architecture and Training. For the Colored MNIST dataset, we use MobileNet Sandder et al.
2018) for the teacher models and a simple CNN with two convolutional layers as for the student model. For the real-world vision tasks (CelebA and WaterBirds), following prior work (Twari et al., 2024 Lee & Lee
2023).
we use the Resnet-18 (He et al. 2016) architecture for the student model and the Resnet-50 (He et al,
2016 architecture for the teacher models. Both networks are pretrained on the ImageNet-1K: (Russakovsky et al.
2019) model for teachers and the DistilBERT (Sanh et al. 2019) for the student model; and language models are pretrained on Book Corpus and English Wikipedia. Following related works on KD (Du et al.
2015) dataset. For the language task, we use the BERT (Devlin et al.)
2020 Fukuda et. al. 2017 Chen et al. 2022). we set the temperature hyperparameter T = 4 (Eq. ! and show in an ablation study in Supplementary (Figure 5) that increasing 7 can exert positive effect on WGA up to certain values. We provide further details about hyperparameters in the Supplementary
Basclines. In addition to the standard training process using the onc-hot class label for training cach teacher model (Section 2) we consider other ensemble knowledge distillation methods aiming to improve the student's performance. In particular, we consider the following baseline:
8

---

Published in Transactions on Machine Learning Research (11/2025)
Deep Ensemble (Deep Ens.): This bascline corresponds to deep ensembling using a majority voting
scheme of models with the same capacity the student model. In particular, given set of models, the predicted class label represents the class that received the most votes for models in the ensemble.
One-hot: A student model trained with ground truth labels without knowledge distillation.
Random Fukuda et al. 2017): During each mini-batch training, this method randomly selects a teacher model from the ensemble to train the student model. Fukuda et al. (0117) referred to this technique as switched switcheddtrainnig as the weights of the student model are updated by switching across teacher labels at the minibatch level.
AVER (KD with averaged teachers' outputs You et al. 2017 ): Here, we perform standard knowledge distillation following equation 3 by minimizing the KL loss between the student's softmax output and the averaged softened outputs (dark knowledge) from teachers.
AE-KD (Du et al. 2020): This method is an adaptive ensembling distillation technique closest to ours. However, the method postulates that when teachers have conflicting gradient. directions, a multi-objective optimization problem solved select the gradient direction (teachers) that satisfies most of the teachers in the ensemble.
5.2 Results
We train each model using three random seeds and report the means and standard deviations. We consider ensemble distillation with ten teacher models randomly sampled from a poll of pretrained (biased) teachers across independent random seeds. In addition to the teacher's performance and deep ensemble approach, we report the performance of the "student" trained only using the ground truth class labels (One-hot), i..,, without knowledge distillation. We report the average accuracy and WGA. We consider models trained using ERM or with last-layer retraining for debiasing. Specifically, for ensemble knowledge distillation methods, we train the student model with biased teachers (ERM model) and debiased teachers (last layer retrained with DFR Kirichenko et al., 2022
5.2.1 Results on the synthetic dataset
We considered different settings of the Colored MNIST (CMNIST) dataset by varying the ratios of bias- aligned samples in the training dataset, i.c., the proportion of samples where the color and digit correspond (99.5%,999 98%, and 95%]. This means for configuration with a ratio of 99.5%, only 0.005% of samples will have a digit-color mismatch, and decreasing the ratio reduces the strength of the spurious correlation in the training dataset. We train every baseline under each bias ratio configuration and report the average and worst-group test accuracy.
Table D presents the performance of different approaches on Colored MNIST under varying levels of spurious correlation. Several important observations can be drawn.
The teacher models illustrate the critical role of debiasing. Without debiasing, teachers perform poorly the minority group (WGA as low as 30.5% under 99.5% spurious correlation), even average accuracy appears lower (37.4%) since the test set mainly contains samples where spurious correlation does not apply. Once debiased, however, teacher models show clear improvements in WGA across all regimes, confirming that debiased supervision is key to counteracting extreme spurious correlations.
Deep Ensembles models already provide a substantial performance boost: simply averaging biased models significantly boosts robustness compared to single teacher, leading to WGA improvements of 20-40 points in extreme regimes (e.g., 29. 29.2% to 69.8% at 99.5%). Interestingly, when ensemble predictions used for knowledge distillation (AVER, AEKD, or AGRE-KD with biased teachers), performance patterns diverge. In highly biased seltings (CMNIST-99.5%% 99,,, CMNIST-99%. ensemble KD tends to underperform relative to the raw Deep Ensemble, often worsening WGA (e.g., AVER and AEKD around 49-54% WGA vs. 69.8% for Deep Ens. in CMNIST-99.5%% However, in less extreme regimes (CMNIST-98% and CMNIST-95%) ensemble KD begins to catch up and even surpass the teacher ensemble, suggesting that distillation stabilizes and refines knowledge transfer when correlations milder.
9

---

Published in Transactions on Machine Learning Research (11/2025)
CMNIST-99..
CMNIST-99%
CMNIST-958
CMNIST-95%
Modela
Debiased
KD
Average
WGA
Average
WGA
Average
WGA
Average
WGA
x
37. 47.400.00
30.5+0.61
55.3
50.4+1.00
71.720.45
68.720..6
87.T
86.3+0.14
Teacher
/
58.41184
54.712.31
66.0_0.97
63. 1.00
80.310.60
10.70
83.4
81.7.0.58
Deep Ens.
x
36.210.a8
29.2.0.44
26.2-0.25
51.4
74.4
71.5
89.640.09
88.4
/
72.810.27
69.810.23
77.7=0.11
75.3
84.220.16
82.510.19
90.020.09
88.920..0 0.000
One-hot
x
x
26.916.52
19.0
29.3_6.80
21.617,93
34.517.85
27.4 27.4
47.017.01
41.418.07
/
x
53. 53.646.29
52. 22.6646.11
51.9=
51.745..6
81.5g1.58
80.0 1.82
61.841.88
60.9
Random
x
25.0+2.41
16.7 58
26.5_3.31
18.5
29. 29.444424
21.724.61
41.1+3.02
34.624.29
53.2+5.35
48.5+0.02
49.7=5.00
44.5
39. 39.3+.400
32 32.6g4.45
41.824.67
35.525.12
AVER
x
V
33.940.48
26 26.5+0.01
51.6=u.14
46.2+0.21
72.2
69. #0.1
89. 89.34
88. 88.1+0.54
14.11
49.010.28
62.8_o.42
58.710.48
86.510.55
85.010.83
88.510.19
87.310.22
AEKD
x
16.210.76
29.2 29.210.81 10.81
35.0 _o83
50.010.84
73.410.78
70.510.81
89.710.09
88.610.07
/
58.820.80
54.5
69.4_1.50
66.3
86.640.65
85.1
88.140.45
86.920.53
AGRE-KD
x
30.3.0.05
29. 19.222..0 65
35.5_0..0
51.5
74.2+0.21
71.40.06
90.2+0.31
89. 89.
63.9
59.9
70.7 #0.43
67.5
87.5
86.2
89.0
88.7 =0.49
Table 1: Results on the Colored MNIST dataset. We report the average and worst- group test accuracy (WGA) for different ratios of bias-aligned samples (199.5%, 5%, 99%, 98%, and 95%)). The Debiased column indicates whether the model involves debiasing with DFR or whether teacher models are debiasod when the KD column checked (V). Bolded represents the best-performing student with the debiased teacher ensemble and underlined represents the best-peeforming with the biased teacher ensemble.
Both AVER and AEKD, which attempt leverage ensemble outputs in different ways, offer moderate improvements over naive random strategies. For example, AEKD achieves 66.3% WGA at 99% with debiased teacher, substantially higher than One-hot or Random. However, their effectiveness remains inconsistent: in extreme regimes, they still fail to match the robustness of the Deep Ensemble, and in milder regimes, they are often outperformed by stronger approaches.
Across all bias regimes, AGRE-KD consistently yields the best performance when combined with debiased teachers. It achieves the highest WGA at every correlation level, from 59.9% under the most extreme bias (CMNIST-99.5%) to 88.7% in CMNIST-95%. Notably, AGRE-KD is the only KD method that not only closes the gap with the raw Deep Ensemble in highly biased regimes but surpasses it in less extreme regimes. Even with biased teachers, AGRE-KD matches or exceeds other KD baselines, demonstrating its robustness to spurious correlation during the distillation process.
5.2.2 Results on real-world datasets
Table summarizes the main results of the paper on real-world datasets, from which we draw the following observations:
Ensomble knowledge distillation can amplify bias. From Table 2 we first observe that deep ensembles improve worst-group accuracy (WGA) relative to single models (one-hot). For example, on Waterbirds, WGA increases from 54.1% (biased model) to 59.3% (biased deep ensemble) and from 86.7% (debiased model) to 90.0% (debiased deep ensemble). On CclebA, the jump is even larger: WGA improves from 32.3% (biased model) to 37.2% (biased deep ensemble) and from 88.9% (debiased teacher) to 90.3% (debiased deep ensemble). This confirms that ensembling stabilizes predictions and mitigates reliance on spurious features.
However, when ensemble predictions are used for knowledge distillation, the picture changes. In highly biased regimes, ensemble KD often reduces WGA. For instance, on Waterbirds, Random KD drops WGA to 39.6%, nearly 20 points below the biased deep ensemble (59.3%). Similarly, AVER KD achieves only 46.4% WGA on Waterbirds and 28.8% on CelcbA, representing losses of more than 10 points compared to their ensemble teachers. These results suggest that switching between teachers or naively averaging teachers' output during distillation amplifies bias, particularly harming
10

---

Published in Transactions on Machine Learning Research (11/2025)
minority groups, even if average accuracy remains high (e.g., CelebA average accuracy stays around 95.5%) When comparing to the one-hot baseline, we observe mixed results, but generally, ensemble KD can increase average accuracy compared to the "One-hot' baseline, but they are also more prone to bias amplification. This reinforces our finding that while ensemble KD offers advantages, debiasing mechanisms are essential to make it competitive with simpler approaches like direct training or DFR.
DFR teachers lead to robust student models. When we train the teacher models using ERM and then retrain the classification layer with the group-balanced set, the resulting student models achieve significantly better worst-case group accuracy compared to those trained only with ERM leacher models. We only train the student model using the aggregated teachers' outputs without any feature distillation. For example, on CelebA, Random KD training with debiased teachers improves WGA from 27.2% (biased) to 85.1% (debiased), a remarkable +57.9 point gain. Similar patterns hold for the other ensemble KD methods with AVER KD (28.8% + 83.4%), AE-KD (46.9% + 85.0%), and AGRE-KD (55.0% + 87.9%).
These results demonstrate that the retrained classification layer can provide pseudo-label distribution (dark knowledge) that reduces students' reliance on spurious features. This shows that by mimicking the teacher's s outputs, the classifier layer of the student model also downweights the spurious features in its last layers (Izmailov et al.) 2022). As the temperature parameter (7) also alters the teacher's 's output by softening its prodicted probability distribution, we provide an ablation study (Appendix C.2) showing that incrcasing т positively impacts the worst-group accuracy. We also show in Appendix ?? experiments with different values of a (Eq. 3 & 7 controlling the balance between updaling the student model with ground truth label and ensemble knowledge distillation loss. The results show that the student model gets less biased as more weight is given to the knowledge distillation loss. For example, on the Waterbirds dataset, the WGA increases linearly by up to +25% with increased values of a.
On the other hand, the improved students' WGA across ensemble distillation methods still does not match the WGA of the debiased teachors or debiased deep ensemble. For example, on Waterbirds, AVER KD with debiased teachers achieves 82.9% WGA, 7% below the 90.9% WGA of the debiased teacher. We attribute this to the smaller capacity of the student model, which we discuss in the next experiment.
Our weighting scheme consistently improves WGA. Finally, our proposed AGRE-KD achieves the most robust results across datasets. With debiased teachers, AGRE-KD attains the highest WGA in all three benchmarks: 87.9% on Waterbirds, 91.9% on CelebA, and 75.9% on CivilComments. This represents improvements of 2.9, 3.0, and 0.9%, respectively, over the best competing KD methods (AEKD AVER). Importantly, on CelebA, AGRE-KD even outperforms the deep ensemble itself (91.9% vs. 90.3% WGA), showing that our weighting scheme extracts more bias-free knowledge than naive ensembling. When all teachers are biased, AGRE-KD still identifies and upweights the least biased members. For example, on Waterbirds, it achieves 55.0% WGA, outperforming AVER KD(46.4%) and Random KD (39.6%). On CelebA, it reaches 37.6%, slightly higher than the biased teacher baseline (37.5%). While the improvements in this setting are modest since all teachers share similar spurions gradients- the method still avoids the large collapses observed with other KD strategics. In the Supplementary D (Table 7 we provide the group-wise accuracy of cach bascline method on the Waterbirds and CelebA datasets, showing how each subgroup is impacted by each method.
5.2.3 Impact of the model capacity.
In this experiment, we study whether the student's network capacity is source of the implication of spurious feature learning in ensemble KD. The capacity here represents the number of trainable parameters of the model. For example, our ResNet-50 teacher models have around 25.6M parameters, substantially larger than ResNet-18 student models with approximately 11M parameters. We perform the same experiments as previously, but using sesf-distillation, i.c., we use the same architecture for the student and teacher models (e.g., ResNet-50 on image tasks). Results in Table show that ensemble KD methods with self-distillation
11

---

Published in Transactions on Machine Learning Research (11/2025)
Waterbirds
CelebA
CivilComment
Models
Debiased
KD
Average
WGA
Average
WGA
Average
WGA
X
85.7
65.6
95.4=0.07
37. 37.5+2.27
90.0
75.9+1.15
Teacher
94.2+0.53
90.9+1.00
93.7=2.14
10.1
85.920.19
77-9+0.19
84.4+0.00
59.3+0.44
95.6=0.02
37.2
90.620.02
76.1+0.10
Deep Ensemble
93.5
90.0+0.56
92.4
90.3 00.5
85.9
76.6+0.09
x
83.8
54.1+2.21
95.5.0.07
32.3
20.1
75.5
One-hot
x
11.4
1..7
92.3
88.9
85.7
76.5+1.09
80.0
39.643.63
20.2
27.2
20.8
75.2
Random
89.6
77.3
92.5 1:0.9
85. 10.84
91.0.to.18
74.2
79.2
26.99
95.5.1000
28.8
90.9
74.7
AVER
90.8 44
1..9+1.23
92.4
83.4
90.8
75.0
/
80.7
46.9
95.3
30.5+1.88
90.8
73.1+3.05
AE-KD
90.9 /
85.0+1.22
92.340.26
87.5+1.17
90.7
74.8+1.11
82.2
55.4
95.4
37.6
89.3
74.7
AGRE-KD(Ours)
40.34
87.9+1.23
90.7
91.9
90.2
75.9
Table 2: Comparison of ensemble KD methods. We report the average and worst-group test accuracy (WGA) on each dataset. The Debiased column indicates whether the model involves debiasing with DFR whether teacher models are debiased when the KD column is checked (V). Bolded represents the best- performing student with the debiased teacher ensemble and underlined represents the best-performing with the biased Leacher ensemble.
Waterbirds
CelebA
Models
Debiased
KD
Average
WGA
Average
WGA
X
84.4
59.31
95.610.02
37. 37.700.00
Deep Ensemble
/
93.5
90.0+0.56
92.440.18
88.840.55
X
x
85.7
65.6
95.440.07
37. 37.542.27
One-hot.
/
x
10.53
90.911.00
92.510.34
88.2
x
83.5 1.668
64.0
95.5110.00
35.8
Random
/
92.8
90.3
92.640.26
87.940.64
x
83.5 20.55
63. 6..722.72
95.610.02
3..3
AVER
92.8+0.64
90.2+0.23
92.6+0.20
88.3+1.66
X
84.2
61.0
95.640.02
36.9
AE-KD
91.9+1189
89. 39.0
92.3+0.01
89.4 19.441.11
X
84.9+1.40
66.3 1.76
94.5+3.22
39.2
AGRE-KD
91.4
91.1 12.56
91.1
91.9
Table 3: Results on self-distillation. The student and the teacher models have the same network architecture (ResNet-50). The student's worst group test accuracy increases when we distill to a network with higher capacity.
significantly improve the worst-group test accuracy. The gap between the teacher models and students is reduced compared to KD settings in Table Ø where the students' models have smaller capacity. The results indicate that the students' higher capacity can help the network learn more core features and reduce the inffuence of spurious features in the last layer. On the other hand, AGRE-KD outperforms other baselines, showing that our adaptive weighting scheme effectively guides the student. models to focus minimizing
12

---

Published in Transactions on Machine Learning Research (11/2025)
worst-case group error during training. We further illustrate the effectiveness of our weighting scheme in the next experiment by adjusting the number of debiased teachers in the ensemble.
5.2.4
Effect of the number of debiased teachers.
90
.92
80
Acmirny (%)
900
80
80
70
194
86
40
84
50
0.8
0.4
Proportion debsased
Proportion debiased
Proportion debaased teachers
Proportion teachers
(a) Waterbirds
(b) CelebA
One-bot (FRM)
One-bot (DFR)
Deep
RANDOM
AVER
AB-KD
ACRE-XD (Ours)
Figure 3: Results on the proportion of debiased teachers in the ensemble. We trained the student model using an ensemble of 5 teachers with different ratios of debiased teachers within the ensemble ({0.2, 0.4, 0.6, 0.8, 1}). AGRE-KD effectively upweights and favors the least biased teachers in the ensemble, while other ensembble methods rely more on biased teachers' output and decrease the WGA. AGRE-KD maintains significantly higher WGA, despite having only a single debiased model in the ensemble.
In this experiment, we study how an ensemble with different proportions of biased and debiased models impacts the WGA of the student model. We use the same training process as previously and consider an ensemble of five teachers with different proportions of debiased teachers (i.e., {0.2, 0.4, 0.6, 0.8, 1}); we report the average and WGA in Figure 3 for the Waterbirds and CelebA datascts. The results show that students' WGA increases the proportion of debiased models in the ensemble increases. When the ensemble contains single debiased model, the aggregation process of other ensemble KD methods relies more the majority of biased teachers, leading the student model to capture spurious correlation. On the other hand, our method can implicitly identify and adaptively rely on the knowledge of debiased teachers while reducing reliance on biased teachers. This reflected by the significantly higher WGA in settings where the ensemble contains a single debiased teacher. Additionally, AGRE-KD outperforms or matches the performance of the deep ensembling model, where the last layer each model in the ensemble directly retrained.
5.2.5 Ablation on teacher architecture heterogeneity
Debiased Teachers
ERM Teachers
Models
Architecture
KD
Avg
WGA
Avg
WGA
VIT
X
96. 90.30
90.1
92.710.00
73.8
Teacher
NesNet-50
X
94.3+0.8
89.5+2.5
88. 88.7+0.09
68.0+2.20
Con2NeXtV2
92.440.6 X
85.741.5
86.440.41
62.0
RANDOM
94.3 10. 18
80.642.90
88.9
57.7 43.21
AVER
94.2+0.21
80.7+1.10
84.240.
41.7
ResNet-18
AE-KD
93.5
84.942.81
89.141.4
57.9
AGRE-KD
93.540..3
87.941.25
89.5
61.642.10
Table 4: Results with heterogencous teacher ensemble on the Waterbirds dataset. We train teachers with different architectures (ViT (base), ResNet-50, and ConvNeXtV2 (base)) and perform knowledge distillation to ResNet- 18 student model.
13

---

Published in Transactions on Machine Learning Research (11/2025)
Experiments in Section 5.2.3 revealed that the capacity of the student model plays an important role in bias amplification of ensemble KD. Prior work has demonstrated that increasing the diversity of the ensembles results in better generalization performance. We enforced diversity by initializing the weights with independent random scods (Allen-Zhn) & Li 2020p. We now explore the heterogeneity of teacher models by building ensembles with diverse work architectures and analyzing the impact on the WGA accuracy of the student. models. We train teacher models using three different network architectures, i.c., ViT (Dosovitskly al., 2020), ConvNeXtV2 (Woo et al. 2023), and ResNet-50 (He et al.) 2016). Using the same experimental setup previously, built ensemble of nine models with equal proportion of each model architecture, i.e., three models with each architecture, all trained using different random initialization. This ensures diversity between and within teacher architectures.
The results in Table[[show, for the Waterbirds dataset, the average and WGA accuracy when the heterogeneous ensemble contains ERM-trained or debiasod teacher models. We observe that even under more heterogeneous teacher ensemble, student models trained with classical ensemble KD methods can match the teacher's average accuracy but incur a significant drop in WGA. Compared to student models trained with homogeneous ensembles of ResNet-50 models (Table 2), we observe that using more diverse teacher's backbone marginally improves performance across basclines, yet the drop in WGA remains important across ensemble KD methods. This highlights that the bias amplification in ensemble KD does not necessarily depend on ensemble diversity, further supporting our hypothesis of student. capacity as a source of the disparity. On the other hand, the orchestration process in AGRE-KD shows better performance as its distillation process encourages better WGA. Despite the teachers' architecture being heterogencous, we use a ResNet-50 biased model to compute teachers' weights in AGRE-KD during the distillation process, demonstrating the robustness of the proposed approach to the choice of teacher model architecture. The results on the CelebA datasct show the same trends, which we provide in Table g in the Appendix.
6 Limitation and conclusion
In this paper, we studied bias in ensemble knowledge distillation (KD) and demonstrated that, unlike deep ensemble models that reduce bias, traditional ensemble KD methods can amplify it. We proposed AGRE-KD, an adaptive gradient-based weighting method that improves group robustness ensemble KD by guiding the student model to learn core features and boosting worst-group accuracy. Our experiments across several benchmarks retenstrated the effectiveness of our approach in distilling knowledge with reduced spurious correlations. While our results highlight AGRE-KD's advantages over existing methods, several limitations and open questions remain. Specifically, the study focused on unsupervised KD using the teachers' logits alone, without access to group or class labels. Although our primary interest was in examining the ability of an ensemble of teachers to distill more robust knowledge, exploring scenarios where class labels are available could enhance the performance of the student further. Moreover, we observed that the WGA improvement in the proposed method is less pronounced when all teachers in the ensemble are biased. This indicates an opportunity to leverage class labels in this context to further enhance group robustness, e-g., by considering the teachers' misclassifications. Finally, additional evaluation using more complex datasets, such as health datasets, is necessary to validate the approach across wider range of applications.
References
Sumyeong Ahn, Seongyoon Kim, and Se- Young Yun. Mitigating dataset bias by using per-sample gradient. arXiv preprint Xxx:22205.15744 2022.
Zeyuan Allen-Zhn and Yuanzhi Li. Towards understanding ensemble, knowledge distillation and self-distillation deep learning. arXiv preprinl arXiv:2012.098 099.66 2020.
Pedro RAS Bassi, Andrea Cavalli, and Sergio Decherchi. Explanation is all you need in distillation: Mitigating bias and shortcut learning. XrXiv preprint arXim:2407.09788 09788, 2024.
Defang Chen, Jian-Ping Mci, Hailin Zhang, Can Wang. Yan Feng, and Chun Chen. Knowledge distillation with the reused teacher classifier. In Proceedings of the IEEE/CVF conference on computer vision and pallern recognition, pp. 11933 11942, 2022.
14

---

Published in Transactions on Machine Learning Research (11/2025)
Jacob Devlin, Ming-Wer Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In North American Chapler of the Associalion for Compudaaioonl Linguistics, 2019. URL heeaa//ccpp..emmttcccchhoooaa De
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:0010.11929 2020.
Shangchen Du, Shan You, Xiaojie Li, Jianlong Wu, Fei Wang, Chen Qian, and Changshui Zhang, Agree to disagree: Adaptive ensemble knowledge distillation in gradient space, advances in newral information processing systems, 33:12345-12355, 2020.
William Falcon and The PyTorrc Lightning team. Pytorch lightning, 2019. URL https: //ww pytorchlightning. ail Available at: https:://i /ttp:bb.ggtu.......m com/Lightning cow./LLghhonnngg////ittttti
Takashi Fukuda, Masayuki Suzuki, Gakuto Kurata, Samuel Thomas, Jia Cui, and Bhuvana Ramabhadran. Efficient knowledge distillation from an ensemble of teachers. In Interspeech, pp. 3697-3704 2017.
Mudasir A Ganaie, Minghui Hu, AK Malik, M Tanveer, and PN Suganthan. Ensemble deep learning: A review. Engineering Applications of Artificial Intelligence, 115:105151, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770 778, 2016.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02555 2015.
Sara Hooker, Nyalleng Moorosi, Gregory Clark, Samy Bengio, and Emily Denton. Characterising bias in compressed models. arXie preprint wxX::2210000305,, 2020.
Pavel Izmailov, Polina Kirichenko, Nate Gruver, and Andrew G Wilson. On feature learning in the presence of spurious correlations. Advances in Neural Information Processing Systems, 35:38516 38532, 2022.
Patrik Joslin Kenfack, Adil Mehmood Khan, SM Ahsan Kazmi, Rasheed Hussain, Alma Oracevic, and Asad Masood Khattak. Impact of model ensemble on the fairness of classifiers in machine learning. In 2021 International conference applied artificial intelligence (ICAPAI), pp. 1-6. IEEE, 2021.
Patrik Joslin Kenfack, Samira Ebrahimi Kahou, and Ulrich Aivodji. A survey on fairness without demographics. Transactions on Machine Learning Research, 2024a. ISSN 2835-8856. URL https:://peerrever. net/
forum?1d==3E44PPPTE
Patrik Joslin Kenfack, Samira Ebrahimi Kahou, and Ulrich Aivodji. Fairness under demographic scarce regime. Transactions on Machine Learning Research, 2024b. ISSN 2835-8856. URL htttps://openreviil
tett/forrummdd-7718800ddddd
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. arXiv preprint orXiv:2204 02937, 2022.
Wei-Yin Yin Ko, Daniel D'souza, Karina Nguyen, Randall Balestriero, and Sara Hooker. Fair-ensemble: When fairness naturally emerges from deep ensembling. arXiv preprint arXiv:2903.008: 2023.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasuaga,, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In International conference on machine learning, pp. 5637-5666 PMLR, 2021.
Kisoo Kwon, Hwidong Na, Hoshik Loe, and Nam Soo Kim. Adaptive knowledge distillation basod on entropy. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7409 -413. IEFF, 2020.
15

---

Published in Transactions on Machine Learning Research (11/2025)
Tyler LaBonte. Milkshake: Quick and extendable experimentation with classification models. //github. //github.com/taee wow////laaaotteeeilllaaaaaaaaaaa 2023.
http:
Tyler LaBonte, Vidya Muthukumar, and Abhishek Kumar. Towards last-layer retraining for group robustness with fewer annotations. Advances in Neural Information Processing Systems, 36, 2024.
Yann LeCun, Léon Botton, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applicd to document recognition. Proceedings of the IEEE, 86(11):2278 2324, 2002.
Jiwoon Lee and Jaeho Lee. Debiased distillation by transplanting the last layer. arXiv preprint 11X1:22002.1187,, 2023.
Yi Li and Nuno Vasconcelos. Repair: Removing reprosentation bias by dataset resampling. In Proceedings of the IEEE/CVE conference on computer vision and pallern recognilion, pp. 9572 9581, 2019.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. International Conference on Machine Learning, pp. 6781-6792. PMLR, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730 3738, 2015.
Ilya Loshchilov, Frank Hutter, et al. Fixing weight decay regularization in adam. International Conference Learning Representations (ICLR), 2019.
Michal Lukasik, Srinadh Bhojanapalli, Aditya Krishna Menon, and Sanjiv Kumar. Teacher's pet: under- standing and mitigating biases in distillation. arXie preprint arXiv:2106.10494 2021.
Michal Lukasik, Srinadh Bhojanapalli, Aditya Krishna Menon, and Sanjiv Kumar. Teacher's pet: under- standing and mitigating biases in distillation. Transactions on Machine Learning Research, 2023.
Aida Mohammadshahi and Yani loannou. What is left after distillation? how knowledge transfer impacts fairness and bias. arXiv preprint arXiv:2410. 08407, 2024.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahm, Jacho Lee, and Jinwoo Shin. Learning from failure: De-biasing classifier from biased classifier. Advances in Newral Information Processing Systems, 33:20673 20684, 2020.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. NIPS Workshop Autodiff, 2017.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, ZZeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances neural information processing systems, 32, 2019.
Shikai Qiu, Andres Potapezynski, Pavel Izmailov, and Andrew Gordon Wilson. Simple and fast group robustness by automatic feature reweighting. In International Conference Machine Learning, pp- 28448 28467. PMLR, 2023.
Ahmed Radwan, Layan Zaafarani, Jetana Abudawood, Faisal AlZahrani, and Fares Fourati. Addressing bias through ensemble learning and regularized fine-tuning. arXiv preprint arXiv:2402.0000 00910, 2024.
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.655// 2014.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Sathoesh, Scan Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-211 252, 2015.
16

---

Published in Transactions on Machine Learning Research (11/2025)
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Perey Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. Xi preprinl arXiv:1911.08791, 2019.
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted rosiduals and linear bottlenecks. In Proceedings of the IEEE conference computer vision and pollern recognilion, pp. 4510 4520, 2018.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, distilled version of bert: smaller, faster, cheaper and lighter. ArXiv, abs/1910.01 108, 2019. URL http:s////pp...saaatttcccccoobot
prg/CorppuID::20622577
Masashi Sugiyama, Matthias Krauledat, and Klaus-Robert Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5), 2007.
Rishabh Tiwari and Pradeep Shenoy. Overcoming simplicity bias in deep networks using feature sieve. In International Conference on Machine Learning, pp. 34330 34343. PMLR, 2023.
Rishabh Tiwari, Durga Sivasubramanian, Anmol Mckala, Ganesh Ramakrishnan, and Pradcep Shenoy. Using early readouts to mediate featural bias in distillation. In Proceedings of the IEEE/CVF Winler Conference on Applications of Computer Vision, pp.
2024.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. California Institute of Technology, 2011.
Serena Wang, Harikrishna Narasimhan, Yichen Zhou, Sara Hooker, Michal Lukasik, and Aditya Krishna Menon. Robust distillation for worst-class performance: on the interplay between teacher and student objectives. In Uncertainty in Artificial Intelligence, pp. 2237-2247. PMLR, 2023.
Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders. In Proceedings of the IEFE/CV CCVF conference on computer vision and pattern recognition, pp. 16133-16142 2023.
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey. International Journal of Computer Vision, 132(12):5635 5662, 2024.
Wenqian Ye, Guangtao Zheng, Xu Cao, Yunsheng Ma, and Aidong Zhang. Spurious correlations in machine learning: A survey. arXiv preprint arXiv:2402.1271 12715 2024.
Shan You, Chang Xu, Chao Xu, and Dacheng Tao. Learning from multiple teacher networks. In Proceedings of the 23nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1295-1294, 2017.
Hailin Zhang Delang Chen, and Can Wang. Confidence-awware multi-teacher knowledge distillation. In ICASSP 2022-20 2022 IEEE International Conference on Acoustics, Speech and Signol Processing (ICASSP), pp. 4498 4502. IEEE, 2022.
Wenjie Zhou, Zhenxin Ding, Xiaodong Zhang, Haibo Shi, Junfeng Wang, and Dawei Yin. Govern: Gradient orientation vote ensemble for multi-teacher reinforced distillation. arXiv preprint 2xXi:2205.03764, 2024.
17

---

Published in Transactions on Machine Learning Research (11/2025)
A AGRE-KD: Algorithm and method overview
In Algorithm | we provide high-level description of the AGRE-KD methodology for training and validation
of group-robust distilled models. We also provide an overview our method Figure 2
Algorithm Adaptive Group Robust Ensemble Knowledge Distillation (AGRE-KD)
1: Input: Ensemble of pretrained teachers T = , ......, "2....,TmT] TMr, biased model T., student model with parameters 0, dataset D, distillation coefficient a, temperature parameter T for each minibatch [(x, y:) R from D do
for each leacher 7, E 7 do Compute knowledge distillation loss (0) for sample between S and T₃
Equation 1
Compute biased model distillation loss (0) for sample between S and Ti Equation
6:
Compute gradient aligmment G; (O). (e) & Dot product of normalized gradient vectors.
Table showw the importance of ignoring the magnitude of the gradients.
7:
Compute the adaptive sample weight for teacher t on sample it W((z;) = --
8:
end for Compute weighted knowledge distillation loss:
Ee -1(z))).400 Σ, Wz()))
LwKD
10:
Compute classification loss (if labeled data available): Lels
11:
Compute final loss:
C = aLakD (1 Cels
12: Update student model parameters 0 using gradient descent on C
13:
end for
B Hyperparameters
We train all models using standard hyperparameters from previous work [aBonte et al. 2024 Kirichenko lt al. 2022 Izmailov et al. 2022) and keep their value fixed across experiments. For the vision tasks, we used an initial learning rate 1 10-3 with a cosine learning rate scheduler; we used a batch size 32 and 100 for the Waterbirds and the CelebA datasets, respectively. For the CivilComments dataset, we use an initial learning rate of 10-5 with linear learning rate scheduler, batch size 16, and train for ten epochs. We keep all hyperparameters fixed to train the teacher and student models. For the optimizer, AdamW (Loshchilov et al. 2019) and SGD for the language and vision datasets, respectively, with weight decay of x 10-4, Our implementation uses PyTorch (Paszke et al. 2017 2019). Torch Lightning (Falcon &
used
team
2019 and Milkshake [LaBonte 2023
C Ablation studies
C.1
Ablation on a
Equations 3 and define the student training objective as weighted combination of the cross-entropy loss with ground-truth labels and the ensemble knowledge distillation loss. The hyperparameter a E [0, 1] controls this weighting: smaller values place more emphasis the ground-truth loss, while larger values prioritize the distillation signal. Since our primary focus is to study how bias propagates through ensemble KD, all main paper experiments fixed = 1.0, thereby relying entirely on distillation.
To investigate the impact a, we performed an ablation with a E {0.11 0.5, 0.7, 0.9, 1.0] on the Waterbirds and CelebA datasets. We used ensembles of five ResNet-50 DFR teachers and a ResNet-18 student, and report results averaged over three random seeds. Figure presents the trends for both average accuracy and worst-group accuracy (WGA).
18

---

Published in Transactions on Machine Learning Research (11/2025)
We observe that WGA increases nearly monotonically with a, indicating that when the KD loss dominates, students inherit more of the debiasing effect from the DFR teachers. For example, in Waterbirds with AGRE-KD, WGA rises from 62.5% at a = 0.1 to 84.6% at = 1.0. The best-performing baseline (Average KD) improves from 62.3% to 81.3% across the same range, yet still falls short of the One-hot model trained directly with DFR (83.9% WGA).
These results reinforce two key points: (i) ensemble KD benefits from stronger reliance on distillation when teachers are debiased, and (ii) without dedicated mechanisms such AGRE-KD, ensemble KD can lag behind simpler approaches like direct DFR training. This highlights the importance of explicitly addressing bias amplification in ensemble distillation.
Moan Acc.
WGA
Mean Acc.
WGA
80
95
Acciraey
90
70
94
888
93
60
84
92
40
0.1
0.3
0.5
0.7
0.91.0
0.1
0.3
0.5
0.7
0.9 1.9
0.1
0.3
0.5
0.7
0.91.0
0.1
0.3
0.5
0.7
0.91.0
(a) Waterbirds
(b) CelebA
...- One-hol (ERM)
Onc bot (DFR)
RANDOM
AVER
AE-KD
AGRE KD (Ours)
Figure 4: Effect the parameter a (Eq. 3 on the worst-group group accuracy DER teachers.
C.2 Ablation temperature parameter
Our experiments using the DFR teacher models revealed that reweighting the feature by retraining the last classification layer provides a 'dark knowledge' (soft target) to the student model that is more robust to spurious correlation. Since DFR teachers share the same feature representation as their ERM-trained counterparts but differ only in their final classification layer, the enhanced robustness the student when using DFR teachers indicates that the probability distribution over all classes predicted by the teacher significantly influences the student's dependence spurious correlations. Since the temperature hyperparameter (Fquation 1 softens the probability distribution of the teacher to better expose dark knowledge. We perform ablation study by varying the temperature parameter (T E {1, 21,2,44 2, 10}) and analyze its influence on the spurious correlation captured by the student model.
Figure shows the WGA averaged across three independent random seeds in WaterBirds and CelebA datasets. We considered ensembles of five ResNet-50 DFR teachers and ResNet-118 student. The results show that WGA increases with temperature and then remains steady or decreases after higher values of temperature (r 10 and 8 for the Waterbirds and CelebA datascts, respectively). The smallest WGA is achieved when 1, which means the training uses a regular cross-entropy loss; it increases when we soften the probability distribution (T > 1) and eventually decreases when the distribution gets almost uniform (r > 1). These findings suggest that the dark knowledge provided by the softened probability distribution of the teacher models roduces students' reliance on spurious correlation during training. These results align with recent. work by MMohamnasshahi & Toannou (2024) studying the benefit of increased temperature for fairness in knowledge distillation.
Supplemental results
19

---

Published in Transactions on Machine Learning Research (11/2025)
Waterbirds
CelebA
91
88
90
86
89
WGA (%)
84
88
82
86
80
85
78
84
1
10
14
2
8
14
Temperature
Temperature
RANDOM
AVER
AE-KD
AGRE KD (Ours)
Figure 5: Effect of the temperature parameter on the worst grost accuracy.
Mean Acc.
WGA
Mean Acc.
WCA
Accuracc (%)
Resnet Versian
Resnot Version
Resnet Versian
Resnot Version
(a) Waterbirds
(b) CelebA
RANDOM
AVER
AE-KD
AGRE-KD (Ours)
Figure 6: Results on student model capacity. We perform experiments on WaterBirds and CelebA using ResNet-50 teachers and varying the capacity of the student network (ResNet-18, ResNet34, and ResNet-50). We plot the average and the worst-case accuracy of different ensemble distillation methods across three random seeds. The WGA tends to increase as the student model has more capacity for lcarning the core features. Most baselines match or outperform the teachers when the student. model has the same capacity as the teachers (i.e., ResNet-50), and our method remains superior terms of WGA. These results suggest that the reduced capacity the student model source the disparity observed.
Mean Acc.
WGA
Mean Acc.
WGA
Accuracy (%)
Ensemble Size
Ensemble Size
Ensemble Size
Ensemble Size
(a) Waterbirds
(b) CelebA
RANDOM
AVER
AB KD
AGRE-KD (Ours)
Figure 7: Effect of ensemble size. We train the ResNet--18 18 student model with different ensemble sizes of ResNet-50 teachers (5, 10, 20, ..., 50). Inereasing ensemble size exerts a positive effect on both the average and the worst-group performance. However, the Random KD method tends to get worse as increase the number of teachers.
20

---

Published in Transactions on Machine Learning Research (11/2025)
Waterbirds
CelebA
Biased model
Average
WGA
Average
WGA
ResNet-50
90.640.49
86.7
91.840.20
90.510.24
Resnet34
90.0+1.63
85.2
91. 91.84u.14
90.5+0.24
ResNet-18
90.2
86.8
91.5
89.8
Table 5: Sensitivity of AGRE-KD to the biased model architecture. In the main paper, we used biased model with the same architecture the teacher models (i.e., ResNet-50). We experiment with different network backbones for the biased models in AGRE-KD. The results below do not show significant differences across biased model choices, demonstrating the robustness of the proposed method to the choice of biased model. These results suggest that the gradient direction of any biased pretrained model can provide sufficient guidance for debiased distillation.
Waterbirds
CelebA
Model
Average
WGA
Average
WGA
AGRE-KD w/ grad norm
90.6
86.8+1.86 1.86
91.744.20
90.9
AGRE-KD w w/o grad norm
91.0+1.32
81.544.07
93. +0.21
87.9
Table 6: Training AGRE-KD using gradient direction with (w/) and without (w/o) gradient normalization. On the Waterbirds and CelebA datasets, study how using the gradient magnitude in the weighting scheme impact the results. We trained our AGRE-KD method without normalizing the gradient vectors in the dots product, i.e, accounting for the gradient magnitude of the losses. The results below show that accounting for the gradient magnitude in the weighting scheme reduces the WGA performance. This shows the importance using normalized dot products to compare directional changes in gradients, making the computed weights independent of the gradient scales themsclves, which are generally very noisy.
Datasets
Sub Groups
#Samples
FRM
Teacher
Random
AVER
AEKD
AGRE-KD
(landbirds,1an land)
3498
99.3
95.5
96.740.37
96.8
96.1
94.3+1.46
Landbirds,water
184
74.0
94.3
86.2
87.4
86.4g3.88
87.543.14
Waterbirds
(1aterbirds,lanc)
56
54.1+1.80
92. 11.30
82.3
82, 82,
84.643..7
86.7+2.82
(waterbirds,water
1057
93.4+0.72
90.9+0.81
91.5+
90.9 2.67
01.4
90.6+1.86
(nemblond.female)
71629
96.0+0.39
91.1
91.8
91.8
91.3
90.4+0.77
(nonblond, male))
66874
99.5
92.9+0.20
94.340.37
94.2
93. #0.70
92.440.53
CelebA
(blond, female)
22880
85. 85.2+1.21
94.440.30
93. 93.8
93.9-1.18
94.620.34
95.020.83
(blond, male)
1387
32.3+2.7
90.1
88.340.78
87.5
89.4+1.03
91 91.6+0.78
Table Group-wise accuracy comparison. We report the group-wise accuracy of different ensemble KD methods the Waterbirds and CelebA datasets. As in previous experiments, average the performances across three different random seeds and considered ensembles of five teachers.
21

---

Published in Transactions on Machine Learning Research (11/2025)
Debiased Teacher
ERM Teacher
Models
Architecture
KD
Avg
WGA
Avg
WGA
ViT
X
93.940.48
85.0+
95.8
38.840.78
Teacher
ResNet-50
X
93.2
90.1 10.86
95.8
42.2
ConvNeXtV2
92.9+0.07 X
86.940.39
95.9=0.05
44.440.78
RANDOM
94.1
79.4+3.14
95.9
36.340.39
AVER
94.1+
80.0+3.10
95.9-0.09
36.1 36.1+0.00
Res.Ntt-1
AE-KD
93.9 10.19
82.2
95.910.10
40.2
AGRE-KD
90.4+0.69
87.640.94
95.9
41.2+11.7
Table Results with heterogeneous teacher ensemble on the CelebA dataset. We train teachers with different
architectures (ViT (base), ResNct-50, and ConvNeXtV2 (base)) and perform knowledge distillation to ResNet-18 student model.
22


Reviews:
[{"review_id": "9GIxKILDHq", "reviewer": "Reviewer_SyT8", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work highlights how standard ensemble distillation methods of teaching a single student model with larger teacher models may lead the student more biased against underrepresented groups with spurious correlations. To address this, the authors propose AGRE-KD that modifies the teacher ensemble's signal to the student by down-weighting the contribution of teachers that align more with a separate biased model. Experimental results across a variety of architectures and classification domains indicate that this method can improve performance upon other ensemble KD methods, especially for the worst-performing groups in the dataset domain.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Most claims are supported with accurate, convincing, and clear evidence. However, there are some gaps in communicating the full story to the audience.\n\n- It is misleading to not include \"One-hot\" in your table bolding/underlining scheme. Especially for audience members from outside the distillation community, it is important to see when ensemble KD methods can outperform standard ground truth training. In particular, in Table 1, One-hot's results are consistently beating or within the confidence bounds of the best ensemble KD results. \n- The datasets and architectures used came directly from related works, but some are becoming somewhat dated. Also, only repeating previously used datasets and architectures may lead to overfitting on these domains, so we don't know how your findings may generalize.", "audience_yn": "Yes", "audience_explanation": "In it's current state, this paper is mostly interesting to other members of the ensemble KD and debiasing communities. However, it requires further transparency (both in table highlighting and in qualitative discussion) on the performance compared to standard training without ensemble KD, as discussed both above and below. Why should a practitioner consider debiased ensemble KD if they may be able to get comparable performance much more cheaply just training the target model directly on their training data (possibly with DFR debiasing)?", "requested_changes": "Please include \"One-hot\" in your table bolding/underlining scheme, as detailed above. \n\nPlease add an experimental analysis of $\\alpha$, since it is introduced in the methods but then set to $1$ without further experimentation (except for \"One-hot\" technically using $\\alpha=0$. As your analysis of $\\tau$ shows uniform effects on different algorithms, a similar experimental exploration of $\\alpha$ may reveal important results for applying these methods in \"the real world\".\n\nThe different datasets seem to have different effects on the algorithms' results: in CelebA, debiasing the student always results in reduced overall performance and increased WGA performance, while for CivilComment this is much less consistent. Can you provide qualitative discussion of these? Further, are there other datasets (even not used by prior works) that can be studied to provide further understanding of these methods? \n\nTables 1-2: the column names \"Debiased\" and \"KD\" aren't very clear even with the given definitions. When is the student/target model debiased, and when are the teachers debiased?\n\nFigure 3 would be more interesting if you included a ratio of $\\frac05$ to see how each algorithm performs with non-debiased models, as well as add a horizontal line for the performance of One-hot in each subplot.\n\nFigure 6: \"Effect of teacher size\" implies the model size of each teacher, so \"Effect of ensemble size\" would be more clear.\n\nTypos:\n- Period missing in last sentence of Paragraph 2 of Section 2. Background.\n- Occasional misuses of `\\citet` that should be `\\citep`, such as \"...by mimicking the teacher model $f^t$ ~Hinton et al. (2015)~(Hinton et al., 2015).\"\n- Naming consistency: replace all of \"resnet50\", \"Resnet-50\", \"Resnet50\", etc. with one consistent name (\"ResNet-50\" is used in the original work) both in the text and in plot labels, and check other architecture/dataset/etc. names for similar issues.", "broader_impact_concerns": "No significant ethical concerns beyond those for standard deep learning research.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "oKWyq5kEjs", "reviewer": "Reviewer_tNEq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a study on the biases arising in ensemble knowledge distillation as well as a method to mitigate them, based on following the gradient direction in the opposite direction of a biased teacher. Experiments are conducted on simple vision and text datasets such as Waterbirds, CelebA and CivilComment, using simple architectures such as Resnets and BERTs, demonstrating the biases and the effectiveness of the approach.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Strenghts:\n\n- The experiments show in a vision and a language setup, that distilling from multiple debiased teachers can lead to a biased student. The experiments also demonstrate that the proposed teacher weighting mechanism is effective and produces a less biased student.\n\n- Additional ablations on model capacity and architecture and numbers of teachers are conducted and further validate the conclusions.\n\nWeaknesses:\n\n- Overall I found the description of the different experimental setups unclear. The information is often either scattered across different sections in the text, or incomplete:\n\n- The presentation of Table 1 and of the baselines could be improved. It would be more clear if one bullet point in the text corresponds to one row in the table, in the same order. Right now “one-hot” is barely mentioned in a very unclear way, “AVER” is before “random” in text but appears after in the text, and the lines separating the rows in the table are inconsistent.\n\n- It is not very clear for each experiment how the biased model is obtained. Section 4 mentions “AGRE-KD relies on a model pretrained with ERM that captured the dataset’s spurious correlation (biased model)” but more details can only be found later in the experimental section. I believe those details are very important and should already be described in Section 4.\n\n- Only later the paper mentions: “For the biased model used by our method to compute teachers’s weights, we randomly select one teacher model trained with ERM and without DFR” But does this mean that some teachers are trained with DFR and some others are not ? Can you provide more details on that ?\n\n- The experimental setup is a bit outdated/toyish. The vision models used are old ResNet-18 and ResNet-50. Table 3 presents results with ViT and ConvNeXt v2 but does not mention the model size. The text models are also old: BERT and DistillBERT are not much used nowadays in the era of LLMs. The datasets are also fairly simple and old. Have you thought about a more concrete setup where Ensemble-KD really unlocks a result that was not possible without ?\n\n- The paper mentions: “we consider ERM-trained (biased) and debiased teachers obtained by deep feature reweighting (DFR).” Do you measure how effective this is in practice ?\n\n- In general in the experiments, how is the WGA determined ?\n\n- If my understanding of the “one-hot” baseline is correct, KD, in particular Ensemble-KD should give better results than “one-hot”, otherwise what is the point of distillation ? The experiments show that this is not the case, do you have an intuition on why ? Maybe the setup is not that interesting ?\n\n- The abstract and introduction of the paper do not mention the experimental protocol or the quantitative gains on the protocol. Good ideas can only be validated by strong experiments and I believe the experiments should be part of the main findings and discussed early in the paper.\n\n- In Section 5.2, when describing the results in the 3 bullet points : it would be much clearer to refer to concrete numbers in the table when describing the findings, rather than only saying “From the results in Table 1” and letting the reader infer the conclusion from the number themselves.\n\n- Small typo in the introduction, there is a missing space: “model.Figure 1 illustrates the gradient”.", "audience_yn": "Yes", "audience_explanation": "Distilling knowledge from multiple teachers is a practical and valuable approach in deep learning. Although bias has rarely been studied in this context—except in deep ensembles—it turns out that bias can still exist even when all teachers are debiased. This finding alone is quite interesting. Moreover, the paper goes further by proposing a solution to address this issue. This work will be of interest to practitioners.", "requested_changes": "Improve the clarity when describing the method and experimental setups. See weaknesses.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "s8d1iApGZB", "reviewer": "Reviewer_2r2L", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the situation where knowledge distillation (i.e. a smaller-capacity neural net learning to reproduce a large capacity network) is applied to an ensemble of teachers on a domain where there are spurious correlations between features and the outputs (e.g. the background natural scene in a waterbirds vs. landbirds discrimination task). Knowledge distillation on ensembles can worsen error rates on rare subgroups (e.g. pictures of waterbirds on land, where they normally do not live). The authors present a method to prevent this poor performance on rare subgroups. They use a biased model (one which has learned the spurious correlation) to guide the student network away from the biased model, by guiding it towards the gradient of the teacher in the ensemble which is most different from that of the biased model. The authors also show that knowledge distillation is the key driver of the poor performance on rare subgroups, by showing that the problem does not occur when the student has the same capacity as the teacher. The authors also discuss a method for retraining the last layer only of teachers so as to mitigate bias. Positive results (as compared to competing methods) on 3 benchmarks are presented.\n\nA key strength of the paper is the clear win by their method over competing methods on the 3 benchmarks. The method is also novel, to the best of my knowledge.\n\nOne significant weakness is the failure of the paper to situate the research in the context of preexisting covariate shift research, such as the work of Sugiyama. Relatedly, the authors refer many times to spurious correlations without defining 'spurious' in a rigorous way.\n\nAnother concern is that while the results in some ways are strong, they are not as broad-based as I would like. 3 benchmarks seems like to small-to-moderate amount of evidence for a journal paper, where there is more time for authors to generate evidence than for a conference paper.", "claims_and_evidence_yn": "Yes", "claims_explanation": "I think I may be able to get to the point where I would answer 'yes' after a revision. However, for now, I would answer 'no', partly because I think 3 benchmarks is not quite enough. \n\nI also don't feel comfortable accepting this paper as-is without an explanation from the authors about how their research relates to the evidence on covariate shift. \n\nTo illustrate the importance of thinking in terms of covariate shift, let's look at the waterbirds vs. landbirds example. I concede that ideally, a perfect classifier of waterbirds vs. landbirds would only look at the traits of the birds themselves and not the backgrounds of the pictures. However, suppose perfect classification is not possible based on the apperance of the birds themselves...maybe the birds are partially occluded in some pictures, or maybe the pictures are low-resolution enough that you can't always distinguish based on the appearance of the birds themselves. Suppose also that we can be confident that we will only need to classify photos taken from nature rather than e.g. from a zoo where a waterbird might appear against a land background. In this scenario, the optimal classifier probably would use the background...the correlation, while technically spurious, would nonetheless improve classification performance, assuming that the input distribution did not change, i.e., that we continued to classify natural images. It would only be under covariate shift, where waterbirds appearing on land actually starts occurring at a significant non-zero rate, that we would want to avoid using the 'spurious' correlation.", "audience_yn": "Yes", "audience_explanation": "Yes, while the topic is fairly specific (knowledge distillation from an ensemble), some readers would be interested.", "requested_changes": "1) Positive results on at least one more benchmark.\n\n2) Discussion of the covariate shift literature and how it relates to this research.\n\n3) A little more rigor and detail on what it means for a student to have smaller 'capacity' than a teacher. I imagine it's usually assumed that the student has both fewer hidden layers and fewer hidden units per layer. However, the paper should try to define it more carefully. Does a student network with 5 hidden layers but 1000 hidden units per layer have a smaller or larger capacity than a teacher network with 10 hidden layers but only 50 hidden units per layer? Are machine learning theory concepts like VC dimension relevant here?", "broader_impact_concerns": "No concerns. This is a good paper from the point of view of trying to prevent harm to rare and overlooked subgroups.", "additional_comments": "Typos:\n\nPage 1: but not causally related -> but are not causally related\nPage 2: whose only the last layer -> whose last layer was the only one\nPage 3: using a given the training -> using a training set\nPage 3: of the a given sample -> of the given sample\nPage 6 improve the performances-> improve the performance\nPage 6 word group accuracy -> worst group accuracy \nPage 7 for the forvision tasks -> for the vision tasks\nPage 7 is spuriously correlated gender-> is spurious correlated with gender\nPage 9 can consistently improves -> improve", "extra_scores": {}, "extra_text": {}}, {"review_id": "TQB0agZUt5", "reviewer": "Reviewer_6dLt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a novel adaptive group robust ensemble knowledge distillation (AGRE-KD) method designed to address the shortcomings of traditional ensemble knowledge distillation in handling group bias. The core idea is to use a biased model as a reference and dynamically adjust the weights of the teacher models based on the cosine similarity between the gradients of the student model's loss with respect to each teacher and the biased model. This selective distillation helps the student model overcome group bias. The experimental results demonstrate the effectiveness of this method on multiple benchmark datasets. The research has strong practical significance and is highly innovative. The paper is well-written, logically structured, and the experiments are relatively comprehensive. I recommend this paper for publication after some revisions.\n\n## Major Strengths:\n\n- High Innovation: The core idea of the paper is very novel. Using a biased model as \"negative guidance\" and comparing gradient directions to filter and weight the knowledge from teacher models is an original approach to help students overcome group bias. This method provides a new perspective for the field of knowledge distillation, especially in improving robustness.\n- Clear Problem Definition: The paper clearly identifies the limitations of existing ensemble knowledge distillation methods in improving group robustness and empirically validates this issue, laying a solid foundation for the proposed solution.\n- Solid Methodology: The theoretical basis and implementation details of the AGRE-KD method are well-explained. In particular, the formula for the gradient-based weighting scheme, is concise and intuitive, perfectly embodying the method's core idea.\n- Comprehensive Experiments: The paper conducts thorough experiments on three different datasets (two visual tasks and one language task), using the key metric of worst-group accuracy (WGA) to evaluate model robustness, which makes the conclusions more convincing.\n- High-Quality Writing: The paper is well-written and logically structured. Figure 1 (first cited) intuitively illustrates the working principle of the adaptive weighting mechanism through vector directions, while Figure 2 (first cited) clearly outlines the overall flow of the AGRE-KD method. These figures greatly assist readers in understanding the approach.\n\n## Major Weaknesses:\n- Deeper Theoretical Analysis Needed: While the paper empirically proves the effectiveness of AGRE-KD, a more in-depth theoretical discussion is needed on why the similarity of gradient directions effectively captures model bias. For example, can it be theoretically proven that this weighting mechanism minimizes the risk on the worst-performing group?\n- Sensitivity to the Number of Teachers: The experiments in the paper use a fixed number of teacher models (e.g., 10 for the Waterbirds dataset). Is the performance of the AGRE-KD method sensitive to the number of teachers? How does the method perform with a smaller or larger number of teachers? Exploring this would help to better understand the method's applicability.\n- In the Related Work section, under Bias in Knowledge Distillation, there is a punctuation error. The sentence \"However, their method also requires class labels to derive the misclassified samples..\" has two periods. Please check and correct any similar errors.\n- Analysis with Labeled Data:The paper uses unsupervised methods to learn knowledge directly from the teacher model. It is recommended that the author further investigate how the model can utilize label information to mitigate or correct biases in the case of labeled supervision.\n- Insufficient Methodological Detail:The methodology section of the paper is brief, making it difficult to clearly understand the model's construction and training process. It is recommended to expand this section and provide more details.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Good paper.", "audience_yn": "Yes", "audience_explanation": "Interesting paper.", "requested_changes": "See the weaknesses part.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)