Paper: Towards Fair Knowledge Distillation using Student Feedback (ICLR-2024)

Reviewer comment (Experiments):
I am curious about the results when we apply BIRD to existing methods, and test their performance according to the conventional criteria? eg. The Top-1 and Top-5 accuracy on CIFAR-100 and ImageNet. Will it degrade when pursuing fairness? I want to see more comparison results.

Author response:
We are very excited to learn that the reviewer finds our work well motivated/presented, our method novel as compared to existing works, and our results remarkably enhancing fairness. We appreciate your helpful feedback and provide further details to answer your question below.

**Including more conventional metrics**

We thank the reviewer for this valuable suggestion and agree that the inclusion of the aforementioned criteria will account for a more thorough representation of the accuracy results in our work. We would like to note that the number of classes in our benchmark datasets is 2 (for CelebA) and 4 (for UTKFace), respectively. Following the reviewers’ feedback, we report the top-1 accuracies for the experiments in Table 1 and Table 2 in the Appendix of the updated manuscript (**see Table 12-13**). We are excited to share that our results follow the same trend and BIRD improves the fairness of the student model without sacrificing its predictive performance. Further, as explained below we cannot calculate metrics like top-5 accuracy in our experiments due to the limited number of classes in the fairness datasets.

We have included this clarification in the revised manuscript and hope we have addressed all your questions adequately. In light of these clarifications, we kindly request you consider increasing your score. We are happy to answer any further questions.

Reviewer comment (Novelty):
While the paper's overarching framework appears to draw heavily from the "learning how to teach" paradigm (as detailed in Park et al., 2021; Liu et al., 2021; Zhou et al., 2021), its overall contribution may be perceived as somewhat incremental. This perception arises from the adaptation of a pre-established framework to address knowledge distillation. Despite this, the proposed solution stands out due to its technical novelty and apt alignment with the problem statement. A deeper dive by the authors into the technical distinctions between their work and prior studies would further solidify their contribution.

Author response:
We thank the reviewer for acknowledging the proposed framework as compelling and our results comprehensive. We greatly appreciate your feedback on solidifying the technical contributions of our work with the related works and extending our ablation studies. 

**Learning how to teach paradigm and technical differences with related works**

We thank the reviewer for recognizing the *technical novelty* and *alignment* with the problem statement. While we agree with the reviewer that the referenced works utilize some variant of learning how to teach framework, they focus on fine-tuning/re-training the teacher model, **which is not the goal of our work** as we assume black-box access of the teacher and only use its output representations. Below we provide a deeper dive into the technical distinctions between the referenced works and Bird.

*Liu Et.al 2021: Learning to teach with student feedback*

The paper proposes IKD to establish a bidirectional interaction between the teacher and the student models. **Unlike traditional KD methods, where the teacher model is fixed** and provides static soft targets for the student model, IKD enables the teacher model to adapt its teaching strategy based on feedback from the student. Precisely, the **teacher model is trained to generate specific soft targets at each training step using meta-learning**, especially MAML (Finn, Abbeel, and Levine 2017). In contrast, our proposed framework, Bird, does not modify the weights of the original teacher model.

*Park Et. al 2021: Learning Student-Friendly Teacher Networks for Knowledge Distillation (SFTN)*

Their goal is to learn networks that would serve as better teachers to the students. To achieve this, they add **proxy student branches to the teacher model** during its training and then **optimize the teacher**, such that it minimizes the representation difference between the teacher and student branches. The trained teacher is consequently used for knowledge distillation to standard student models using Vanilla KD (Hinton et. al). Their overarching principle is to learn a model that is better teacher in general and not optimize Knowledge Transfer with respect to a specific student.  Again, they modify the teacher model and its learned weights, **which is non-trivial and computationally expensive for state-of-the-art foundation models.**

*Zhou et al 2022: BERT Learns to Teach: Knowledge Distillation with Meta-Learning*

They introduce MetaDistil, which combines knowledge distillation with meta-learning. Unlike traditional knowledge distillation frameworks where the teacher model is fixed, MetaDistil allows the teacher model to be trainable and adjust based on the student model’s performance using meta-learning (similar to Liu et al).

One key distinction between the above methods and Bird is that we do not fine-tune or modify the weights of the teacher model and only use the student feedback to re-weigh **penultimate** teacher representations during distillation. As rightly pointed out by the reviewer, our approach is aligned and motivated but addresses an unexplored area of fairness in knowledge distillation. While existing works utilize the learning how-to-teach paradigm, **they do so primarily for optimizing the students' predictive performance**, which often comes at the cost of retraining the entire teacher model.

Reviewer comment (Experiments):
For the benefit of practitioners, the authors might consider expanding their ablation section to detail not just the memory overhead, but also the time overhead associated with the distillation process. This is particularly relevant given the well-documented time-intensive nature of meta-learning-based approaches.

Author response:
**Time Ablation**

We thank the reviewer for this insightful question and agree that performing a time overhead study in addition to the memory overhead will be very helpful. In response to the reviewer’s suggestion, we perform this and share the time taken per iteration by BIRD (*note that this number is computed by averaging the time for all iterations in an epoch*). As rightly pointed out by the reviewer, *meta-learning is usually time-intensive.* However, we would like to highlight that even then BIRD achieves competitive time per iteration in addition to the fairness performance gains it offers. Our analysis indicates the time complexity can be attributed to the library used for meta-learning (*higher library*), which can be further optimized and remains a part of our future work (as discussed in the limitation).

|     **Teacher --> Student**     |   **MFD**   | **Base KD** |   **BIRD**  |
|:-------------------------------:|:-----------:|:-----------:|:-----------:|
| CLIP-Vit32 --> CLIP-Vit32       | 0.1192 s/it | 0.0986 s/it | 3.1507 s/it |
| CLIP-ResNet50 --> CLIP-ResNet50 | 0.0985 s/it | 0.0762 s/it | 3.1124 s/it |
| CLIP-Vit32 --> ResNet18         | 0.2585 s/it | 0.1346 s/it | 4.5481 s/it |
| CLIP-ResNet50 --> ResNet18      | 0.2321 s/it | 0.1193 s/it | 4.3356 s/it |

Reviewer comment (Theory):
This paper proposes a meta-learning framework to solve the fairness issue in KD. I don't see why meta-learning can be used to resolve fairness here; the intuition and motivation are unclear.

Author response:
**The intuition behind a Meta-Learning framework**

We would first like to thank the reviewer for appreciating the problem of fairness in knowledge distillation (KD). Further, we would like to highlight that none of the existing works in KD literature **adapt incoming biased teacher knowledge based on student feedback**, which can **lead to a fairer student model**. Here, the intuition is to identify and select teacher features that do not enhance the bias in the student model. We utilize a meta-learning framework as it provides **structure to the process of learning from feedback** and helps better optimization. This means, that when the incoming features from the teacher model are biased, it gets that feedback through the fairness properties of an inner-learner (a proxy/copy student model; Line 7: Algorithm 1), and it is subsequently updated (Line 8: Algorithm 1). The updated teacher model is then used to optimize the main student fairly. Thus, the teacher first uses the proxy student to understand the student’s optimization process, which it leverages for fair teaching.  

Further, we conducted an ablation on the importance of the meta-step (research question 4 in Sec. 5.2 of the original and revised draft) in our BIRD framework and showed that the meta-step allows us to update the parameters of FairDistill such that the biased features from the teacher do not impact the student. More specifically, results show that Bird with meta-gradients shows a **6.45% improvement in the fairness** over Bird without meta-gradients, providing empirical evidence that the meta-gradients improve the fairness of the student model.

Reviewer comment (Experiments):
The experiments are mainly conducted on CelebA and UTKFace, two small datasets. I think it is necessary to evaluate a larger benchmark, such as a dataset that is a mixup of gender/racial images with other non-biased images, i.e., animals, to verify the effectiveness of the proposed method on a larger scale setting. The same issue is on the choice of student models; more models should be evaluated, such as ViT (not as a teacher but as a student model).

Author response:
**Evaluation of More Student Models**

We thank the reviewer for this experiment suggestion. Indeed an evaluation with more student model architecture choices would provide a deeper understanding of our approach. Further, we would like to point out to the reviewer that **Table 4** in our attached supplementary section includes the results of BIRD with students of different sizes: ResNet34, ShufflenetV2, and FLAVA. In addition, we perform the additional experiments suggested by the reviewer choosing CLIP/ViT-32 and CLIP/RN-50 as students and RN18, and RN34 as teachers on the CelebA dataset. Please find the results for the aforementioned ablation in **Table 11** of the updated draft. We note that BIRD comfortably outperforms all baselines by a large margin, proving its efficacy across model sizes.

**Larger Datasets**

The two main datasets – UTKFace and CelebA – we utilize in this work are very popular and are widely employed in fairness research to date. For instance, several recent works in fairness published at **NeurIPS, AAAI, ICCV, CVPR, ECCV, and UAI conferences in 2022-23** have employed these datasets both to evaluate the efficacy of newly proposed methods, as well as to study the behavior of existing methods [1-10]. In addition, previous work in knowledge distillation, like MFD [1] and FWD [2], also rely on these datasets. Given these past works, we follow suit and employ these datasets in our benchmarking efforts. We agree with the reviewer that the necessity of evaluating models on a larger benchmark, such as a dataset with a mixup of images with different attributes, to verify the effectiveness of fairness methods is certainly important, and we aim to explore them in our follow-up work as they become available.

Reviewer comment (Experiments):
It is disappointing that this paper is only focused on the fairness of classification. The application of classification is very limited and has been extensively explored over the past decades. I think it is essential to test KD fairness on more settings, such as multi-turn vqa. Otherwise, I suggest changing the title to a more specific topic, i.e., "Towards Fair Knowledge Distillation on Image Classification".

Author response:
**Ambiguous Title of the Paper**

While we acknowledge the reviewers' feedback that KD has been extensively studied in classification, we would like to highlight that research on fairness for KD in a classification setting using **foundation models** is still in its nascent stages, where **there has been no distillation work** that analyzes the fairness issues of distillation using foundation models. We agree that experimenting with KD tasks considerably different from classification, such as Multi-Turn VQA or image retrieval, involving both text and image input datasets is crucial. However, we would like to kindly mention to the reviewer that in the interest of the discussion period, we look forward to exploring such an adaptation as an interesting future direction. We also apologize to the reviewer for the ambiguity arising from our current paper title and appreciate the suggestion for the new name.

We will include all the above clarifications and details in the final version. We hope we addressed all your questions/concerns/comments adequately. In light of these clarifications, we kindly request you to consider increasing your score. We are very happy to answer any further questions.

**References**

[1] Jung, Sangwon, et al. "Fair feature distillation for visual recognition" In CVPR, 2021

[2] Chai, Junyi, Taeuk Jang, and Xiaoqian Wang. "Fairness without demographics through knowledge distillation" In NeurIPS, 2022

[3] Dooley, S., Wei, G. Z., Goldstein, T., & Dickerson, J. Robustness Disparities in Face Detection. NeurIPS, 2022

[4] Xu, Y., Guo, W., & Wei, Z. Conformal risk control for ordinal classification. In UAI, 2023

[5] Lin, X., Kim, S., & Joo, J. Fairgrape: Fairness-aware gradient pruning method for face attribute classification. In ECCV, 2022

[6] Qraitem, M., Saenko, K., & Plummer, B. A. Bias Mimicking: A Simple Sampling Approach for Bias Mitigation. In CVPR, 2023

[7] Zhu, J., Gu, L., Wu, X., Li, Z., Harada, T., & Zhu, Y. People taking photos that face never share: privacy protection and fairness enhancement from camera to user. In AAAI, 2023

[8] Wu, T. H., Su, H. T., Chen, S. T., & Hsu, W. H. Fair Robust Active Learning by Joint Inconsistency. In ICCV, 2023

[9] Teo, C. T., Abdollahzadeh, M., & Cheung, N. M. Fair generative models via transfer learning. In AAAI, 2023

[10] Um, S., & Suh, C. A fair generative model using lecam divergence. In AAAI, 2023

Reviewer comment (Theory):
One major problem with the proposed method is that it needs the internal features of the teacher model for distillation (Eq. 4). However, as the paper mentioned in the motivation, most of the FMs in the real world only provide APIs. I.e., their features are barely accessible. This limitation seems to undermine the practical value of the proposed method severely.

Author response:
**Practical Value of BIRD (Need for internal features)**

We would like to clarify a potential misunderstanding here. Most cloud service provider (e.g., OpenAI, Google, and Amazon) pre-train a general-purpose feature extractor (called an encoder) and deploys it as a cloud service called an **Encoder As a Service (EAS)** [1,2,3,4], where a client/end-users queries the cloud service APIs for the final layer feature embeddings of its training/testing inputs when training/testing a downstream classifier. For safety and security-critical applications, a client aims to build an accurate and fair downstream classifier to safeguard algorithmic decisions and promote fairness. Further, Knowledge Distillation of Foundational models is usually performed at two ends: 1) developers who train student models and have access to everything about the model and 2) end-users who use the foundation model as a service (EAS).

Further, we would like to clarify that **we do not use the internal feature representations of the foundation models** in Eqn. 4. Following the premise of EAS, BIRD **only utilizes the final layer representation of the models** and trains a single one-dimensional feature vector to re-weigh the teacher representations using FairDistill operator and improve the fairness of the distilled student model. Hence, using our proposed framework, an end-user can leverage the foundational model representations (from APIs) and use BIRD to distill its knowledge in a student network they are building to develop an accurate and fairer model.

Reviewer comment (Evaluation):
Another concern is that the fairness is improved but in many cases at a price of degraded accuracy. E.g., CLIP- ResNet50 with UTKFace in Tab. 1, CLIP-ViT-32 −→ ResNet18 with UTKFace in Tab. 2, CLIP-R50 −→ ResNet18 with UTKFace in Tab. 2. Namely, the proposed method is not very strong. The fairness issue may be a concern, while accuracy also matters.

Author response:
**Importance of Fairness and its Tradeoff with Predictive Performance**

Studying fairness and its trade-off with predictive performance is imperative within the context of AI regulations and AI executive orders passed throughout the world. While accuracy is crucial, addressing fairness concerns is equally vital to prevent discriminatory outcomes. AI regulations like **the General Data Protection Regulation (GDPR) in the EU, the Personal Information Protection and Electronic Documents Act (PIPEDA) in Canada, the AI Executive Order in the USA, and the Data Protection Act (DPA) in the UK**, increasingly emphasize the importance of fairness, demanding that AI researchers and developers strike a balance between accurate predictions and equitable treatment of individuals. Executive orders underscore the need for transparency and accountability in AI systems, urging a thorough examination of how fairness considerations may impact predictive performance. Focusing solely on accuracy without regard for fairness can lead to biased outcomes, legal repercussions, and erosion of public trust in AI technologies, emphasizing the urgency of a nuanced understanding and integration of fairness principles into AI development practices.

Reviewer comment (Evaluation):
A side concern is that, as seen above, the method does not perform well on the UTKFace dataset. Why?

Author response:
**Performance on UTKFace Dataset**

We thank the reviewer for bringing up this insightful observation. We would like to report that BIRD, on average, **improves** fairness by **58.78%** (DEO-max) and **7.31%** (DEO-mean), while the F1 score **decreases** by **1.27%** and AUROC decreases by **6.34%** on the UTKFace dataset (see Table 1-2) compared to the baseline. Our results show that BIRD improves model fairness without sacrificing its predictive performance. 

Further, UTKFace is a small dataset that contains only 20,000 facial images with four ethnicities as opposed to the 200,000 images in the CelebA dataset. We hypothesize that one possible reason for the relatively poor performance of BIRD  on the UTKFace dataset could be the limited training dataset size. To this end, in response to the reviewers’ feedback, we perform an additional experiment and run BIRD on the UTKFace dataset with image augmentations (randomly flip the image horizontally, apply color jitter (using a $\sigma=0.2$), randomly rotate the image by 15 degrees and apply the random affine transformation). On average, we observe an increase of **+0.36% in AUROC** and **+1.17% in F1-score** for approximately the same fairness performance. Another notable observation is the massive performance boost when UTK augmentation is used with Clip-RN50 teacher – **+0.51% in AUROC** and **+1.53% in F1-score**, **-6.40% in DEO-mean**, and **-11.86% in DEO-max**. We added the exact ablation numbers in the appendix of the updated manuscript (see Table 10).

Reviewer comment (Novelty):
This paper seems quite relevant: https://aclanthology.org/2022.gebnlp-1.27.pdf. It reports a similar observation to Sec. 5.2 that KD amplifies biases.

Author response:
**Relevant related work**

Thank you for sharing this work. We have cited and discussed the work in the revised related work section in our updated draft.

We are very grateful to the reviewer for all their suggestions, as they have helped us improve our paper significantly. We tried to incorporate all the reviewer suggestions in our write-up and include all clarifications in the revised version. We would kindly request the reviewer to consider increasing their score.