SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $\mathcal{L}_{task}$  represents the loss of specific user modeling task,  $\lambda_1, \lambda_2$  and  $\lambda_3$  are hyperparameters. We study the influence of these hyperparameters in the experiment section (RQ3).
Discriminator. To guide the filter  $\mathcal{F}$  training, the goal of the discriminators is to predict the corresponding sensitive attribute from the filtered user representations  $\hat{\theta}$ . Therefore, for the loss function of  $\mathcal{D}$ , we optimize the discriminator with data  $U_{L}$  using the following cross-entropy:
$$
\min  _ {\mathcal {D}} \mathcal {L} _ {\mathcal {D}} = - \frac {1}{l} \sum_ {s} \sum_ {1 \leq i \leq l: s _ {i} = s} \log \mathcal {D} _ {s} (\hat {\theta} _ {i}), \tag {10}
$$
where  $\hat{\theta}_i$  is the filtered representation of  $u_i$ . Here we omit  $H(S)$  in Eq. (11) since it is a constant..
Training Algorithm. Since the discriminator can not be optimized to the best directly, following the traditional adversary learning training methods [12], we adopt mini-batch training in implementation. Specifically, 1)  $T$  mini-batch updates minimizing  $\mathcal{L}_F$  with the  $\mathcal{D}$  fixed; 2)  $T$  mini-batch updates minimizing  $\mathcal{L}_D$  with the  $\mathcal{F}$  fixed. Here  $T = 10$  in our implementation. The whole optimization algorithm for FairLISA is shown in Appendix B.
4.5 Relation to Existing Adversarial Methods
In this paper, we provide a novel mutual information perspective to interpret the strategies for achieving fairness in existing adversarial methods, and unify the treatment of data with known and unknown sensitive attributes. As a result, FairLISA can easily generalize most existing fairness-aware adversarial methods with data with unknown sensitive attributes. For example, if we set  $\lambda_3$  in Eq. (9) to 0, which means ignoring the data in  $U_N$ , FairLISA degenerates to ComFair [3], a classical fairness-aware adversarial method. Moreover, our fairness mutual information objective is compatible with existing adversarial methods and can theoretically expand them to the limited sensitive attribute situation. For example, if we combine Eq. (8) from FairLISA with FairGo [45], a state-of-the-art fair model which considers graph structure, we could enhance its fairness performance. We will show the performance in experiments (denoted as FairGo+FairLISA).
In this section, we first introduce the dataset and experimental setup. Then, we conduct extensive experiments to answer the following questions:
- RQ1: Does FairLISA outperform the fairness-aware baselines in limited sensitive situations?
- RQ2: Can FairLISA achieve more robust fairness results on different missing ratios?
- RQ3: How do the data with known and unknown sensitive attributes impact the results?
- RQ4: How is the performance of FairLISA on several classical group fairness metrics?
The code is released at https://github.com/zhengz99/FairLISA.
5.1 User Modeling Tasks
To explore fairness in user modeling with limited sensitive information, we choose two representative issues in real-world user modeling scenarios. The first is user capability modeling in areas such as education which is called cognitive diagnosis [40]. The user  $u$ , item  $v$  in our general user modeling refer to student and exercise correspondingly, and interaction result  $r$  is the score that the student got on exercise. The goal is to model the capability of students. To achieve this, we train models on student performance prediction task. We choose several representative models: IRT [30], MIRT [4], NCD [41]. The second is user preference modeling in recommender system [39]. The user  $u$ , item  $v$  here refer to the customer and product respectively. The interaction result in  $r$  is the user evaluation of the product, such as rating behavior. By user rating prediction task, we can mine user preference. We choose several representative models: PMF [34], NCF [15], LightGCN [16]. Detailed basic user model descriptions will be shown in Appendix C.1.
5.2 Dataset Description
We use two real-world datasets to verify the effectiveness and robustness of FairLISA. Specifically, we use PISA2015² for cognitive diagnosis and MovieLens-1M³ for a recommender system. Detailed descriptions can be found in Appendix C.2. Both of these datasets are publicly available and do not contain any personally identifiable information. In our experiments, we split each dataset into training, validation, and testing sets, with a ratio of 7:1:2. All baseline models were trained and evaluated using the same datasets.
5.3 Experimental Setup and Baselines
Evaluation. The evaluation of our methods can be divided into accuracy evaluation and fairness evaluation. For accuracy evaluation, we select different accuracy metrics for different tasks. For cognitive diagnosis, following previous works from cognitive diagnosis [41, 8], we adopt different metrics from the perspectives of both regression (MAE, RMSE) and classification (AUC, ACC). For recommender system task, we adopt RMSE.
For fairness evaluation, our fairness goal refers to user modeling results do not leak any user's sensitive information. To measure the fairness performance, following the settings in fair user modeling such as [3, 23, 45], we train an attacker who has the same structure as the discriminator. Specifically, after training our framework, we use the filtered user representations as input and the corresponding sensitive attributes as labels to train the attacker. If the attacker can distinguish sensitive features from the user representations, it indicates that the sensitive information is exposed by user modeling results. To train and evaluate attackers, we split the data into train (80%) and test sets (20%) and calculate the AUC score and apply their macro-average to make the result insensitive to imbalanced data. The smaller values of AUC denote better fairness performance with less sensitive information leakage, which be used as the unfairness metric in the following experiments.
$^{2}$ https://www.oecd.org/pisa/data/2015database/
<sup>3</sup>https://grouplens.org/datasets/movielens/1m/
Table 1: The fairness and accuracy performance on recommender system task. AUC represents AUC scores of all attackers. The smaller values of AUC denote better fairness performance with less sensitive information leakage (the fairer). G, A, O represent gender, age, and occupation. RMSE represents accuracy performance. The best fairness and accuracy results methods are highlighted in bold. The runner-up accuracy results are represented by underline.
PMF	NCF	LightGCN
AUC-G	AUC-A	AUC-O	RMSE	AUC-G	AUC-A	AUC-O	RMSE	AUC-G	AUC-A	AUC-O	RMSE
Origin	0.6862	0.7235	0.6656	0.8670	0.6915	0.7153	0.6625	0.8635	0.7112	0.7053	0.6939	0.8301
ComFair	0.5389	0.5554	0.5398	0.9027	0.5379	0.5414	0.5374	0.8994	0.5498	0.5498	0.5411	0.8704
FairGo	0.5592	0.5592	0.5371	0.8933	0.5351	0.5412	0.5319	0.8941	0.5457	0.5435	0.5408	0.8658
FairGNN	0.5288	0.5370	0.5212	0.9025	0.5294	0.5305	0.5256	0.9009	0.5289	0.5315	0.5225	0.8648
FairLISA	0.5193	0.5240	0.5210	0.8935	0.5281	0.5220	0.5222	0.8853	0.5273	0.5291	0.5214	0.8564
FairGo+FairLISA	0.5132	0.5270	0.5112	0.8927	0.5273	0.5204	0.5214	0.8867	0.5241	0.5260	0.5208	0.8628
Implementation detail. For task-specific loss  $\mathcal{L}_{task}$  in Eq.(9), we use cross-entropy loss for cognitive diagnosis tasks and MSE loss for recommendation tasks. For all datasets and models, we set the learning rate to 0.001 and the dropout rate to 0.2. We select the best model based on its original performance on the valid set and use it to conduct the fairness test. We set adversarial coefficient  $\lambda_{1} = 1$ ,  $\lambda_{2} = 2$ ,  $\lambda_{3} = 1$  for cognitive diagnosis, and  $\lambda_{1} = 1$ ,  $\lambda_{2} = 20$ ,  $\lambda_{3} = 10$  for recommendation tasks. For the adversarial architecture, the filter module is two-layer perceptrons and we use ReLU as the activation function. The discriminators and attackers are three-layer perceptrons with the activation function of LeakyReLU. We set the dropout rate to 0.1 and the slope of the negative section for LeakyReLU to 0.2 for them. We implement all models with PyTorch and conduct all experiments on four 2.0GHz Intel Xeon E5-2620 CPUs and a Tesla K20m GPU.
Baseline approaches. To validate the effectiveness of FairLISA, we compare FairLISA with the following baselines. Origin (i.e., basic user models without fairness consideration), ComFair [3], FairGo [45], FairGNN [5]. Further, to validate the generalizability of FairLISA, we expand FairGo to FairGo+FairLISA (introduced in section 4.5). A detailed description will be shown in Appendix C.3.
5.4 Experimental Results
Overall results (RQ1). The fairness and accuracy results of the recommendation task are shown in Table 1. The results of cognitive diagnosis are shown in Appendix C.4. We have several observations.
- From the perspective of model fairness, the AUC of attackers on origin user models is significantly higher than 0.5, which means that the sensitive information is exposed by user modeling results even if such information is not explicitly used. Further, models which can simultaneously apply data with known and unknown sensitive information (i.e., FairGNN, FairLISA, FairGo+FairLISA) get better fairness performance. It shows that data with unknown sensitive attributes benefit fair user modeling. More importantly, we find our methods (i.e., FairLISA, FairGo+FairLISA) outperform others, suggesting the effectiveness of our work.
- From the perspective of model accuracy, all fairness-improving methods lose some accuracy compared with origin user models. The reason is that the fairness-aware methods are aiming at filtering out the information of certain sensitive features from user modeling results, which will to some extent reduce the information contained, thus decreasing accuracy [3, 33, 45]. Among these fairness-aware methods, we could find FairLISA, FairGo+FairLISA have the best accuracy performance, and they could even maintain comparable accuracy with origin models in some cases.
Performance on different ratios of missing sensitive attributes (RQ2). In this part, we study the impacts of sensitive attributes missing ratios and investigate the robustness of FairLISA in extremely limited sensitive situations. We choose the representative models which achieve the best performance result in RQ1 (i.e., NCD, LightGCN) and vary the ratio of missing sensitive attributes as  $\{20\%, 40\%, 60\%, 80\%, 95\% \}$  in the training set. From the results in Figure 2, we have the following findings.
- The sensitive attributes missing ratios have a huge impact on the fairness results. As the missing ratios increase, the fairness-aware methods are not effective enough. Especially, when the missing
Figure 2: Fairness performance on different missing sensitive attributes ratios ( the lower, the fairer).
ratio reaches  $95\%$ , several fairness-aware baselines converge to the Origin baseline. This observation indicates that in the absence of extremely sensitive attributes, other baseline methods have essentially lost their effectiveness in achieving fairness and perform similarly to the case where fairness was not considered at all. This highlights the point that fair user modeling in limited sensitive situations is worth studying.
- The models that can deal with unknown sensitive attributes, i.e., FairGNN, FairLISA, FairGo+FairLISA, perform better in most cases. It shows that making use of data with unknown sensitive attributes can alleviate the limited known sensitive information problem to some extent.
- Our methods, i.e., FairLISA and FairGo+FairLISA, consistently achieve the best performance. As the missing ratio increases, the improvements of our methods over FairGNN become more prominent. When the missing ratio reaches  $95\%$ , FairLISA can achieve an  $8\%$  improvement in fairness compared to FairGNN. This is due to FairGNN's reliance on the accuracy of the estimator. When numerous sensitive attributes are present, the accuracy of the sensitive estimator can be ensured. Nevertheless, FairGNN encounters notable constraints when access to sensitive information is limited. Our methods, based on information theory, can directly apply data with unknown sensitive attributes to improve fairness and are not greatly affected by the missing ratios, demonstrating the robustness of our methods.
Influence of  $U_{L}$ ,  $U_{N}$  (RQ3). FairLISA can directly leverage  $U_{L}$  and  $U_{N}$  to improve fairness. In this part, we investigate their influence by adjusting the hyperparameter  $\lambda_{2}(U_{L})$ ,  $\lambda_{3}(U_{N})$  as defined in Eq. (9) on two user models (i.e., NCD, LightGCN). Additional results are included in Appendix C.5. Specifically, we vary the values of  $\lambda_{2}$ ,  $\lambda_{3}$  on NCD as  $\{0, 0.5, 1, 1.5\}$  and the values of  $\lambda_{2}$ ,  $\lambda_{3}$  on LightGCN as  $\{0, 5, 10, 15\}$ . The results are shown in Figure 5 (the lower, the fairer). We can find that  $\lambda_{2}$  has a greater impact than  $\lambda_{3}$ , indicating that  $U_{L}$  has a
more significant role in promoting fairness. This is reasonable because sensitive labels in  $U_{L}$  can guide the filtering process. Moreover, as the values of hyperparameters increase, the results become fairer, suggesting that both  $U_{L}$  and  $U_{N}$  contribute to fairness.
Figure 3: Effects of  $\lambda_{2}$ ,  $\lambda_{3}$  on user models.
This is reasonable because sensitive labels in  $U_{L}$  can be uses of hyperparameters increase, the results become close to fairness.
Relation to classical group fairness metrics (RQ4). Previous experiments have demonstrated the effectiveness of FairLISA in terms of preventing the leakage of sensitive information from representations. In this section, we present the results of our proposed model on two classical group fairness metrics, namely Demographic Parity [7] and Equal Opportunity [13] (the lower, the better). The specific formulas for these metrics and additional results are included in Appendix C.6. In short, the results shown in Figure 7 indicate that our goal of removing the effect of sensitive attributes
Figure 4: Performance of different group fairness metrics (the lower, the better).
also benefits the classical group fairness metrics, and our models achieve the best performance.
In this paper, we focused on the fairness problem of user modeling and proposed a general framework FairLISA as a solution in limited sensitive information situations. We first provide a novel theoretical perspective to unify data with known and unknown sensitive attributes with the fair mutual information objective. Building on this unification, we propose an adversarial learning-based framework that enables us to effectively leverage the entire user dataset for fair user modeling. We further showed that FairLISA could be seen as a generalized framework from traditional adversarial methods and take better advantage of data with unknown sensitive attributes. Experimental results on different user modeling tasks showed the effectiveness of FairLISA.
7 Border Impact and Future Work
User modeling is a crucial task in various applications and has a significant impact on various aspects of our daily lives. In this paper, we discuss the social responsibilities of user modeling and consider a situation where limited sensitive attributes are available. Our model could better help to design fairness-aware user modeling tasks and also protect users' privacy in the user modeling process. However, we notice that despite considering a limited sensitive situation, there is still a need to collect part of users' private information as model input, which may increase the burden on users as well as incurs possible privacy leakage in the data storage and transfer process. In the future, we will try to combine the privacy and fairness concerns so as to explore fairness and privacy aware user modeling.
This research was partially supported by grants from the National Key Research and Development Program of China (Grant No. 2021YFF0901003) and the National Natural Science Foundation of China (Grants No. 62337001, U20A20229).
[1] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. A reductions approach to fair classification. In International Conference on Machine Learning, 2018.
[2] Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In International Conference on Machine Learning, 2018.
[3] Avishek Bose and William Hamilton. Compositional fairness constraints for graph embeddings. In International Conference on Machine Learning, 2019.
[4] R Philip Chalmers. mirt: A multidimensional item response theory package for the environment. Journal of statistical Software, 2012.
[5] Enyan Dai and Suhang Wang. Say no to the discrimination: Learning fair graph neural networks with limited sensitive attribute information. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, 2021.
[6] Yushun Dong, Jian Kang, Hanghang Tong, and Jundong Li. Individual fairness for graph neural networks: A ranking based approach. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021.
[7] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proc. of ITCS, 2012.
[8] Weibo Gao, Qi Liu, Zhenya Huang, Yu Yin, Haoyang Bi, Mu-Chun Wang, Jianhui Ma, Shijin Wang, and Yu Su. Rcd: Relation map driven cognitive diagnosis for intelligent education systems. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pages 501-510, 2021.
[44] Jinze Wu, Qi Liu, Zhenya Huang, Yuting Ning, Hao Wang, Enhong Chen, Jinfeng Yi, and Bowen Zhou. Hierarchical personalized federated learning for user modeling. In Proceedings of the Web Conference 2021, 2021.
[45] Le Wu, Lei Chen, Pengyang Shao, Richang Hong, Xiting Wang, and Meng Wang. Learning fair representations for recommendation: A graph-based perspective. In Proceedings of the Web Conference 2021, 2021.
[46] Peizhi Wu, Yi Tu, Zhenglu Yang, Adam Jatowt, and Masato Odagaki. Deep modeling of the evolution of user preferences and item attributes in dynamic social networks. In Proceedings of the The Web Conference 2018, 2018.
[47] Sirui Yao and Bert Huang. Beyond parity: Fairness objectives for collaborative filtering. Advances in neural information processing systems, 30, 2017.
[48] Renzhe Yu, Qiujie Li, Christian Fischer, Shayan Doroudi, and Di Xu. Towards accurate and fair prediction of college success: Evaluating different sources of student data. International Educational Data Mining Society, 2020.
[49] Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In International conference on machine learning, pages 325-333. PMLR, 2013.
[50] C. Zhao, H. Zhao, X. Li, M. He, J. Wang, and J. Fan. Cross-domain recommendation via progressive structural alignment. IEEE Transactions on Knowledge Data Engineering, (01): 1-16, oct 5555. ISSN 1558-2191. doi: 10.1109/TKDE.2023.3324912.
[51] Tianxiang Zhao, Enyan Dai, Kai Shu, and Suhang Wang. Towards fair classifiers without sensitive attributes: Exploring biases in related features. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, 2022.
[52] Yuqiang Zhou, Qi Liu, Jinze Wu, Fei Wang, Zhenya Huang, Wei Tong, Hui Xiong, Enhong Chen, and Jianhui Ma. Modeling context-aware features for cognitive diagnosis in student learning. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 2420-2428, 2021.
[53] Huaisheng Zhu and Suhang Wang. Learning fair models without sensitive attributes: A generative approach. arXiv preprint arXiv:2203.16413, 2022.
[54] Yan Zhuang, Qi Liu, Zhenya Huang, Zhi Li, Binbin Jin, Haoyang Bi, Enhong Chen, and Shijin Wang. A robust computerized adaptive testing approach in educational question retrieval. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 416-426, 2022.
[55] Yan Zhuang, Qi Liu, Yuting Ning, Weizhe Huang, Rui Lv, Zhenya Huang, Guanhao Zhao, Zheng Zhang, Qingyang Mao, Shijin Wang, et al. Efficiently measuring the cognitive ability of llms: An adaptive testing perspective. arXiv preprint arXiv:2306.10512, 2023.
[56] Ingrid Zukerman and David W Albrecht. Predictive statistical models for user modeling. User Modeling and User-Adapted Interaction, 2001.
In the appendix, we start from the proofs of Lemma 4.1 and Lemma 4.2 in Section A. After that, we introduce the whole optimization algorithm for FairLISA in Section B. Finally, the supplementary experimental introduction and results are reported in Section C.
FairLISA is an adversarial learning-based framework designed to promote fair model training by efficiently utilizing data with known and unknown sensitive attributes. It consists of two modules: a filter module and a discriminator module. Specifically, the filter module  $\mathcal{F}$  is trained to filter out the information of sensitive attributes (e.g., gender) of  $\theta$ . We denote the filtered user modeling results of a user  $u$  as:  $\hat{\theta} = \mathcal{F}(\theta)$ . The discriminator module  $\mathcal{D}$  tries to predict the corresponding sensitive attribute from the filtered user representations. The fundamental concept behind FairLISA is to establish a connection between two types of data and the fairness objective (i.e.,  $I(\hat{\theta}, S) = 0$ ) through Lemma 4.1 and Lemma 4.2. By doing so, it effectively utilizes data with both known and unknown sensitive attributes to facilitate fair model training. In the following sections, we will provide a detailed explanation of Lemma 4.1 and Lemma 4.2, respectively.
In this proof, we explain how FairLISA builds the relationship between fairness goal  $I(\hat{\theta}, S) = 0$  with user data with known sensitive attributes  $U_{L}$  through Lemma A.1. We define  $\mathcal{D}_s(\hat{\theta})$  as the probability that the discriminator  $\mathcal{D}$  recognizes the value of the sensitive attribute  $S$  as  $s$  according to  $\hat{\theta}, p(\hat{\theta})$  be the distribution of users' filtered representations.
Lemma A.1. Given the optimal discriminator  $\mathcal{D}_s^* (\hat{\theta})$  and let  $p(\hat{\theta} |s)$  be the distribution of users' filtered representations whose sensitive attribute is  $s$ , then  $I(\hat{\theta},S)$  can be derived as
$$
I (\hat {\theta}, S) = \sum_ {s} p (s) \mathbb {E} _ {\hat {\theta} \sim p (\hat {\theta} | s)} [ \log \mathcal {D} _ {s} ^ {*} (\hat {\theta}) ] + H (S). \tag {11}
$$
Proof. We use  $\mathcal{L}_D$  represents the loss of discriminator  $D$ , we first prove that when  $\mathcal{L}_D = -\sum_s p(s) \mathbb{E}_{\hat{\theta} \sim p(\hat{\theta} | s)} [\log \mathcal{D}_s(\hat{\theta})]$ , the discriminator can achieve the optimal.
Consider the constraints of the discriminator  $\sum_{s}\mathcal{D}_{s}(\hat{\theta}) = 1$ , we can use Lagrange multipliers:
$$
\mathcal {L} _ {D, \lambda} = - \sum_ {s} p (s) \mathbb {E} _ {\hat {\theta} \sim p (\hat {\theta} | s)} \log \left[ \mathcal {D} _ {s} (\hat {\theta}) \right] + \sum_ {\hat {\theta}} \lambda_ {\hat {\theta}} \left(\sum_ {s} \mathcal {D} _ {s} (\hat {\theta}) - 1\right) \tag {12}
$$
Differentiate  $\mathcal{L}_{D,\lambda}$  with respect to  $\mathcal{D}_s(\hat{\theta})$  and  $\lambda_{\hat{\theta}}$  and make the corresponding result 0, we get:
$$
- p (s) \frac {p (\hat {\theta} | s)}{\mathcal {D} _ {s} (\hat {\theta})} + \lambda_ {\hat {\theta}} = 0, \forall s, \hat {\theta}, \quad \sum_ {s} \mathcal {D} _ {s} (\hat {\theta}) - 1 = 0, \forall \hat {\theta}, \tag {13}
$$
therefore, we can calculate that  $\mathcal{D}_s^* (\hat{\theta}) = p(s|\hat{\theta})$ . Then we substitute it into the right side of the formula and get,