COFFEE: Counterfactual Fairness for Personalized Text Generation in Explainable Recommendation
Authors: Nan Wang
Qifan Wang
Yi-Chia Wang
Maziar Sanjabi
Jingzhou Liu
Hamed Firooz
Hongning Wang
Shaoliang Nie
Venue: EMNLP 2023 Main (2023)
Keywords: personalized text generation
fairness
bias
explanation for recommendation
human evaluation
counterfactual fairness
TL;DR: We investigate the fairness issues in personalized text generation and propose a general framework called COFFEE for improving fairness.

Abstract: As language models become increasingly integrated into our digital lives, Personalized Text Generation (PTG) has emerged as a pivotal component with a wide range of applications. However, the bias inherent in user written text, often used for PTG model training, can inadvertently associate different levels of linguistic quality with users' protected attributes. The model can inherit the bias and perpetuate inequality in generating text w.r.t. users' protected attributes, leading to unfair treatment when serving users. In this work, we investigate fairness of PTG in the context of personalized explanation generation for recommendations. We first discuss the biases in generated explanations and their fairness implications. To promote fairness, we introduce a general framework to achieve measure-specific counterfactual fairness in explanation generation. Extensive experiments and human evaluations demonstrate the effectiveness of our method.

COFFEE: Counterfactual Fairness for Personalized Text Generation in Explainable Recommendation
Nan Wa Qifan Wang', Yi-Chia Wang", Maziar Sanjabi', Jingzhou Liu²
Hamed Firooz', Hongning Wang'. Shaoliang Nie²
Netflix Inc., Los Gatos, California, USA
Meta AI, Menlo Park, CA, USA University of Virginia, VA, USA
Abstract
Ground-ruth explanations
Generated explarations
As language models become increasingly inte- grated into our digital lives, Personalized Text Generation (PTG) has emerged a pivotal component with wide range of applications. However, the bias inherent in user written text, often used for PTG model training, can inad- vertently associate different levels of linguistic quality with users' protected attributes. The model can inherit the bias and perpetuate in- equality in generating text w.r.t. users' pro- tected attributes, leading unfair treatment when serving users. In this work, we investi- gate fairness of PTG in the context person- alized explanation generation for recommenda- tions. We first discuss the biases in generated explanations and their fairness implications. To promote fairness, we introduce general framework to achieve measure-specifie coun- terfactual fairness in explamation generation. Extensive experiments and human evaluations demonstrate the effectiveness of our method.
RootDov
9330
3:30
mole
femele
male
female
Figure 1: Left: average FeatCov of ground-truth ex- planations in the train set. Right: average FeatCov of explanations generated the test set by PETER.
recommendations (Wang al., 2018; Chen et al., 2021; Yang al., 2021; et al., 2021a la; Yang al., 2022). Personalized explanation generation aims to provide a user with descriptive paragraphs on recommended items that align with his/her pref- erences, enabling more informative and accurate decision-making The generators typically lan- guage models trained on user written reviews from e-commerce platforms (Zhang et al., 2018; Ni al., 2019; Yang et al., 2021), where sentences related to item descriptions are retained to construct the ground-truth explanations. However, due to histori- cal, social, or behavioral reasons, inherent bias may exist within the review text, associating specific linguistic characteristies with the users' protected attributes such gender race (Newman et al., 2008; Alim et al., 2016; Volz et al., 2020). While certain linguistic features that capture the diversity of language use (Newman et al., 2008; Groenwold et al., 2020) are suitable for personalization, others pertaining to the linguistic quality of explanations, such as informativeness or detailedness, (Louis, 2013), should be excluded. Failure to do so can result in unfair treatment when serving users.
1 Introduction
Personalized text generation (PTG) has extensive applications, such as explainable recommendation (Zhang and Chen, 2020; Chen et al., 2021), post generation (Yuan and Huang, 2019; He et al., 2021) I), and conversational systems (Zhang et al., 2018 88 2019; Lee et al., 2021). The auto-generated text, functioning at the frontier of human machine inter- action, influences users' decisions and transforms their way of thinking and behaving. However, due its immense power and wide reach, PTG can inadvertently give rise to fairness issues and lead to unintended consequences (Alim et al., 2016; Bor- dia and Bowman, 2019; Blodgett et al., 2020).
As an example, we investigate the explanation generation on Amazon Movies' with the personal- ized transformer model PETER (Li et al., 202 1a). We adopt feature coverage (FeatCov, the number of unique features mentioned about movie) an automatically measurable metric of explanation
In this work, we investigate the fairness issues in PTG, focusing on one of the mostly studied set- tings: generating natural language explanations for
The research was conducted when the author was an in- tern at Meta and a PhD candidate at the University of Virginia. Correspondence to nanw@net flix...com com
TTee details about experiment setup is in Section 6.

---

30
Infermativeness
explanations should differentiate user in the real world vs. in the counterfactual world where only the user's protected attribute (e-g., gender) is changed. This problem is essential and unique com- pared to fairesss problems in the literature (Sec- tion 2), and imposes specific technical challenges (Section 4). To achieve the goal, we develop a general framework, COFFEE, for COunterFactuur FairnEss in Explanation generation. COFFEE treats a user's protected attribute value as a sep- arate token input to the model, and disentangles its representation from the user's representation (Ma et al., 2019; Locatello et al., 2019b; Zheng et al.. 2021). By controlling the input attribute val- ues for counterfactual inference (CI), we impose measure-specifie CF constraint (Russell et al., 2017) generated explanations. Then novel fair policy learning scheme is developed opti- mize CF with carefully designed rewards, which generalizes to any single or combination of qual- ity measures. We use user-side fairness by default in discussions, but COFFEE is general to ensure fairness on either user or item side in the two-sided market (Wang and Joachims, 2021), and be adapted to different models. Extensive experiments and rigorous
Detailedness
Helpfuiness
FeatCov
not
someehal
modesate Human evaluated quality
uxtreneyy
Figure 2: FeatCov vs. human evaluated quality mea- sures. Each measure is rated in a five-category scale. p-valus < 0.05 in Kruskal-Walln H test for all three criteria (see Appendix D for detailed test results).
quality. As shown in Figure 1 (left), reviews from male users generally have higher FeatCov the target movies than those from female users (Fan- Osuala, 2023) A model trained on such data can inherit the bias and generate explanations discrim- inately when serving users users-higher FeatCov for males than females, as indicated in Figure (right). To further substantiate the bias issue, we conducted user study and found strong positive correlation, shown in Figure 2, between FeatCov and human evaluated quality criteria including informativeness, detailedness and helpfulness. The observations re- main consistent when results from male and female human evaluators are analyzed separately, affirm- ing that both genders consider FeatCov signifi- cant indicator of quality. This study demonstrates concerning issue for PTG.
studies demonstrate COFFEE's periority in achieving fairness and maintain high generation performance compared to baselines.
the bias in FeatCov Details of the user study are in Section 7.
Background
In particular, the bias observed in training data originates from various factors such as different demographic behavior patterns that are beyond our control. But the system should not inherit the bias and act discriminately by generating explanations of different quality for different users lack of informative reviews from users of demographic group should not prevent them from receiving infor- mative explanations. Without proper intervention, the system can exhibit such bias, thus adversarially affecting the users' experience, reliance and trust in the system (Tintarev and Masthoff, 2015). From a broader perspective, the bias reinforce itself by influencing how users write online, and jeopar- dize fairness the long run (Schwartz al., 2016; Alim et al., 2016; Bordia and Bowman, 2019).
Uniqueness and Importance: Fairness in machine learning (ML) is originally studied in automatic decision-making systems that directly impose "sig- nificant" or "legal" effeets on individuals (Voigt and Bussche, 2017). Such fairness considerations often revolve around resource allocation by ML models, exemplified in contexts like loan assessments (Lee and Floridi, 2021) job applications (Singh and Joachims, 2018), where model predictions can un- favorably affect a protected group (Du et al., 2021; Mehrabi et al., 2021).
contrast to resource allocation fairness, NLP researchers primarily examine the representational fairness (Blodgett et al., 2020; Liang et al., 2021; Sheng et al., 2021) regarding how language models shape social biases and stereotypes through natu- ral language understanding (NLU) generation (NLG). In the realm of NLG, fairness particularly concerns how generated text may contain biased information about a specific demographic group. For instance, Huang et al. (2020) analyze the sen- tence completion by GPT-2 model, and find dif-
To mitigate the issue, we take a causal perspec- tive to dissect the bias and enforce counterfactual fairness (CF) (Kusner et al., 2017) in personalized explanation generation: the quality of generated
'W use binary gender for case study, but our work gener- alizes to any binary or non-binary attributes.

---

ferent sentiment distributions of completed sen- tences when the occupation word is counterfactu- ally changed in the prompts. Bordia and Bowman (2019) revealed a more frequent co-occurrence of the 'doctor' with male pronouns and 'nurse' with female pronouns in generated text. However, these biases, directly encapsulated within the text, can be more easily analyzed. To the best of our knowl- edge, our work is pioneering in exploring how per- sonalization, bridging NLP and recommendation, associates the bias in NLG with protected attributes users.
where A is the of possible attribute values, e.g.. A = {male, female, other) for gender. Each dataset entry is a tuple of (u, a,,, correspond- ing to user ID, item ID, observed attribute value, and ground-truth explanation. The explanation gen- crator Gg is a language model parameterized by 0. Given user u, an item i, and observed attribute value a, an explanation can be sampled from the generator as Y Gg(u, i a)- The linguis- tic quality of any explanation y is measured by a function (yy) Notably, we treat Q as black box oracle-a a quality measure that can only be queried. This is essential in practice and offers the flexibility to arbitrarily tailor Q based on the fairness require- ments of the application. An explanation can be gauged in various ways by customizing Q, such as using an explicit function, human evaluation, tool provided by authorities. We assume, without loss of generality, that higher Q values represent superior quality. CF on any measure Q of explana- tions is achieved when, given a user u and an item i,
Fairness Notions: Various fairnsss notions exist in the literature, with group-wise fairness notions be- ing the firstly studied ones (Zafar al., 2015; Hardt al., 2016; Zafar et al., 2017). Yet, group-wise fairness has different quantitative definitions that are generally incompatible (Kleinberg et al., 2017; Berk et al., 2021). Some definitions can even exac- erbate discrimination (Kusner et al., 2017). Individ- ual faimess (Zemel et al., 2013; Joseph et al., 2016) requires similar users to receive similar predictions. But it relies on carefully chosen domain-specifis similarity metries (Dwork et al., 2012). In contrast, counterfactual fairness (CF) (Kusner et al., 2017), 77,, considering fairness from a causal perspective, has gained prominence recently as a more robust fair- ness notion (Russell et al., 2017; Wu et al., 2019; Makhlouf et al., 2020), which can also enhance group-wise fairness in certain scenarios (Zhang and Bareinboim, 2018; Khademi et al., 2019).
P(Q(Ya-o)lu, = =(Q(Yaa=>))uu...) (1)
where YAea ~ Go(u, i|A a') is the expla- nation generated when we counterfactually as- sign the value of the user's protected attribute by A f- d', a' * The right side of Eq. (1) evaluates the quality distribution of explanations generated had the user's protected attribute value been a', given that the observed attribute value is a (Kusner et al., 2017: Li et al., 2021b).
Though CF has been studied in some non- personalized NLP tasks (Huang et al., 2020; Garg et al., 2019), most existing works study the depen- dency of model outputs on attribute-specific words within the input text (Blodgett et al., 2020; Liang et al., 2021: Sheng et al., 2021). In such cases, CI can be easily performed the input text it- self, such as changing male pronouns to female pronouns (Huang et al., 2020; Garg et al., 2019). However, CF PTG necessitates CI on the pro- tected attributes of users being served-an an area yet to be thoroughly explored.
Denote the loss of the generator for a given user u and item i by EGen(Ga(u, i = a), c), which is typically the negative log- (NLL) loss or a combination of several losses (Li et al., 2017; Yang et al., 2021; Li et al., 2021a). We consider training the generator for fair explanation genera- tion as a constrained optimization problem:
minCgen (aaggeenGGGooo (G(u i|. = a)
s.l. Ey [Q i,
(2)
Ey [Q(Y lu,
For ease of presentation, we consider a single user- item pair, and the total loss on a dataset is simply summed over all user-item pairs with the constraint upplied to every pair. In this work, apply the first-order moment of the quality of generated ex- planations to construct the constraint, and leave the extension to other moment-matching constraints for future work. We further simplify the expression of the constraint as [E(((Ae-a)) E[Q(YA (Y)()))))))))
3 Problem Formulation
In the following discussions, consider a sin- gle protected attribute on the user side for sim- plicity, but our proposed framework is versatile to accommodate multiple attributes on either the user or the item side. The value of user's pro- tected attribute is denoted by a variable A E A.

---

Policy Learning
YA=a
Explanation
Fairness Constraint between
CLA - d'
Generator G3 (Transformers, RNN, etc.)
((ar-a) and Q A-aa
Ym-a
Figure 3: Given user with attribute value and a recommended item, COFFEE performs CI by switching the attribute value to a' to get the counterfactual user, which achieved by disentangled attribute embeddings. Explanations Y for both the real user and the counterfactual COFFEE then updates the generator's parameters by policy learning from the fairness constraint.
sampled from the generator, evaluated by Q.
4 COFFEE
4.1 Disentangled Attribute Representation
For given tuple (u, a), we denote the represen- tation for the attribute value ra- the user's pref- erence representation (independent from the pro- tected attribute) as ru and item representation as ri. The complete user representation r = [a, ru]. Correspondingly, when performing < a' on user u, we change the input attribute token from to a', and the new user representation becomes Iu Ae-' = Irg.Tal Note that each attribute value has its representation, and is shared across all users having that same attribute value. For instance, all male users' attribute representation is the same vector mmale- We can do the same for item-side attributes as r. (ra, r:].²
User and item IDs are conventionally input to the generator for explanation generation, represented as learned embedding vectors (Li et al., 2017; Wang et al., 2018; Li et al., 2021a). However, the user's protected attribute, entangled with their preference, is implicitly encoded in the representations (Lo- catello et al., 2019a), hindering its direct manipu- lation for CI. One way explored for CF in person- alization involves removing all information about the protected attribute representations via dis- criminators (Zemel al., 2013; Li et al., 202 bb; Wang et al., 2022). Although this improves fairness, it detrimentally affects personalization and elimi- nates desired characteristics linked to the protected attribute. In contrast, we aim to enforce the inde- pendence of the protected attribute from any speci- fied quality measure Q on generated explanations, while preserving explanation content-dependence sustain high personalization performance.
Simply separating the user's protected attribute and preference representations does not guarantee that r. will not contain any information about the protected attribute, inhibiting the accuracy of CL.
To further enforce the disentanglement, we intro- duce a discriminator D(rw), and add an adversarial loss on ra in Eq. (2) as
To enable CI, COFFEE considers a user's pro- tected attribute value as a separate token input to the model, along with user and item IDs, and learns disentangled attribute representations from the user representation to encapsulate the effect of each altribute value in explanation generation (Locatello et al., b;; Ma et al., 2019; Zheng et al., 2021). This mirrors methods controllable text generation (Oraby et al., 2018; Shu et al., 2020; Dathathri et al., 2020), where disentangled attribute tokens are utilized as inputs to modulate the topic, sentiment, or style of generated text. CI is then achieved by altering the attribute token imput to the model. Subsequently, we enforce fairness con- straint based on the explanations generated pre and post CI, and establish a policy learning method for optimizing this constraint. An illustration of the COFFEE framework shown in Figure 3.
min Ceen ilA a). + Ao log((((())))) s.t. B[Q(YAea) = (8[((Xa-))), A, #a,
(3)
where D(ru: a) is the probability of predicting the correct attribute value a. In this way, we adver- sarially remove the protected attribute information from ru. and enforce to capture all the attribute information. During mini-batch optimization, we alternate between the parameter updates of the model and the discriminator follows: (1) X batches of updates minimizing the loss Eq. (3) with D fixed, and (2) Z batches of updates maximizing the the loss Eq. (3) with the generator Ge fixed.
We can introduce K 2 attribute tokens, each mapped to its disentangled representations. Sum instead of concatenation of embeddings can he used when K is large.

---

4.2 Policy Learning for Fairness
their utility in practice. To address this, we fur- ther design weighting mechanism to calibrate the rewards such that the optimization focuses more on improving (or decreasing) the quality measure for an attribute value for achieving fairness. This empowers the designer to better control the opti- mization and the utility-fairness trade-off. Specif- ically, we introduce a quality promotion weight n [0, 1] to re-weigh the rewards of explanations in the world with lower expected quality, and use ny to reweigh the rewards of explanations in the other world. The resulting calibrated rewards are:
Once the user, item, and attribute representations are learned and fixed, we can optimize the gen- crator w.n.t. the CF constraint. Due to the non- convexity of the constrained fairness optimization problem, closed-form solutions are unattainable. We add the constraint with Lagrangian relaxation regularization in the loss (Russell et al., 2017)
min Cau X|E -B
(4)
with fall = Lyen ( (Go(u, i| = e)
AD log((rr,))) a)) in Eq. (3) denotes all other losses except for the CF constraint, and À is hyper- parameter for fairness-utility trade-off.
Tw (Vā-a))
N sign(A) w())
However, standard gradient methods cannot be directly applied to optimize the constraint: expla- nations are discretely sampled from the generator, and Q is an oracle. Instead, we consider policy learning for fairness optimization, where the ex- planation distribution imposed by the generator is considered as the policy for explanation generation, and sampled explanations for estimating the expectations in the regulariza- tion, we sample N explanations and calculate the average quality of explanations sampled in both the real-world and the counterfactual world. Denote the regularization term as & fair, the expectations in the regularization can be estimated as:
1 To (vAAta)) sign( (isnn))((((-aaa (1 w(A))
where w sign(A)+ (1 (1-1) 17) + (1- sign( ( 2
2 77-
We can leverage this weight to guide the faimess optimization: if 17 0.5, the algorithm will focus more on improving the quality of low-quality expla- nations for fairness. Finally, for stability and faster convergence, we apply the advantage trick (Mnih et al., 6), and each reward is its difference from the average reward in its corresponding world:
actions. Concretely,
rodu (V'A-a) rw (VA-a) Fw (VÂ-a) r'dde (vā-w:) ru (Vim-r) Fo (Vāro'))
(6)
The policy gradient (Sutton et al., 1999; Yang al., 2021) on 0 can then be estimated as:
faarr= [[Q(Y~-a) E(Q(Ya-o))]) N
ss)gg((A)
Σ ((v N
8
(5) Σ E N
1
VoLrai
Vo log Go (yhra)) (yāmo))
(7)
A = For an explanation Vira sampled in the real- world, its contribution to the unfairness in regu- larization term is thus N (igg(A)Q())
EN' (v
Vo log (adu
Intuitively, during optimization, the probability of sampled explanations that lead to the unfairness will be demoted, while the probability of those that contribute to fairness will be promoted.
To im- prove fairness by minimizing the regularization, the reward for v V is considered as r ((vā-)) = sign A)Q A)Q(2) (v) Similarly, the reward for a' in the counterfactual world is r (1hic ) = x (ign(A)Q((u))) (1))(((aar)))))))))))) v)). In this way, we convert the CF optimization into maximiz- ing the rewards of generated explanations.
sampled explanation ghacaa
To train COFFEE model, we first pre-train the model without the fairness constraint. Then we fix the latent representations of users, items and attributes, and add the faimess constraint in the objective function for fine-tuning the model.
Although the rewards are designed to minimize the difference in the qualities of explanations from the real vs. counterfactual world, we lack direct control over how the difference should be mini- mized during optimization. The optimization may arbitrarily improve or decrease the quality of ex- planations to achieve fairness, e.g., always gener- ating low quality explanations, which greatly hurt
5 Applying COFFEE to Existing Models
Existing models for personalized explanation gen- eration mostly adopt either Transformer (Li et al., 2021a) or RNN (Li et al., 2017: Chen et al., 2021; Yang et al., 2021). In this section, we demon- strate the application of COFFEE to PETER (Li

---

Table 1: Statistics of the datasets and the protected attributes used in experiments. The candidate values of the attributes are indicated in parentheses.
where a restaurant's price range (8. $8, $5$) is used as the protected attribute, as the system may not discriminate a restaurant simply because the restau- rant owner sets lower prices for equally good food and service than its counterparts. The statistics of the datasets are summarized in Table 1. The details of data processing are described in Appendix B,
#Uscrs
#ltems
#Reviews
Autrbbtt Values
Video Games 9,511
5,173
59,374
male, female male, female
Movies TV
28,001
12,888
265,646
Yelp 35,714
24,900
1,499,291
S, SS, S8$
et al., 2021a), which is an explanation generation model based on personalized Transformers. In Ap- pendix A, also illustrate the application of COF- FEE NRT (Li et al., 2017), which is based on RNN. However, it is worth noting that COFFEE is a general framework that can be applied to most of existing explanation generation models, as long as the model is based on user/item representation learning and can be fine-tuned.
On both Amazon Games and Movies, the raw model tends to generate higher FeatCov explana- tions for male users than female users, aligned with the existing findings that male users usually write more detailed reviews (Fan-Osuala, 2023). On Yelp, the model generates more thorough fea- ture descriptions for pricier restaurants than their less expensive counterparts, as shown in Table 4 in Appendix B. This could be a reflection of users' heightened attention to experiences in more expen- sive restaurants due to higher expenditure. These observations also align with the biases present in the respective training datasets.
In PETER (Li et al., 2021a), user and item IDs, represented as special tokens, are appended to the start of each explanation sequence before being in- putted transformer layers. A context prediction loss is introduced for enhanced personalization in addition to the NLL loss, which predicts the words in the explanations regardless of their order. We denote the overall loss as Epeter. which is the sum of the NLL loss and context prediction loss (Li et al., 2021a). To apply COFFEE to PETER, we introduce an attribute token at the start of each se- quence for disentangled attribute embedding. The discriminator applied at the input embedding layer to enhance disentanglement. User, item, and attribute tokens can attend to each other, while the explanation words can only attend to past tokens. When applying COFFEE to PETER, the loss is constructed by replacing Lgen in Eq. (3) by Cpeter.
Baselines. As we are the first to study the quality fairness in PTG, there is no existing work that di- rectly addresses this problem. We adapt popular methods in the fairness literature for the purpose. Below we briefly introduce them and describe their detailed implementations in Appendix B.
RAW: Original PETER model without modifica- tion, which is also the base for other baselines. ADV (in-processing): Adversarially removing the sensitive information in user or item represen- tations by adding diseriminator (Li et al., 2021b).
NORM (pre-processing): Normalizing the train- ing data to remove the bias on group-level.
BT (pre-processing): Back-translation has been shown to help normalize text and reduce bias (Ra- binovich et al., 2017: Christiansen et al., 2021). We pre- pre-process the training data by translating the ex- planations Chinese and then back English. ATTR: To evaluate the effectiveness of the disen- tanglement and optimizing the fairness constraint in COFFEE, we use the model trained without the constraint (equivalent to X = in COFFEE) but with disentangled attribute representations.
6 Experiments
We conduct experiments on three public review datasets: Amazon Video Games, Amazon Movies & TV, and Yelp Restaurant, and compare COFFEE with multiple baseline models. Due to space limit, we present the results based on PETER, and put example explanations and results based on NRT in Appendix E.
NATTR (post-processing):: We disable the pro- tected attribute token in ATTR for generating ex- planations during inference.
6.1 Experiment Setup
Datasets. In Amazon Games and Amazon Movies review d we use users' binary gender (male, female) as the protected attribute. We study the fairness on the item side on the Yelp d
We train each model on the training set, tune papre-parraeters on the validation set, and report the results on the testing set. Each reported result point is averaged over 3 runs. We put the detailed model specifications and parameter tuning in Ap-
gitb. io/ aaaaaammiiineeeeee html
hhtps//wwww.cdp..omddaass

---

pendix C.
NORN
WATM
AmT
correc
Metries, Our evaluation consists of two parts: fair- ness and utility. Fairness requirements are subjec- tive, contingent on both the application and the the designer's 's objective (Hu and Chen, 2020; Khademi et al., 2019). COFFEE is designed to accommodate different faimess specifications by appropriately defining Q. To select meaningful quality evalua- tion metrics in experiments, we evaluated expla- nations with different measures such as FeatCov, length, grammar, redundancy, structure (Zhu and Bhat, 2020). We found that only FeatCov con- sistently shows significant correlations to human perceptions of explanation quality such as infor- mativeness, detailedness, and helpfulness. The de- tailed correlation results are shown in Table 6 in Appendix D. Therefore, we specify the quality mea- sure function in COFFEE as fent, which directly corresponds to an explanation's 's FeatCov. Further- more, experiment findings suggest a substantial influence of the number of tokens in explanations on FeatCov. Hence, we additionally explore a com- bination of FeatCov and number of tokens to facil- itate the fairness optimization on FeatCov, speci- fying Q as feat + Quumtoken. We refer to this variant, with Q augmented by the number of tokens, as COFFEE-NT. This modification also helps vali- date COFFEE's s ability to adapt to different quality measures optimizing fairness.
BERTS.core
20:25
38.50
0.1
13 FeetCov (Movies TV)
52
6.
FeatCov (Telp)
Figure 4: Trade-off between fairness and utility on Ama- zon Movies & TT (left) and Yelp (right) datasets. A method should give low DDP and high BERTScore (top- left corner of each plot) for better trade-offs.
(Lin, 2004) scores, also employ the increasingly popular BERTScore (Zhang et al., 2020), known for better semantic alignment between target and generated text and stronger correlation with hu- man judgments. We compute BERTScore using RoBERTa-large with re-scaled scores, and report F- scores for ROUGE and BERTScore. All utility metrics are higher the better. Lastly, we incorpo- rate FeatCov into the utility metrics evaluate the impact of various fairness attainment methods.
6.2 Comparison to Baselines
An ideal method should enhance fairness without hurting explanation utility. Due to space limits, we display complete results on Amazon Games in Ta- ble 2, and sample results for Amazon Movics and Yelp in Figure 4; full results are in Appendix E. As indicated in Table 2, COFFEE excels in fairness im- provement across all metrics, significantly outper- forming baselines. Morcover, comparing COFFEE and ATTR in terms of Ind-CI and Grp-CF validates the effectiveness of our fair policy learning scheme in optimizing CF constraints. Additionally, intro- ducing Qnumtoken and Q feat in COFFEE-NT bol- sters fairness optimization in FeatCov, suggesting that enhanced quality measures can aid fairness op- timization, echoing findings in (Bose and Hamilton, 2019). Crucially, COFFEE secures strong fairness outcomes while preserving high utility in explana- tion generation.
Faimesss metrics. We adopt both individual and group-wise fairness metrics evaluated FeatCov, with and without counterfactual perspectives. De- tailed metric formulas are in Appendix B. The first metric, Ind-CF, assesses individual CF as defined by (Kusner et al., 2017), which is equivalent to the regularization term in Eq. (4) averaged over all test set user-item pairs. We also evaluate the counter- factual effects on group-level (Coston et al., 2020). For a demographic group with attribute value a, counterfactually alter all group members' attribute value a' * a, and compute the change of the average FearCov of the group. The average change is denoted as Grp-CI, Additionally, we evaluate COFFEE's generalization to improve group-wise fairness evaluated by demographic disparity (DDP) (Zafar et al., 2015). Metrics based on CF can only be evaluated on baselines that feature disentangled attribute representations, such as ATTR and COF- FEE. All fairnsss metrics are lower the better.
We plot results on Amazon Movies and Yelp in Figure 4, where we use DDP on FeatCov for fair- ness and BERTScore for utility. Notably, COFFEE achieves the best trade-off by drastically reducing DDP with minimal impact on BERTScore. ADV and NORM moderately improve fairness, albeit less significantly than COFFEE or at the cost of a greater decrease on utility. Lastly, BT also ef- fectively enhances fairness, showing its interesting
Utility metrics. Besides commonly used BLEU- (1,4) (Papineni et al., 2002) and (OUGE-(112,L)

---

Table 2: Comparison between COFFEE and baselines. BL stand for BLEU and RG denotes ROUGE. BLBU, ROUGE and BERTScore are in percentage values and others are in absolute values. The best results are boldfaced, and the second best are underlined. indicates p < 0.05 for significance test over the second best baseline.
Fairness on FeatCov
Utility
PETER
Ind-CFi
Grp-CF4 DDP,
BLIt
BL41
RGI1
RG21
RGL1
BERTY
FeatCov
Amazon Games (User's s Gender Protected Attribute)
RAW
1.18
8.82
1.49
19.95*
5.37
15.54
14.14
6.53
ADV
0.62
8.78
1,50
16.13
3.96
13.06
12.84
6.12
NORM-F
1.13
8.83
1.51
18.20
4.66
14.42
13.91
4.28
BT
0.65
10.43*
1.55
13.42
2.08
10.44
9.55
5.42
NATTR
0.63
8.76
1,52
16.55
3.86
13.02
12.72
5,26
ATTR
2.68
0.75
1.40
8.91
1.57
17.73
4.45
14.13
14.08
5.78
COFFEE
1.26
0.32
0.68
9.77
1.58
16.73
3.98
13.43
14.12
6.34
COFFEE-NT
1.15
0.01
0.08
10.23
1.24
17.38
3.70
13.62
14.02
6.27
Fairness & Utility Change vs.
Table 3: Faimess and quality based on human evalu- ations of explanations. Smaller Ind-CF means better fairness. indicates p-value <0.05 for paired t-test.
Relative Metric (%)
80
Base 0)
Ind-CF FeatCov
Ind-CFi
Averaget
60
BERTScore
RAW
COFFEE
RAW
COFFEE
Infor
0.8476
0,5673*
2.6381
2.6712
Detail
0.8190
0.6442
2.5238
2.3788
0.0
0.1
0.4
Helpful
0.8857
0.5786
2.5904
2.5577
Figure 5: Relative Ind-CF and BERTScore ratio w.r.t.
to À = 0 in COFFEE. The Ind-CF and BERTScore of ATTR-PET are 12.42 and 14.08, respectively.
more universally understood. In each questionnaire, present the title of a recommended movie for user, randomly sampled from the dataset, along with two explanations: one generated for the real user and the other for a counterfactual user with a modified gender. Evaluators then assess each explanation from three criteria: informativeness, detailedness, and helpfulness. Each criterion is rated on scale of 5: 1: "not at all", 2: "somewhat", 3: "moderate". 4: "very", 5: "extremely". We also collect evaluators' gender (only for aggregated analysis) examine if humans with different gen- ders evaluate the explanations differently, Rigorous privacy protection and quality control mechanisms are implemented. We compare the explanations generated from the raw model and COFFEE, with 150 valid questionnaires collected for each.
normalization effect. But BT causes a dramatic sacrifice on utility, as the translation may eliminate much information about personal preference.
6.3 Effect of Tuning À in COFFEE
We further study the fairness-utility trade-off in COFFEE by tuming the weight on faimess con- straint in Eq. (4). We use Ind-CF on FeatCov for fairness evaluation which corresponds to the con- straint that COFFEE directly optimizes, and use BERTScore for utility evaluation. We plot the re- sults on Amazon Movies in Fig. 5, where the y-axis is the metric values ratio to the base results when A = 0 (equivalent to ATTR). As X increases from 0 to 0.5, COFFEE significantly improves fairness, with drop of about 77% in Ind-CF, while barely hurts the BERTScore, with drop of about 3%. This study shows COFFEE's outstanding efficiency and effectiveness in fairness optimization.
We measure fairness in the three human evalu- ated criteria of explanations. The results are shown in Table 3, where we use Ind-CF as the fairness metric. COFFEE can effectively achieve better faimess on human evaluated quality measures by improving faimess FeatCov. Moreover, as in- dicated by the average quality measure values, the quality of explanations generated by COFFEE is comparable that of the RAW model. These re- sults demonstrate COFFEE's ability and practical use in achieving faimess in explanation generation.
7 Human Evaluation
We conduct human evaluations to justify the use of FeatCov for fairness and confirm COFFEE'S efficacy. Human evaluators, recruited on Mechan- ical Turk, are asked to assess explanations gener- ated from Amazon Movies" test data, as movies are

---

8 Conclusion
Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, and Aaron Roth. 2021. Faireess in criminal justice risk assessments: The state of the art. Socio- logical Methods & Research.
We investigate the fairness problem in PTG by fo- cusing on the bias in FeatCov of explanation gener- ation. We propose COFFEE for achieving counter- factual faimess in explanation generation. Compre- hensive experiments and human evaluations show the efficacy of COFFEE in achieving fairness with- out hurting the utility of explanations.
Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of "bias" in nip. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Com- putational Linguistics.
This work opens the new direction of quality fairness in PTG. We anticipate that this work will encourage researchers to investigate novel fairness notions in this problem, based on different quality measures, and conduct in-depth analyses on their social impacts. A promising direction is to involve humans in the loop for direct fairness optimization on human evaluated measures. We also plan to generalize the COFFEE to other PTG settings, such as conversational systems.
Shikha Bordia and Samuel R. Bowman. 2019. Identify- ing and reducing gender bias in word-level language models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Student Research Work- shop, Minneapolis, Minnesota. Association for Com- putational Linguistics.
Avishek Joey Bose and William Hamilton. 2019, Com- positional fairness constraints for graph embeddings.
Hanxiong Chen, Xu Chen, Shaoyun Shi, and Yongfeng Zhang. 2021. Generate natural language explana- tions for recommendation.
Limitations
In this study, we formulated counterfactual fair- ness constraint on explanation qualities, which is grounded in sampled explanations from both real and counterfactual worlds. We employed straight- forward first moment matching to minimize the dis- parity between these two sample sets. Future work could explore the use of higher moment matching to more effectively align the quality distributions from the two worlds. For our experiments, lized FeatCov of explanations as a quality measure, given its simplicity and significance in the context of explainable recommendations. However, ex- ploring other domain-specific measures (e.g., senti- ment, emotion) to define Q and assess COFFEE's versatility in different PTG settings would be in- triguing. We have also achieved encouraging re- sults in terms of sentiment fairness using COF- FEE. Nevertheless, more detailed discussion on this falls outside the scope of this paper's focus on quality fairness, and thus we earmark it for future investigation.
Jonathan Gabel Christiansen, Mathias Gammelgaard, and Anders Søgaard. 2021. The effect of round-trip translation on fairness sentiment analysis. In Pro- ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing. Association for Computational Linguistics.
Amanda Coston, Alan Mishler, Edward H. Kennedy, and Alexandra Chouldechova, 2020. Counterfac- tual risk assessments, evaluation, and fairness. In Proceedings of the 2020 Conference on Fairness, Ac- countability, and Transparency. FAT '20. Associa- tion for Computing Machinery.
uti-
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach to controlled text generation. In International Conference on Learning Representa- tions.
Mengnan Du, Fan Yang, Na Zou, and Xia Hu. 2021. Fairness in deep learning: A computational perspec- tive. IEEE Intelligent Systems.
Acknowledgements
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd Inno- vations in Theoretical Computer Science Conference, ITCS 12. Association for Computing Machinery.
This work was partially supported by NSF IIS- 2007492 and NSF 1838613.
References
Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguisties ( (Volame I: Long Papers). Association for Computational Linguistics.
H. Samy Alim, John R. Rickford, and Arnetha F. Ball. 2016. Raciolinggistics: How Language Shapes Our Ideas About Race. Oxford University Press.

---

Onochie Fan-Osuala. 2023.
Women's 's online opinions are still not as influential as those of their male peers in buying decisions. Humanities and Social Sciences Communications, 10:40.
Matt J Kusner, Joshua Loftus, Chris Russell, and Ri- cardo Silva. 2017. Counterfactual fairness. In Ad- vances in Neural Information Processing Systems. Curran Associates, Inc.
Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H. Chi, and Alex Beutel. 2019. Counterfactual fairesss in text classification through robustness. In
Jing Yang Lee, Kong Aik Lee, and Woon Seng Gan. 2021. Divgen: A dual latent variable approach to personalized dialogue generation.
Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, AIES 19, New York, NY, USA. Association for Computing Machinery.
Michelle Seng Ah Lee and Luciano Floridi. 2021. Algo- rithmic fairness in mortgage lending: From absolute conditions to relational trade-offs. Minds Mach.
Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honmavalli, Sharon Levy, Diba Mirza, and William Yang Wang. 2020. Investigating african- american vernacular english transformer-base text generation. In Proceedings of the 2020 Con- ference Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics.
Lei Li, Yongfeng Zhang, and Li Chen. 2021a. la. Person- alized transformer for explainable recommendation. In Proceedings of the 59th Annual Meeting of the As- sociation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing ( (Volume Long Papers). Association for Computational Linguisties.
Piji Li, Zihao Wang. Zhaochun Ren, Lidong Bing, and
Moritz Hardt, Eric Price, Eric Price, and Nati Srebro. 2016. Equality of opportunity in supervised learn- ing. In Advances in Neural Information Processing Systems. Curran Associates, Inc.
Wai Lam. 2017. Neural rating regression with ab- stractive tips generation for recommendation. In Pro- ceedings of the 40th International ACM SIGIR Con- ference on Research and Development in Information Retrieval, SIGIR '17. Association for Computing Machinery.
Bing He, Mustaque Ahamad, and Srijan Kumar. 2021. Petgen: Personalized text generation attack on deep sequence embedding-baas based classification models. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. KDD '21, New York, NY, USA. Association for Computing Machinery.
Yunqi Li, Hanxiong Chen. Shuyuan Xu, Yingqiang Ge, and Yongfeng Zhang. 2021b. Towards personalized fairness based on causal notion. In Proceedings of the 44th International ACM SIGIR Conference on Rexearch and Development in Information Retrieval, SIGIR '21, New York, NY, USA. Association for Computing Machinery.
Lily Hu and Yiling Chen. 2020. Fair classification and social welfare. In Proceedings of the 2020 Confer-
ence on Fairness, Accountability, and Transparency, FAT* '20. Association for Computing Machinery.
Paul Pu Liang, Chiyu Wu. Louis-Philippe Morency, and Ruslan Salakhutdinov. 2021. Towards understanding and mitigating social biases in language models. 1CML.
Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stan- forth, Johannes Welbi, Jack Rac, Vishal Maini, Dani Yogatama, and Pushmeet Kohli. 2020. Reducing sen- timent bias in language models via counterfactual evaluation. In Findings of the Association for Com- putational Linguisties: EMNLP 2020. Association for Computational Linguistics.
Chin- Yew Lin. 2004, Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out. Association for Computational Lin- guistics.
Francesco Locatello, Gabriele Abbati, Thomas Rain- forth, Stefan Bauer, Bernhard Schölkopf. and Olivier Bachem. 2019a. On the fairness of disentangled representations. In Advances in Neural Information Processing Systems. Curran Associates, Inc.
Matthew Joseph, Michael Kearns, Jamie H. Mor- genstem, Seth Neel, and Aaron Roth. 2016. Rawlsian fairness for machine learning. ArXiv, abs/1610.09559.
Francesco Locatello, Stefan Bauer, Mario Lucic, Gun nar Rätsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. 2019b. Challenging common as- sumptions in the unsupervised learning of disentan- gled representations. In ICML.
Aria Khademi, Sanghack Lee, David Foley, and Vas- ant Honavar. 2019. Fairness algorithmic decision making: An excursion through the lens of causal- ity. In The World Wide Web Conference, WWW '19. Association for Computing Machinery.
Annie Louis. 2013. Predicting text quality: Metrics for content, organization and reader interest. Disserta- tions available from ProQuest.
Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2017. Inherent trade-offs in the fair de- termination of risk scores. In 8th Innovations in Theoretical Computer Science Conference (ITCS 2017), Leibniz International Proceedings in Infor- matics. Schloss Dagstuhl-Leitn -eituull--einii---- Leitnii-ZZZetuu fuer In- formatik.
Jianxin Ma, Chang Zhou, Peng Cui, Hongxia Yang, and
Wenwu Zhu, 2019. Learning Disentangled Represen- tations for Recommendation. Curran Associates Inc.. Red Hook, NY, USA.

---

Karima Makhlouf, Sami Zhioua, and Catuscia Palamidessi. 2020. Survey on causal- based machine learning fairness notions.
Emily Sheng, Kai-Wei Chang, Premkumar Natarajan. and Nanyun Peng. 2021. Societal biases in language generation: Progress and challenges. In Proceedings of the Conference of the 59th Annual Meeting of the Association for Computational Linguistics (ACL).
Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A sur- vey on bias and fairness in machine learning. ACM Comput, Sure.
Lei Shu, Alexandros Papangelis, Yi-Chia Wang. Gokhan Tur, Hu Xu, Zhaleh Feizollahi, Bing Liu, and Piero Molino. 2020. Controllable text generation with focused variation, In Findings of the Associa- fion for Computational Linguistics: EMNLP 2020. Association for Computational Linguisties.
Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asyn- chronous methods for deep reinforcement learning.
Ashudeep Singh and Thorsten Joachims. 2018. Fairness of exposure in rankings. In Proceedings of the 24th ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining, KDD '18. Associ- ation for Computing Machinery.
Matthew Newman, Carla Groom, Lori Handelman, and James Pennebaker. 2008. Gender differences in lan- guage use: An analysis of 14,000 text samples. Dis- course Processes- DISCOURSE PROCESS.
Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Jus- tifying recommendations using distantly-labeled re- views and lin-ggrined aspects. Proceedings of the 2019 Conference Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing DEMNLP-IJCNLID Hong Kong, China. Association for Computational Linguistics.
Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999. Policy gradient methods for reinforcement leamnngg with function approxima- tion. In Advances in Neural Information Processing Systems. MIT Press.
Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Na- man Goyal, Vishrav Chaudhary, Jiatao Gu, and An- gela Fan. 2020. Multilingual translation with extensi- ble multilingual pretraining and finetuning.
Shereen Oraby, Lena Reed, Shubhangi Tandon, Sharath T.S., Stephanic Lukin, and Marilyn Walker. 2018. Controlling personality-base based stylistic variation with neural natural language generators. In Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue. Association for Computational Linguistics.
Nava Tintarev and Judith Masthoff. 2015. Explaining Recommendations: Design and Evaluation. Springer US.
Paul Voigt and Axel von dem Bussche. 2017. The EU General Data Protection Regulation (GDPR): A Practical Guide. Springer Publishing Company, Incorporated.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computa- tional Linguistics, ACL '02, USA. Association for Computational Linguistics.
Sarah Volz, Marc-André Reinhard, and Patrick Müller. 2020. Why don't you believe me? detecting de- ception in messages written by nonnative and native speakers. Applied Cognitive Psychology.
Ella Rabinovich, Raj Nath Patel, Shachar Mirkin, Lu- cia Specia, and Shuly Wintner. 2017. Personalized machine translation: Preserving original author traits. Proceedings of the 15th Conference of the Euro- pean Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Association for Computational Linguistics.
Lequn Wang and Thorsten Joachims. 2021. User fair- ness, item fairness, and diversity for rankings in two- sided markets. In Proceedings of the 2021 ACM SIGIR International Conference Theory of Infor- mation Retrieval, ICTIR '21. Association for Com- puting Machinery.
Chris Russell, Matt Kusner, Joshua R. Loftus, and Ricardo Silva. 2017. When worlds collide: Integrat- ing different counterfactual assumptions in fairness. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17. Curran Associates Inc.
Nan Wang, Lu Lin, Jundong Li, and Hongning Wang. 2022. Unbiased graph embedding with biased graph observations. Proceedings of the ACM Web Con- ference 2022, wWW '22, New York, NY, USA. As- sociation for Computing Machinery.
H. Schwartz, Maarten Sap, Margaret Kern, Johannes Eichstaedt, Adam Kapelner, Megha Agrawal, Ed- uardo Blanco, Lukasz Dziurzynski, Gregory Park, David Stillwell, Michal Kosinski, Martin Seligman, and Lyle Ungar. 2016. Predicting individual well- being through the language of social media.
Nan Wang, Hongning Wang, Yiling Jia, and Yue Yin. 2018. Explainable recommendation via multi-task learning in opinionated text data. In The 41 Inter- national ACM SIGIR Conference Research and Development in Information Retrieval, SIGIR '18 Association for Computing Machinery.

---

Yongkai Wu, Lu Zhang, Xintao Wu, and Hanghang Tong. 2019. PC-Fairness: A Unified Framework for Measuring Causality-Based Fairness. Curran Asso- ciates Inc.
CIKM '18. New York, NY, USA. Association for Computing Machinery.
Yu Zheng, Chen Gao, Xiang L., Xiangnan He, Yong Li, and Depeng Jin. 2021. Disentangling user interest and conformity for recommendation with causal em- bedding. In Proceedings of the Web Conference 2021, WWW 21. Association for Computing Machinery.
Aobo Yang, Nan Wang. Renqin Cai, Hongbo Deng, and Hongning Wang. 2022. Comparative explanations of recommendations. In Proceedings of the ACM Web Conference 2022, WWW '22. Association for Computing Machinery.
Wanzheng Zhu and Suma Bhat. 2020. GRUEN for evaluating linguistic quality of generated text. Findings of the Association for Computational Lin- guistics: EMNLP 2020, Online. Association for Com- putational Linguistics.
Aobo Yang. Nan Wang. Hongbo Deng, and Hongning Wang. 2021. Explanation as defense of recommen- dation. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, WSDM '21, New York, NY, USA. Association for Computing Machinery.
Chenhan Yuan and Yi-Chin Huang. 2019. Personal- ized sentence generation using generative adversarial networks with author-specific word usage. CoRR, 09444..
Muhammad Bilal Zafar. Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. 2017. Fairness beyond disparate treatment and disparate impact: Learning classification without disparate mistreatment. Proceedings of the 26th Interna- rional Conference on World Wide Web, WWW '17. International World Wide Web Conferences Steering Committee.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. 2015. Learning fair classifiers. arXiv: Machine Learning.
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. 2013. Learning fair representations. In Proceedings of the 30th International Confer- ence on Machine Learning, Proceedings of Machine Learning Research. PMLR.
Junzhe Zhang and Elias Bareinboim. 2018. Equality of opportunity in classification: A causal approach. In Advances Neural Information Processing Systems. Curran Associates, Inc.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2020. Bertscore: Eval- uating text generation with bert. In International Conference on Learning Representations.
Wei-Nan Zhang, Qingfu Zhu, Yifa Wang, Yanyan Zhao, and Ting Liu. 2019. Neural personalized response generation domain adaptation. World Wide Web. 22(4).
Yongfeng Zhang and Xu Chen. 2020. Explainable recommendation: A survey and new perspectives. Found. Trends Inf. Retn. 14:1-101.
Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W. Bruce Croft. 2018. Towards conversational search and recommendation: System ask, user respond. In Proceedings of the 27th ACM International Confer- ence on Information and Knowledge Management,

---

A Applying COFFEE tO NRT
Table 4: Average FeatCov of explanations generated for restaurants of different price tags.
There are mainly two modules in NRT (Li et al., 2017)-a MLP for rating prediction, and an RNN for explanation generation. Both modules share the same user and item embeddings mapped from input user and item IDs for personalization. In particular, the user and item embeddings are used in the initial hidden state of the RNN for explanation generation (see Figure in (Li et al., 2017)). To apply COFFEE to NRT, we simply need to use the concatenation of disentangled user preference embedding and the attribute embedding instead of the holistic user embedding, exactly as introduced in Section 4.1. Similar to PETER, there are also three loss terms in the original NRT, corresponding to the explanation, context, and rating prediction (see Section 3 in (Li et al., 2017)). We denote the total loss of NRT by Lart, which is used to replace Egen in Eq. (3) for applying COFFEE.
FeatCov
$$
$$$
Ground-Truth 3.15
3.72
4.38
RAW PETER
3.26
3.68
4.03
will also leave the lower-priced restaurants at disadvantage.
We split each dataset into training (80%), valida- tion (10%), and testing (10%) sets, and ensure that there is at least one record in each subset for every user and item.
B.2 Details of Baselines
ADV (in-processing): Adversarially removing the sensitive information in user or item represen- tations by adding a discriminator (Li et al., Ib). We add discriminators on the user's (or item's 's) em- bedding in PETER and NRT to remove the infor- mation about protected attributes.
B Experiment Setup
B.1 Dataset Processing
NORM (pre-processing): We normalize the train- ing data to remove the bias on group-level. With two demographic groups, remove the explana- tions with the higher (lower) quality in the group with more reviews, until the difference of average quality between the two groups below 10% of the original difference. With more than two groups, we recursively apply the procedure to the two groups with the maximum difference until the maximum difference is below the threshold. NORM removes less than 2% of training data on Amazon datasets, and less than 18% on Yelp.
The first two datasets are the Video Games and the Movies TV categories from Amazon reviews. These two datasets only provide the names of users. We use a gender classification tool to predict the gender of users from their names. To guarantee the accuracy of the attribute values, we only re- serve the users whose names classified male or female and remove those clas- sified as unknown. In the Yelp dataset, we use a restaurant's s price range as the protected attribute. There are originally four price ranges ($, $$, S$$, $$S$) provided. However, we found that there are much less dollar restaurants than in the other price ranges, and thus we merge the four-dollar restaurants into the group of three-dollar ones for experiments. Using restaurant price as sensitive attribute is based on the counterfactual argument: if a restaurant chose to raise/lower the price of its dishes without other changes, the quality of gener- ated explanations shouldn't 't change. The rationale is that users may care more about experiences in ex- pensive restaurants since they are paying more, and thus writing more detailed reviews. As shown in Table 4, the explanations for expensive restaurants have higher FeatCov and thus are more informa- tive and detailed, which helps recommendations on them be accepted more often. However, this
be confidently
We pre-process the training data by translating the explanations to Chinese and then back to En- glish. We use the multilingual model mBART (Tang et al., 2020) from EasyNMT to perform the translation.
ATTR: In order to evaluate the effectiveness of optimizing the fairness constraint in COFFEE, we use the model trained without the constraint (A = 0 in Eq. (4)) baseline. We call the model ATTR, which means the attribute is disentangled from the user's or item's 's representations as introduced Scction 5 but without adding the fairness constraint. NATTR (post-processing) We disable the pro- tected attribute token in ATTR for generating expla- nations during inference. Specifically, for NATTR- NRT, replace the learned attribute embeddings
https://gituuh.om//UKLLLaaaaa

---

with random embeddings with values uniformly sampled in |-1,1] for explanation generation. For NATTR-PETER, we disable the other tokens' at- tention on the attribute token during inference.
PETER
COFFEE
Games
Movies
Yelp
Games
Movies
Yelp
0.2
0.2
0.2
0.3
0.1
0.1
0.6
0.6
0.5
0.5
0.5
0.5
Table Weight À for the faimess constraint and the promotion weight 1) in reward calibration of COFFEE when applied PETER and NRT on the three datasets.
B.3 Evaluation Metrics
We evaluate both individual- and group-level fairness, with and without counterfactual perspec- tives. The first metric denoted as Ind-CF evaluates the originally defined CF on individuals (Kusner et al., 2017), which is the same as the regularization term in Eq. (4) averaged over all user-item pairs on the test set D:
C Experiment Settings
Here we elaborate the experiemntal protocols, model specifications, and hyper-parameters: For all models, we set the size of vocabulary to 20,000 by keeping the most frequent words. By default, we use top-5 sampling (Fan et al., 2018) the decod- ing strategy, and the maximum decoding sequence length is 128.
Ind-CF
1
ElQ(ua--a)lu,
E[Q()A-)))u. i]|.
D
For the raw PETER model (Li et al.. 2021al we mostly follow the original paper's hyper- parameters. The token embedding dimension is 512 and the dimension of feed-forward network is 2, 048. The number of transformer layers and attention heads are both two. The dropout rate dur- ing training is 0.2, For NRT (Li et al., 2017), we follow the original paper and set the embedding di- mension to 300 for all users, items and words. The dimension of hidden layers is 100. The dropout rate during training is 0.1. COFFEE consists of pre-training the ATTR models with disentangled at- tribute representations and then fine-tune them with the counterfactual fairness constraints. For ATTR- PETER, we just add additional attribute token to PETER, with the same embedding dimension for the attribute. For ATTR-NRT, we disentangle the attribute representation by concatenating to the user or item embedding an attribute embedding of dimension 100. The other model specifications for ATTR-PETER and ATTR-NRT are the same as raw PETER and NRT. When applying the discriminator for removing the information about the protected attribute from user or item embeddings, we use 2- layer MLP with hidden size of 512 as the attribute discriminator. For both ATTR-PETER and ATTR- NRT, the weight AD of the adversarial loss is set to 0.5 on Amazon Games and Yelp datasets, and 0 on Amazon Movies. We iterate between one epoch B[Q(y)|u a of model training and one epoch of discriminator training until convergence. After the pre-training of ATTR models, we fix the user, item and attribute embeddings, remove the loss on rating prediction but keep the loss on explanation generation, and tune the model with the fairness constraint. The
We also evaluate the counterfactual effect group-level (Coston et al., 2020). For each attribute value a and the corresponding demographic group, we counterfactually change the attribute value of all users in this group by a' * a, and calculate the change on the average quality of this group. We call this metric Group-CF:
Group-CF
1
LAI(IA] -
[Dal
a'yta
BIQ(vs-a)lu)
Σ B[Q(AA+
B,iEDo
MiED
where D, denotes the demographic group with at- tribute value a.
Finally, besides counterfactual metrics, also evaluate COFFEE's generalization to improve group-wise fairness by the popular notion of de- mographic disparity (DDP) (Zafar al., 2015);
DDP
(10)
Σ
= LAI(IAI 1)
0,''EA
E[Q(y)lu,
Σ LiiED
Dal
IDai
U.EED
All fairness metrics are lower the better, and the quality expectation on each user-item pair is calcu- lated over = 3 sampled explanations.

---

Table 6: Kruskal- Wallis H test between FeatCov and quality and utility evaluated by human.
model in Table Importantly, COFFEE achieves strong fairesss results while maintaining high util- ity on explanation generation. Although COFFEE may slightly drop the utility compared to the base- lines, it can sometimes even outperform the base- lines, as shown in the results on Amazon Movies and Yelp datasets. This is because the disentan- glement mechanism enables better representation learning (Ma et al., 2019; Zheng et al., 2021) and in- creases the flexibility and accuracy of interactions between the user and item for better explanation generation.
K-W H test
Infor
Detail
Help
H-stats
39.44
45.67
23.83
All
p-value
5.6e
2.9e-9
8.6e
H-stats
24.28
28.07
14.37
Male
p-value
7.0c
1.2e-5
6.2c
H-stats
16.58
15.74
12.24
Female
p-value
2.3e
3.4c
5.6c
We also present some examples of generated explanations on Amazon Games in Table In par- ticular, we select a male user Chadwick who wrote detailed and informative reviews with high Feat- Cov and female user Noemi who wrote generic reviews with low FeatCov for the same item "Wii Nunchuk Controller". The raw PETER model fol- lows the ground reviews and generate detailed explanation for Chadwick but short and generic ex- planation for Noemi. contrast, after applying COFFEE to improve fairness, the model tend to generate and more descriptive and informative ex- planation for Noemi. These examples demonstrate the effectiveness of COFFEE in generating fair and high-quality explanations for users with different protected attribute values.
weight À for the fairness constraint and the pro- motion weight y in reward calibration are tuned for different models and datasets, and are listed in Table 5.
For model training, we use Adam as optimizer and the initial learning rate for training raw mod- els and ATTR models is lc-4. The initial learning rate is le-5 for COFFEE during fine tuning by the fairness constraint. By default, the batch size is 16. But use batch size of 8 during the tuning phase of COFFEE when applying to PETER mod- els, mainly due to memory issues. For training raw models and ATTR models, we evaluate the total loss on the validation set after each epoch, and use the epoch checkpoint when the total loss is higher in the next 5 consecutive epochs. When COFFEE fine-tunes the pre-trained ATTR models, we always use the checkpoint after a single epoch, which al- ready yields promising results from COFFEE.
E.1 Entanglement of Protected Attribute with Preference
D Results from Kruskal- Wallis H Tests
In Section 4.1, we employ a disentanglement ap- proach where each user's representation is sepa- rated into two components: the attribute representa- tion and the preference representation. This setup enables us perform counterfactual inference by switching the attribute representation from one at- tribute value to another. Following this method, we analyzed how much changing a user's learnt at- tribute representation will influence the FeatCov of generated explanations. Take the Amazon Games dataset for example, without imposing any faimess constraints, we observed that the average FeatCov change is 2.68 when altering the attribute represen- tation. Furthermore, over 92% of users exhibit a FeatCov change greater than 1. This indicates that, in most cases, the attribute and preference repre- sentations are intertwined, and the disentanglement approach effectively separates these two aspects.
We perform Kruskal-Wallis H test between Feat- Cov and three human evaluated measures including Informativeness, Detailness, and Helpfulness. The results are shown in Table 6, where H-stats larger than one means strong correlation, and a p-value smaller than 0.05 means the correlation is signifi- cant. FeatCov strongly and significantly correlate with humans' (both male and female) judgement of the quality and helpfulness of explanations. Thus, the bias can lead to unfair treatments to users of different genders, as the quality of explanations, evaluated by FeatCov in this case may affect the utility of explanations to users.
E Additional Results
We show the complete results on the Amazon Movies and Yelp datasets based on the PETER

---

Table 7: Comparison between COFFEE and basclines based on the PETER model. BL stand for BLEU and RG denotes ROUGE. BLEU, ROUGE and BERTScore are in percentage values and others are in absolute values. The best results are boldfaced, and the second best are underlined. * 'ndicates p < 0.05 for significance test over the second best baseline.
Faimesss FeatCov
Utility
PETER
Ind-CF4 Crp-CFy DDP,
BLI1
BL.4
RGIT
RG21
RGLF
BERT
FeatCov
Amazon Movies
TV
(User's Gender Protected Attribute)
RAW
0.57
11.25
2.20
18.04
4.53
15.48
19.93
5.61
ADV
0.26
11.08
2.16
17.20
4.32
14.37
18.94
5.14
NORM
0.27
11.23
2.17
17.46
4.42
15.00
19.67
4.34
BT
0.16
11.25
2.18
12.91
1.66
11.02
15.01
4.13
NATTR
0.27
11.24
2.19
17.35
4.40
14.92
19.34
4.71
ATTR
1.47
0.10
0.37
11.19
2.16
17.43
4.63
15.98
19.72
5.31
COFFEE
1.13
0.01
11.28
2.12
17.77
4.60
15.35
20.13
5.45
COFFEE-NT
1.16
0.01
0.00
10.05
1.39
19.34
4.56
15.86
19.52
5.08
Yelp (Restaurant's s Price Protected Attribute)
RAW
0.51
10.24
1.57
18.71
3.11
14.64
19.82
3.78
ADV
0.28
1.60
17.25
2.64
13.99
18.34
3.18
NORM
0.09
9.96
1.54
17.56
2.81
14.10
19.67
3.06
BT
0.27
10.69
1.61
16.42
2.36
13.29
18.89
3,20
NATTR
0.32
10.08
1.55
18.53
3.06
14.57
19.75
3.37
ATTR
1.50
0.44
0.64
10.00
1.53
18.83*
3.16
14.75
19.81
3.52
COFFEE
0.84
0.08
0.13
11.17
1.77
17.97
3.00
14.39
20.58
3.68
COFFEE-NT
0.54*
0.03
0.03*
11.61
1.72
17.41
2.93
14.31
20.21
3.61
Table Generated explanations by different models on Amazon Games. We select one male user Chadwick and one female user Noemi, and present the explanations for them on the same item "Wi Nunchuk Controller".
PETER
user: Chadwick, item: Wii Nunchuk Controlle
user: Noemi, item: Wii Nunchuk Controller
this review for the white wi nunchuk controller. it is necessary component most wii games, and attaches bottom port every wii remote. well-designed. sturdy, and confortable to hold. the price relatively low for such controller.
Ground-truth
great controller, arrived as expected.
bought it to play wii and this is great addition to the wii the gamepad is nice uddition the wii ganepud. must have for any bought this controller to use the nunchuck. this is great controller for wii owners especially the wii u. the controller is very responsive and the triggers awesome.
RAW
the controller worked very well.
console owner.
bought this controller for my wil u. it was exactly what would be expected. love the feel of the box and it works great.
COFFEE

---

E.2 Results based on NRT
We present the complete results based on the NRT model in Table Similar to the results on PE- TER, COFFEE when applied to NRT can signif- icantly outperform the baselines in terms of fair- ness improvement. Again, when optimizing both Quamtoken and feat together, COFFEE achieves the best fairness improvements, verifying the effect of correlations between different quality measures. In general, PETER can generate better explana- tions than NRT, e.g.. on BERTScore, with or with- out fairness optimizations, showing the capability of transformers over RNNs. However, COFFEE still maintains high generation utility based on NRT when compared to baselines, showing its advantage in generalizing the results different models.
The observations and conclusions are similar to the PETER based results. These results show COF- FEE's ability to generalize the fairness improve- ment to different types of models, and indicate its flexibility and practicality in real-world uses.

---

Table 9: Comparison hetween COFFEE and baselines based the NRT model. BL stand for BLEU and RG denotes ROUGE. BLEU, ROUGE and Score are in percentage values and others are in absolute values. The results boldfaced, and the second are underlined. * indicates p < 0.05 for significance test over the second best baseline.
Fairness on Jeat
Utility
NRT
Ind-CF|
Grp-CF) DDP1
BLIf
BL41
RGI1
RG21
RGL1
BERT
FeatCov
Amazon Games
(User's 's Gender as
Protected Attribute)
RAW
1.42
8.33
1.30
19.21
3.97
14.35
13.79
6.48
ADV
1.25
8.13
1.29
17.67
3.37
13.27
12.85
6.08
NORM
1.79
8.17
1.26
19,89*
4.27*
13.65
13.49
4.80
BT
1.47
8.34
1.17
16.44
2.71
12.23
10.73
5.29
NATTR
0.12
8.39
1.39
13.07
2.71
10.83
11.11
5.83
ATTR
6.70
1,46
1.65
8.98
1,29
18.94
3.89
14.29
13.84
6.23
COFFEE
1.81
0.52
0.51
8.91
1.30
17.94
3.50
13.63
13.83
6.30
COFFEE-NT
0.77c
0.02*
0.04
8.67
1.26
17.11
3.11
12.46
13.69
6.27
Amazon Movies
(User's Gender as Protected Attribute)
RAW
0.46
10.55
1.86
18.37
3.98
15.16
19.20
5,46
ADV
0.24
10.48
1.83
18.44
3.95
15.10
19.09
5.18
NORM
0.46
10.53
1.91
18.09
3.89
14.96
19.03
4.62
0.43
10.63
1.62
14.84
1.87
11.95
14.36
5.06
NATTR
0.33
9.84
2.07
15.90
3.37
13.77
16.57
4.75
ATTR
1,74
0.14
0.48
10.53
1.84
18.68*
4.10
15.34
19.21
5.07
COFFEE
1.23
0.04
0.24
11.05
2.08
17.90
4.02
15.11
19.27
5.48
COFFEE-NT
1.10
0.04
0.19*
10.52
2.02
17.04
3.92
14.77
19.35*
5.27
Yelp (Restaurant's 's Price Protected Attribute)
RAW
0.56
10.09
1.49
19.14
3.17
14.83
19.50
3.76
ADV
0.21
10.35
1.97
16.78
2.49
13.14
17.88
3.24
NORM
0.15
9.88
1.42
19.20
3.14
14.85
19.48
3.04
BT
0.30
10.56*
1.52
16.44
2.31
13.24
18.95
3.10
NATTR
0.19
10.38
2.23*
15.89
2.26
12.58
17.26
3.04
ATTR
1.58
0.42
0.50
10.08
1.85
18.92
3.09
14.72
19.74
3.22
COFFEE
0.93
0.17
0.10
10.25
1.75
18.34
2.92
14.33
19.49
3.70
COFFEE-NT
0.69*
0.09*
0.02
10.45
2.07
18.65
2.86
14.25
19.67
3.52


Reviews:
[{"review_id": "l0DzTk0BOg", "reviewer": "Reviewer_VQ9B", "summary": "", "questions": "A.\tLine 126: Technically, any numerical quality measure can be plugged into COFFEE; however, this paper lacks experiments with a second quality measure other than FeatCov, so it might be insufficient to claim the generality of COFFEE.\n\nB.\tLine 258: I wonder how much the user’s protected attribute is entangled with their preference and is implicitly encoded in the representations – are they significant enough? Possible experiments to do include measuring the percentage of entangling cases, and ablation studies on “Sec 4.1 Disentangled Attribute Representation”. \n\nC.\tLine 314: What if there are more than two attributes? Formula (3) seems to work for binary attributes only.\n\nD.\tLine 412: I’m not sure if PETER is still the SOTA model in 2023 – there are a few follow-up works (listed below) of PETER, which I guess are supposed to be superior to PETER. More experiments with those advanced models are needed to verify the effectiveness of COFFEE more completely. (a. https://aclanthology.org/2022.acl-long.20/; b. https://dl.acm.org/doi/abs/10.1145/3477495.3532039; c. https://dl.acm.org/doi/abs/10.1145/3523227.3546767; d. https://dl.acm.org/doi/abs/10.1145/3580488.)\n\nE.\tLine 460: Are there any experimental results that can support this claim?\n\nF.\tLine 515: Given that Figure 2 shows a strong positive correlation between FeatCov and human-evaluated quality measures, after adding $Q_{numOfToken}$, have authors verified the existence of that correlation – does it remain strong or become weak?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer VQ9B, \n\n\nThank you very much for your time and effort in reviewing our paper! We believe most of the concerns are due to lack of clarity and missing details that we decided not to include due to the page limit. Your questions also help us further clarify things and improve the paper. Below please find our detailed responses to your questions and we will revise the paper accordingly. We sincerely hope our responses can address your concerns and help you consider increasing the score. \n\n\n**Regarding generality of COFFEE**\n>Technically, any numerical quality measure can be plugged into COFFEE; however, this paper lacks experiments with a second quality measure other than FeatCov, so it might be insufficient to claim the generality of COFFEE.\n\n\nFirst of all, our primary objectives for fairness optimization revolve around enhancing human-evaluated quality criteria such as *informativeness*, *detailedness*, and *helpfulness* of explanations, as extensively discussed in Section 7. We ultimately opted for FeatCov because it demonstrates a strong correlation with these human-evaluated quality criteria, as evidenced in Figure 2.\n \nActually, we also conducted experiments using the length of explanations as another quality metric. Notably, in the Amazon datasets, we observed that generated explanations tend to be longer for male users compared to female users. Similarly, in the Yelp dataset, explanations are generally longer for higher-priced restaurants than their less pricey counterparts. To address fairness concerns related to explanation length, we incorporated Q_{numToken} into COFFEE for fairness optimization. The results obtained for the Amazon Games dataset are presented below.\n\n### Fairness on Length of explanations\n|         | Ind-CF  | Grp-CF | DDP   | BERT  |\n|---------|---------|--------|-------|-------|\n| RAW     | -       | -      | 5.84  | 14.14 |\n| ADV     | -       | -      | 3.04  | 12.84 |\n| NORM    | -       | -      | 5.67  | 14.08 |\n| BT      | -       | -      | 4.00  | 9.55  |\n| NATTR   | -       | -      | 3.24  | 12.72 |\n| ATTR    | 12.42   | 3.63   | 6.81  | 14.08 |\n| COFFEE  | 3.87    | 0.47   | 1.09  | 14.09 |\n\nAs illustrated, COFFEE successfully mitigated the bias in length of explanations while preserving the quality of the generated content, as assessed by BERTScore. Similar results are observed across the other two datasets. While we did not include these results initially, it's important to note that explanation length does not exhibit as strong a correlation with human evaluations as FeatCov. Nevertheless, to demonstrate the broad applicability and effectiveness of COFFEE, we will incorporate these results into the appendix as supplementary evidence of COFFEE's capabilities.\n\n\n**Regarding experiments on the entanglement of protected attribute with preference**\n>I wonder how much the user’s protected attribute is entangled with their preference and is implicitly encoded in the representations – are they significant enough? Possible experiments to do include measuring the percentage of entangling cases, and ablation studies on “Sec 4.1 Disentangled Attribute Representation”\n\n\nThank you for the suggestion. In Section 4.1, we employ a disentanglement approach where each user's representation is separated into two components: the attribute representation and the preference representation. This setup enables us to perform counterfactual inference by switching the attribute representation from one attribute value to another.\n\n\nFollowing this method, we analyzed how much changing a user’s learnt attribute representation will influence the FeatCov of generated explanations. Take the Amazon Games dataset for example, without imposing any fairness constraints, we observed that the average FeatCov change is 2.68 when altering the attribute representation. Furthermore, over 92% of users exhibit a FeatCov change greater than 1. This indicates that, in most cases, the attribute and preference representations are intertwined, and the disentanglement approach effectively separates these two aspects. In our revision, we’ll provide more comprehensive and detailed ablation studies to further elucidate these findings.\n\n\n**Regarding the cases with non-binary attributes**\n>What if there are more than two attributes? Formula (3) seems to work for binary attributes only.\n\nThank you for your question. We'd like to clarify that COFFEE is designed to accommodate non-binary attributes with any number of value choices, as explicitly mentioned in Section 4.1 and reiterated in footnote 3 after line 331. Specifically, the constraint outlined in Eq. (3) is articulated as $\\mathbb E[Q(Y_{A\\leftarrow a})] = E[Q(Y_{A\\leftarrow a’})], \\forall a’\\in\\mathcal A, a’ \\neq a$. Here, it's important to note that the set of attribute values $\\mathcal A$ is not limited to binary choices. We will further clarify this in the revision.\n\n\n**Regarding the use of PETER as SOTA for experiments**\n>I’m not sure if PETER is still the SOTA model in 2023 – there are a few follow-up works (listed below) of PETER, which I guess are supposed to be superior to PETER.\n\n\nWe will revise the claims accordingly. At the same time, it is worth noting that our goal in this paper is to develop a general framework for improving fairness that can be applied to most of existing explanation generation models, as long as the model is based on user/item representation learning and can be fine-tuned. To that end, we applied COFFEE to both a transformer-based model PETER and RNN-based model NRT (Table 8 in appendix) to demonstrate its generality and effectiveness for different types of models. COFFEE can also be applied to those newer models referenced by the reviewer, since they also depend on learned representations and can be fine-tuned. \n\n\n**Regarding the detailed bias results on Yelp**\n>Line 460: Are there any experimental results that can support this claim?\n\n\nYes, below are the detailed results supporting our claim in lines 460-464 and we will include them in the appendix. In particular, ground-truth explanations extracted from user reviews show the bias that users generally write detailed and more informative explanations with high FeatCov. When we train a raw PETER model on such data, it also generates higher FeatCov explanations for pricer restaurants than cheaper ones.\n### Average FeatCov of explanations generated for restaurants of different price tags\n| | $ | $$ | $$$ |\n|--------|----------|---------|---------|\n| Ground-Truth | 3.15 | 3.72 | 4.38 |\n| RAW PETER | 3.26 | 3.68 | 4.03 |\n\n\n**Regarding the correlation to human evaluation after adding Q_{numOfToken}**\n>Given that Figure 2 shows a strong positive correlation between FeatCov and human-evaluated quality measures, after adding Q_{numOfToken}, have authors verified the existence of that correlation – does it remain strong or become weak?\n\n\nThanks for the question. We confirmed that after adding Q_{numOfToken}, the correlation to human evaluated quality measures remains strong. Similar to Table 5 in appendix D, the table below shows the overall correlation results from Kruskal-Wallis H test between human evaluation and the automatic metric of Q_{sum} = FeatCov + Q_{numOfToken}. All H values are greater than 1 and p-values are much smaller than 0.05 in the test, indicating the significance of the correlations.\n\n\n### Kruskal-Wallis H test between Q_{sum} = Q_{feat} + Q_{numOfToken} and human evaluated quality measures\n| K-W H test |   | Infor        | Detail       | Help         |\n|------------|---------|--------------|--------------|--------------|\n| $H$ stats | $Q_{sum}$ | 19.16        | 29.47        | 13.87       |\n| $p$-value | $Q_{sum}$ | $7.3e^{-5}$  | $6.26e^{-6}$ | $7.7e^{-4}$ |\n\n\n\n**Regarding the presentation**\n\nThank you for pointing out the issues regarding our presentation. COFFEE is designed to be a general framework that applies to different personalized text generation (PTG) settings. Focusing on explanation generation does not impose any application-driven assumptions or specific adaptations of COFFEE specific to this setting. We only experimented with explanation generation settings mainly because it is a typical scenario for personalized text generation and widely studied with various datasets and models to experiment with. We will carefully re-examine the scope and revise the paper accordingly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThis paper is the first attempt at the quality fairness problem in explanation generation for recommendations.\n\n2.\tThe mathematics looks sound, and the experiments are extensive.\n\n3.\tThis paper is well-organized, and the writing is easy to follow.", "reasons_to_reject": "(Please check the questions for details)\n\n1.\tThe generality of the proposed framework appears to be limited.\n\n2.\tThe experiments lack ablation studies.\n\n3.\tSome parts of the experiments are incomplete.\n\n4.\tSome parts of the paper presentation can be improved.", "questions_to_authors": "A.\tLine 126: Technically, any numerical quality measure can be plugged into COFFEE; however, this paper lacks experiments with a second quality measure other than FeatCov, so it might be insufficient to claim the generality of COFFEE.\n\nB.\tLine 258: I wonder how much the user’s protected attribute is entangled with their preference and is implicitly encoded in the representations – are they significant enough? Possible experiments to do include measuring the percentage of entangling cases, and ablation studies on “Sec 4.1 Disentangled Attribute Representation”. \n\nC.\tLine 314: What if there are more than two attributes? Formula (3) seems to work for binary attributes only.\n\nD.\tLine 412: I’m not sure if PETER is still the SOTA model in 2023 – there are a few follow-up works (listed below) of PETER, which I guess are supposed to be superior to PETER. More experiments with those advanced models are needed to verify the effectiveness of COFFEE more completely. (a. https://aclanthology.org/2022.acl-long.20/; b. https://dl.acm.org/doi/abs/10.1145/3477495.3532039; c. https://dl.acm.org/doi/abs/10.1145/3523227.3546767; d. https://dl.acm.org/doi/abs/10.1145/3580488.)\n\nE.\tLine 460: Are there any experimental results that can support this claim?\n\nF.\tLine 515: Given that Figure 2 shows a strong positive correlation between FeatCov and human-evaluated quality measures, after adding $Q_{numOfToken}$, have authors verified the existence of that correlation – does it remain strong or become weak?", "paper_topic_and_main_contributions": "This paper studies the quality fairness problem in explanation generation for recommendations. Due to the bias of users’ protected attributes (e.g. gender) that exists in the training data, the quality of model-generated explanations may vary accordingly as attributes change. A general framework named COFFEE is proposed to achieve quality-measure-specific counterfactual fairness. In COFFEE, the user’s protected attribute value is disentangled from the user’s representation via adversarial training, and is instead modeled with a separate control token; the explanation generator is then trained according to a counterfactual fairness constraint applied to model-generated explanations, so that the explanation quality is encouraged to change minimally when the attribute value is altered counterfactually; that constraint is expressed as rewards for RL and the model is optimized via policy gradient. Experiments show the effectiveness of COFFEE in achieving counterfactual fairness while maintaining the utility of explanations.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "a. https://aclanthology.org/2022.acl-long.20/;\n\nb. https://dl.acm.org/doi/abs/10.1145/3477495.3532039;\n\nc. https://dl.acm.org/doi/abs/10.1145/3523227.3546767;\n\nd. https://dl.acm.org/doi/abs/10.1145/3580488.", "Typos_Grammar_Style_and_Presentation_Improvements": "I think this paper does not have much to do with causality, so it might be misleading to mention the causal perspective in Line 103. Similarly, as explanation generation for recommendations is only one of the various use cases of personalized text generation, overclaiming the research scope by mentioning PTG in the paper title and introduction might hurt the paper's readability."}}, {"review_id": "eVJUnI89bP", "reviewer": "Reviewer_yP6b", "summary": "", "questions": "N/A", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer yP6B, \n\nThank you very much for your efforts in reviewing our paper and sharing your comments and concerns! We believe that there are significant mis-understandings of the fairness literature and our method. Thus we kindly request that the reviewer can carefully check our rebuttal and the corresponding sections in the submission. Below please find detailed responses to your concerns. We sincerely hope you can increase the score if the concerns are addressed. Thanks! \n\n**Regarding the counterfactual inference (CI) module**\n\nWe would like to provide clarifications regarding some potential misunderstandings of the fairness notions and our proposed solution.\n\nFirst of all, not all fairness-aware methods are designed to remove sensitive attributes [7,8]. Most fairness works aim to improve specific fairness notions, such as group-wise demographic parity, individual fairness, and counterfactual fairness [2-7,12]. There are mainly three categories of fairness methods in literature, pre-, in-, and post-processing methods [4,7,8]. Neither pre- nor post processing methods require removing sensitive attributes. Only a small portion of in-processing methods propose to remove sensitive attribute information to learn attribute-independence representations [11,13], which, however, does not have guarantees in optimizing a specific fairness notion. In fact, the most popular in-processing method to achieve fairness is to impose a constraint or regularization corresponding to a specific fairness notion during optimization [1,2,3,10,12], which is also what our method COFFEE is designed for. \n\nIn the literature of Counterfactual Fairness (CF), removing sensitive information has been explored in some papers to achieve CF, which however does not follow the exact CF definition and thus does not have any guarantee in optimizing CF [5,10]. \n\nIn lines 262-272, we discussed the methods that simply remove sensitive attribute information from representations, and why we are taking a more controlled way to improve fairness. In particular, although directly removing sensitive information from representations can potentially improve fairness in general, it also blindly removes all desired or undesired characteristics from the sensitive attribute and detrimentally affects the utility and performance of these representations for downstream tasks. For example, when explaining why a user might like a movie, a male user and a female user may prefer different aspects/factors (related to their gender) of the movie being highlighted in the explanation. Thus the personalized explanations should be related to their gender, but they should not receive explanations of different quality (informativeness, detailedness, helpfulness, etc.) because of their gender. \n\nOur goal here is to make sure that the explanations are still personalized to their gender, but the *quality* of the explanations should not depend on their gender. Therefore, we are taking a more controlled approach in improving fairness through CI with respect to any specific quality measures of generated explanations (as formulated in Section 3), while maintaining the performance of personalized explanation generation. The advantage of our approach is also shown in the experiment results. In Table 2, compared to ADV which simply removes sensitive attribute information from representations, COFFEE achieves superior performance in improving fairness while maintaining high utility of the explanations. \n\nThe sensitive attribute is not exactly a direct input to the model for explanation generation, but is essentially a value token for manipulating the representations which are used for the generation. We use this sensitive attribute token to control how the representation is changed to reflect the attribute value change for CI, which is a commonly used method in controlled text generation (see lines 274-291). \nMore specifically, the manipulation of the sensitive attribute value is required for performing CI to study how the model output changes according to the attribute value change [1,6,10,11]. In our method, we need to manipulate the sensitive attribute value on the learned representations. To achieve this, in Section 4.1 (lines 293-306), we propose to first learn disentangled representations for each user $r_u^a = [r_a, r_u]$, which consist of two parts: $r_a$ for the attribute representation of value $a$, and $r_u$ for the preference representation that is orthogonal to $r_a$. When performing CI of $A \\leftarrow a’$ on user $u$ with true attribute value $a$, we swap this user’s attribute representation from $r_a$ to $r_a’$ to form $r_u^{A \\leftarrow a’} = [r_a’, r_u]$. This modified representation $r_u^{A \\leftarrow a’}$ is used to generate the counterfactual explanation and construct the CF constraint. Note that the input of the input sensitive attribute token is mainly used to indicate the CI performed on the user’s representation, which is used for explanation generation. \n\n[1] Huang, Po-Sen, et al. \"Reducing Sentiment Bias in Language Models via Counterfactual Evaluation.\" Findings of the Association for Computational Linguistics: EMNLP 2020. 2020. \\\n[2] Zemel, Rich, et al. \"Learning fair representations.\" International conference on machine learning. PMLR, 2013.\\\n[3] Jiang, Ray, et al. \"Wasserstein fair classification.\" Uncertainty in artificial intelligence. PMLR, 2020.\\\n[4] Du, Mengnan, et al. \"Fairness in deep learning: A computational perspective.\" IEEE Intelligent Systems 36.4 (2020): 25-34.\\\n[5] Kusner, Matt J., et al. \"Counterfactual fairness.\" Advances in neural information processing systems 30 (2017).\\\n[6] Hardt, Moritz, Eric Price, and Nati Srebro. \"Equality of opportunity in supervised learning.\" Advances in neural information processing systems 29 (2016).\\\n[7] Mehrabi, Ninareh, et al. \"A survey on bias and fairness in machine learning.\" ACM computing surveys (CSUR) 54.6 (2021): 1-35.\\\n[8] Li, Yunqi, et al. \"Fairness in Recommendation: Foundations, Methods and Applications.\" ACM Transactions on Intelligent Systems and Technology.\\\n[9] Garg, Sahaj, et al. \"Counterfactual fairness in text classification through robustness.\" Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. 2019.\\\n[10] Russell, Chris, et al. \"When worlds collide: integrating different counterfactual assumptions in fairness.\" Advances in neural information processing systems 30 (2017).\\\n[11] Bose, Avishek, and William Hamilton. \"Compositional fairness constraints for graph embeddings.\" International Conference on Machine Learning. PMLR, 2019.\\\n[12] Madras, David, et al. \"Learning adversarially fair and transferable representations.\" International Conference on Machine Learning. PMLR, 2018.\\\n[13] Elazar, Yanai, and Yoav Goldberg. \"Adversarial removal of demographic attributes from text data.\" arXiv preprint arXiv:1808.06640 (2018).\n\n**Regarding the generality claim in introduction section**\n\nOur proposed method COFFEE is general to address fairness issues in personalized text generation (PTG). Focusing on explanation generation does not impose application-driven assumptions or specific adaptations of our method specific to this setting. We focused on the experiments in explanation generation mainly because it is a typical and widely studied setting of PTG, and there exists various models and datasets to experiment with. \n\nCOFFEE can be easily applied to other PTG settings as well. For example, in personalized post generation, we can also apply COFFEE by learning disentangled representations (Section 4.1) of users and impose the counterfactual constraint for post generation model training (Section 4.2). \n\nWe thank the reviewer for the great suggestion. We will revise accordingly to further clarify this.\n\n**Regarding the experiment setup on Yelp dataset**\n\nWe consider the price range in Yelp restaurants because we observed that *the quality of generated explanations* are generally higher for pricer restaurants than for cheaper restaurants. Again, please note that we are not trying to remove the price information in generated explanations, but to make sure the quality of explanations (informativeness, detailedness, helpfulness) are agnostic to the price of restaurants, which otherwise will unfavor restaurants with a lower price. Customers can still choose restaurants according to the price, but the quality of system-generated explanations should not affect their decisions. In other words, the price range of restaurants is a sensitive attribute for *the quality of explanations* generated for different restaurants.  \n\n\nThanks again for your time reviewing our work! Please let us know if you have any further questions, and we are happy to discuss further.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The topic of counterfactual fairness in explainable recommendations is worth studying.\n2. Extensive experiments prove the effectiveness of the COFFEE model.\n3. Based on my experience, the disentangled learning module in the proposed model appears to be technically sound. I have confidence in the validity of the experimental results.\n", "reasons_to_reject": "1.\tMy biggest concern is the Counterfactual Inference (CI) module. As far as I know, all fairness-aware models are designed to remove sensitive attribute information, but here the proposed model takes a joint input of sensitive attributes and user-product IDs, which seems counterintuitive. \nAdditionally, I'm puzzled by the fact that existing research has shown that even models that do not input sensitive attributes can still contain sensitive information in the user representations. From a causal perspective, this would mean that sensitive attributes (such as gender) have a causal effect on user representation, and user representation has a causal effect on the output. Therefore, the explicit input of sensitive attributes in this paper should be understood as 'sensitive attributes have a direct causal effect on user representation, and both user representation and sensitive attributes have a direct causal effect on the output'. I believe that the edge 'sensitive attributes have a direct causal effect on user representation' objectively exists. Therefore, if we want to perform CI, simply modifying the input sensitive attribute value without considering changes on user representation would not truly constitute CI. \n2.\tGiven my doubts about the validity of counterfactual inference, I am also skeptical of the metrics used in the experimental section that involve CI.\n3.\tI also have concerns about the introduction section. The authors claim to have solved a problem in the field of personalized text generation, but in reality, the proposed model seems to be specific to the sub-field of generating natural language explanations for recommendations. It's important for the authors to explain the difference between these two areas or clarify why they chose to write it this way.\n4.\tI have some concerns regarding the experimental setup, particularly regarding why the price range ($, $$, $$$) of a restaurant is considered protected information in the Yelp dataset. In my experience, price is an essential factor to consider when evaluating a restaurant as it can determine its affordability. It would be helpful to have more information on why the authors consider the price range as protected information.\n", "questions_to_authors": "N/A", "paper_topic_and_main_contributions": "This paper addressed fairness in personalized explanation generation for explainable recommendations. The authors conducted an analysis of unfairness in personalized explanation generation and proposed a COFFEE model, which took a causal perspective to enforce fairness. The COFFEE model consisted of a counterfactual inference module, a disentangled learning module, and a policy learning module. Extensive experiments showed the effectiveness of the COFFEE model.\n\nWhile the topic of counterfactual fairness in explainable recommendation is worth studying, I have some concerns with this paper. Specifically, I have identified some incorrect statement in this manuscript, which make me unable to recommend it for acceptance.\n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "N/A"}, "extra_text": {}}, {"review_id": "zeSQS8MfJp", "reviewer": "Reviewer_iBGj", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer iBGj, \n\n\nThank you very much for your comments and suggestions for our paper! We believe that most of your concerns/questions can be addressed properly based on our user study presented in the paper. Below please find detailed responses to your concerns. We sincerely hope our responses can address your concerns. \n\n\n**Regarding the study of different interaction patterns with Amazon reviews**\n\n\nThank you for the insightful comment. We would like to provide some justification here. \n\nOur claims and motivations are supported by both empirical observations and comprehensive user studies. We first observed the different behavior patterns of Amazon reviews from male and female users, which are inherited by the model generated explanations that have higher FeatCov for male users than female users. To confirm whether such bias in explanations is harmful, we performed rigorous user studies (firstly introduced in Figure 2 and detailed in Section 7 and Appendix D) to examine how real humans evaluate the usefulness of explanations. The human evaluation results suggest that humans’ judgments of explanation quality are highly correlated with FeatCov (Figure 2), and the bias is also observed in human evaluation measures including informativeness, detailedness, helpfulness (Table 3). Thus we confirmed the harmfulness of such bias which motivated our research for fairness-aware explanation generation. The proposed COFFEE generally addresses a class of such fairness issues in the broad personalized text generation setting. As suggested in Section 6, we experimented with different datasets and models and showed that COFFEE is able to address fairness issues with respect to different sensitive attributes (besides gender). \n\n\n**Regarding using FeatCov as a quality measure**\n\n\nWe actually experimented with different automatic evaluation metrics for evaluating explanations, including FeatCov, length, grammar, redundancy, structure, etc. Then we correlated these metrics with human evaluations of explanation quality (informativeness, detailedness, helpfulness), and found FeatCov aligns well and highly correlated with human evaluations. Therefore, we chose FeatCov as an automatic quality evaluation measure for fairness improvement. Thank you for pointing it out. We will provide further discussion in our revision.\n\n\n**Regarding structural assumptions for counterfactual fairness**\n\n\nWe agree that achieving counterfactual fairness requires casual structure assumptions. In fact, in practical applications of counterfactual fairness (CF), we usually approximate CF by imposing a fairness constraint in the optimization objective (see Section 3.2, 3.3 in [1]). We followed this recipe in developing CF for explanation generation as in Section 3 and Section 4.2 in our paper. Thank you for the suggestion, and we will make this clearer in the revision of the paper.\n\n\n[1] Russell, Chris, et al. \"When worlds collide: integrating different counterfactual assumptions in fairness.\" Advances in neural information processing systems 30 (2017).\n\n\n**Regarding other Presentation Improvements**\n\nThank you for the great suggestions. We will revise accordingly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper provides a very interesting and tractable structural model of counterfactual fairness in recommendations. Technically, the use of disentangled representations and policy learning are nice contributions. ", "reasons_to_reject": "I really like the theoretical part of this paper, but the application is not great.\n\nThis paper shows that men and women are different in how they interact with Amazon reviews, which results in differences in the features of  recommendations presented. This is immediately interpreted as harmful to women users, and so an extremely invasive system is proposed to try to give men and women identical recommendations. \n\nIs FeatCov really quality? That is not justified nearly enough. Do men like a higher FeatCov than women? That possibility is not even entertained. \n\nFinally, counterfactual fairness relies on strong structural assumptions. Those are not justified well in this paper.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper investigates the fairness of Personalized Text Generation (PTG) in the context of personalized explanation generation for recommendations. The authors identify that bias inherent in user-written text, often used for PTG model training, can inadvertently associate different levels of linguistic quality with users' protected attributes, leading to unfair treatment. To address this, they introduce a general framework to achieve measure-specific counterfactual fairness in explanation generation. The effectiveness of their method is demonstrated through extensive experiments and human evaluations.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Figure 1 needs to fix the Y axis scale so you can see the difference between the left and right plot.  \n\nNot every paper needs a cute acronym like COFFEE."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper examines the issue of potential bias and unfairness in personalized text generation models. Specifically, it looks at personalized explanation generation for recommendations. The authors find that biases in the user-written training data can lead models to associate different levels of linguistic quality with different user attributes. This results in unfair treatment when serving users with different protected attributes. To address this problem, the paper proposes a new framework called COFFEE to optimize counterfactual fairness constraints during training. The key ideas are disentangling user attribute representations and using policy learning to impose fairness rewards.

The proposed framework COFFEE shows promise in mitigating quality disparities for personalized explanation generation. In particular, experiments on explanation generation datasets demonstrate the system's ability to reduce quality disparities across user attributes. 
Looking at the reviews, the reviewers generally acknowledge the importance of studying counterfactual fairness for personalized text generation, particularly in the area of explanation systems. They find the technical contributions of disentangled representations and policy learning to be novel and promising. However, concerns exist around justifying quality metrics as proxies for human preferences and quantifying the harmfulness of observed disparities. The reviewers consistently seek more evidence that differential quality is inherently problematic rather than simply representing user preference differences. Additionally, two reviewers raise doubts about assumptions in the counterfactual inference module, suggesting sensitive attributes may not be fully disentangled from representations by removing them from the input. The framing also overstates the generalizability of the approach beyond the specific explanation generation setting based on the experiments shown. While the reviewers are in agreement that the paper is technically sound overall, there is a general recommendation for authors to enhance the problem formulation, making explicit the required assumptions, and situating the work within the broader literature on algorithmic fairness. In my opinion, the authors have provided satisfactory clarification on most technical concerns, which equally address my own worries about the work. The authors also demonstrated a sufficient understanding of general fairness literature and methodology. I think the technical approach appears novel and promising.

In summary, the paper tackles an important emerging problem at the intersection of fairness, personalization, and natural language generation. The initial results are positive, but further analysis of the assumptions and framework generalizability would strengthen the contribution.
Decision: Accept-Main