SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Erik Miehling
Rahul Nair
Elizabeth Daly
Karthikeyan Natesan Ramamurthy
Robert Redmond
erik.miehling@ibm.com
{rahul.nair,elizabeth.daly}@ie.ibm.com
{knatesa,rredmond}@us.ibm.com
Cookies are designed to enable more accurate identification and tracking of user behavior, in turn allowing for more personalized ads and better performing ad campaigns. Given the additional information that is recorded, questions related to privacy and fairness naturally arise. How does a user's consent decision influence how much the system can learn about their demographic and tastes? Is the impact of a user's consent decision on the recommender system's ability to learn about their latent attributes uniform across demographics? We investigate these questions in the context of an engagement-driven recommender system using simulation. We empirically demonstrate that when consent rates exhibit demographic-dependence, user consent has a disparate impact on the recommender agent's ability to estimate users' latent attributes. In particular, we find that when consent rates are demographic-dependent, a user disagreeing to share their cookie may counterintuitively cause the recommender agent to know more about the user than if the user agreed to share their cookie. Furthermore, the gap in base consent rates across demographics serves as an amplifier: users from the lower consent rate demographic who provide consent generally experience higher estimation errors than the same users from the higher consent rate demographic, and conversely for users who choose to withhold consent, with these differences increasing in consent rate gap. We discuss the need for new notions of fairness that encourage consistency between a user's privacy decisions and the system's ability to estimate their latent attributes.
Increased regulation surrounding cookies has emerged in recent years in response to growing concerns over online privacy and data protection. In the EU, cookie policy is driven primarily by the General Data Protection Regulation (13) and the ePrivacy Directive (10) which dictate that if a website wishes to use cookies to identify and track users, beyond "strictly necessary cookies," then consent must be explicitly obtained from the user. There is currently no federal law regulating the use of cookies in the US, however, several states have imposed regulation that requires websites to disclose their data collection practices to users (5, 40).
Marketers use cookies to narrow in on target audiences most likely to buy their products, primarily via improved user tracking and enhanced personalization. The actual mechanisms for obtaining user consent can introduce some additional fairness and privacy concerns. Many cookie consent interfaces are designed such that web content is at least partially obscured until the user makes a
consent decision. These interfaces often exhibit deceptive design patterns (38, 34, 15, 16) – design practices that incentivize users to make choices that lower their privacy. While some websites do offer a clear “reject all” button, many present the user with a decision between a simple “I agree” or “accept all” and a much less immediate choice of “manage preferences.” This leads to many users agreeing to cookie tracking simply out of convenience (17). Furthermore, and an observation that forms the basis of the current paper, evidence suggests that users exhibit inherent differences in their likelihood of agreeing to cookie tracking. A recent study by YouGov (45) revealed that a user’s consent rate is influenced by their age and culture/geographical location. Of the markets surveyed in the study, users ranged from  $64\%$  agree (Poland) to  $32\%$  agree (US), with older individuals being less likely to provide consent than younger individuals, perhaps partially due to their overall lowered trust in tech companies when it comes to their personal data (46).
In this paper, we investigate how demographic-dependent consent rates impact the ability of the recommender system to learn users' latent attributes and offer targeted recommendations. The main finding of our paper is that when user consent rates differ by demographics, the recommender system possesses disparate estimation accuracies across users in different demographics. The finding is based on a simple model of recommendations where, prior to the interaction, users provide cookie consent according to demographic-dependent probabilities. Users' consent decisions, along with the cookie for those who have provided consent, allow the recommender agent to form refined beliefs on users' demographics and preferences. Through sequential interaction with the users, the agent learns to personalize content to maximize engagement. We empirically illustrate the following:
1) Disparate impact of consent: Under demographic-dependent consent rates, users' consent decisions can have a disparate impact on the recommender agent's estimation accuracy of users' latent attributes. In particular, we find that withholding consent can lead to lower estimation errors for users in the lower consent rate population. In other words, a recommender system may know more about a person who opts not be tracked compared to those who willingly provide tracking information. Additionally, users from a population with a low consent rate who provide consent experience higher estimation errors than the same users from a high consent rate population.
2) Amplification effects: The difference in consent rates across cohorts serves as an amplifier for the above effects, with the disparities in estimation errors across cohorts increasing in the consent rate gap.
Note that our model is simple: ads are described by a single feature, the user pool is fixed, and user affinities/preferences do not change over time. This simplicity is intentional, with the goal being to isolate and understand the effects of specific model aspects (i.e., impact of different user consent rates) without the additional noise and confounding factors present in real-world settings. The observations on disparate impact that emerge from our setting should serve as an indicator that the same negative results may also exist in more complex recommender systems. Additionally, given the fundamental nature of the observed disparities, we believe that the search for mitigation strategies in this simplified setting can provide useful insights for the design of more complex systems.
Recommender systems. The main challenge in recommender systems is making recommendations when only a sparse set of preference data is available. Broadly, recommender systems address this missingness via two approaches: content-based filtering, and collaborative filtering. Content-based filtering methods use known information about users and items to suggest content. For example, a content-based movie recommender system may compare known features of a movie (degree of humor, action, drama) with known user preferences to suggest movies to users that yield the greatest similarity. Collaborative filtering, on the other hand aims to learn these relevant features based on patterns in the observed preference/response data. Items in a collaborative filtering-based system are recommended to users based on what content other users with similar behavior have consumed (hence the term collaborative).
<sup>1</sup>Hybrid approaches, as well as methods based that leverage more sophisticated learning methods, are also popular. For a comprehensive review see (4).
A popular collaborative-filtering algorithm is matrix factorization. Matrix factorization asserts that user data is not missing uniformly at random, but rather according to some low rank structure of which it aims to discover. The learned factors can then be used to infer unobserved ratings for other user-item pairs, in turn driving recommendations. Associating each user with factor  $u_{i} \in \mathbb{R}^{k}$  and each ad with factor  $v_{a} \in \mathbb{R}^{k}$ , latent factor estimates  $(\hat{\pmb{u}},\hat{\pmb{v}}) = ((u_i)_i,(v_a)_a)$  are obtained under standard matrix factorization via solution of a regularized loss-minimization on the known user responses
$$
(\hat {\boldsymbol {u}}, \hat {\boldsymbol {v}}) = \underset {(\boldsymbol {u}, \boldsymbol {v})} {\operatorname {a r g m i n}} \left[ \sum_ {(i, a) \in \mathcal {R}} (u _ {i} ^ {\top} v _ {a} - r _ {i, a}) ^ {2} + \lambda \left(\sum_ {i} \| u _ {i} \| _ {2} + \sum_ {a} \| v _ {a} \| _ {2}\right) \right]
$$
where  $\mathcal{R}$  is the set of user-item pairs  $(i, a)$  where a response  $r_{i,a}$  has been recorded. A variety of modifications of standard matrix factorization exist (25), including incorporating ranking biases, user and item side information, varying confidence weights, and temporal effects. The above optimization is typically carried out on a mix of both historical data and new interaction data, with the latter data incorporated periodically via retraining.
Cookies, consent, and behavioral advertising. Cookies are small pieces of data that are stored on a user's device with the specific purpose ranging from storing login credentials (first-party cookies) to tracking user behavior for targeted advertising (third-party cookies). Given their capability of enabling inference of potentially sensitive user information (9, 39), regulation has emerged to increase transparency around their use. While specific laws vary, many countries now require web publishers to present some form of banner or pop-up either informing the user that information is being recorded or requesting the user to indicate their cookie consent decision prior to browsing the page.
Behavioral advertising describes the practice of using learned user behavior (e.g., user preferences) to make personalized recommendations. Cookies help to facilitate inference of this information via tracking user behavior across websites (e.g., search history, clicks, purchases). The end goal of behavioral advertising is to increase relevancy of recommendations, in turn increasing user engagement and the return of the ad campaign.
Algorithmic feedback loops. Much of the analysis of algorithmic feedback loops in the literature is centered on recommender systems and the trade-off between recommendation accuracy and topic diversity. It is generally accepted that as the recommender system learns to generate high engagement recommendations, topic diversity suffers which in turn may cause filter bubbles and echo chambers to emerge (51, 26, 33, 22, 6, 27, 24, 11, 18, 30, 50, 14). While a variety of mitigation techniques have been proposed – including techniques to identify and remove these effects (37), slow degeneracy (24), improve user heterogeneity (43), and disentangle user interest from user conformity (49) – the question of how to best balance accuracy with diversity, and addressing broader fairness concerns, is still very much an active area of research.
The above papers illustrate the many potential downsides of engagement-driven recommendation and collaborative filtering. To the best of the authors' knowledge, our paper is the first to analyze the impact of consent rates on the dynamics of the recommendation process.
Privacy, fairness, and bias considerations in recommender systems. Given that recommender systems are designed to personalize content to users, questions related to privacy, fairness, and bias naturally arise (23, 12, 7). Many notions of recommender system bias have been studied in the literature including participation inequality where system usage varies across a user's attributes (e.g., gender, race, language, etc.), selection bias where the users' choice behavior leads to non-representative item feedback, conformity bias where users in similar groups act similarly, exposure bias where suppliers/items exhibit fundamentally different levels of visibility, and popularity bias where popular items are recommended more often than long tail items, among various others. Furthermore, it is widely recognized that the inherent feedback nature of recommender systems amplifies these biases (6, 30).
Research on the privacy and fairness of recommender systems aims to gain a deeper understanding of how the structure of the recommender system influences these biases and subsequently propose mitigation strategies via modified recommendation algorithms, with a large thread of research concerning the design of privacy-preserving recommender systems (1, 31, 47, 3). Most related to our
paper are the structural questions of characterizing the ability of the recommender system to recover protected attributes from ratings of both a homogeneous user pool (42) and a mix of public/private users (44). Notably, (44) demonstrates that only a small number of public users (users willing to share preference information) with a large number of ratings in a pool of private users (users with hidden preferences) is sufficient to produce accurate system-wide recommendations. In contrast, our paper studies how inherent differences in user consent rates influences the system's ability to estimate users' latent attributes across groups.
Simulators for recommender systems. Given the complexity of the interactions between users and the learning behavior of the agent, simulators have become an increasingly popular tool for understanding the dynamics of recommender systems. Agent-based modeling allows for simulation of fine-grained user interactions, facilitating useful insights without running costly field tests. A variety of open-source recommender system simulators have emerged in recent years, namely RecoGym (35), RecSim (21), PyRecGym (36), Surprise (20), RecSim NG (32), and others (48), (8). The simulator developed for purposes of this study augments RecSim with the ability for the learning agent to maintain asymmetric (Bayesian) uncertainty over users.
4 A Simple Recommendation Model with Cookie Consent
Consider a recommendation environment consisting of a recommender, termed the agent, sequentially interacting with a fixed population of  $n$  individuals or users (see Fig. 1). Before the interaction begins, users make consent decisions according to known cohort-dependent probabilities. The agent uses the consent decisions and revealed cookies to form refined (interim) beliefs on the users' cohorts, which in turn guide recommendations. The agent is periodically retrained using the updated history of recommendation-response pairs, with the overall goal of recommending content that maximizes engagement.
Figure 1: The recommendation process. The advertisement sampler generates the ad pool (assumed to be refreshed in each round  $t$ ). The user sampler initializes both latent attributes of users and consent decisions. In each round, the agent generates recommendations (ads) and records user responses ( clicks).
Advertisement model. Each ad is described by a single topic feature  $\tau \in \mathcal{T} = \{\tau^1,\dots ,\tau^m\}$ , where  $m$  is the number of possible ad topics. In each round  $t$ , a subset of  $l < m$  ad topics are sampled to form the current set of recommendable ads  $\mathcal{A}_t$ , termed the ad pool.
User model. Each user  $i \in [n]$  is described by three features: a cookie  $\phi_i \in \Phi = \{\phi^1, \dots, \phi^c\}$ , a demographic or cohort  $\theta_i \in \Theta = \{\theta^1, \dots, \theta^d\}$ , and a vector of topic affinities  $\alpha_i \in \mathbb{R}_+^m$ . Each user's cookie-cohort pair is drawn from a known joint prior  $\mu \in \Delta(\Phi \times \Theta)$ , i.e., knowledge of a user's cookie is at least partially informative for its cohort. Consent decisions are dictated by cohort-dependent probabilities  $q_\theta$ ,  $\theta \in \Theta$ , with each user  $i$  in cohort  $\theta$  revealing its cookie with the agent according to  $X_i \sim \mathrm{Bernoulli}(q_\theta)$ . Topic affinities are generated according to log-normal distributions, with each user  $i$  from a given cohort  $\theta$  possessing affinities  $\alpha_i \sim \mathrm{LogNormal}(\rho_\theta, \sigma_\theta^2)$ , where  $\rho_\theta \in \mathbb{R}^m$  and  $\sigma_\theta \in \mathbb{R}_+^m$  are the cohort-dependent means and standard deviations of  $\alpha_i$ 's natural logarithm.
Note that we model every user as possessing an underlying cookie, with the agent only being aware of the user's cookie if the user decides to share, i.e.,  $x_{i} = 1$ .
In each round  $t$ , each user is faced with a single ad and makes a binary decision to either click or not click. Specifically, when user  $i$  is recommended an ad  $a$ , the user first scores the ad via the utility function  $u(a; \alpha_i) = \alpha_i^\top \mathbb{1}_{\tau_a}$  where  $\mathbb{1}_{\tau_a} \in \mathbb{R}^m$  is the indicator vector on topics (a vector of zeros with a one in location of the topic of ad  $a$ ). User  $i$ 's choice to click on  $a$ , denoted by  $c_{i,a} \in \{0,1\}$ , is stochastic and is dictated by  $c_{i,a} \sim \text{Bernoulli}(p_{i,a})$ , where  $p_{i,a}$  is the click probability given by the following logit model
$$
p _ {i, a} = \frac {\exp (u (a ; \alpha_ {i}))}{\exp (u _ {0}) + \exp (u (a ; \alpha_ {i}))} \tag {1}
$$
where  $u_{0}$  is the no-click-mass used to model the possibility of the user not clicking on  $a$ , assumed to be homogenous across users.
Recommender agent. The agent faces a fundamental trade-off between recommending high-engagement content and learning more about users' tastes. Our recommendation model combines two foundational models from the literature, namely incorporation of confidence weights (19, 25) into an online matrix factorization procedure (41). Online matrix factorization addresses the conflicting objectives of the agent - exploration to improve estimates of latent factors, and exploitation of known high engagement content - by interleaving estimation of the latent factors with specification of recommendations. Including confidence weights allow us to model the agent's heterogeneous uncertainty on users due to their consent decisions and personalized recommendations. Note that while this is a stylized recommender model, it possesses the main features seen in more complex systems, specifically online learning and heterogeneous uncertainty across users.
Formally, the recommendation process evolves as follows. Given the agent's prior belief,  $\mu \in \Delta (\Phi \times \Theta)$ , the users' consent decisions are used to form interim cohort beliefs  $\tilde{\mu}_i\in \Delta (\Theta)$  for each user  $i$ . If user  $i$  gives consent  $(x_{i} = 1)$  then the agent is informed of the user's cookie,  $\phi_i$ , and its updated belief of the user's cohort is formed as  $\tilde{\mu}_i(\vartheta) = p(\vartheta \mid x_i = 1,\phi_i) = \frac{q_\vartheta\mu(\phi_i,\vartheta)}{\sum_{\vartheta'}q_{\vartheta'}\mu(\phi_i,\vartheta')}$  for each  $\vartheta \in \Theta$ . If user  $i$  does not provide consent  $(x_{i} = 0)$ , no cookie information is revealed and the agent's belief of the user's cohort is given by  $\tilde{\mu}_i(\vartheta) = p(\vartheta \mid x_i = 0) = \frac{(1 - q_\vartheta)\sum_\varphi\mu(\varphi,\vartheta)}{\sum_{\vartheta'}(1 - q_{\vartheta'})\sum_{\varphi'}\mu(\varphi',\vartheta')}$ ,  $\vartheta \in \Theta$ . Interim beliefs  $\tilde{\mu} = (\tilde{\mu}_1,\ldots ,\tilde{\mu}_n)$  are used to form beliefs  $\mu_0 = (\mu_{0,1},\dots,\mu_{0,n})$  via an offline response set  $\mathcal{L}_0$  of recommendation-response pairs of the form  $\{(a_i,c_{i,a})\}$  across users.
The recommender model at round  $t$  is represented by a pair of user-ad latent factor estimates  $(\hat{\pmb{u}}_t, \hat{\pmb{v}}_t)$ . Recommendations are generated via an  $\varepsilon$ -greedy bandit: with probability  $\varepsilon$ , the ad recommended to user  $i$  in round  $t$ , denoted by  $a_{i,t}$ , is chosen uniformly at random,  $a_{i,t} \sim \mathrm{U}(\mathcal{A}_t)$ , and with probability  $1 - \varepsilon$ , the recommendation is the ad in the current ad pool with the highest estimated value  $a_{i,t} = \operatorname{argmax}_{a \in \mathcal{A}_t} \hat{u}_{i,t}^\top \hat{v}_{a,t}$ . Responses to the recommendations are appended to the response set,  $\mathcal{L}_{t+1} = \mathcal{L}_t \cup \{(a_i, c_{i,a}) | i \in [n], a_i \in [m]\}$ , and are used to maintain a cumulative count of clicks  $r_{i,a,t}$  across user-ad pairs.
Retraining consists of updating cohort beliefs and latent factor estimates using recommendation-response pairs, and is performed every  $T_{b}$  rounds. If  $t$  is a retraining round, cohort beliefs are first updated according to a Bayesian update  $\mu_{i,t} = f(\mu_{i,t-1}, \mathcal{L}_{t} \setminus \mathcal{L}_{t-T_{b}})$  where  $\mathcal{L}_{t} \setminus \mathcal{L}_{t-T_{b}}$  is the set of responses since the previous retraining round. The heterogeneous beliefs on user cohorts give rise to a weighted procedure. For a given set of cohort beliefs  $\mu_{t}$ , the agent computes confidence weights  $\bar{w}_{t} = (\bar{w}_{i,a,t})_{i,a}$  as the expected probability for seeing the current response counts. Specifically, each weight  $\bar{w}_{i,a,t}$  is the expected binomial probability, given  $\mu_{t}$ , for seeing  $r_{i,a,t}$  defined as
$$
\bar {w} _ {i, a, t} = \mathbb {E} _ {\vartheta_ {i} \sim \mu_ {i, t}} \left[ p _ {t} \left(I _ {i, a, t}, r _ {i, a, t}, \vartheta_ {i}\right) \right]
$$
where  $p_t(I, r, \vartheta)$  is the binomial probability at round  $t$  given impressions  $I$ , positive response counts  $r$ , and cohort  $\vartheta$ . Updated latent factor estimates are computed via a confidence-weighted matrix factorization procedure as
$$
\left(\hat {\boldsymbol {u}} _ {t}, \hat {\boldsymbol {v}} _ {t}\right) = \underset {(u, v) \in \mathcal {U} \times \mathcal {V}} {\operatorname {a r g m i n}} \left[ \sum_ {\left(a _ {i}, c _ {i, a}\right) \in \mathcal {L} _ {t}} \bar {w} _ {i, a, t} \left(u _ {i} ^ {\top} v _ {a} - r _ {i, a, t}\right) ^ {2} + \lambda \left(\sum_ {i \in [ n ]} \| u _ {i} \| _ {2} + \sum_ {a \in [ m ]} \| v _ {a} \| _ {2}\right) \right] \tag {2}
$$
where  $\mathcal{U} = \{(u_1, \ldots, u_n) \mid u_i \in \mathbb{R}^k, i \in [n]\}$ ,  $\mathcal{V} = \{(v_1, \ldots, v_m) \mid v_a \in \mathbb{R}^k, a \in [m]\}$  are the latent factor spaces,  $\mathcal{L}_t$  is the current history of responses, and  $\lambda > 0$  is a regularization weight. If  $t$  is not a retraining round, then cohort beliefs and latent factor estimates are propagated as is from the previous round. Expressions for the Bayesian updates and expected binomial probabilities can be found in Appendix A, with pseudocode of the recommendation process in Appendix B.
The following experiments study the dynamics of the recommendation process described in Section 4 particularly as it relates to users' consent rates. Empirical results were obtained via a simulator based on RecSim (21). Full details of the simulator and the experimental setup can be found in the supplementary material (Appendix C). Base model parameters assumed throughout this section are: number of users  $n = 1000$ , number of ads  $m = 200$ , ad pool size  $l = 50$ , and number of cohorts  $d = 2$ . Sensitivity analyses can be found in Appendix D.
The following notation/terminology will be used throughout this section to aid explanations. Let  $N_1^\vartheta = \{i \in [n] \mid x_i = 1, \theta_i = \vartheta\}$  denote the set of users in cohort  $\vartheta$  who have decided to share their cookie, with  $N_0^\vartheta$  defined analogously. We refer to  $N_1^\vartheta$  as the consent group (in cohort  $\vartheta$ ) and  $N_0^\vartheta$  as the non-consent group. Estimation errors on cohorts for each group are quantified by log loss, i.e., for a set of users  $N$  in group  $(\vartheta, x)$ , the (average) log loss is denoted by  $L_x^\vartheta(\mu) = -\frac{1}{|N|} \sum_{i \in N} \sum_{\vartheta \in \Theta} \mathbf{1}(\theta_i = \vartheta) \log(\mu_i(\vartheta))$ . Estimation errors on topic affinities are given by MSE.
5.1 Impact of consent on estimation accuracy