SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The following set of experiments investigates the impact of cohort-dependent consent rates on the agent's estimates of the users' cohorts and topic affinities. Given users' consent decisions, and cookies for those who decided to provide consent, the agent forms interim beliefs on each user,  $\tilde{\mu}_i$ ,  $i \in [n]$ , as per the model description in Section 4. Fig. 2 illustrates the impact of these consent decisions on cohort and affinity estimation errors, for  $\Theta = \{\theta, \theta'\}$ , in two cases: homogeneous consent rates (Fig. 2(a)) and heterogeneous consent rates (Fig. 2(b)).
(a) Homogeneous consent,  $q_{\theta} = q_{\theta^{\prime}} = 0.5$ .
Fig. 2(a) illustrates that when users' consent rates are identical across cohorts, estimation errors for the non-consent group  $N_0^\theta$  are higher than the errors for the consent group  $N_1^\theta$  regardless of their cohort  $\theta$ .
(b) Heterogeneous consent,  $q_{\theta} = 0.25$ $q_{\theta^{\prime}} = 0.75$
Figure 2: Estimation errors over training rounds  $t$  ( $T_{b} = 1$ ) as a function of consent rates under binary cookie space  $\Phi = \{\phi, \phi'\}$  and symmetric prior  $\mu(\phi, \theta) = \mu(\phi', \theta') = 0.4$ .
As seen in Fig. 2(b), this intuitive result breaks down when users' consent rates differ across cohorts. While withholding consent leads to a higher estimation error for the higher consent rate group,  $\theta'$ , withholding consent can lead to a lower estimation error for the lower consent rate group,  $\theta$ . This inversion effect persists for the lower consent rate population for any sufficiently large separation of consent rates across cohorts, as illustrated by Fig. 3. These cohort-dependent effects of consent are summarized by Observation 1.
Observation 1 (Comparison within cohorts). Let  $\Theta = \{\theta, \theta'\}$  and let  $\mu$  be any non-fully informative prior. If  $q_{\theta} = q_{\theta'}$  (consent rates are independent of cohort) then non-consent leads to a higher estimation error than consent. If  $q_{\theta} < q_{\theta'}$  (consent rates are cohort-dependent with users in cohort  $\theta$  providing consent as a lower rate than users in cohort  $\theta'$ ), then:
a. There exists a (prior-dependent) constant  $\delta_{\mu}$  such that for  $q_{\theta^{\prime}} - q_{\theta} > \delta_{\mu}$ , the non-consent group in cohort  $\theta$ ,  $N_0^\theta$ , experiences lower estimation errors than the consent group,  $N_1^\theta$ .
b. The non-consent group in cohort  $\theta'$ ,  $N_0^{\theta'}$ , experiences higher estimation errors than the consent group,  $N_1^{\theta'}$ .
Observation 1 holds for any non-fully informative prior, meaning that there exists at least one cookie that does not reveal the user's cohort with certainty - a property that is almost certainly satisfied in practical (i.e., noisy) recommender systems.
For partially informative priors (each cookie only partially reveals the user's cohort), the statement of the above observation can be understood by looking at the agent's sources of information. Consider the following signals: i) the cookie value in the event that the user provided consent and, ii) under heterogeneous consent rates, the consent decision itself. The interplay between these two sources of information means that, under some priors, the act of a user withholding consent can itself be more informative than providing consent and revealing their cookie. Fig. 3 illustrates the estimation errors for different consent rate regimes under symmetric prior  $\mu (\theta ,\phi) = \mu (\theta^{\prime},\phi^{\prime}) = 0.45$
(a) Case:  $q_{\theta} < q_{\theta^{\prime}} < 0.5$
Figure 3: Impact of consent versus non-consent on the agent's cohort estimation error in each cohort for various consent rate regimes under symmetric prior  $\mu (\phi ,\theta) = \mu (\phi^{\prime},\theta^{\prime}) = 0.45$
(b) Case:  $q_{\theta} < 0.5 < q_{\theta'}$
(c) Case:  $0.5 < q_{\theta} < q_{\theta'}$
The above figure, specifically Fig. 3(a), illustrates that under a very informative prior, a small gap in consent rates,  $(q_{\theta}, q_{\theta'}) = (0.2, 0.4)$ , yields a consent signal that contains relatively low informativeness compared to the revealed cookie value, causing consent to yield lower estimation errors for both cohorts. However, as the absolute consent rates increase in both cohorts, this same consent rate gap (compare Figs. 3(a) and (c)) yields a consent signal that is sufficiently informative to yield lower estimation errors for withholding consent for the lower consent rate group.
Generally the larger the gap in consent rates, the more information the consent signal carries for the user's cohort. Apart from very informative priors, the consent signal carries sufficiently rich information to yield lower estimation errors for withholding consent for the lower consent rate population even for small gaps in the consent rates.
Observation 1 compares the impact of consent decisions within a given cohort under homogeneous and heterogeneous consent rates. A comparison of the impact of the same consent decision across users in different cohorts is summarized by the following observation.
Observation 2 (Comparison across cohorts). Let  $\Theta = \{\theta, \theta'\}$  and let  $\mu$  be any partially informative prior. If  $q_{\theta} = q_{\theta'}'$ , then the consent group experiences the same estimation errors regardless of cohort (similarly for non-consent). If  $q_{\theta} < q_{\theta'}$ , then:
a. There exists a (prior-dependent) constant  $\eta_{\mu}$  such that for  $q_{\theta^{\prime}} - q_{\theta} > \eta_{\mu}$ , the non-consent group in cohort  $\theta^{\prime}$ ,  $N_0^{\theta'}$ , experiences higher estimation errors than the non-consent group in cohort  $\theta$ ,  $N_0^\theta$ .
b. The consent group in cohort  $\theta^{\prime}$ ,  $N_{1}^{\theta^{\prime}}$ , experiences lower estimation errors than the consent group in cohort  $\theta$ ,  $N_{1}^{\theta}$ .
5.2 Amplification effects
The following experiments investigate how the disparities observed in (O.1) and (O.2) are influenced by the relative consent rates across cohorts. For  $\Theta = \{\theta, \theta'\}$ , define  $\delta_t^\vartheta = \bar{L}_0^\vartheta(\mu_t) - \bar{L}_1^\vartheta(\mu_t)$  as the difference in mean cohort errors between non-consent and consent for users in cohort  $\vartheta \in \Theta$ . Additionally, define  $\gamma_t^x = \bar{L}_x^{\theta'}(\mu_t) - \bar{L}_x^\theta(\mu_t)$  as the difference in mean cohort errors for consent decision  $x$ .
In words, the quantities  $\delta_t^\vartheta$ ,  $\vartheta \in \Theta$ , represent how much worse the agent's estimation error is under non-consent versus consent for users in cohort  $\vartheta$ . The quantity  $\gamma_t^0$  (resp.  $\gamma_t^1$ ) represents how much higher the agent's estimation error is for a user who withheld (resp. provided) consent if they belonged to cohort  $\theta'$  versus if they belonged to cohort  $\theta$ . Figs. 4 and 5 illustrate how these quantities differ as a function of consent rates for a given number of training rounds ( $\tau = 10$ ).
(a) Cohort  $\theta$ $\delta_{\tau}^{\theta}$
(b) Cohort  $\theta^{\prime}$ $\delta_{\tau}^{\theta '}$
Figure 4: Relative errors for non-consent versus consent as a function of consent rates  $(q_{\theta}, q_{\theta'})$  for each cohort and fixed number of training rounds  $\tau = 10$ .
(a) Non-consent group,  $\gamma_{\tau}^{0}$ .
Figure 5: Relative errors for a given consent decision across cohorts as a function of consent rates  $(q_{\theta}, q_{\theta'})$  for fixed number of training rounds  $\tau = 10$ .
(b)Consent group,  $\gamma_{\tau}^{1}$
Fig. 4 illustrates that the relative loss in accuracy for non-consent (compared to consent) is greatest in cohort  $\theta$  when  $q_{\theta}$  is minimal and  $q_{\theta'}$  is maximal, with the opposite effects observed in cohort  $\theta'$ . Fig. 5 illustrates that the users who withhold consent are on average impacted greater by a given gap in consent rates than users who consent, e.g., compare the magnitude of the values at  $(q_{\theta}, q_{\theta'}) = (0.25, 0.75)$  for non-consent versus consent.
In summary, the gap in consent rates serves as an amplifier for an individual user's consent decision. Users from the higher consent rate cohort who withhold consent experience higher estimation errors than the same users from the lower consent rate cohort, with the opposite holding for users in the consent group.
Our results demonstrate that even in a simple recommendation model with reasonable modeling assumptions, the impact of a user's cookie consent decision is not necessarily straightforward. Particularly, when consent rates exhibit demographic-dependence, the user's consent decision itself acts as a signal, carrying information beyond the information contained in the actual cookie value. This leads to a demographic-dependent impact of consent, resulting in a user's privacy-protective decision (of disagreeing to share their cookie) potentially causing the system to know more about them than if the user were to intentionally reveal their cookie information.
To draw a comparison with the large body of existing literature in the fairness of recommender systems, many of the existing papers (see the papers referenced in Section 3) draw attention to the way in which the system learns similarities among users over the recommendation process as a primary source of unfairness. Indeed, this is fundamental to how matrix factorization algorithms (and many more complex recommender systems) operate: users who behave similarly, i.e., how they respond to the same recommendations, will be deemed as being similar (even falsely so). This will lead to these users seeing similar recommendations which can, in turn, lead to undesirable outcomes.
However, one of the goals of our paper is to draw attention to the informational effects of consent as a potential additional source of unfairness. When consent is a conscious decision of the user, and the rate of a given consent decision differs based on sensitive attributes of the user, e.g., age, then the system can use the consent decision to learn more about the user's sensitive attributes. When embedded within an algorithmic system, platforms may unintentionally profile users based on the absence of data.
These observations have possible implications in the design of recommender systems. Any information that the system can use to draw similarities will be used, most notably how users respond to recommendations, but also, and the focus of our paper, the inherent likelihood of a user to provide consent to cookie sharing. These similarities are used to infer to which group a user belongs and subsequently drive recommendations. The implications for algorithm design is that one must be mindful of the complexities of information revelation especially as it relates to how the algorithm will use the revealed information.
New notions of fairness. The design of recommender systems that are more aware of the informational effects of users' decisions may require new notions of fairness. It is well-known in the fairness community that static fairness metrics are insufficient for dynamic settings, and can actually actively harm the group they intend to protect (28).
In our setting, whether the system possessing heterogeneous accuracy about users results in a fair or unfair outcome depends on the product being recommended. Given our observations that the specific impact of a user's consent decision varies depending on the user's demographic, it is not hard to see that such accuracy disparity may lead to unfair situations. For example, consider an ad recommendation setting where some of the ads may be predatory to older individuals. According to our model, an older individual's desire to remain more private by deciding to disagree to cookie sharing may allow the system to know more about them, simply by virtue that more older individuals choose to not share their cookies, and subsequently make them a more likely target for such ads. Consequently, we emphasize the need for development of fairness metrics that ensure consistency of an individual user's privacy-protective decision and the amount (or accuracy) of information that the system knows about that user.
It is worth noting that this informational consistency property is distinct from the literature on privacy-preserving recommender systems. For instance, differentially-private recommender systems (31, 29) ensure individual-anonymity, via careful injection of noise, while still enabling extraction of aggregate-level information used to feed the recommendation algorithm. In contrast, a consistency-based approach would hypothetically still allow for some individuals to be (more accurately) identified if they choose to be (e.g., to receive more targeted recommendations) while honoring the privacy of individuals who chose to not share their information.
Towards industry reform. Our results contribute to the growing sentiment that the digital advertising space is in need of reform, particularly due to the unintended consequences of "the collection of personal data, tracking, and massive-scale profiling" (2). As the move away from cookie-based tracking gains momentum, alternative mechanisms for user profiling are becoming more prevalent.
In the context of an advertising environment without cookies (often referred to as the cookieless future), the shift to these alternative data collection methods may lead to informational disparities across companies. Given that the current advertising ecosystem contains players of vastly different sizes, ending the use of cookies may give asymmetric power to companies that have already collected significant amounts of personal data. Indeed, tools that facilitate access to user information that would otherwise be tracked by cookies are already in use, e.g., websites that allow users to create an account by using their Google/Microsoft credentials (a practice known as "auto-linking").
In general, these issues point to the fact that controls and regulations should be based on the downstream effects of user interactions with the advertising system. In particular, policy must carefully consider how much information has been extracted from a given user, whether it be from individual decisions/responses of the user or gleaned from inferred similarities with other users, and its privacy-compromising effects. To foster transparency, it would be beneficial for the industry to derive estimates of the monetary value of user information and share them with the public. In general, the advertising industry can be more transparent, not just about user data but also concerning the downstream effects on users, since real-world advertising engines are vastly more complex than our simplified setting and may not lend themselves to simple mathematical modeling.
Limitations. Our findings on the disparate impact of consent should be interpreted with the understanding that our model necessarily has some limitations. Broadly, our model uses a simplified definition of cookies in which cookies serves as a proxy for the users' cohorts. While the reason for this simplicity is to extract insights that depend directly on the user's cookie consent decisions, extending the definition of a cookie to more realistic settings by including user click/behavioral information would likely generate additional insights. Secondly, to capture the core aspects of recommender systems, our recommendation model is based on foundational recommendation algorithms (namely online + confidence-weighted matrix factorization). While these algorithms form the basis for modern recommender systems, it would be worthwhile to see how the insights extend to more modern algorithms. Lastly, we consider simplified ad and user pools; consideration of additional ad features and a more realistic dynamic user pool could influence the findings.
7 Concluding remarks and future directions
We've investigated the question of how heterogeneous cookie consent rates among users influence the recommender agent's ability to learn users' latent features (e.g., their demographics and tastes). We've empirically discovered, through construction of a simulator, that disparities in the agent's estimation accuracy across users emerge when consent rates exhibit demographic-dependence. Our observations show that seemingly simple informational decisions by users (i.e., whether to share their cookie) can have complex effects on the agent's information. Consideration of informational effects of users' decisions is crucial in the design of recommender systems. We encourage the development of new fairness metrics for dynamic settings that enforce consistency between a user's privacy decision and the amount of information that the system knows about them.
Future work focuses on studying the downstream effects of our observations via validation in real-world recommender systems. An additional direction is quantification of the aforementioned fairness consistency property such that it can be embedded as a constraint in the design of recommender system algorithms.
Acknowledgments. This research was funded in part by the Horizon Europe project AutoFair (grant agreement ID: #101070568).
[49] Yu Zheng, Chen Gao, Xiang Li, Xiangnan He, Yong Li, and Depeng Jin. Disentangling user interest and conformity for recommendation with causal embedding. In Proceedings of the Web Conference 2021, pages 2980-2991, 2021.
[50] Meizi Zhou, Jingjing Zhang, and Gediminas Adomavicius. Longitudinal impact of preference biases on recommender systems' performance. Kelley School of Business Research Paper, (2021-10), 2021.
[51] Tao Zhou, Zoltán Kuscsik, Jian-Guo Liu, Matús Medo, Joseph Rushton Wakeling, and Yi-Cheng Zhang. Solving the apparent diversity-accuracy dilemma of recommender systems. Proceedings of the National Academy of Sciences, 107(10):4511–4515, 2010.
A Omitted expressions
Derivation of interim belief update. Given user consent decisions, the agent updates the prior  $\mu$  to form interim beliefs as follows. For a user  $i$  who provided consent  $(x_{i} = 1)$ , with its revealed cookie  $\phi_{i} = \varphi$ , the interim belief  $\tilde{\mu}_{i,0} \in \Delta(\Theta)$  is given elementwise by
$$
\begin{array}{l} \tilde {\mu} _ {i, 0} (\vartheta) = \mathbb {P} \left(\theta_ {i} = \vartheta \mid x _ {i} = 1, \phi_ {i} = \varphi\right) \\ = \frac {\mathbb {P} (x _ {i} = 1 \mid \phi_ {i} = \varphi , \theta_ {i} = \vartheta) \mathbb {P} (\phi_ {i} = \varphi , \theta_ {i} = \vartheta)}{\mathbb {P} (x _ {i} = 1 , \phi_ {i} = \varphi)} \\ = \frac {P (x _ {i} = 1 \mid \theta_ {i} = \vartheta) p (\phi_ {i} = \varphi , \theta_ {i} = \vartheta)}{\sum_ {\vartheta^ {\prime}} \mathbb {P} (x _ {i} = 1 \mid \theta_ {i} = \vartheta^ {\prime}) \mathbb {P} (\phi_ {i} = \varphi , \theta_ {i} = \vartheta^ {\prime})} \\ = \frac {q _ {\vartheta} \mu (\varphi , \vartheta)}{\sum_ {\vartheta^ {\prime}} q _ {\vartheta^ {\prime}} \mu (\varphi , \vartheta^ {\prime})}. \\ \end{array}
$$
Similarly, for a user  $i$  who did not provide consent  $(x_{i} = 0)$ , its cookie is not revealed and thus interim beliefs are given by
$$
\begin{array}{l} \tilde {\mu} _ {i, 0} (\vartheta) = \mathbb {P} \left(\theta_ {i} = \vartheta \mid x _ {i} = 0\right) \\ = \frac {\mathbb {P} (x _ {i} = 0 \mid \theta_ {i} = \vartheta) \mathbb {P} (\theta_ {i} = \vartheta)}{\mathbb {P} (x _ {i} = 0)} \\ = \frac {\mathbb {P} (x _ {i} = 0 \mid \theta_ {i} = \vartheta) \sum_ {\varphi} \mathbb {P} (\phi_ {i} = \varphi , \theta_ {i} = \vartheta)}{\sum_ {\vartheta^ {\prime}} \mathbb {P} (x _ {i} = 0 \mid \theta_ {i} = \vartheta^ {\prime}) \sum_ {\varphi^ {\prime}} \mathbb {P} (\phi_ {i} = \varphi^ {\prime} , \theta_ {i} = \vartheta^ {\prime})} \\ = \frac {\left(1 - q _ {\vartheta}\right) \sum_ {\varphi} \mu (\varphi , \vartheta)}{\sum_ {\vartheta^ {\prime}} \left(1 - q _ {\vartheta^ {\prime}}\right) \sum_ {\varphi^ {\prime}} \mu \left(\varphi^ {\prime} , \vartheta^ {\prime}\right)}. \\ \end{array}
$$
Cohort belief update. Given a set of cohort beliefs across users  $\mu = (\mu_1, \dots, \mu_n)$  and a set of recommendation-response pairs  $\mathcal{L}$ , the agent forms updated beliefs  $\mu' = (\mu_1', \dots, \mu_n')$  via a Bayesian update,  $\mu' = f(\mu, \mathcal{L})$ . This update is carried out independently for each user  $i$ . Let  $\mu_i$  denote the agent's current belief on user  $i$ 's cohort  $\theta_i$ , and denote  $\mathcal{L}_i = \{(a_i, c_{i,a})\}$  as the pairs that correspond to user  $i$ . The updated belief  $\mu_i'$  is given by  $\mu_i' = (\mu_i'(\theta^1), \dots, \mu_i'(\theta^d)) = f_i(\mu_i, \mathcal{L}_i)$  where each  $\mu_i'(\vartheta)$  is given by
$$
\begin{array}{l} \mu_ {i} ^ {\prime} (\vartheta) = \mathbb {P} \left(\theta_ {i} = \vartheta \mid \mathcal {L} _ {i}\right) \\ = \frac {\prod_ {(a _ {i} , c _ {i , a}) \in \mathcal {L} _ {i}} \mathbb {P} (C _ {i , a} = c _ {i , a} \mid A _ {i} = a _ {i} , \theta_ {i} = \vartheta) \mathbb {P} (\theta_ {i} = \vartheta)}{\sum_ {\vartheta^ {\prime}} \prod_ {(a _ {i} , c _ {i , a}) \in \mathcal {L} _ {i}} \mathbb {P} (C _ {i , a} = c _ {i , a} \mid A _ {i} = a _ {i} , \theta_ {i} = \vartheta^ {\prime}) \mathbb {P} (\theta_ {i} = \vartheta^ {\prime})} \\ = \frac {\prod_ {(a _ {i} , c _ {i , a}) \in \mathcal {L} _ {i} ^ {1}} \bar {p} _ {i , a _ {i}} (\vartheta) \prod_ {(a _ {i} , c _ {i , a}) \in \mathcal {L} _ {i} ^ {0}} (1 - \bar {p} _ {i , a _ {i}} (\vartheta)) \mu_ {i} (\vartheta)}{\sum_ {\vartheta^ {\prime}} \prod_ {(a _ {i} , c _ {i , a}) \in \mathcal {L} _ {i} ^ {1}} \bar {p} _ {i , a _ {i}} (\vartheta^ {\prime}) \prod_ {(a _ {i} , c _ {i , a}) \in \mathcal {L} _ {i} ^ {0}} (1 - \bar {p} _ {i , a _ {i}} (\vartheta^ {\prime})) \mu_ {i} (\vartheta^ {\prime})} \\ \end{array}
$$
where  $\mathcal{L}_i^1$  (resp.  $\mathcal{L}_i^0$ ) are the responses where the user clicked (resp. did not click) on the recommendation and  $\bar{p}_{i,a}(\vartheta)$  is the expected click probability given  $\vartheta$  defined as
$$
\bar {p} _ {i, a} (\vartheta) = \mathbb {E} _ {\alpha_ {i} \sim \operatorname {L o g N o r m a l} \left(\rho_ {\vartheta}, \sigma_ {\vartheta} ^ {2}\right)} \left[ p _ {i, a} \left(\alpha_ {i}\right) \right]
$$
where the click probability  $p_{i,a}$  from (1) has been written as  $p_{i,a}(\alpha_i)$  to make the dependence on user  $i$ 's topic affinities explicit.
**Confidence weights.** The confidence weights  $\bar{w}_{i,a,t}$  in (2) are computed as the expected binomial probability for seeing the current response counts  $r_{i,a,t}$  given the agent's current belief on user  $i$ 's cohort  $\vartheta_i$ , i.e.,  $\bar{w}_{i,a,t} = \mathbb{E}_{\vartheta_i \sim \mu_{i,t}}[p_t(I_{i,a,t}, r_{i,a,t}, \vartheta_i)]$ . The binomial probability  $p_t(I_{i,a,t}, r_{i,a,t}, \vartheta_i)$  is defined as
$$
p _ {t} (I _ {i, a, t}, r _ {i, a, t}, \vartheta_ {i}) = \binom {I _ {i, a, t}} {r _ {i, a, t}} \bar {p} _ {i, a} (\vartheta_ {i}) ^ {r _ {i, a, t}} (1 - \bar {p} _ {i, a} (\vartheta_ {i})) ^ {I _ {i, a, t} - r _ {i, a, t}}
$$
where  $I_{i,a,t}$  is the current impression count for user-ad pair  $(i,a)$ ,  $r_{i,a,t}$  is the current click click for  $(i,a)$ , and  $\bar{p}_{i,a}(\vartheta_i) = \mathbb{E}_{\alpha_i \sim \mathrm{LogNormal}(\rho_{\vartheta_i}, \sigma_{\vartheta_i}^2)}[p_{i,a}(\alpha_i)]$  is the expected click probability.
B Recommendation procedure