SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 1: Recommendation procedure.
Input parameters.  $(n,l,m,\Theta ,\Phi ,\mathcal{T},\{\rho_{\theta},\sigma_{\theta}\} ,u_0,\{q_\theta \} ,\mu ,k,\lambda ,\varepsilon ,\mathcal{L}_0,T_b,T)$    
Initialize users. For each user  $i\in [n]$  sample:   
cookie-cohort pairs  $(\phi_i,\theta_i)\sim \mu$    
topic affinities  $\alpha_{i}\sim \mathrm{LogNormal}(\rho_{\theta_{i}},\sigma_{\theta_{i}}^{2})$    
consent decision  $x_{i}\sim \mathrm{Bernoulli}(q_{\theta_{i}})$    
Form interim beliefs. For each user  $i\in [n]$  .. if  $x_{i} = 1$  then // user provided consent  $\tilde{\mu}_{i}(\vartheta)\leftarrow \frac{q_{\vartheta}\mu(\phi_{i},\vartheta)}{\sum_{\vartheta^{\prime}}q_{\vartheta^{\prime}}\mu(\phi_{i},\vartheta^{\prime})},\vartheta \in \Theta$  else //user withheld consent  $\tilde{\mu}_{i}(\vartheta)\leftarrow \frac{(1 - q_{\vartheta})\sum_{\varphi}\mu(\varphi,\vartheta)}{\sum_{\vartheta^{\prime}}(1 - q_{\vartheta^{\prime}})\sum_{\varphi^{\prime}}\mu(\varphi^{\prime},\vartheta^{\prime})},\vartheta \in \Theta$  end   
Offline responses. Form priors using  $\mathcal{L}_0:\mu_{i,0}\gets f(\tilde{\mu}_i,\mathcal{L}_0),i\in [n]$    
Online recommendations.   
for  $t = 1,\ldots ,T$  do Define current ad pool  $\mathcal{A}_t$  by sampling  $l$  items uniformly without replacement from T if mod  $(t,T_b) = 0$  then // retraining step Update cohort beliefs:  $\mu_{i,t}\gets f(\mu_{i,t - 1},\mathcal{L}_t\setminus \mathcal{L}_{t - T_b}),i\in [n]$  Compute weights:  $\bar{w}_{i,a,t}\gets \mathbb{E}_{\vartheta_i\sim \mu_i,t}[p_t(r_i,a,t,\vartheta_i)],i\in [n]$  Update factor estimates:  $(\hat{u}_t,\hat{v}_t)\gets \operatorname {argmin}_{(u,v)\in \mathcal{U}\times \mathcal{V}}\Bigg[\sum_{(a_i,c_i,a)\in \mathcal{L}_t}\bar{w}_{i,a,t}(u_i^\top v_a - r_{i,a,t})^2 +$ $\lambda \Big(\sum_{i\in [n]}\| u_i\| _2 + \sum_{a\in [m]}\| v_a\| _2\Big)\Big]$  else Propagate cohort beliefs:  $\mu_t\gets \mu_{t - 1}$  Propagate factor estimates:  $(\hat{u}_t,\hat{v}_t)\gets (\hat{u}_{t - 1},\hat{v}_{t - 1})$    
end Recommend ads: for each  $i\in [n]$  , recommend at  $a_{i,t}$  via  $a_{i,t} = \left\{ \begin{array}{ll}\operatorname *{argmax}_{a\in A_t}\hat{u}_{i,t}^\top \hat{v}_{a,t} & \mathrm{w.p.}1 - \varepsilon \\ a\sim \mathrm{U}(A_t) & \mathrm{w.p.}\varepsilon \end{array} \right.$    
Append responses:  $\mathcal{L}_{t + 1}\gets \mathcal{L}_t\cup \{a_{i,t},c_{a,i,t}\}$    
end
C Simulator and experiments
Our simulator was built upon RecSim (21) (source code at https://github.com/emiehling/cookie-consent/). The high-level architecture of our simulator is illustrated in Fig. 1 of Section 4. Additional details (with references to objects in the source code) are provided below.
Advertisement and user samplers. The advertisement sampler object (AdvertisementSampler) defines the distribution of each ad feature, here assumed to simply be the ad's topic. Similarly, the user sampler object (UserStateSampler) defines the distribution of each user feature, described by the joint cookie-cohort prior, the opt-in distribution, and the statistics of the user's topic affinities. The ad sampler and user sampler objects are used to define a gym environment for the recommendation procedure (via MultiUserEnvironment and RecSimGymEnv). Ads are resampled in each round whereas users remain fixed for the duration of the episode.
Users and the recommender agent. Each user is described by the class RSUserModel. This class contains the user's choice model, i.e., the logit model dictating the binary click decision given the recommended ad (see the method simulate_response).
Given the collections of ads and users, the recommender agent makes recommendations according to an  $\varepsilon$ -greedy bandit (see the pseudocode in Section B).
Retraining consists of first updating the cohort beliefs (via the methods update_cohort-beliefs and get.click probabilities, see Appendix A for the expressions), updating weights  $\bar{w}_{i,a,t}$  and recomputing the matrix factor estimates (via get_estimated_factors, see (2)). Estimation is carried out via stochastic gradient descent with a learning rate of 0.01, regularization weight of 0.01, and a stopping threshold on the mean-squared error of  $\varepsilon_{\mathrm{thresh}} = 0.001$ . Latent factors are assumed to be of dimension  $k = 50$ .
Experimental setup. Simulations were run in Python 3.8 on an Intel(R) Xeon(R) CPU E5-2667 v2 (3.30GHz). Unless otherwise stated, baseline parameters of the simulation environment were as follows:  $n = 1000$  users,  $m = 200$  ads, ad candidate size  $l = 50$ , batch size  $T_{b} = 1$ , offline response set  $\mathcal{L}_0 = \varnothing$ , exploration probability  $\varepsilon = 0.1$ , and binary cookie and cohort spaces. Simulations were averaged over 500 runs/episodes.
The experimental setup can be extended in a variety of directions to investigate additional interesting questions. One direction is to extend the feature description of the ads (beyond topic) to include features that reflect ad quality and location. This would enable studying how the recommender system treats minority populations (compared to majority populations). Additionally, augmenting the simulator with the ability to handle a changing user pool would allow for analysis of the cold start problem.
D Sensitivity analyses
Sensitivity analyses on cohort errors were carried out by independently varying the prior  $\mu$ , batch size  $T_{b}$ , offline response set  $\mathcal{L}_{0}$ , and topic affinity means  $(\rho_0, \rho_1)$  with fixed parameters  $n = 200$  users,  $m = 200$  topics, ad pool size  $l = 50$ , latent factor dimension  $k = 50$ , and  $|\Theta| = |\Phi| = 2$ . Plots illustrate error means and standard deviations (red:  $\theta = 0$ , blue  $\theta = 1$ , solid: consent group, dashed: non-consent group) over 50 episodes. Baseline parameters are:
$$
\mu = \left[ \begin{array}{c c} 0. 4 & 0. 1 \\ 0. 1 & 0. 4 \end{array} \right], \qquad T _ {b} = 1, \qquad \mathcal {L} _ {0} = \varnothing , \qquad (\rho_ {0} / m, \rho_ {1} / m) = (0. 3, 0. 7)
$$
Prior. The impact of the prior  $\mu$  was studied in three cases dictated by the degree of informativeness of the cookie for inferring the cohort.
Partially informative:  $\mu = \begin{bmatrix} 0.4 & 0.1 \\ 0.1 & 0.4 \end{bmatrix}$
Fully informative:  $\mu = \begin{bmatrix} 0.5 & 0 \\ 0 & 0.5 \end{bmatrix}$
Uninformative:  $\mu = \begin{bmatrix} 0.25 & 0.25 \\ 0.25 & 0.25 \end{bmatrix}$
For a partially informative prior (any  $\mu$  that does not have identical rows), knowledge of a user's cookie partially reveals the user's cohort. The resulting cohort errors are consistent with Fig. 2 of Section 5.1. For the fully informative prior, the agent is completely certain of users' cohorts for users who opted-in (the agree group), but still possesses uncertainty for users who did not opt-in (the disagree group). Lastly, for the uninformative prior, revelation of a user's cookie does not inform the user's cohort (as the likelihoods of seeing cookie values are identical across cohorts) and the agent must infer cohorts solely from differences in response behavior.
Batch size. The batch size,  $T_{b}$ , dictates how many responses to collect from each user before retraining. Values for the batch size were varied in the range  $T_{b} \in \{1,2,10\}$ .
$T_{b} = 1$
$T_{b} = 2$
$T_{b} = 10$
Intuitively, for the same number of observations, cohort estimation errors are the same across various batch sizes (e.g., updates may be less frequent but they contain more data). However, waiting until a batch update  $(T_b > 1)$  results in more interactions where the users face greater disparate estimation errors (compared to the  $T_b = 1$  case).
Size of offline response set. The offline response set,  $\mathcal{L}_0$ , is a set of recommendation-responses that are available before the online recommendation process. Recommendations in the offline set,  $\mathcal{L}_0$ , were generated uniformly at random with responses generated by the users' choice models. Simulations were run for  $|\mathcal{L}_0| = \{0,1,5\}$ , differing in the number of offline responses assumed available from each user.
$$
\left| \mathcal {L} _ {0} \right| = 0:
$$
$(q_0,q_1) = (0.25,0.25)$
$(q_0,q_1) = (0.25,0.5)$
$(q_0,q_1) = (0.25,0.75)$
$$
\left| \mathcal {L} _ {0} \right| = 1:
$$
$$
\left| \mathcal {L} _ {0} \right| = 5:
$$
The impact of the offline response set does not appear to have a significant effect on the evolution of the cohort errors.
Similarity of topic affinities. The sensitivity to similarity of topic affinities was studied by increasing similarities of the synthetic affinities via means (normalized by topic count  $m$ ) ( $\rho_0 / m, \rho_1 / m$ )  $\in\{(0.3, 0.7), (0.4, 0.6), (0.5, 0.5)\}$ .
$$
\left(\rho_ {0} / m, \rho_ {1} / m\right) = (0. 3, 0. 7):
$$
$$
\left(\rho_ {0} / m, \rho_ {1} / m\right) = (0. 4, 0. 6):
$$
$$
\left(\rho_ {0} / m, \rho_ {1} / m\right) = (0. 5, 0. 5):
$$
The agent's ability to distinguish users based on their responses depends on the similarities of affinities across users in different cohorts. Intuitively, as the topic affinities across cohorts become more similar, the agent requires more responses to reach the same level of estimation error (since users from different cohorts behave more similarity as topic similarity grows). The extreme case of identical statistics of users' affinities across cohorts  $(\left(\rho_0 / m,\rho_1 / m\right) = (0.5,0.5))$  results in the agent being unable to resolve any uncertainty over users' cohorts (since user responses are uninformative for their cohort).