SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lamy, A., Zhong, Z., Menon, A. K., and Verma, N. Noise-tolerant fair classification. In Advances in Neural Information Processing Systems, 2019.
Larson, J., Mattu, S., Kirchner, L., and Angwin, J. How We Analyzed the COMPAS Recidivism Algorithm, 2016. URL https://www_propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm.
Lattimore, T. and Szepesvári, C. Bandit Algorithms. Cambridge University Press, 2020.
Li, X. and Ye, Y. Online linear programming: Dual convergence, new algorithms, and regret bounds. Operations Research, 2021.
Li, X., Sun, C., and Ye, Y. Simple and fast algorithm for binary integer and online linear programming. Mathematical Programming, 2022.
Lipton, Z., McAuley, J., and Chouldechova, A. Does mitigating ml's impact disparity require treatment disparity? In Advances in Neural Information Processing Systems, 2018.
Liu, X., Li, B., Shi, P., and Ying, L. An efficient pessimistic-optimistic algorithm for stochastic linear bandits with general constraints. In Advances in Neural Information Processing Systems, 2021.
Molina, M. and Loiseau, P. Bounding and approximating intersectional fairness through marginal fairness. In Advances in Neural Information Processing Systems, 2022.
Nasr, M. and Tschantz, M. C. Bidding strategies with gender nondiscrimination constraints for online ad auctions. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 2020.
Orabona, F. A modern introduction to online learning, 2019. Available as arXiv:1912.13213.
Perchet, V. and Rigollet, P. The multi-armed bandit problem with covariates. The Annals of Statistics, 41(2), 2013.
Rockafellar, R. T. Convex Analysis. Princeton University Press, 1970.
Slivkins, A. Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning, 12 (1-2), 2019.
Speicher, T., Ali, M., Venkatadri, G., Ribeiro, F. N., Arvanitakis, G., Benevenuto, F., Gummadi, K. P., Loiseau, P., and Mislove, A. On the potential for discrimination in online targeted advertising. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency, 2018.
Vershynin, R. High-Dimensional Probability: An Introduction with Applications in Data Science. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2018.
Zhao, T., Dai, E., Shu, K., and Wang, S. Towards fair classifiers without sensitive attributes: Exploring biases in related features. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, 2022.
A Fairness Penalty
In the main part of the paper, we have considered a penalty with a general form  $R(\sum_{t}a_{t}x_{t} / T)$ . Here we give examples of fairness notions that can be encoded through this general penalty.
Statistical parity Notions like statistical parity are defined as constraints on the independence between protected attributes and selection decisions. More precisely, if  $X$  denotes a treatment variable, and  $A$  the protected attributes of an individual, we say that  $X$  and  $A$  are fair with respect to statistical parity if  $X$  and  $A$  are independent.
If  $X \in \mathcal{X}$  and  $A \in \mathcal{A}$  can take a finite number of values, this can be equivalently rewritten using probability distribution:  $X$  satisfies statistical parity if for all  $(x, a) \in \mathcal{X} \times \mathcal{A}$ :
$$
\Pr (A = a, X = x) = \Pr (A = a) \Pr (X = x). \tag {11}
$$
To encode statistical parity in our framework, we consider that the  $a_{t}$  are one-hot encoding of the protected attributes (i.e.,  $a_{t}$  is a vector with  $d = |\mathcal{A}|$  dimensions that is equal to 0 everywhere except on the coordinate corresponding to the protected attribute of individual  $t$ , that equals 1). For the decision variables  $x_{t} \in \{0,1\}$ ,  $\sum_{t}x_{t}$  is the number of individual selected and  $\sum_{t}(a_{t})_{i}x_{t}$  is the number of selected individual whose protected attributed is  $i \in [d]$ . Hence, (11) rewrites as: for all  $i \in [d]$ :
$$
\frac {\sum_ {t} (a _ {t}) _ {i} x _ {t}}{T} = \left(\frac {\sum_ {t} (a _ {t}) _ {i}}{T}\right) \left(\frac {\sum_ {t} x _ {t}}{T}\right).
$$
Recall that the variables  $a_{t}$  are i.i.d. and let us denote by  $\alpha \in \mathcal{P}_d$  the probability distribution vector of the  $a_{t}$ . For large  $T$ , we have  $\sum_{t} a_{t} / T \approx \alpha$ . Which means that we will say that an allocation vector  $x$  satisfies statistical parity if  $\sum_{t} a_{t} x_{t} / T = \sum_{t} \alpha x_{t} / T$ , which can be equivalently rewritten as
$$
\frac {\sum_ {t} a _ {t} ^ {\prime} x _ {t}}{T} = 0,
$$
with  $a_{t}^{\prime} = a_{t} - \alpha$
Penalties  $R$  corresponding to statistical parity In practice, achieving strict statistical parity is very constraining, and most of the time some relaxation of this constraint is allowed, at the price of some penalty (Ahani et al., 2021). In our paper, we penalize the non-independence through the function  $R$ . There are different choices of penalty function that can be used:
- A natural choice can be to penalize as a function of the distance between joint empirical distribution  $a_{t}^{\prime}x_{t} / T$  and 0. For instance, one could use  $R(a_t^\prime x_t / T) = ||a_t^\prime x_t / T||^\beta$  for any given norm  $\lvert\lvert\cdot\rvert\rvert$  and any parameter  $\beta >0$
- It is also possible to measure the non-independence with the empirical distribution of the  $x_{t}$  conditioned on  $a_{t} = a$ , which should be equal regardless of the protected group  $a \in \mathcal{A}$ . In this case, the empirical distribution conditioned on the protected group  $i \in [d]$  is  $\sum_{t} a_{ti} x_{t} / \sum_{t} a_{ti} \approx \sum_{t} a_{ti} x_{t} / (\alpha_{i} T)$ . Similarly we the previous case,  $R$  can penalize how far is this vector from 0.
In the two cases, the fairness penalty can be written as  $TR(\sum_{t}a_{t}^{\prime}x_{t} / T)$ , for some modified  $a_{t}^{\prime}$  that are not necessarily one-hot encoder. This explains why we choose the set  $\mathcal{A}$  to be more general than the set of one-hot encoders.
If we want to target another specific proportion for each protected group, we can replace  $\alpha$  by any other distribution  $\nu$ . Note that another possibility is to instead use the empirical distribution of the  $a_{t}$  conditioned on the  $x_{t}$ . This time the form of the vector is slightly different, it is  $\sum_{t} a_{t} x_{t} / \sum_{t} x_{t}$ ; this is the type of penalty tackled and discussed in Section 4.1.
Equality of opportunity Another popular fairness notion is the one of equality of opportunity (Hardt et al., 2016), where the probability distribution is further conditioned on another finite random variable  $y_{t}$  taking some discrete value in  $\mathcal{V}$  that indicates some fitness of the individual. For  $Y$  a
random variable with an outcome  $y \in \mathcal{Y}$  considered as a positive opportunity, we say that  $X$ ,  $A$  and  $Y$  are fair with respect to equality of opportunity, if
$X$  and  $A$  are independent given  $[Y = y]$
In order to adapt this notion in the online allocation setting, is sufficient to consider the cartesian product  $\mathcal{A}' = \mathcal{A} \times \mathcal{Y}$  as an extended new set. Then if we would like to penalize with a metric related to equality of opportunity, we can for instance compose  $R$  with the projection over the coordinates corresponding to  $y_{t} = 1$ .
Overall, this shows that the choice of  $R$ , the type of fairness vectors, and the choice of  $\mathcal{A}$  are all crucial when designing a fairness penalty.
B A More General Online Allocation Setting
In the main body of the paper, we choose to simplify the exposition and to restrict ourselves to  $x_{t} \in \{0,1\}$  and to formulate utilities as scalar variables  $u_{t}$ . In the following, the regret bounds of Algorithm 1 are obtained in a setting that is more general than the one previously introduced:
- We now consider that the allocations decisions  $x_{t}$  are in some set  $\mathcal{X} \subset \mathbb{R}_{+}^{n}$ . This makes it possible to tackle bipartite matching problems, by letting  $\mathcal{X} = \{x \in \mathbb{R}_{+}^{n} \mid \sum_{i=1}^{n} x_{i} \leq 1\}$  be a matching polytope.
- Instead of a random scalar  $u_{t}$  being drawn, we suppose that a random function  $f_{t}: \mathcal{X} \to \mathbb{R}$  is drawn. The functions  $f_{t}$  are supposed u.s.c. and are all bounded in absolute value by some  $\bar{f}$ . This could simply be a deterministic concave function of  $u_{t}$  for instance. Remark that when we assume that the conditional expectation can be computed, the decision maker's required knowledge is more demanding, as she needs to be able to compute it for all  $x \in \mathcal{X}$ .
- Instead of a random vector, a continuous random function  $a_t: \mathcal{X} \to \mathcal{A}$  can be drawn, with  $\|a_t(x)\|_2$  uniformly bounded for all  $a_t$  and  $x$ , and we define  $\Delta = \operatorname{Conv}(\mathcal{A})$ . We suppose that  $\|a_t(x)\|_2 \leq 1$ , which can be done without loss of generality.
The setting of Section 2.1 is recovered by using  $f_{t}(x) = u_{t}x$ ,  $a_{t}(x) = a_{t}x$  and  $\mathcal{X} = \{0,1\}$ . For the more general setting, we need to redefine the virtual value with  $f_{t}$  and  $a_{t}$  as:
$$
\varphi (\lambda , c _ {t k}, k) = \max  _ {x \in \mathcal {X}} \mathbb {E} [ f _ {t} (x) - \langle \lambda_ {t}, a _ {t} (x) \rangle \mid c _ {t k} ] - p _ {k} \tag {12}
$$
This virtual reward is well defined by Lemma D.1. We rewrite Algorithm 1 using  $f_{t}$  and this new virtual value to obtain Algorithm 2. There are two differences with Algorithm 1: First we use the argmax function in (13) instead of the explicit form of Equation (6); Second we use the virtual value function (12) in Equation (14) instead of the virtual value function of Equation (5).
C Reduction from auctions
In the rest of the paper, we assume that the decision variables are the values  $x_{t}$ . In this section, we explain how our setting can be adapted to the case of targeted advertisement, that are usually allocated through auction mechanisms. In the later, the decision variables are the bids but not the  $x_{t}$ : the variables  $x_{t}$  are a consequence of the bids of the decision maker and of the bids of others.
Model The model that we consider is as follows: for user  $t$ , after choosing a source of information  $k_{t}$ , the decision maker observes  $c_{tk_{t}}$  and decides to bid  $b_{t} \in \mathcal{B}$ . The allocation  $x_{t} = 1$  is given if her bid is higher than the maximum bid of the other bidders. If the bidder wins the auction, it then pays a price that depends on the auction format (e.g.,  $b_{t}$  for first price auctions and the second highest bid for second price auctions). Assuming that the bid of the others are not affected by the decision maker's strategy, the bids of the others are i.i.d. (see Iyer et al. (2011); Balseiro et al. (2014); Flajolet & Jalillet (2017) for a discussion about the stationarity assumption). In this case, one falls back to our model with i.i.d. users, except that the decision variables are the variables  $b_{t}$  and we have  $x_{t}(b_{t})$ . We can apply our algorithm to this setting, by replacing the set of strategies  $\mathcal{X}$  by the set of possible bids  $\mathcal{B}$  which is also compact. For that, we need to consider a new utility function  $b \mapsto u_{t}(b)x_{t}(b)$  which is a bounded u.s.c. function, and the new protected attributes function  $b \mapsto a_{t}x_{t}(b)$ . If  $\mathbb{E}[a_{t}x_{t}(b) \mid c_{tk}]$  is continuous in  $b$ , this falls under the setting that we described in Appendix B.
Algorithm 2 Online Fair Allocation with Source Selection — General algorithm
Input: Initial dual parameter  $\lambda_0$ , initial source-selection-distribution  $\pi_0 = (1 / K, \dots, 1 / K)$ , dual gradient descent step size  $\eta$ , EXP3 step size  $\rho$ , cumulative estimated rewards  $S_0 = 0 \in \mathbb{R}^K$ . for  $t \in [T]$  do
Draw a source  $k_{t} \sim \pi_{t}$ , where  $(\pi_{t})_{k} \propto \exp(\rho S_{(t-1)k})$  and observe  $c_{tk_{t}}$ .
Compute the allocation for user  $t$ :
$$
x _ {t} = \underset {x \in \mathcal {X}} {\arg \max } \mathbb {E} \left[ f _ {t} (x) - \left\langle \lambda_ {t}, a _ {t} (x) \right\rangle \mid c _ {t k _ {t}} \right]. \tag {13}
$$
$$
S _ {t k} = S _ {t - 1, k} + \hat {\varphi} \left(\lambda_ {t}, c _ {t k}, k _ {t}\right), \tag {14}
$$
Compute the expected protected group allocation  $\delta_t = \mathbb{E}[a_t(x_t) \mid c_{tk_t}, x_t]$  and compute the dual protected groups allocation target and update the dual parameter:
$$
\gamma_ {t} = \underset {\gamma \in \Delta_ {\delta_ {t}}} {\arg \max } \{\langle \lambda_ {t}, \gamma \rangle - \bar {R} (\gamma) \},
$$
$$
\lambda_ {t + 1} = \lambda_ {t} - \eta (\gamma_ {t} - \delta_ {t}).
$$
Examples of utilities: first-price and second price auctions Let us denote by  $v_{t}$  the value of the user  $t$  for the decision maker, and by  $d_{t}$  the highest bid of the others for this user. The decision maker will win the auction if  $b_{t} \geq d_{t}$ . Hence, the allocations will be  $x_{t}(b_{t}) = \mathbb{1}_{b_{t} > d_{t}}$ . Moreover, the utility function will be:
-  $f_{t}(b_{t}) = (v_{t} - b_{t})x_{t}(b_{t})$  for first price auctions.
-  $f_{t}(b_{t}) = (v_{t} - d_{t})x_{t}(b_{t})$  for second price auctions.
These functions are u.s.c with respect to  $b_{t}$ . Moreover, under mild assumption that the distribution of the highest bid of the others  $d_{t}$  has a density, the function  $\mathbb{E}[a_t x_t(b)|c_{tk_t}]$  is continuous. Hence, our algorithm can be applied.
Difficulty: Knowledge is power To apply the algorithm, the main difficulty is to be able to compute the value  $b_{t}$  that maximizes Equation (12). For that it is sufficient for the decision maker to have access to  $\mathbb{E}[f_t(b_t)|c_{tk_t}]$  and  $\mathbb{E}[a_tx_t(b_t)|c_{t,k_t}]$ . This presupposes a lot of distributional knowledge from her part. In the case of full information with  $a_{t}$  and  $v_{t}$  directly given, Balseiro et al. (2021); Celli et al. (2022) were able to obtain an efficient reduction from online allocations to second price auctions, using the incentive compatible nature of this kind of auction. Similarly, Flajolet & Jalillet (2017) were able to derive efficient bidding strategies using limited knowledge for repeated auctions with budget constraints under some independence assumptions. Translating what is done in these two lines of work to our general model is difficult because we would need to rely on the assumption that  $a_{t}$  and  $d_{t}$  are independent conditioned on  $c_{tk_t}$ . While we think that this assumption is not reasonable because different agents might have access to different information sources, and because as the  $a_{t}$  is common to all bidders it is correlated with all the bids hence also with  $d_{t}$ , we explain in the remark below how assuming independence can lead to a simple algorithm for second price auctions.
- For our model, the first price and second price auctions are equally difficult: The amount of knowledge needed to compute  $b_{t}$  and  $\pi_{t}$  is the same for both cases and first price auction is not more difficult than the second-price auction setting (contrary to what usually happens).
- During this whole reduction from auctions, we have not assumed that the  $d_t$  (or  $x_t$ ) are observed as a feedback (whether the bidder wins the auction or not), and thus they are not assumed to be observed in the benchmark either. This is because observing  $d_t$  or  $x_t$  could give us access to information on  $a_t$  that are not contained in  $c_{tk_t}$ .
- If we do make the assumption that  $(u_t, a_t)$  is independent of  $d_t$  conditionally on all  $c_{tk}$ , then we do recover an easy algorithm for second price auctions, and we can allow for a
benchmark taking into account the feedback  $d_{t}$  (simply because now this feedback does not give any additional information for the estimation of  $a_{t}$ ). Indeed, in that case the optimal bid is  $b_{t} = \mathbb{E}[v_{t} - \langle \lambda_{t},a_{t}\rangle |c_{tk}]$ . This follows from similar arguments made in Flajolet & Jaillet (2017), as for  $b_{t}$  some  $\mathcal{H}_t$  measurable bidding strategy we have:
$$
\begin{array}{l} \mathbb {E} \left[ v _ {t} x _ {t} - \left\langle \lambda_ {t}, a _ {t} \right\rangle x _ {t} - d _ {t} x _ {t} \mid c _ {t k _ {t}} \right] = \mathbb {E} \left[ \mathbb {E} \left[ v _ {t} x _ {t} - \left\langle \lambda_ {t}, a _ {t} \right\rangle x _ {t} - d _ {t} x _ {t} \mid \mathcal {H} _ {t}, d _ {t} \right] \mid c _ {t k _ {t}} \right] \\ = \mathbb {E} \left[ \mathbb {E} \left[ v _ {t} \mid c _ {t k _ {t}} \right] x _ {t} - \left\langle \lambda_ {t}, \mathbb {E} \left[ a _ {t} \mid c _ {t k _ {t}} \right] \right\rangle x _ {t} - d _ {t} x _ {t} \mid c _ {t k} \right] \\ = \mathbb {E} [ x _ {t} (\mathbb {E} [ v _ {t} \mid c _ {t k _ {t}} ] - \langle \lambda_ {t}, \mathbb {E} [ a _ {t} \mid c _ {t k _ {t}} ] \rangle - d _ {t}) \mid c _ {t k} ] \\ \leq \mathbb {E} [ \max  (\mathbb {E} [ v _ {t} \mid c _ {t k _ {t}} ] - \langle \lambda_ {t}, \mathbb {E} [ a _ {t} \mid c _ {t k _ {t}} ] \rangle - d _ {t}, 0) \mid c _ {t k} ], \\ \end{array}
$$
and this last inequality is reached for  $b_{t} = \mathbb{E}[v_{t} - \langle \lambda_{t},a_{t}\rangle \mid c_{tk}]$ .
D Technical & Useful Lemmas
In this section, we prove several lemmas that are useful for the proof of the main Theorem 3.1.
D.1 Preliminary lemmas
We first provide some simple lemmas. For completeness, we provide proofs of all results, even if they are almost direct consequences of the definitions.
Lemma D.1. The virtual value function  $\varphi$  is well defined by Equation (12), as the arg max is non-empty and the maximum is reached.
Proof. We simply need to have that  $\mathbb{E}[f_t(x) \mid c_{tk}]$  and  $-\langle \lambda, \mathbb{E}[a_t(x) \mid c_{tk}] \rangle$  are u.s.c. in  $x$ . The continuity of  $a_t$  is used to guarantee that  $-\langle \lambda, a_t(x) \rangle$  is u.s.c. regardless of the values of  $\lambda$ . This is immediate from the definition of upper semi-continuity and the Reverse Fatou Lemma. We detail it here for  $f_t$  for completeness. We say that  $f_t$  is u.s.c. if and only if
$\forall x \in \mathcal{X}$  if there is a sequence  $(x_{n})_{n \in \mathbb{N}}$  so that  $x_{n} \underset{n \to \infty}{\longrightarrow} x$ , then  $\lim \sup_{n \to \infty} f_{t}(x_{n}) \leq f_{t}(x)$ .
For  $x \in \mathcal{X}$ , let  $(x_{n})_{n \in \mathcal{N}}$  be a sequence so that  $x_{n} \to x$ . Let  $P$  be a probability distribution over  $f_{t}$  conditionally on  $c_{tk}$ . The function  $|f_{t}(x_{n})| \leq \bar{f}$ ,  $f_{t}(x_{n})$  is integrable, we can apply the Reverse Fatou Lemma for conditional expectation. If  $f_{t}(x_{n})$  is not positive, we can apply it to  $f_{t}(x_{n}) + \bar{f}$  which is positive. Using the Reverse Fatou Lemma and the upper semi-continuity of  $f_{t}$  we obtain:
$$
\limsup _ {n \rightarrow \infty} \int f _ {t} (x _ {n}) d P \leq \int \limsup _ {n \rightarrow \infty} f _ {t} (x _ {n}) d P \leq \int f _ {t} (x) d P.
$$
This means that  $x\mapsto \mathbb{E}[f_t(x)\mid c_{tk}]$  is u.s.c. over  $\mathcal{X}$
Because  $\mathcal{X}$  is compact, and because the maximum of an u.s.c. function is reached over a compact set, the maximum is well defined.
Lemma D.2. Let  $\bar{R}:\delta \mapsto \inf_{\delta^{\prime}\in \Delta}\{R(\delta^{\prime}) + L\| \delta -\delta^{\prime}\| _2\}$ . Then  $\bar{R} = R$  over  $\Delta$ ,  $Lip(R) = Lip(\bar{R})$ , and  $\bar{R}$  is convex.
Proof. The function  $\bar{R}$  is obtained through the explicit formula of the Kirschbraun theorem, which states that we can extend functions over Hilbert spaces with the same Lipschitz constant. Hence the equality over  $\Delta$  and the same Lipschitz constant.
It remains to prove the convexity. Note that because  $R$  is proper, it is lower semi-continuous,  $\| \delta - \cdot \|_2$  as well, and  $\Delta$  is compact, hence the inf is reached and it is actually a minimum. Let  $\delta_1, \delta_2$  in  $\mathbb{R}^d$  and  $\alpha \in [0, 1]$ , and let  $u_1$  and  $u_2$  be the minimums of the optimization problem for respectively  $\delta_1$  and  $\delta_2$ . Let  $u = \alpha u_1 + (1 - \alpha)u_2$ . We have  $u \in \Delta$  because  $\Delta$  is convex, and using that  $R$  is convex and we have
$$
\begin{array}{l} R (u) + L \| (\alpha \delta_ {1} + (1 - \alpha) \delta_ {2}) - u \| _ {2} \leq \alpha (R (u _ {1}) + L \| \delta_ {1} - u _ {1} \| _ {2}) + (1 - \alpha) (R (u _ {2}) + L \| \delta_ {2} - u _ {2} \| _ {2}) \\ = \alpha \bar {R} (\delta_ {1}) + (1 - \alpha) \bar {R} (\delta_ {2}), \\ \end{array}
$$
and taking the inf over  $\Delta$  on the left hand-side we obtain the convexity of  $\bar{R}$ .
D.2 Conditional expectation: notation and abuse of notations
In the analysis of the algorithm, we will consider conditional expectation by fixing some of the random variables to specific values. To simplify notations, we will use the following notation: if  $X$  and  $Y$  are random variables and  $g$  is a measurable function, then we denote  $E_{X}[g(X,Y)]$  the expectation with respect to  $X$ , that is:
$$
E _ {X} [ g (X, Y) ] = E [ g (X ^ {\prime}, Y) | Y ],
$$
where  $X'$  is a random variable independent of  $Y$  that has the same law as  $X$ . This is equivalent to the random variable obtained by fixing  $Y$  and taking the expectation with respect to the law of  $X$  only.
In Algorithm 2, we have assumed that the decision maker is able to compute  $\mathbb{E}[f_t(x_t) \mid c_{tk_t}]$  and  $\mathbb{E}[a_t(x_t) \mid c_{tk_t}]$ . While the law of  $c_{tk_t}$  is complicated (because it involves all past decision up to time  $t$ ), what the decision maker needs to compute is the value  $\zeta(c_{tk_t}, x, k_t)$ , where  $\zeta(c, x, k) = \mathbb{E}[f_t(x) \mid c_{tk} = c]$ . Those two quantities are equal because users are i.i.d. and  $k_t$  and  $f_t$  are independent (conditioned on  $z_t$  and the history).
This manipulation is properly stated by Lemma D.3. Indeed by denoting  $\mathcal{H}_{t-1}$  the whole history up to time  $t - 1$ , and applying this Lemma: for  $x_{t}$  the decision generated by the algorithm which is  $(\mathcal{H}_{t-1}, k_{t}, c_{tk_{t}})$  measurable, we have that
$$
\mathbb {E} \left[ f _ {t} \left(x _ {t}\right) \mid c _ {t k _ {t}}, \mathcal {H} _ {t - 1}, k _ {t} \right] = \zeta \left(c _ {t k _ {t}}, x _ {t}, k _ {t}\right) = \mathbb {E} _ {f _ {t}} \left[ f _ {t} \left(x _ {t}\right) \mid c _ {t k _ {t}} \right]. \tag {15}
$$
It is identical for  $a_{t}(x_{t})$ : for  $\zeta^{\prime}(c,x,k) = \mathbb{E}[a_t(x)\mid c_{tk} = c]$  we have the following equality
$$
\mathbb {E} \left[ a _ {t} \left(x _ {t}\right) \mid c _ {t k _ {t}}, \mathcal {H} _ {t - 1}, k _ {t} \right] = \zeta^ {\prime} \left(c _ {t k _ {t}}, x _ {t}, k _ {t}\right) = \mathbb {E} _ {a _ {t}} \left[ a _ {t} \left(x _ {t}\right) \mid c _ {t k _ {t}} \right]. \tag {16}
$$
The following Lemma is a modification of the so-called "Freezing Lemma" (see Lemma 8 of Cesari & Colomboni (2020)) which helps simplifying conditional expectations by allowing you to "fix" the random variable if it is measurable with respect to the conditioning of the expectation:
Lemma D.3. Let  $X, Y$  and  $Z$  be random variables with  $(X, Z)$  independent of  $Y$ , and  $g$  some bounded measurable function. Let us denote by  $\sigma(X)$  the sigma-algebra generated by the random variable  $X$ . Then
$$
\mathbb {E} [ g (X, Y, Z) \mid \sigma (Y, Z) ] = \mathbb {E} _ {X} [ g (X, Y, Z) \mid \sigma (Z) ].
$$
Proof. Let  $V(Y,Z) = \mathbb{E}_X[g(X,Y,Z) \mid \sigma(Z)]$ . To show that  $W$  is the conditional expectation of  $g(X,Y,Z)$  given  $\sigma(Y,Z)$ , we need to show that  $V$  is  $\sigma(Y,Z)$  measurable and that for any bounded  $W$  that is  $\sigma(Y,Z)$  measurable we have
$$
\mathbb {E} [ V W ] = \mathbb {E} [ g (X, Y, Z) W ].
$$