SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 3 (see Appendix D) outlines a family of mechanisms parametrized by  $\epsilon >0$  for this setting. As we will see shortly, with parameter  $\epsilon$ , the mechanism can achieve a PR of  $(1 + \epsilon)$ . This mechanism computes agent  $i$ 's estimate for  $\mu$  as follows. First, let  $Y_{-i}$  be the union of all datasets submitted by the other agents. Similar to Algorithm 1, the algorithm individually adds Gaussian to each  $Y_{-i}$  to obtain  $Z_{i}$  (line 10). Unlike before, this noise is added to the entire dataset and the variance  $\eta_i^2$  of this noise depends on the difference between the sample means of the agent's submission  $Y_{i}$  and all of the other agents' submissions  $Y_{-i}$ . It also depends on two  $\epsilon$ -dependent parameters defined in line 6. Finally, the mechanism deploys the sample mean of  $Y_{i} \cup Z_{i}$  as the estimate for  $\mu$ . The recommended strategies  $s_i^\star = (n_i^\star, f_i^\star)$  for the agents is to simply collect  $n_i^\star = \sigma / \sqrt{cm}$  points and submit it as is  $f_i^\star = \mathbf{I}$ . The following theorem states the main properties of the mechanism.
Theorem 3. Let  $\epsilon >0$ . The following statements about Algorithm 3 and the strategy profile  $s^{\star}$  given in the paragraph above are true: (i)  $s^{\star}$  is a Nash equilibrium. (ii) The mechanism is individually rational at  $s^{\star}$ . (iii) At  $s^{\star}$ , the mechanism is approximately efficient with  $\mathrm{PR}(M,s^{\star})\leq 1 + \epsilon$ .
The above theorem states that it is possible to obtain a social penalty that is arbitrarily close to the global minimum under the given restriction of the strategy space. However, this mechanism is not NIC if agents are allowed to design their own estimator. For instance, if the mechanism returns  $A_{i} = Z_{i}$  (line 10), then using a weighted average of the data in  $X_{i}$  and  $Z_{i}$  yields a lower estimation error than simple average used by the mechanism (see Appendix D). An agent can leverage this insight to collect and submit less data and obtain a lower overall penalty at the expense of other agents. Cai et al. [11] study a setting where agents are incentivized to collect data and submit it truthfully via payments. Interestingly, their corruption method can be viewed as a special case of Algorithm 3 with  $k_{\epsilon} = 1$  and only achieves a  $1.5 \times$  factor of the global minimum social penalty. Moreover, when applied to the more general strategy space, it shares the same shortcomings as the mechanism in Theorem 3.
We studied collaborative normal mean estimation in the presence of strategic agents. Naive mechanisms which only look at the quantity of the dataset submitted, can be manipulated by agents who under-collect and/or fabricate data, leaving all agents worse off. To address this issue, when sharing the others' data with an agent, our mechanism  $M_{\mathrm{c3D}}$  corrupts this dataset proportional to how much the data reported by the agent differs from the other agents. We design minimax optimal estimators for this corrupted dataset to achieve a socially desirable Nash equilibrium.
Future directions: We believe that designing mechanisms for other collaborative learning settings may require relaxing the exact NIC guarantees to make the analysis tractable. For many learning problems, it is difficult to design exactly optimal estimators, and it is common to settle for rate-optimal (i.e up to constants) estimators [24]. For instance, even simply relaxing to high dimensional distributions with bounded variance,  $M_{\mathrm{c3D}}$  can only provide an approximate NIC guarantee.
This project is supported in part by NSF grants 1545481, 1704117, 1836978, 2023239, 2041428, 2202457, ARO MURI W911NF2110317, and AF CoE FA9550-18-1-0166.
[1] Ads Data Hub. https://developers.google.com/ads-data-hub/guides/intro. Accessed: 2022-05-10.
[38] Tianhao Wang, Johannes Rausch, Ce Zhang, Ruoxi Jia, and Dawn Song. A principled approach to data valuation for federated learning. Federated Learning: Privacy and Incentive, pages 153-167, 2020.
[39] Xinyi Xu, Lingjuan Lyu, Xingjun Ma, Chenglin Miao, Chuan Sheng Foo, and Bryan Kian Hsiang Low. Gradient driven rewards to guarantee fairness in collaborative machine learning. Advances in Neural Information Processing Systems, 34:16104-16117, 2021.
[40] Wenting Zheng, Raluca Ada Popa, Joseph E Gonzalez, and Ion Stoica. Helen: Maliciously secure cooptitive learning for linear models. In 2019 IEEE symposium on security and privacy (SP), pages 724-738. IEEE, 2019.
In this section, we prove Theorem 1. This section is organized as follows. First, in §A.1, we consider the case where  $m \leq 4$ . In the remainder of this section, we will assume  $m \geq 5$ . First, in §A.2, we will show that (7) can be solved for  $\alpha$  and state some properties about the solution. Then, in §A.3, we will prove the Nash incentive compatibility result, in §A.4 we will prove individual rationality, and in §A.5, we will prove the result on efficiency.
A.1 When  $m \leq 4$
First, consider the (easy) case  $m \leq 4$ . At  $s_i^\star$ , the total amount of data collected is  $\sigma / \sqrt{c}$  as each agent will be collecting  $n_i^\star = \frac{\sigma}{m\sqrt{c}}$  (see (8)). As there is no corrupted dataset,  $h_i^\star$  simply reduces to the sample mean of  $X_i \cup Y_{-i}$ . The individual rationality property follows from the following simple calculation:
$$
p _ {i} (M _ {\mathrm {C 3 D}}, s ^ {\star}) = \left(1 + \frac {1}{m}\right) \sqrt {c} \sigma <   2 \sqrt {c} \sigma = p _ {\mathrm {m i n}} ^ {\mathrm {I R}}.
$$
Similarly, the bound on the ratio between the penalties can also be obtained via the following calculation:
$$
\mathrm {P R} = \frac {m \left(1 + \frac {1}{m}\right) \sqrt {c} \sigma}{2 \sigma \sqrt {c m}} <   \sqrt {m} \leq 2.
$$
Finally, to show NIC, consider agent  $i$  and assume that all other agents have followed the recommended strategies, i.e. collected  $\sigma / (m\sqrt{c})$ . Then, the agent will have an uncorrupted dataset  $Y_{-i} = \bigcup_{j \neq i} X_j$  of  $n_{-i}^{\star} = (m - 1)\sigma / (m\sqrt{c})$  points with no corruption. Regardless of what she chooses to submit, the best estimator she could use with the union of this dataset  $Y_{-i}$  and the data she collects  $X_i$  and will be the sample mean as it is minimax optimal. The number of points that minimizes her penalty is,
$$
\underset {n _ {i}} {\operatorname {a r g m i n}} \left(\sup _ {\mu} \mathbb {E} \left[ (h _ {i} (X _ {i}, Y _ {i}, Y _ {- i}) - \mu) ^ {2} \Big | \mu \right] + c n _ {i}\right) = \underset {n _ {i} \in \mathbb {R}} {\operatorname {a r g m i n}} \left(\frac {\sigma^ {2}}{n _ {i} + n _ {- i} ^ {\star}} + c n _ {i}\right) = \frac {\sigma}{m \sqrt {c}}
$$
Finally, as  $A_{i}$  does not depend on  $f_{i}$  under these conditions, there is no incentive to fabricate or falsify data, i.e. choosing anything other than  $f^{\star} = \mathbf{I}$  does not lower her utility.
In the remainder of this section, will study the harder case,  $m \geq 4$ .
A.2 Existence of a solution to (7) and some of its properties
In this section, we show that  $G\left(\frac{\sigma^{1/2}}{(cm)^{1/4}}\right) < 0$  and  $G\left(\left(1 + \frac{C_m}{m}\right)\frac{\sigma^{1/2}}{(cm)^{1/4}}\right) > 0$ , where  $C_m = 20$  when  $m \leq 20$  and  $C_m = 5$  when  $m > 20$ . This means equation  $G(\alpha) = 0$  has solution in  $\left(\frac{\sigma^{1/2}}{(cm)^{1/4}}, \left(1 + \frac{C_m}{m}\right)\frac{\sigma^{1/2}}{(cm)^{1/4}}\right)$ .
First, in Lemma 12, we derive an asymptotic expansion of the Gaussian complementary error function, and construct lower and upper bounds for  $G(\alpha)$  that are easier to work with. We have restated these lower (ErfcLB) and upper (ErfcUB) bounds below.
$$
\operatorname {E r f c} _ {\mathrm {U B}} (x) := \frac {1}{\sqrt {\pi}} \left(\frac {\exp (- x ^ {2})}{x} - \frac {\exp (- x ^ {2})}{2 x ^ {3}} + \frac {3 \exp (- x ^ {2})}{4 x ^ {5}}\right) \tag {11}
$$
$$
\operatorname {E r f c} _ {\mathrm {L B}} (x) := \frac {1}{\sqrt {\pi}} \left(\frac {\exp (- x ^ {2})}{x} - \frac {\exp (- x ^ {2})}{2 x ^ {3}}\right) \tag {12}
$$
We can now use this to derive the following lower  $(G_{\mathrm{LB}})$  and upper  $(G_{\mathrm{UB}})$  bounds on  $G$ . Here, we have used the fact that  $4(m + 1)\frac{\alpha^2}{\sigma\sqrt{m / c}} - 1 > 0$  when  $\alpha \geq (\sigma /\sqrt{cm})^{1 / 2}$ . We have:
$$
\begin{array}{l} G _ {\mathrm {L B}} (\alpha) := \left(\frac {m - 4}{m - 2} \frac {4 \alpha^ {2}}{\sigma / \sqrt {c m}} - 1\right) \frac {4 \alpha}{\sqrt {\sigma} (m / c) ^ {1 / 4}} \\ - \left(4 (m + 1) \frac {\alpha^ {2}}{\sigma \sqrt {m / c}} - 1\right) \sqrt {2 \pi} \exp \left(\frac {\sigma \sqrt {m / c}}{8 \alpha^ {2}}\right) \mathrm {E r f c _ {U B}} \left(\frac {\sqrt {\sigma} (m / c) ^ {1 / 4}}{2 \sqrt {2} \alpha}\right), \\ \end{array}
$$
Figure 1: Plot for  $G\left(\left(1 + \frac{C_m}{m}\right)\frac{\sigma^{1/2}}{(cm)^{1/4}}\right)$ . See G_em.plot.py. The discontinuity at  $m = 20$  is due to the different values for  $C_m$  when  $m \leq 20$  and when  $m > 20$ .
$$
\begin{array}{l} G _ {\mathrm {U B}} (\alpha) := \left(\frac {m - 4}{m - 2} \frac {4 \alpha^ {2}}{\sigma / \sqrt {c m}} - 1\right) \frac {4 \alpha}{\sqrt {\sigma} (m / c) ^ {1 / 4}} \\ - \left(4 (m + 1) \frac {\alpha^ {2}}{\sigma \sqrt {m / c}} - 1\right) \sqrt {2 \pi} \exp \left(\frac {\sigma \sqrt {m / c}}{8 \alpha^ {2}}\right) \operatorname {E r f c} _ {\mathrm {L B}} \left(\frac {\sqrt {\sigma} (m / c) ^ {1 / 4}}{2 \sqrt {2} \alpha}\right). \\ \end{array}
$$
By first, substituting  $\sigma / \sqrt{cm}$  for  $\alpha$  in the expressions for  $G_{\mathrm{UB}}$  and  $\mathrm{Erfc}_{\mathrm{UB}}$ , and then via a sequence of algebraic manipulations, we can verify that
$$
\begin{array}{l} G \left(\frac {\sigma^ {1 / 2}}{(c m) ^ {1 / 4}}\right) \leq G _ {\mathrm {U B}} \left(\frac {\sigma^ {1 / 2}}{(c m) ^ {1 / 4}}\right) \\ = \frac {4 \left(\frac {4 (m - 4)}{m - 2} - 1\right) \left(\frac {\sigma}{\sqrt {c m}}\right) ^ {1 / 2}}{\sqrt {\sigma} \left(\frac {m}{c}\right) ^ {1 / 4}} - \sqrt {2} \left(\frac {4 (m + 1)}{\sqrt {\frac {m}{c}} \sqrt {c m}} - 1\right) \left(\frac {2 \sqrt {2} \left(\frac {\sigma}{\sqrt {c m}}\right) ^ {1 / 2}}{\sqrt {\sigma} \left(\frac {m}{c}\right) ^ {1 / 4}} - \frac {8 \sqrt {2} \left(\frac {\sigma}{\sqrt {c m}}\right) ^ {3 / 2}}{\sigma^ {3 / 2} \left(\frac {m}{c}\right) ^ {3 / 4}}\right) \\ = - \frac {1 2 8}{(m - 2) m ^ {5 / 2}} <   0. \\ \end{array}
$$
Next, we will show that  $G\left(\left(1 + \frac{C_m}{m}\right)\frac{\sigma^{1/2}}{(cm)^{1/4}}\right) > 0$  by studying the lower bound  $G_{\mathrm{LB}}$ . For  $m \in [5,500]$ , we can verify individually that  $G\left(\left(1 + \frac{C_m}{m}\right)\frac{\sigma^{1/2}}{(cm)^{1/4}}\right) > 0$  (See Figure 1). For  $m > 500$ , we have:
$$
\begin{array}{l} G \left(\left(1 + \frac {C _ {m}}{m}\right) \frac {\sigma^ {1 / 2}}{(c m) ^ {1 / 4}}\right) = G \left(\left(1 + \frac {5}{m}\right) \frac {\sigma^ {1 / 2}}{(c m) ^ {1 / 4}}\right) \geq G _ {\mathrm {L B}} \left(\left(1 + \frac {5}{m}\right) \frac {\sigma^ {1 / 2}}{(c m) ^ {1 / 4}}\right) \\ = \frac {4 \left(\frac {4 \left(\frac {5}{m} + 1\right) ^ {2} (m - 4)}{m - 2} - 1\right) \left(\frac {5}{m} + 1\right) \left(\frac {\sigma}{\sqrt {c m}}\right) ^ {1 / 2}}{\sqrt {\sigma} \left(\frac {m}{c}\right) ^ {1 / 4}} \\ - \sqrt {2} \left(\frac {4 \left(\frac {5}{m} + 1\right) ^ {2} (m + 1)}{\sqrt {\frac {m}{c}} \sqrt {c m}} - 1\right) \left(\frac {2 \sqrt {2} \left(\frac {5}{m} + 1\right) \left(\frac {\sigma}{\sqrt {c m}}\right) ^ {1 / 2}}{\sqrt {\sigma} \left(\frac {m}{c}\right) ^ {1 / 4}} - \frac {8 \sqrt {2} \left(\frac {5}{m} + 1\right) ^ {3} \left(\frac {\sigma}{\sqrt {c m}}\right) ^ {3 / 2}}{\sigma^ {3 / 2} \left(\frac {m}{c}\right) ^ {3 / 4}} \right. \\ \left. + \frac {9 6 \sqrt {2} \left(\frac {5}{m} + 1\right) ^ {5} \left(\frac {\sigma}{\sqrt {c m}}\right) ^ {5 / 2}}{\sigma^ {5 / 2} \left(\frac {m}{c}\right) ^ {5 / 4}}\right) \\ = \frac {6 4 (m + 5) ^ {3} \left(m ^ {6} - 1 9 1 m ^ {5} - 1 5 6 6 m ^ {4} - 3 9 2 0 m ^ {3} + 2 1 0 0 m ^ {2} + 1 9 5 0 0 m + 1 5 0 0 0\right)}{(m - 2) m ^ {2 1 / 2}}. \\ \end{array}
$$
When  $m > 500$
$$
m ^ {6} - 1 9 1 m ^ {5} - 1 5 6 6 m ^ {4} - 3 9 2 0 m ^ {3} = m ^ {3} \left(m ^ {3} - 1 9 1 m ^ {2} - 1 5 6 6 m - 3 9 2 0\right)
$$
$$
\begin{array}{l} > m ^ {3} \left(\left(2 0 0 + 2 0 0 + 1 0 0\right) m ^ {2} - 1 9 1 m ^ {2} - 1 5 6 6 m - 3 9 2 0\right) \\ > m ^ {3} \left(2 0 0 m ^ {2} + 1 0 ^ {5} m + 2. 5 \times 1 0 ^ {7} - 1 9 1 m ^ {2} - 1 5 6 6 m - 3 9 2 0\right) > 0. \\ \end{array}
$$
Combining the results from the two previous displays, we have,  $G\left(\left(1 + \frac{C_m}{m}\right)\frac{\sigma^{1/2}}{(cm)^{1/4}}\right) > 0$  which completes the proof for this section.
A.3 Algorithm 1 is Nash incentive compatible
In this section, we will prove the following lemma which states that  $s_i^\star$ , as defined in (8) is a Nash equilibrium in  $M_{\mathrm{C3D}}$ .
Lemma 4 (NIC). The recommended strategies  $s^{\star} = \{(n_{i}^{\star}, f_{i}^{\star}, h_{i}^{\star})\}_{i}$  as defined in (8) in mechanism  $M_{\mathrm{C3D}}$  (Algorithm 1) satisfies:
$$
p _ {i} \left(M _ {\mathrm {C 3 D}}, s ^ {\star}\right) \leq p _ {i} \left(M _ {\mathrm {C 3 D}}, \left(s _ {i}, s _ {- i} ^ {\star}\right)\right)
$$
for all  $i\in [m]$  and  $s_i\in \mathbb{N}\times \mathcal{F}\times \mathcal{H}$
The Proof of Lemma 4 relies on the following two lemmas:
Lemma 5 (Optimal Estimation and Submission). For all  $i \in [m]$  and  $(n_i, f_i, h_i) \in \mathbb{N} \times \mathcal{F} \times \mathcal{H}$ ,
$$
p _ {i} \left(M _ {\mathrm {C 3 D}}, \left(\left(n _ {i}, f _ {i} ^ {\star}, h _ {i} ^ {\star}\right), s _ {- i} ^ {\star}\right)\right) \leq p _ {i} \left(M _ {\mathrm {C 3 D}}, \left(\left(n _ {i}, f _ {i}, h _ {i}\right), s _ {- i} ^ {\star}\right)\right).
$$
See the Proof of Lemma 5 in  $\S A.3.1$
Lemma 6 (Optimal Sample Size). For all  $i \in [m]$  and  $n_i \in \mathbb{N}$ ,
$$
p _ {i} \left(M _ {\mathrm {C 3 D}}, \left(\left(n _ {i} ^ {\star}, f _ {i} ^ {\star}, h _ {i} ^ {\star}\right), s _ {- i} ^ {\star}\right)\right) \leq p _ {i} \left(M _ {\mathrm {C 3 D}}, \left(\left(n _ {i}, f _ {i} ^ {\star}, h _ {i} ^ {\star}\right), s _ {- i} ^ {\star}\right)\right).
$$
See the Proof of Lemma 6 in  $\S A.3.2$
Proof of Lemma 4. By Lemma 5 and 6, we have, for all  $i \in [m]$  and  $s_i' = (n_i, f_i, h_i) \in \mathbb{N} \times \mathcal{F} \times \mathcal{H}$ ,
$$
\begin{array}{l} p _ {i} \left(M _ {\mathrm {C 3 D}}, s ^ {\star}\right) = p _ {i} \left(M _ {\mathrm {C 3 D}}, \left(\left(n _ {i} ^ {\star}, f _ {i} ^ {\star}, h _ {i} ^ {\star}\right), s _ {- i} ^ {\star}\right)\right) \leq p _ {i} \left(M _ {\mathrm {C 3 D}}, \left(\left(n _ {i}, f _ {i} ^ {\star}, h _ {i} ^ {\star}\right), s _ {- i} ^ {\star}\right)\right) \\ \leq p _ {i} \left(M _ {\mathrm {C 3 D}}, \left(\left(n _ {i}, f _ {i}, h _ {i}\right), s _ {- i} ^ {\star}\right)\right) = p _ {i} \left(M _ {\mathrm {C 3 D}}, \left(s _ {i} ^ {\prime}, s _ {- i} ^ {\star}\right)\right) \\ \end{array}
$$
A.3.1 Proof of Lemma 5
In this section, we will prove Lemma 5, which, intuitively states that, regardless of the amount of data collected, agent  $i$  should submit the data as is  $(f_i^* = \mathbf{I})$  and use the weighted average estimator in (8) to estimate  $\mu$ . We will do so via the following three step procedure, inspired by well-known techniques for proving minimax optimality of estimators (e.g. see Theorem 1.12, Chapter 5 of Lehmann and Casella [24]).
1. First, we construct a sequence of prior distributions  $\{\Lambda_{\ell}\}_{\ell \geq 1}$  for  $\mu$  and calculate the sequence of Bayesian risks under the prior distributions:
$$
R _ {\ell} := \inf  _ {f _ {i} \in \mathcal {A}, h _ {i} \in \mathcal {H}} \mathbb {E} _ {\mu \sim \Lambda_ {\ell}} \left[ \mathbb {E} \left[ \left(h _ {i} \left(X _ {i}, f _ {i} \left(X _ {i}\right), A _ {i}\right) - \mu\right) ^ {2} \mid \mu \right] \right], \quad \ell \geq 1.
$$
2. Then, we will show that  $\lim_{\ell \to \infty}R_{\ell} = \sup_{\mu}\mathbb{E}\big[(h_i^{\star}(X_i,f_i^{\star}(X_i),A_i) - \mu)^2 |\mu \big]$ .
3. Finally, as the Bayesian risk is a lower bound on maximum risk, we will conclude that  $(f_i^\star, h_i^\star)$  is minimax optimal.
Without loss of generality, we focus only on the deterministic  $f_{i}$  and  $h_{i}$ . If either of them are stochastic, we can condition on the external source of randomness and treat them as deterministic functions. Our proof holds for any realization of this external source of randomness, and hence it will hold in expectation as well. Similarly,  $Z_{i}$  is randomly chosen in Algorithm 1. In the following, we condition on this randomness and the entire proof will carry through.
Note that  $Y_{i} = f_{i}(X_{i})$ . We will use both of them interchangeably in the subsequent proof.
Step 1 (Bounding the Bayes' risk under the sequence of priors): We will use a sequence of normal priors  $\Lambda_{\ell} \coloneqq \mathcal{N}(0, \ell^{2})$  for all  $\ell \geq 1$ . To bound the Bayes' risk under these priors, we will first note that for a fixed  $f_{i} \in \mathcal{F}$ ,
$$
x \mid \mu \sim \mathcal {N} (\mu , \sigma^ {2}) \quad \forall x \in X _ {i} \cup Z _ {i}; \tag {13}
$$
$$
x \mid \mu , \eta_ {i} ^ {2} \sim \mathcal {N} \left(\mu , \sigma^ {2} + \eta_ {i} ^ {2}\right) \quad \forall x \in Z _ {i} ^ {\prime}. \tag {14}
$$
Here, recall that  $\eta_i^2$  is a function of  $Y_{i}$  and  $Z_{i}$ . Because both  $Y_{i} = f_{i}(X_{i})$  and  $\eta_i^2$  are deterministic functions of  $X_{i}, Z_{i}$  when  $f_{i}$  is fixed, the posterior distribution for  $\mu$  conditioned on  $(X_{i}, Y_{i}, A_{i})$  can be calculated as follows:
$$
\begin{array}{l} p \left(\mu \mid X _ {i}, Y _ {i}, A _ {i}\right) = p \left(\mu \mid X _ {i}, Y _ {i}, Z _ {i}, Z _ {i} ^ {\prime}, \eta_ {i} ^ {2}\right) = p \left(\mu \mid X _ {i}, Z _ {i}, Z _ {i} ^ {\prime}\right) \\ \propto p (\mu , X _ {i}, Z _ {i}, Z _ {i} ^ {\prime}) = p \left(Z _ {i} ^ {\prime} \mid X _ {i}, Z _ {i}, \mu\right) p \left(X _ {i}, Z _ {i} \mid \mu\right) p (\mu) = p \left(Z _ {i} ^ {\prime} \mid X _ {i}, Z _ {i}, \mu\right) p \left(X _ {i} \mid \mu\right) p \left(Z _ {i} \mid \mu\right) p (\mu) \\ \propto \exp \left(- \frac {1}{2 (\sigma^ {2} + \eta_ {i} ^ {2})} \sum_ {x \in Z _ {i} ^ {\prime}} (x - \mu) ^ {2}\right) \exp \left(- \frac {1}{2 \sigma^ {2}} \sum_ {x \in X _ {i} \cup Z _ {i}} (x - \mu) ^ {2}\right) \exp \left(- \frac {\mu^ {2}}{2 \ell^ {2}}\right) \\ \propto \exp \left(- \frac {1}{2} \left(\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}} + \frac {1}{\ell^ {2}}\right) \mu^ {2}\right) \exp \left(\frac {1}{2} 2 \left(\frac {\sum_ {x \in Z _ {i} ^ {\prime}} x}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {\sum_ {x \in X _ {i} \cup Z _ {i}} x}{\sigma^ {2}}\right) \mu\right) \\ = \exp \left(- \frac {1}{2} \left(\frac {1}{\sigma_ {\ell} ^ {2}} \mu^ {2} - 2 \frac {\mu_ {\ell}}{\sigma_ {\ell} ^ {2}} \mu\right)\right) \propto \exp \left(- \frac {1}{2 \sigma_ {\ell} ^ {2}} (\mu - \mu_ {\ell}) ^ {2}\right), \\ \end{array}
$$
$$
\mu_ {\ell} = \frac {\frac {\sum_ {x \in Z _ {i} ^ {\prime}} x}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {\sum_ {x \in X _ {i} \cup Z _ {i}} x}{\sigma^ {2}}}{\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}} + \frac {1}{\ell^ {2}}}, \quad \text {a n d} \quad \sigma_ {\ell} ^ {2} = \frac {1}{\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}} + \frac {1}{\ell^ {2}}}. \tag {15}
$$
We can therefore conclude that (despite the non i.i.d nature of the data), the posterior for  $\mu$  is Gaussian with mean and variance as shown above. We have:
$$
\mu | X _ {i}, Y _ {i}, A _ {i} \sim \mathcal {N} (\mu_ {\ell}, \sigma_ {\ell} ^ {2}).
$$
Next, following standard steps (See Corollary 1.2 in Chapter 4 of [24]), we know that  $\mathbb{E}_{\mu}\Big[(h_i(X_i,Y_i,A_i) - \mu)^2 |X_i,Y_i,A_i\Big]$  is minimized when  $h_i(X_i,Y_i,A_i) = \mathbb{E}_{\mu}[\mu |X_i,Y_i,A_i] = \mu_{\ell}$ . This shows that for any  $f_{i}\in h_{i}$ , the optimal  $h_i$  is simply the posterior mean of  $\mu$  under the prior  $\Lambda_{\ell}$  conditioned on  $(X_{i},f_{i}(X_{i}),A_{i})$ . We can rewrite the minimum averaged risk over  $\mathcal{H}$  by switching the order of expectation:
$$
\begin{array}{l} \inf _ {h _ {i} \in \mathcal {H}} \mathbb {E} _ {\mu \sim \Lambda_ {\ell}} \left[ \mathbb {E} \left[ (h _ {i} (X _ {i}, Y _ {i}, A _ {i}) - \mu) ^ {2} | \mu \right] \right] \\ = \inf _ {h _ {i} \in \mathcal {H}} \mathbb {E} _ {X _ {i}, Z _ {i}, Z _ {i} ^ {\prime}} \Big [ \mathbb {E} _ {\mu} \Big [ (h _ {i} (X _ {i}, Y _ {i}, A _ {i}) - \mu) ^ {2} | X _ {i}, Z _ {i}, Z _ {i} ^ {\prime} \Big ] \Big ] \\ = \mathbb {E} _ {X _ {i}, Z _ {i}, Z _ {i} ^ {\prime}} \Big [ \mathbb {E} _ {\mu} \Big [ (\mu_ {\ell} - \mu) ^ {2} | X _ {i}, Z _ {i}, Z _ {i} ^ {\prime} \Big ] \Big ] = \mathbb {E} _ {X _ {i}, Z _ {i}, Z _ {i} ^ {\prime}} \big [ \sigma_ {\ell} ^ {2} \big ] \\ = \mathbb {E} _ {X _ {i}, Z _ {i}} \left[ \frac {1}{\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}} + \frac {1}{\ell^ {2}}} \right], \tag {16} \\ \end{array}
$$
the expectation in the last step involves only  $X_{i}$ ,  $Z_{i}$  because  $\sigma_{\ell}^{2}$  depends only on  $X_{i}, Z_{i}$  and  $|Z_i^{\prime}|$ , but not the instantiation of  $Z_{i}^{\prime}$ .
Next, we will show that (16) is minimized for the following choice of  $f_{i}$  which shrinks each points in  $X_{i}$  by an amount that depends on the prior  $\Lambda_{\ell}$ 's variance  $\ell^2$ :
$$
f _ {i} \left(X _ {i}\right) = \left\{\frac {\left| X _ {i} \right| / \sigma^ {2}}{\left| X _ {i} \right| / \sigma^ {2} + 1 / \ell^ {2}} x, \text {f o r e a c h} x \in X _ {i} \right\}. \tag {17}
$$
Remark 1. An interesting observation (albeit not critical to the proof) here is that  $f_{i}$  in (17) converges pointwise to  $f_{i}^{\star}$ , i.e.  $\mathbf{I}$ , as  $\ell \to \infty$ . This shows that the optimal submission function under the prior converges to  $f_{i}^{\star}$ . We can make a similar observation about the posterior mean in (15), where  $\mu_{\ell}$  converges to  $h_{i}^{\star}$  as  $\ell \to \infty$ .
To prove (17), we first define the following quantities.
$$
\widehat {\mu} (X _ {i}) := \frac {1}{| X _ {i} |} \sum_ {x \in X _ {i}} x, \quad \widehat {\mu} (Y _ {i}) := \frac {1}{| Y _ {i} |} \sum_ {x \in Y _ {i}} x, \quad \widehat {\mu} (Z _ {i}) := \frac {1}{| Z _ {i} |} \sum_ {s \in Z _ {i}} x.
$$
We will also find it useful to express  $\eta_i^2$  as follows. Here  $\alpha$  is as defined in (7). We have:
$$
\eta_ {i} ^ {2} = \alpha^ {2} (\widehat {\mu} (Y _ {i}) - \widehat {\mu} (Z _ {i})) ^ {2}
$$
The following calculations show that, conditioned on  $X_{i}$ ,  $\widehat{\mu}(Z_{i}) - \mu$  and  $\mu - \frac{|X_{i}| / \sigma^{2}}{|X_{i}| / \sigma^{2} + 1 / \ell^{2}} \widehat{\mu}(X_{i})$  are independent Gaussian random variables<sup>3</sup>: