SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} p \left(\widehat {\mu} \left(Z _ {i}\right) - \mu , \mu \mid X _ {i}\right) \propto p \left(\widehat {\mu} \left(Z _ {i}\right) - \mu , \mu , X _ {i}\right) \\ = p \left(\widehat {\mu} \left(Z _ {i}\right) - \mu , X _ {i} \mid \mu\right) p (\mu) = p \left(\widehat {\mu} \left(Z _ {i}\right) - \mu \mid \mu\right) p \left(X _ {i} \mid \mu\right) p (\mu) \\ \propto \exp \left(- \frac {1}{2} \frac {| Z _ {i} |}{\sigma^ {2}} \left(\widehat {\mu} \left(Z _ {i}\right) - \mu\right) ^ {2}\right) \exp \left(- \frac {1}{2 \sigma^ {2}} \sum_ {x \in X _ {i}} (x - \mu) ^ {2}\right) \exp \left(- \frac {1}{2 \ell^ {2}} \mu^ {2}\right) \\ \propto \underbrace {\exp \left(- \frac {1}{2} \frac {| Z _ {i} |}{\sigma^ {2}} (\widehat {\mu} (Z _ {i}) - \mu) ^ {2}\right)} _ {\propto p (\widehat {\mu} (Z _ {i}) - \mu | X _ {i})} \underbrace {\exp \left(- \frac {1}{2} \left(\frac {| X _ {i} |}{\sigma^ {2}} + \frac {1}{\ell^ {2}}\right) \left(\mu - \frac {| X _ {i} | / \sigma^ {2}}{| X _ {i} | / \sigma^ {2} + 1 / \ell^ {2}} \widehat {\mu} (X _ {i})\right) ^ {2}\right)} _ {\propto p \left(\mu - \frac {| X _ {i} | / \sigma^ {2}}{| X _ {i} | / \sigma^ {2} + 1 / \ell^ {2}} \widehat {\mu} (X _ {i}) | X _ {i}\right)} \\ \end{array}
$$
Thus conditioning on  $X_{i}$  we can write
$$
\left( \begin{array}{c} \widehat {\mu} (Z _ {i}) - \mu \\ \mu - \frac {| X _ {i} | / \sigma^ {2}}{| X _ {i} | / \sigma^ {2} + 1 / \ell^ {2}} \widehat {\mu} (X _ {i}) \end{array} \right) \sim \mathcal {N} \left(\left( \begin{array}{c} 0 \\ 0 \end{array} \right), \left( \begin{array}{c c} \frac {\sigma^ {2}}{| Z _ {i} |} & 0 \\ 0 & \frac {1}{| X _ {i} | / \sigma^ {2} + 1 / \ell^ {2}} \end{array} \right)\right).
$$
which leads us to
$$
\left. \widehat {\mu} \left(Z _ {i}\right) - \frac {\left| X _ {i} \right| / \sigma^ {2}}{\left| X _ {i} \right| / \sigma^ {2} + 1 / \ell^ {2}} \widehat {\mu} \left(X _ {i}\right) \right\rvert   X _ {i} \sim \mathcal {N} \left(0, \underbrace {\frac {\sigma^ {2}}{\left| Z _ {i} \right|} + \frac {1}{\left| X _ {i} \right| / \sigma^ {2} + 1 / \ell^ {2}}} _ {=: \bar {\sigma} _ {\ell} ^ {2}}\right) \tag {18}
$$
Next, we will rewrite the squared difference in  $\eta_i^2$  as follows:
$$
\begin{array}{l} \frac {\eta_ {i} ^ {2}}{\alpha^ {2}} = (\widehat {\mu} (Y _ {i}) - \widehat {\mu} (Z _ {i})) ^ {2} \\ = \left(\underbrace {\widehat {\mu} (Z _ {i}) - \frac {| X _ {i} | / \sigma^ {2}}{| X _ {i} | / \sigma^ {2} + 1 / \ell^ {2}} \widehat {\mu} (X _ {i})} _ {= \tilde {\sigma} _ {\ell} e} + \left(\underbrace {\frac {| X _ {i} | / \sigma^ {2}}{| X _ {i} | / \sigma^ {2} + 1 / \ell^ {2}} \widehat {\mu} (X _ {i}) - \widehat {\mu} (Y _ {i})} _ {=: \phi (X _ {i}, f _ {i})}\right)\right) ^ {2}. \\ \end{array}
$$
Here, we observe that the first part of the RHS above is equal to  $\tilde{\sigma}_{\ell}$ , where  $e$  is a normal noise  $e|X_{i} \sim \mathcal{N}(0,1)$  and  $\tilde{\sigma}_{\ell}$  is as defined in (18). For brevity, we will denote the second part of the RHS as  $\phi(X_{i},f_{i})$ , which intuitively characterizes the difference between  $X_{i}$  and  $Y_{i}$ . Importantly,  $\phi(X_{i},f_{i}) = 0$  when  $f_{i}$  is chosen to be (17).
Using  $e$  and  $\phi$ , we can rewrite (16) using conditional expectation:
$$
\begin{array}{l} \mathbb {E} _ {X _ {i}, Z _ {i}} \left[ \frac {1}{\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}} + \frac {1}{\ell^ {2}}} \right] = \mathbb {E} _ {X _ {i}} \left[ \mathbb {E} _ {Z _ {i} | X _ {i}} \left[ \frac {1}{\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}} + \frac {1}{\ell^ {2}}} \right] \right] \\ = \mathbb {E} _ {X _ {i}} \left[ \mathbb {E} _ {e | X _ {i}} \left[ \frac {1}{\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \alpha^ {2} (\tilde {\sigma} _ {\ell} e + \phi (X _ {i} , f _ {i})) ^ {2}} + \frac {| X _ {i} | + | Z _ {i} |}{\sigma^ {2}} + \frac {1}{\ell^ {2}}} \right] \right] \\ \end{array}
$$
This is akin to the observation that given  $u, v \sim \mathcal{N}(0,1)$ , then  $u - v$  and  $u + v$  are independent.
$$
= \mathbb {E} _ {X _ {i}} \left[ \int_ {- \infty} ^ {\infty} \underbrace {\frac {1}{\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \alpha^ {2} \tilde {\sigma} _ {\ell} ^ {2} (e + \phi \left(X _ {i} , f _ {i}\right) / \tilde {\sigma} _ {\ell}) ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}} + \frac {1}{\ell^ {2}}} _ {=: F _ {1} (e + \phi \left(X _ {i}, f _ {i}\right) / \tilde {\sigma} _ {\ell})} \underbrace {\frac {1}{\sqrt {2 \pi}} \exp \left(- \frac {e ^ {2}}{2}\right)} _ {=: F _ {2} (e)} d e \right], \tag {19}
$$
where we use the fact that  $e|X_i \sim \mathcal{N}(0,1)$  in the last step. To proceed, we will consider the inner expectation in the RHS above. For any fixed  $X_i$ ,  $F_1(\cdot)$  (as marked on the RHS) is an even function that monotonically increases on  $[0,\infty)$  bounded by  $\frac{\sigma}{|X_i| + |Z_i|}$  and  $F_2(\cdot)$  (as marked on the RHS) is an even function that monotonically decreases on  $[0,\infty)$ . That means, for any  $a \in \mathbb{R}$ ,
$$
\int_ {- \infty} ^ {\infty} F _ {1} (e - a) F _ {2} (e) d e \leq \int_ {- \infty} ^ {\infty} \frac {\sigma}{| X _ {i} | + | Z _ {i} |} F _ {2} (e) d e = \frac {\sigma}{| X _ {i} | + | Z _ {i} |} <   \infty .
$$
By a corollary of the Hardy-Littlewood inequality in Lemma 9, we have
$$
\int_ {- \infty} ^ {\infty} F _ {1} \left(e + \phi \left(X _ {i}, f _ {i}\right) / \tilde {\sigma} _ {\ell}\right) F _ {2} (e) d e \geq \int_ {- \infty} ^ {\infty} F _ {1} (e) F _ {2} (e) d e, \tag {20}
$$
the equality is achieved when  $\phi(X_i, f_i) / \tilde{\sigma}_\ell = 0$ . In particular, the equality holds when  $f_i$  is chosen as specified in (17).
Now, to complete Step 1, we combine (16), (19) and (20) to obtain
$$
\begin{array}{l} \inf  _ {h _ {i} \in \mathcal {H}} \mathbb {E} _ {\mu \sim \Lambda_ {\ell}} \left[ \mathbb {E} \left[ \left(h _ {i} \left(X _ {i}, Y _ {i}, A _ {i}\right) - \mu\right) ^ {2} | \mu \right] \right] = \mathbb {E} _ {X _ {i}} \left[ \int_ {- \infty} ^ {\infty} F _ {1} (e + \phi \left(X _ {i}, f _ {i}\right) / \tilde {\sigma} _ {\ell}) F _ {2} (e) d e \right] \\ \geq \mathbb {E} _ {X _ {i}} \left[ \int_ {- \infty} ^ {\infty} F _ {1} (e) F _ {2} (e) d e \right] = \int_ {- \infty} ^ {\infty} F _ {1} (e) F _ {2} (e) d e, \tag {21} \\ \end{array}
$$
where the last step is because conditioning on each realization of  $X_{i}$ , the term inside the expectation is a constant. Using (21), we can write the Bayes risk  $R_{\ell}$  under any prior  $\Lambda_{\ell}$  as:
$$
\begin{array}{l} R _ {\ell} := \inf  _ {f _ {i} \in \mathcal {A}, h _ {i} \in \mathcal {H}} \mathbb {E} _ {\mu \sim \Lambda_ {\ell}} \left[ \mathbb {E} \left[ \left(h _ {i} \left(X _ {i}, Y _ {i}, A _ {i}\right) - \mu\right) ^ {2} \mid \mu \right] \right] = \int_ {- \infty} ^ {\infty} F _ {1} (e) F _ {2} (e) d e \\ = \mathbb {E} _ {e \sim \mathcal {N} (0, 1)} \left[ \frac {1}{\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \alpha^ {2} \tilde {\sigma} _ {\ell} ^ {2} e ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}} + \frac {1}{\ell^ {2}}} \right] \\ \end{array}
$$
Because the term inside the expectation is bounded by  $\frac{\sigma^2}{|X_i| + |Z_i|}$  and  $\lim_{\ell \to \infty} \tilde{\sigma}_{\ell}^{2} = \frac{\sigma^{2}}{|Z_{i}|} + \frac{\sigma^{2}}{|X_{i}|}$ , we can use dominated convergence theorem to show that:
$$
R _ {\infty} := \lim  _ {\ell \rightarrow \infty} R _ {\ell} = \mathbb {E} _ {e \sim \mathcal {N} (0, 1)} \left[ \frac {1}{\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \alpha^ {2} \left(\frac {\sigma^ {2}}{\left| Z _ {i} \right|} + \frac {\sigma^ {2}}{\left| X _ {i} \right|}\right) e ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}}} \right] \tag {22}
$$
Step 2: Maximum risk of  $(f_i^\star, h_i^\star)$ : Next, we will compute the maximum risk of the  $(f_i^\star, h_i^\star)$  (see (8)) and show that it is equal to the RHS of (22). First note that we can write,
$$
\left( \begin{array}{c} \widehat {\mu} (X _ {i}) - \mu \\ \widehat {\mu} (Z _ {i}) - \mu \end{array} \right) \sim \mathcal {N} \Bigg (\left( \begin{array}{c} 0 \\ 0 \end{array} \right), \left( \begin{array}{c c} \frac {\sigma^ {2}}{| X _ {i} |} & 0 \\ 0 & \frac {\sigma^ {2}}{| Z _ {i} |} \end{array} \right) \Bigg).
$$
By a linear transformation of this Gaussian vector, we obtain
$$
\begin{array}{l} \left( \begin{array}{c} \frac {| X _ {i} |}{\sigma^ {2}} (\widehat {\mu} (X _ {i}) - \mu) + \frac {| Z _ {i} |}{\sigma^ {2}} (\widehat {\mu} (Z _ {i}) - \mu) \\ \widehat {\mu} (X _ {i}) - \widehat {\mu} (Z _ {i}) \end{array} \right) = \left( \begin{array}{c c} \frac {| X _ {i} |}{\sigma^ {2}} & \frac {| Z _ {i} |}{\sigma^ {2}} \\ 1 & - 1 \end{array} \right) \left( \begin{array}{c} \widehat {\mu} (X _ {i}) - \mu \\ \widehat {\mu} (Z _ {i}) - \mu \end{array} \right) \\ \sim \mathcal {N} \left(\left( \begin{array}{c} 0 \\ 0 \end{array} \right), \left( \begin{array}{c c} \frac {| X _ {i} | + | Z _ {i} |}{\sigma^ {2}} & 0 \\ 0 & \frac {\sigma^ {2}}{| X _ {i} |} + \frac {\sigma^ {2}}{| Z _ {i} |} \end{array} \right)\right), \\ \end{array}
$$
which means  $\frac{|X_i|}{\sigma^2} (\widehat{\mu}(X_i) - \mu) + \frac{|Z_i|}{\sigma^2} (\widehat{\mu}(Z_i) - \mu)$  and  $\frac{\eta_i}{\alpha} = \widehat{\mu}(X_i) - \widehat{\mu}(Z_i)$  are independent Gaussian random variables. Therefore, the maximum risk of  $(f_i^\star, h_i^\star)$  is:
$$
\begin{array}{l} \sup _ {\mu} \mathbb {E} \left[ (h _ {i} ^ {\star} (X _ {i}, Y _ {i}, A _ {i}) - \mu) ^ {2} | \mu \right] = \sup _ {\mu} \mathbb {E} _ {\eta_ {i}} \left[ \mathbb {E} \left[ \left(\frac {\frac {\sum_ {x \in Z _ {i} ^ {\prime}} x}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {| X _ {i} |}{\sigma^ {2}} \widehat {\mu} (X _ {i}) + \frac {| Z _ {i} |}{\sigma^ {2}} \widehat {\mu} (Z _ {i})}{\frac {| Z _ {i} ^ {\prime} |}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {| X _ {i} | + | Z _ {i} |}{\sigma^ {2}}} - \mu\right) ^ {2} \Bigg | \eta_ {i} \right] \right] \\ = \sup  _ {\mu} \mathbb {E} _ {\eta_ {i}} \left[ \mathbb {E} \left[ \left. \left(\frac {\frac {\sum_ {x \in Z _ {i} ^ {\prime}} (x - \mu)}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {| X _ {i} |}{\sigma^ {2}} (\widehat {\mu} (X _ {i}) - \mu) + \frac {| Z _ {i} |}{\sigma^ {2}} (\widehat {\mu} (Z _ {i}) - \mu)}{\frac {| Z _ {i} ^ {\prime} |}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {| X _ {i} | + | Z _ {i} |}{\sigma^ {2}}}\right) ^ {2} \right| \eta_ {i} \right] \right] \\ = \sup  _ {\mu} \mathbb {E} _ {\eta_ {i}} \left[ \frac {\mathbb {E} \left[ \left(\frac {\sum_ {x \in Z _ {i} ^ {\prime}} (x - \mu)}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {| X _ {i} |}{\sigma^ {2}} (\widehat {\mu} (X _ {i}) - \mu) + \frac {| Z _ {i} |}{\sigma^ {2}} (\widehat {\mu} (Z _ {i}) - \mu)\right) ^ {2} \mid \eta_ {i} \right]}{\left(\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {| X _ {i} | + | Z _ {i} |}{\sigma^ {2}}\right) ^ {2}} \right] \\ = \sup  _ {\mu} \mathbb {E} _ {\eta_ {i}} \left[ \frac {1}{\left(\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}}\right) ^ {2}} \left(\frac {\left| Z _ {i} ^ {\prime} \right| \left(\sigma^ {2} + \eta_ {i} ^ {2}\right)}{\left(\sigma^ {2} + \eta_ {i} ^ {2}\right) ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}}\right) \right] \\ = \mathbb {E} _ {\eta_ {i}} \left[ \frac {1}{\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \eta_ {i} ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}}} \right] = \mathbb {E} \left[ \frac {1}{\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \alpha^ {2} (\widehat {\mu} (Z _ {i}) - \widehat {\mu} (X _ {i})) ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}}} \right] \\ \end{array}
$$
Because  $\widehat{\mu}(Z_i) - \widehat{\mu}(X_i) \sim \mathcal{N}\left(0, \frac{\sigma^2}{|X_i|} + \frac{\sigma^2}{|Z_i|}\right)$ , we can further write the maximum risk as:
$$
\sup _ {\mu} \mathbb {E} \big [ (h _ {i} ^ {\star} (X _ {i}, Y _ {i}, A _ {i}) - \mu) ^ {2} | \mu \big ] = \mathbb {E} _ {e \sim \mathcal {N} (0, 1)} \left[ \frac {1}{\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \alpha^ {2} \left(\frac {\sigma^ {2}}{\left| Z _ {i} \right|} + \frac {\sigma^ {2}}{\left| X _ {i} \right|}\right) e ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}}} \right] = R _ {\infty}
$$
Here, we have observed that the final expression in the above equation is exactly the same as the Bayes' risk in the limit in (22) from Step 1.
Step 3: Minimax optimality of  $(f_i^\star, h_i^\star)$ : As the maximum is larger than the average, we can write, for any prior  $\Lambda_\ell$ , and any  $(f_i, h_i) \in \mathcal{F} \times \mathcal{H}$ ,
$$
\sup  _ {\mu} \mathbb {E} \big [ (h _ {i} (X _ {i}, f _ {i} (X _ {i}), A _ {i}) - \mu) ^ {2} | \mu \big ] \geq \mathbb {E} _ {\Lambda_ {\ell}} \big [ \mathbb {E} \big [ (h _ {i} (X _ {i}, f _ {i} (X _ {i}), A _ {i}) - \mu) ^ {2} | \mu \big ] \big ] \geq R _ {\ell}.
$$
As this is true for all  $\ell$ , by taking the limit we have, for all  $(f_i, h_i) \in \mathcal{F} \times \mathcal{H}$ ,
$$
\sup _ {\mu} \mathbb {E} \big [ (h _ {i} (X _ {i}, f _ {i} (X _ {i}), A _ {i}) - \mu) ^ {2} | \mu \big ] \geq R _ {\infty} = \sup _ {\mu} \mathbb {E} \big [ (h _ {i} ^ {\star} (X _ {i}, f _ {i} ^ {\star} (X _ {i}), A _ {i}) - \mu) ^ {2} | \mu \big ].
$$
That is, the recommended  $(f_i^\star, h_i^\star)$  has a smaller maximum risk than all other  $(f_i, h_i) \in \mathcal{F} \times \mathcal{H}$ . This establishes that for any  $n_i$ ,
$$
p _ {i} (M _ {\mathbb {C 3 D}}, ((n _ {i}, f _ {i} ^ {\star}, h _ {i} ^ {\star}), s _ {- i} ^ {\star})) = \inf  _ {f _ {i} \in \mathcal {A}} \inf  _ {h _ {i} \in \mathcal {H}} p _ {i} (M _ {\mathbb {C 3 D}}, ((n _ {i}, f _ {i}, h _ {i}), s _ {- i} ^ {\star})).
$$
A.3.2 Proof of Lemma 6
In the previous section, we showed that for any  $n_i$ , the optimal  $(f_i, h_i)$  were  $(f_i^\star, h_i^\star)$  as given in (8). Now, we show that for the given  $(f_i^\star, h_i^\star)$ , the optimal number of samples is  $n_i^\star = \sigma / \sqrt{cm}$ . For this, we will show that  $p_i$  is a convex function of  $n_i$  and then show that its gradient is 0 at  $n_i^\star$ .
First, noting that
$$
\widehat {\mu} (Z _ {i}) - \widehat {\mu} (X _ {i}) \sim \mathcal {N} \left(0, \frac {\sigma^ {2}}{| X _ {i} |} + \frac {\sigma^ {2}}{| Z _ {i} |}\right),
$$
we can rewrite the penalty term as:
$$
\begin{array}{l} p (n _ {i}) := p _ {i} \big (M _ {\mathbb {C 3 D}}, ((n _ {i}, f _ {i} ^ {\star}, h _ {i} ^ {\star}), s _ {- i} ^ {\star}) \big) = \mathbb {E} \left[ \frac {1}{\frac {| Z _ {i} ^ {\prime} |}{\sigma^ {2} + \alpha^ {2} (\widehat {\mu} (Z _ {i}) - \widehat {\mu} (X _ {i})) ^ {2}} + \frac {| X _ {i} | + | Z _ {i} |}{\sigma^ {2}}} \right] + c n _ {i} \\ = \mathbb {E} _ {x \sim \mathcal {N} (0, 1)} \left[ \frac {1}{\frac {\left| Z _ {i} ^ {\prime} \right|}{\sigma^ {2} + \alpha^ {2} \left(\frac {\sigma^ {2}}{\left| X _ {i} \right|} + \frac {\sigma^ {2}}{\left| Z _ {i} \right|}\right) x ^ {2}} + \frac {\left| X _ {i} \right| + \left| Z _ {i} \right|}{\sigma^ {2}}} \right] + c n _ {i} \\ = \mathbb {E} _ {x \sim \mathcal {N} (0, 1)} \left[ \underbrace {\frac {1}{\frac {(m - 2) n _ {i} ^ {*}}{\sigma^ {2} + \alpha^ {2} \left(\frac {\sigma^ {2}}{n _ {i}} + \frac {\sigma^ {2}}{n _ {i} ^ {*}}\right) x ^ {2}} + \frac {n _ {i} + n _ {i} ^ {*}}{\sigma^ {2}}}} _ {=: l \left(n _ {i}, x; \alpha\right)} \right] + c n _ {i} \tag {23} \\ \end{array}
$$
Convexity of penalty function: To show that  $p(n_i)$  is convex in  $n_i$ , let us consider  $l(n_i, x; \alpha)$ . Fixing  $\alpha$  and  $x$ , we have
$$
\frac {\partial}{\partial n _ {i}} l \left(n _ {i}, x; \alpha\right) = - \sigma^ {2} \frac {1 + \frac {(m - 2) n _ {i} ^ {\star}}{\left(1 + \alpha^ {2} \left(\frac {1}{n _ {i}} + \frac {1}{n _ {i} ^ {\star}}\right) x ^ {2}\right) ^ {2}} \frac {\alpha^ {2} x ^ {2}}{n _ {i} ^ {2}}}{\left(\frac {(m - 2) n _ {i} ^ {\star}}{1 + \alpha^ {2} \left(\frac {1}{n _ {i}} + \frac {1}{n _ {i} ^ {\star}}\right) x ^ {2}} + n _ {i} + n _ {i} ^ {\star}\right) ^ {2}} = - \sigma^ {2} \frac {1 + \frac {(m - 2) n _ {i} ^ {\star} \alpha^ {2} x ^ {2}}{\left(n _ {i} + \alpha^ {2} \left(1 + \frac {n _ {i}}{n _ {i} ^ {\star}}\right) x ^ {2}\right) ^ {2}}}{\left(\frac {(m - 2) n _ {i} ^ {\star}}{1 + \alpha^ {2} \left(\frac {1}{n _ {i}} + \frac {1}{n _ {i} ^ {\star}}\right) x ^ {2}} + n _ {i} + n _ {i} ^ {\star}\right) ^ {2}} \tag {24}
$$
As  $\frac{\partial}{\partial n_i} l(n_i, x; \alpha)$  is an increasing function of  $n_i$ , we have that  $l(n_i, x; \alpha)$  is a convex function in  $n_i$ . As expectation preserves convexity (see Lemma 10),  $p(n_i)$  is a convex function.
Penalty is minimized when  $n_i = n_i^\star$ . Lemma 13 provides an expression for the derivative of  $p(n_i)$  (obtained purely via algebraic manipulations). Using this, we have
$$
\begin{array}{l} p ^ {\prime} (n _ {i} ^ {\star}) = - \frac {\sigma^ {2}}{6 4 \frac {\alpha^ {2}}{m - 2} \frac {\alpha}{\sqrt {m n _ {i} ^ {\star}}} m n _ {i} ^ {\star}} \left(\frac {4 \alpha}{\sqrt {m n _ {i} ^ {\star}}} \left(\frac {4 \alpha^ {2} m}{(m - 2) n _ {i} ^ {\star}} - 1\right) \right. \\ \left. - \exp \left(\frac {m n _ {i} ^ {\star}}{8 \alpha^ {2}}\right) \left(\frac {4 \alpha^ {2}}{m n _ {i} ^ {\star}} (m + 1) - 1\right) \sqrt {2 \pi} \operatorname {E r f c} \left(\frac {1}{2 \sqrt {2} \sqrt {\frac {\alpha^ {2}}{m n _ {i} ^ {\star}}}}\right)\right) \\ + c \quad (\text {B y} \\ = - \frac {\sigma^ {2}}{6 4 \frac {\alpha^ {2}}{m - 2} \frac {\alpha}{\sqrt {m n _ {i} ^ {\star}}} m n _ {i} ^ {\star}} \left(\frac {4 \alpha}{\sqrt {m n _ {i} ^ {\star}}} \left(\frac {4 \alpha^ {2} (m - 4)}{(m - 2) n _ {i} ^ {\star}} - 1\right) \right. \\ \left. - \exp \left(\frac {m n _ {i} ^ {\star}}{8 \alpha^ {2}}\right) \left(\frac {4 \alpha^ {2}}{m n _ {i} ^ {\star}} (m + 1) - 1\right) \sqrt {2 \pi} \operatorname {E r f c} \left(\frac {1}{2 \sqrt {2} \sqrt {\frac {\alpha^ {2}}{m n _ {i} ^ {\star}}}}\right)\right) \\ = G (\alpha) = 0. \\ \end{array}
$$
Here, the second step uses the fact that  $n_i^\star = \frac{\sigma}{\sqrt{cm}}$ . Finally, we have observed that the expression is equal to  $G(\alpha)$  as defined in (7) which is 0 by our choice of  $\alpha$ . Since  $p'(n_i^\star) = 0$  and  $p(\cdot)$  is convex, we can conclude that  $p(n_i)$  is minimized when  $n_i = n_i^\star$ . Therefore,
A.4 Algorithm 1 is individually rational
As outlined in the main text, the NIC property implies IR since 'working on her own' is a valid strategy in the mechanism. Precisely, if an agent collects any number of points  $n_i$ , chooses not to submit anything  $f_i(\cdot) = \varnothing$ , and then uses the sample average of the points she collected  $h_i(X_i, \varnothing, A_i) = |X_i|^{-1} \sum_{x \in X_i} x$ , then  $(n_i, f_i, h_i) \in S$ .
Below, we will prove this more formally and also show that the agent's penalty is strictly smaller when participating. For any fixed  $n_i$ , without participating in the mechanism, the smallest penalty the agent can achieve is by using empirical mean estimation and the penalty is:
$$
\frac {\sigma^ {2}}{n _ {i}} + c n _ {i}
$$
When participating, the agent gets an additional  $n_i^\star$  number of clean data along with some noisy data, provided that all other agents are following  $s_{-i}^{\star}$ . By using the empirical mean over the clean data, the penalty is:
$$
\frac {\sigma^ {2}}{n _ {i} + n _ {i} ^ {\star}} + c n _ {i} <   \frac {\sigma^ {2}}{n _ {i}} + c n _ {i}
$$
Now, since the weighted average estimator in  $s_i^\star$  is minimax optimal, the agent gets even smaller maximum risk and hence smaller penalty. In other words, for any  $n_i$ ,
$$
p _ {i} (M _ {\mathbb {C 3 D}}, s ^ {\star}) \leq p _ {i} (M _ {\mathbb {C 3 D}}, ((n _ {i}, f _ {i} ^ {\star}, h _ {i} ^ {\star}), s _ {- i} ^ {\star})) \leq \frac {\sigma^ {2}}{n _ {i} + n _ {i} ^ {\star}} + c n _ {i} <   \frac {\sigma^ {2}}{n _ {i}} + c n _ {i}
$$
By minimizing the RHS with respect to  $n_i$ , we get  $p_i(M_{\mathbb{C}\mathbb{D}},s^{\star}) < p_{\min}^{\mathrm{IR}}$ . Thus Algorithm 1 is IR.
A.5 Algorithm 1 is approximately efficient
In this section, we will bound the penalty ratio PR for  $M_{\mathrm{c3D}}$  at the strategy profiles  $s_i^{\star}$ .
First, noting that  $G(\alpha) = 0$  (see (7)), we can rearrange the terms in the equation to obtain:
$$
\exp \left(\frac {m n _ {i} ^ {\star}}{8 \alpha^ {2}}\right) \operatorname {E r f c} \left(\frac {1}{2 \sqrt {2} \sqrt {\frac {\alpha^ {2}}{m n _ {i} ^ {\star}}}}\right) = \frac {1}{\sqrt {2 \pi}} \frac {\frac {4 \alpha}{\sqrt {m n _ {i} ^ {\star}}} \left(\frac {4 \alpha^ {2} (m - 4)}{(m - 2) n _ {i} ^ {\star}} - 1\right)}{\frac {4 \alpha^ {2}}{m n _ {i} ^ {\star}} (m + 1) - 1} \tag {25}
$$
Next, we will use the expression for  $p(n_i) = p_i(M_{\mathbb{C3D}},(s_{-i}^{\star},(n_i,f_i^{\star},h_i^{\star}))$  in Lemma 13 and the equation in (25) to simplify  $p(n_i^\star)$  as follows:
$$
p (n _ {i} ^ {\star}) = \frac {\sqrt {\frac {\alpha^ {2}}{m n _ {i} ^ {\star}}} \sigma^ {2} \left(2 m \sqrt {2 \pi} \sqrt {\frac {\alpha^ {2}}{m n _ {i} ^ {\star}}} - \exp \left(\frac {m n _ {i} ^ {\star}}{8 \alpha^ {2}}\right) (m - 2) \pi \operatorname {E r f c} \left(\frac {1}{2 \sqrt {2} \sqrt {\frac {\alpha^ {2}}{m n _ {i} ^ {\star}}}}\right)\right)}{4 \sqrt {2 \pi} \alpha^ {2}} + c n _ {i} ^ {\star}
$$
(By Lemma 13)
$$
\begin{array}{l} = \frac {\sqrt {\frac {\alpha^ {2}}{m n _ {i} ^ {\star}}} \sigma^ {2} \left(2 m \sqrt {2 \pi} \sqrt {\frac {\alpha^ {2}}{m n _ {i} ^ {\star}}} - (m - 2) \pi \frac {1}{\sqrt {2 \pi}} \frac {\frac {4 \alpha}{\sqrt {m n _ {i} ^ {\star}}} \left(\frac {4 \alpha^ {2} (m - 4)}{(m - 2) n _ {i} ^ {\star}} - 1\right)}{\frac {4 \alpha^ {2}}{m n _ {i} ^ {\star}} (m + 1) - 1}\right)}{4 \sqrt {2 \pi} \alpha^ {2}} + c n _ {i} ^ {\star} \quad (\text {B y} (2 5)) \\ = \frac {\sigma^ {2} \left(m - (m - 2) \frac {\frac {4 \alpha^ {2} (m - 4)}{(m - 2) n _ {i} ^ {\star}} - 1}{\frac {4 \alpha^ {2}}{m n _ {i} ^ {\star}} (m + 1) - 1}\right)}{2 m n _ {i} ^ {\star}} + c n _ {i} ^ {\star} \\ = \frac {\sigma^ {2}}{2 m n _ {i} ^ {\star}} \frac {\frac {4 \alpha^ {2}}{n _ {i} ^ {\star}} (m + 1) - m - \frac {4 \alpha^ {2}}{n _ {i} ^ {\star}} (m - 4) + (m - 2)}{\frac {4 \alpha^ {2}}{n _ {i} ^ {\star}} \frac {m + 1}{m} - 1} + c n _ {i} ^ {\star} \\ = \frac {\sigma^ {2}}{2 m n _ {i} ^ {\star}} \frac {\frac {2 0 \alpha^ {2}}{n _ {i} ^ {\star}} - 2}{\frac {4 \alpha^ {2}}{n _ {i} ^ {\star}} \frac {m + 1}{m} - 1} + c n _ {i} ^ {\star} = \frac {\sigma^ {2}}{m n _ {i} ^ {\star}} \frac {\frac {1 0 \alpha^ {2}}{n _ {i} ^ {\star}} - 1}{\frac {4 \alpha^ {2}}{n _ {i} ^ {\star}} \frac {m + 1}{m} - 1} + c n _ {i} ^ {\star} \\ \end{array}
$$
Algorithm 2  $M_{\mathrm{PCS}}$
1: Mechanism designer publishes:  
2: The allocation space  $\mathcal{A} = \bigcup_{n\geq 0}\mathbb{R}^n$  , and the procedure in lines 6-11.  
3: Each agent i:  
4: Choose strategy  $s_i = (n_i,f_i,h_i)$    
5: Sample  $n_i$  points  $X_{i} = \{x_{i,j}\}_{j = 1}^{n_{i}}$  and submit  $Y_{i} = f_{i}(X_{i})$  to the mechanism.  
6: Mechanism:  
7: For each agent  $i\in [m]$  # can be done simultaneously for all agents  
8:  $A_{i}\gets \bigcup_{j\neq i}Y_{j}$  if  $|Y_{i}|\geq \sigma /\sqrt{cm},\quad A_{i}\gets \emptyset$  otherwise.  
9: Return  $A_{i}$  to each agent.  
10: Each agent i:  
11: Compute estimate  $h_i(X_i,Y_i,A_i)$
$$
= \sigma \sqrt {\frac {c}{m}} \left(\frac {\frac {1 0 \alpha^ {2}}{n _ {i} ^ {*}} - 1}{\frac {4 \alpha^ {2}}{n _ {i} ^ {*}} \frac {m + 1}{m} - 1} + 1\right)
$$
From our conclusion in §A.2, we have  $\alpha^2 > \frac{\sigma}{\sqrt{cm}} = n_i^\star$ , i.e.  $\frac{\alpha^2}{n_i^\star} > 1$ . Therefore, we have: