SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\alpha_ {r} = \frac {\sigma_ {c} ^ {2} - \frac {\epsilon \sigma_ {c} \sigma_ {z} \sqrt {\frac {d}{n}}}{\sqrt {\sigma_ {c} ^ {2} - \epsilon^ {2} + \sigma_ {z} ^ {2} \frac {d}{n}}}}{\sigma_ {c} ^ {2} + \sigma_ {z} ^ {2} \frac {d}{n}} \quad \text {a n d} \quad \alpha_ {j} = \frac {\sigma_ {c} ^ {2}}{\sigma_ {c} ^ {2} + \sigma_ {z} ^ {2} \frac {d}{n} + \sigma_ {w} ^ {2} d}.
$$
Setting  $\alpha_{r} = \alpha_{j}$  and solving for the standard deviation  $\sigma_w$  yields the result.
B.2 Form of perturbations for estimators  $\mathbf{H} = \alpha \mathbf{U}\mathbf{U}^T$
We noted in the main body that for estimators  $\mathbf{H} = \alpha \mathbf{U}\mathbf{U}^T$  worst-case perturbations can be computed in closed form for fixed  $\mathbf{y}$  and  $\mathbf{x}$ . We calculate:
$$
\begin{array}{l} \hat{\mathbf{e}} = \arg \max_{\| \mathbf{e}\|_{2}\leq \epsilon}\| \mathbf{H}(\mathbf{x} + \mathbf{z} + \mathbf{e}) - \mathbf{x}\|_{2}^{2} \\ = \arg \max  _ {\| \mathbf {e} \| _ {2} \leq \epsilon} \| (\mathbf {H} - \mathbf {I}) \mathbf {U c} + \mathbf {H z} + \mathbf {H e} \| _ {2} ^ {2} \\ = \arg \max  _ {\| \mathbf {e} \| _ {2} \leq \epsilon} \left\| (\alpha \mathbf {U} \mathbf {U} ^ {T} - \mathbf {I}) \mathbf {U} \mathbf {c} + \alpha \mathbf {U} \mathbf {U} ^ {T} \mathbf {z} + \alpha \mathbf {U} \mathbf {U} ^ {T} \mathbf {e} \right\| _ {2} ^ {2} \\ = \arg \max  _ {\| \mathbf {e} \| _ {2} \leq \epsilon} \| (\alpha - 1) \mathbf {U} \mathbf {c} + \alpha \mathbf {U} \mathbf {U} ^ {T} \mathbf {z} + \alpha \mathbf {U} \mathbf {U} ^ {T} \mathbf {e} \| _ {2} ^ {2} \\ = \mathbf {U} \arg \max  _ {\| \mathbf {e} ^ {\prime} \| _ {2} \leq \epsilon} \left\| (\alpha - 1) \mathbf {c} + \alpha \mathbf {U} ^ {T} \mathbf {z} + \alpha \mathbf {e} ^ {\prime} \right\| _ {2} ^ {2} \\ = \mathbf {U} \epsilon \frac {(1 - \alpha) \mathbf {c} + \alpha \mathbf {U} ^ {T} \mathbf {z}}{\| (1 - \alpha) \mathbf {c} + \alpha \mathbf {U} ^ {T} \mathbf {z} \| _ {2}}. \\ \end{array}
$$
Thus the perturbation points into the direction of the signal plus noise lying in the signal subspace.
C Theory for general linear inverse problems
In the following, we consider linear inverse problems  $\mathbf{y} = \mathbf{A}\mathbf{x} + \mathbf{z}$ , with a linear forward operator  $\mathbf{A} \in \mathbb{R}^{m \times n}$  and noise  $\mathbf{z} \sim \mathcal{N}(0, \sigma_z^2 / m\mathbf{I})$ . For denoising  $(\mathbf{A} = \mathbf{I})$  we stated an explicit analytical characterization of the optimal worst-case estimator and presented a proof in the appendix. The proof, however, does not generalize in a straightforward manner to more general inverse problems. We conjecture the worst-case optimal linear estimator for large dimensions  $d$  and present numerical simulation results. Moreover, we state the proof for the optimal jittering estimator, and demonstrate that it can yield suboptimal worst-case robust estimators in general.
C.1 Optimal worst-case robust estimator
As formalized by Lemma A.1, the robust-risk (1) of the estimator  $f$  can be written as an expectation involving a minimization problem over a single variable (instead of a maximization over an  $n$ -
dimensional variable, as in the original definition):
$$
R _ {\epsilon} (\mathbf {H}) = \mathbb {E} _ {\mathbf {v}} \left[ \min  _ {\lambda \geq \max  _ {i} \sigma_ {i} ^ {2}} \lambda \epsilon^ {2} + \mathbf {v} ^ {T} \left(\mathbf {I} - \frac {1}{\lambda} \mathbf {H} \mathbf {H} ^ {T}\right) ^ {- 1} \mathbf {v} \right], \quad \mathbf {v} = (\mathbf {H} \mathbf {A} - \mathbf {I}) \mathbf {x} + \mathbf {H} \mathbf {z}. \tag {14}
$$
Here,  $\sigma_{i}$  are the singular values of the matrix  $\mathbf{H}$ . In order to find the optimal robust estimator we wish to solve the optimization problem  $\arg \min_{\mathbf{H}}R_{\epsilon}(\mathbf{H})$ . The difficulty in solving this optimization problem is that we can't solve the minimization problem within the expectation (14) in closed form. In order to prove Theorem 3.1 for denoising (i.e., for  $\mathbf{A} = \mathbf{I}$ ) we derived an upper and a matching lower bound of the risks using several unusual applications of Jensen's inequality. The proof does not generalize in a straightforward manner to the more general case where  $\mathbf{A} \neq \mathbf{I}$ . However, for large  $d$ , the random variable  $\mathbf{v}^T (\mathbf{I} - \frac{1}{\lambda}\mathbf{H}\mathbf{H}^T)^{-1}\mathbf{v}$  concentrates around its expectation, and thus we conjecture that for large  $d$ , we can exchange expectation and minimization, which yields:
$$
R _ {\epsilon} (\mathbf {H}) = \min  _ {\lambda \geq \max  _ {i} \sigma_ {i} ^ {2}} \lambda \epsilon^ {2} + \mathbb {E} _ {\mathbf {v}} \left[ \mathbf {v} ^ {T} \left(\mathbf {I} - \frac {1}{\lambda} \mathbf {H} \mathbf {H} ^ {T}\right) ^ {- 1} \mathbf {v} \right]. \tag {15}
$$
Based on equation (15) we derive a characterization of the optimal worst-case estimator. We proceed similar to the denoising case and start with rearranging the terms in expectation:
$$
\begin{array}{l} \mathbf {v} ^ {T} (\mathbf {I} - \frac {1}{\lambda} \mathbf {H} \mathbf {H} ^ {T}) ^ {- 1} \mathbf {v} = ((\mathbf {H A} - \mathbf {I}) \mathbf {x} + \mathbf {H z}) ^ {T} (\mathbf {I} - \frac {1}{\lambda} \mathbf {H} \mathbf {H} ^ {T}) ^ {- 1} ((\mathbf {H A} - \mathbf {I}) \mathbf {x} + \mathbf {H z}) \\ = \mathbf {c} ^ {T} \left(\left(\mathbf {H} \mathbf {A} - \mathbf {I}\right) \mathbf {U}\right) ^ {T} \left(\mathbf {I} - \frac {1}{\lambda} \mathbf {H} \mathbf {H} ^ {T}\right) ^ {- 1} \left(\mathbf {H} \mathbf {A} - \mathbf {I}\right) \mathbf {U} \mathbf {c} \\ + 2 \mathbf {z} ^ {T} \mathbf {H} ^ {T} (\mathbf {I} - \frac {1}{\lambda} \mathbf {H} \mathbf {H} ^ {T}) ^ {- 1} (\mathbf {H} \mathbf {A} - \mathbf {I}) \mathbf {U c} + \mathbf {z} ^ {T} \mathbf {H} ^ {T} (\mathbf {I} - \frac {1}{\lambda} \mathbf {H} \mathbf {H} ^ {T}) ^ {- 1} \mathbf {H z}. \\ \end{array}
$$
Now, let  $\mathbf{A}\mathbf{U} = \mathbf{W}\boldsymbol{\Lambda}\mathbf{V}^T$  be the singular value decomposition of the matrix  $\mathbf{A}\mathbf{U}$  and  $\mathbf{H} = \mathbf{U}\mathbf{V}\boldsymbol{\Sigma}\mathbf{W}^T$  the singular value decomposition of  $\mathbf{H}$ . We then have:
$$
\mathbf {H} \mathbf {H} ^ {T} = \mathbf {U} \mathbf {V} \boldsymbol {\Sigma} \mathbf {W} ^ {T} \left(\mathbf {U} \mathbf {V} \boldsymbol {\Sigma} \mathbf {W} ^ {T}\right) ^ {T} = \mathbf {U} \mathbf {V} \boldsymbol {\Sigma} \mathbf {W} ^ {T} \mathbf {W} \boldsymbol {\Sigma} \mathbf {V} ^ {T} \mathbf {U} ^ {T} = \mathbf {U} \mathbf {V} \boldsymbol {\Sigma} ^ {2} \mathbf {V} ^ {T} \mathbf {U} ^ {T}
$$
$$
\mathbf {H} \mathbf {A} \mathbf {U} = \mathbf {U} \mathbf {V} \boldsymbol {\Sigma} \mathbf {W} ^ {T} \mathbf {W} \boldsymbol {\Lambda} \mathbf {V} ^ {T} = \mathbf {U} \mathbf {V} \boldsymbol {\Sigma} \boldsymbol {\Lambda} \mathbf {V} ^ {T}.
$$
For the individual parts in the summation it follows:
$$
\begin{array}{l} \mathbf {c} ^ {T} (\mathbf {H A U} - \mathbf {U}) ^ {T} (\mathbf {I} - \frac {1}{\lambda} \mathbf {H H} ^ {T}) ^ {- 1} (\mathbf {H A U} - \mathbf {U}) \mathbf {c} \\ = \mathbf {c} ^ {T} \left(\mathbf {U} \mathbf {V} \boldsymbol {\Sigma} \boldsymbol {\Lambda} \mathbf {V} ^ {T} - \mathbf {U}\right) ^ {T} \left(\mathbf {I} - \frac {1}{\lambda} \mathbf {U} \mathbf {V} \boldsymbol {\Sigma} ^ {2} \mathbf {V} ^ {T} \mathbf {U} ^ {T}\right) ^ {- 1} \left(\mathbf {U} \mathbf {V} \boldsymbol {\Sigma} \boldsymbol {\Lambda} \mathbf {V} ^ {T} - \mathbf {U}\right) \mathbf {c} \\ = \mathbf {c} ^ {T} \left(\mathbf {U} \mathbf {V} \boldsymbol {\Sigma} \boldsymbol {\Lambda} \mathbf {V} ^ {T} - \mathbf {U}\right) ^ {T} \mathbf {U} \mathbf {V} \left(\mathbf {I} - \frac {1}{\lambda} \boldsymbol {\Sigma} ^ {2}\right) ^ {- 1} \mathbf {V} ^ {T} \mathbf {U} ^ {T} \left(\mathbf {U} \mathbf {V} \boldsymbol {\Sigma} \boldsymbol {\Lambda} \mathbf {V} ^ {T} - \mathbf {U}\right) \mathbf {c} \\ = \left(\mathbf {V} ^ {T} \mathbf {c}\right) ^ {T} \left(\boldsymbol {\Sigma} \boldsymbol {\Lambda} - \mathbf {I}\right) \left(\mathbf {I} - \frac {1}{\lambda} \boldsymbol {\Sigma} ^ {2}\right) ^ {- 1} \left(\boldsymbol {\Sigma} \boldsymbol {\Lambda} - \mathbf {I}\right) \mathbf {V} ^ {T} \mathbf {c} = \sum_ {i = 1} ^ {d} \frac {\left(\sigma_ {i} \lambda_ {i} - 1\right) ^ {2} c _ {i}}{1 - \frac {\sigma_ {i} ^ {2}}{\lambda}} \\ \end{array}
$$
where we define  $c_{i} = \mathbf{v}_{i}^{T}\mathbf{c}$  using the  $i$ -th column vector  $\mathbf{v}_{i}$  of the matrix  $\mathbf{V}$ . Similarly, we set  $z_{i} = \mathbf{w}_{i}^{T}\mathbf{z}$ , with  $\mathbf{w}_{i}$  the  $i$ -th column vector of  $\mathbf{W}$ , and get for the other parts:
$$
2 \mathbf {z} ^ {T} \mathbf {H} ^ {T} (\mathbf {I} - \frac {1}{\lambda} \mathbf {H} \mathbf {H} ^ {T}) ^ {- 1} (\mathbf {H} \mathbf {A} \mathbf {U} - \mathbf {U}) \mathbf {c} = \sum_ {i = 1} ^ {d} 2 \frac {z _ {i} \sigma_ {i} (\sigma_ {i} \lambda_ {i} - 1) c _ {i}}{1 - \frac {\sigma_ {i} ^ {2}}{\lambda}}
$$
$$
\mathbf {z} ^ {T} \mathbf {H} ^ {T} (\mathbf {I} - \frac {1}{\lambda} \mathbf {H} \mathbf {H} ^ {T}) ^ {- 1} \mathbf {H} \mathbf {z} = \sum_ {i = 1} ^ {d} \frac {z _ {i} ^ {2} \sigma_ {i} ^ {2}}{1 - \frac {\sigma_ {i} ^ {2}}{\lambda}}.
$$
Hence, for the term in expectation in Eq. (15) we get:
$$
\begin{array}{l} \mathbb {E} _ {\mathbf {v}} \left[ \mathbf {v} ^ {T} (\mathbf {I} - \frac {1}{\lambda} \mathbf {H} \mathbf {H} ^ {T}) ^ {- 1} \mathbf {v} \right] = \mathbb {E} _ {\mathbf {c}, \mathbf {z}} \left[ \sum_ {i = 1} ^ {d} \frac {(c _ {i} (\sigma_ {i} \lambda_ {i} - 1) + z _ {i} \sigma_ {i}) ^ {2}}{1 - \frac {\sigma_ {i} ^ {2}}{\lambda}} \right] \\ = \mathbb {E} _ {\mathbf {g}} \left[ \sum_ {i = 1} ^ {d} \frac {g _ {i} ^ {2} \left(\frac {\sigma_ {i} ^ {2}}{d} (\sigma_ {i} \lambda_ {i} - 1) ^ {2} + \frac {\sigma_ {z} ^ {2}}{m} \sigma_ {i} ^ {2}\right)}{1 - \frac {\sigma_ {i} ^ {2}}{\lambda}} \right], \\ \end{array}
$$
where we note that  $c_{i}(\sigma_{i}\lambda_{i} - 1) + z_{i}\sigma_{i}$  are iid zero-mean gaussian with variance  $\frac{\sigma_c^2}{d} (\lambda_i\sigma_i - 1)^2 +\frac{\sigma_z^2}{m}\sigma_i^2$  and  $g_{i}\sim \mathcal{N}(0,1)$ . From this it follows, assuming the robust risk conjecture (15), that the optimal robust estimator minimizes:
$$
\min  _ {\mathbf {H}} R _ {\epsilon} (\mathbf {H}) = \min  _ {\sigma_ {i}, \lambda \geq \sigma_ {i} ^ {2}} f (\lambda , \sigma_ {1}, \dots , \sigma_ {n}), \tag {16}
$$
$$
f \left(\lambda , \sigma_ {1}, \dots , \sigma_ {n}\right) = \lambda \epsilon^ {2} + \sum_ {i = 1} ^ {d} \frac {\frac {\sigma_ {z} ^ {2}}{m} \sigma_ {i} ^ {2} + \frac {\sigma_ {c} ^ {2}}{d} \left(\sigma_ {i} \lambda_ {i} - 1\right) ^ {2}}{1 - \frac {\sigma_ {i} ^ {2}}{\lambda}}. \tag {17}
$$
We first calculate the unconstrained minimizer of the function  $f$  and get:
$$
\frac {\partial f}{\partial \sigma_ {i}} = 2 \lambda \frac {\lambda \lambda_ {i} ^ {2} \sigma_ {i} \frac {\sigma_ {c} ^ {2}}{d} - \lambda_ {i} (\lambda + \sigma_ {i} ^ {2}) \frac {\sigma_ {c} ^ {2}}{d} + \sigma_ {i} (\frac {\sigma_ {c} ^ {2}}{d} + \lambda \frac {\sigma_ {z} ^ {2}}{m})}{(\lambda - \sigma_ {i} ^ {2}) ^ {2}}.
$$
For  $\lambda_{i} = 0$  we obtain  $\frac{\partial f}{\partial\sigma_i} = 0\Rightarrow \sigma_i = 0$  and for  $\lambda_{i}\neq 0$  ..
$$
\frac {\partial f}{\partial \sigma_ {i}} | _ {\sigma_ {i} = \sigma_ {i, \pm} ^ {*}} = 0 \Rightarrow \sigma_ {i, \pm} ^ {*} = \frac {1 + \lambda_ {i} ^ {2} \lambda}{2 \lambda_ {i}} + \frac {d}{m} \frac {\lambda}{2 \lambda_ {i}} \frac {\sigma_ {z} ^ {2}}{\sigma_ {c} ^ {2}} \pm \sqrt {\left(\frac {1 + \lambda_ {i} ^ {2} \lambda}{2 \lambda_ {i}} + \frac {d}{m} \frac {\lambda}{2 \lambda_ {i}} \frac {\sigma_ {z} ^ {2}}{\sigma_ {c} ^ {2}}\right) ^ {2} - \lambda}.
$$
The solutions  $\sigma_{i,\pm}^{*}$  obtained are for suitable  $a(\lambda)$  of the form:
$$
\sigma_ {i, \pm} = a (\lambda) \pm \sqrt {a (\lambda) ^ {2} - \lambda}.
$$
On further examining the constraint  $\sigma_i^2 \leq \lambda$  in the problem (16) note that for real numbers  $a > b$  it holds  $\sqrt{a^2 - b^2} \geq a - b$ . Hence, it follows:
$$
\sigma_ {i, \pm} - \sqrt {\lambda} = a (\lambda) - \sqrt {\lambda} \pm \sqrt {a (\lambda) ^ {2} - \lambda} \geqslant 0.
$$
Therefore, we can rule out  $\sigma_{i, + }$  and set  $\sigma_{i}^{*} = \sigma_{i, - }^{*}$ . Finally, inserting  $\sigma_{i}^{*}$  for  $1\leq i\leq d$  into the robust risk (17) yields:
$$
\min  _ {\mathbf {H}} R _ {\epsilon} (\mathbf {H}) = \min  _ {\lambda \geq 0} \lambda \epsilon^ {2} + \sum_ {i = 1} ^ {d} \frac {1 - \lambda \lambda_ {i} ^ {2}}{2} \frac {\sigma_ {c} ^ {2}}{d} - \frac {\lambda}{2} \frac {\sigma_ {z} ^ {2}}{m} + \sqrt {\left(\frac {1 + \lambda \lambda_ {i} ^ {2}}{2} \frac {\sigma_ {c} ^ {2}}{d} + \frac {\lambda}{2} \frac {\sigma_ {z} ^ {2}}{m}\right) ^ {2} - \lambda \lambda_ {i} ^ {2} \frac {\sigma_ {c} ^ {4}}{d ^ {2}}}. \tag {18}
$$
C.2 Optimal jittering estimator
We now derive the optimal jittering estimator, i.e., the estimator  $f(\mathbf{y}) = \mathbf{H}\mathbf{y}$  that minimizes the jittering risk
$$
J _ {\sigma_ {w}} (f) = \mathbb {E} _ {(\mathbf {x}, \mathbf {z}, \mathbf {w})} \left[ \| \mathbf {H} (\mathbf {A x} + \mathbf {w} + \mathbf {z}) - \mathbf {x} \| _ {2} ^ {2} \right],
$$
where as before the signal  $\mathbf{x}$  is assumed to lie within a subspace, i.e.,  $\mathbf{x} = \mathbf{U}\mathbf{c}$ . We first calculate the expectation by using that  $\mathbf{c} \sim \mathcal{N}(0, \sigma_c^2 / d\mathbf{I})$ ,  $\mathbf{z} \sim \mathcal{N}(0, \sigma_z^2 / m\mathbf{I})$  and  $\mathbf{w} \sim \mathcal{N}(0, \sigma_w^2\mathbf{I})$  are Gaussian distributed. The jittering risk is
$$
\begin{array}{l} J _ {\sigma_ {w}} (f) = \mathbb {E} _ {(\mathbf {x}, \mathbf {z}, \mathbf {w})} \left[ \| \mathbf {H} (\mathbf {A x} + \mathbf {z} + \mathbf {w}) - \mathbf {x} \| _ {2} ^ {2} \right] \\ = \mathbb {E} _ {\mathbf {x}} \left[ \| (\mathbf {H A} - \mathbf {I}) \mathbf {U c} \| _ {2} ^ {2} \right] + \mathbb {E} _ {\mathbf {z}} \left[ \| \mathbf {H z} \| _ {2} ^ {2} \right] + \mathbb {E} _ {\mathbf {w}} \left[ \| \mathbf {H w} \| _ {2} ^ {2} \right] \\ = \operatorname {t r} \left((\mathbf {H A} - \mathbf {I}) \mathbf {U U} ^ {T} (\mathbf {H A} - \mathbf {I}) ^ {T}\right) \frac {\sigma_ {c} ^ {2}}{d} + \operatorname {t r} \left(\mathbf {H H} ^ {T}\right) \frac {\sigma_ {z} ^ {2}}{m} + \operatorname {t r} \left(\mathbf {H H} ^ {T}\right) \sigma_ {w} ^ {2}. \\ \end{array}
$$
Hence, the optimal jittering estimator minimizes
$$
J _ {\sigma_ {w}} (f) = \operatorname {t r} (\mathbf {H X H} ^ {T}) - 2 \operatorname {t r} (\mathbf {H Y}) + \sigma_ {c} ^ {2},
$$
where  $\mathbf{X} = \frac{\sigma_c^2}{d}\mathbf{A}\mathbf{U}(\mathbf{A}\mathbf{U})^T + (\sigma_w^2 + \sigma_z^2 / m)\mathbf{I}$  and  $\mathbf{Y} = \frac{\sigma_c^2}{d}\mathbf{A}\mathbf{U}\mathbf{U}^T$ . Using matrix calculus we calculate the optimal estimator as
$$
\nabla_ {\mathbf {H}} J _ {\sigma_ {w}} (f) \left| _ {\mathbf {H} = \mathbf {H} ^ {*}} = 2 \mathbf {H} ^ {*} \mathbf {X} - 2 \mathbf {Y} ^ {T} \right. = 0,
$$
which yields  $\mathbf{H}^{*} = \mathbf{Y}^{T}\mathbf{X}^{-1}$ . Now, let  $\mathbf{A}\mathbf{U} = \mathbf{W}\mathbf{A}\mathbf{V}^{T}$  be the singular value decomposition of  $\mathbf{A}\mathbf{U}$ . Then:
$$
\mathbf {X} = \frac {\sigma_ {c} ^ {2}}{d} \mathbf {W} \boldsymbol {\Lambda} ^ {2} \mathbf {W} ^ {T} + \left(\sigma_ {w} ^ {2} + \sigma_ {z} ^ {2} / m\right) \mathbf {I}
$$
$$
\mathbf {Y} = \frac {\sigma_ {c} ^ {2}}{d} \mathbf {W} \boldsymbol {\Lambda} \mathbf {V} ^ {T} \mathbf {U} ^ {T}.
$$
We get the optimal jittering estimator as:
$$
\begin{array}{l} \operatorname {a r g m i n} _ {\mathbf {H}} J _ {\sigma_ {w}} (f) = \mathbf {Y} ^ {T} \mathbf {X} ^ {- 1} = \frac {\sigma_ {c} ^ {2}}{d} \mathbf {U V} \boldsymbol {\Lambda} \mathbf {W} ^ {T} \left(\frac {\sigma_ {c} ^ {2}}{d} \mathbf {W} \boldsymbol {\Lambda} ^ {2} \mathbf {W} ^ {T} + \left(\sigma_ {w} ^ {2} + \sigma_ {z} ^ {2} / m\right) \mathbf {I}\right) ^ {- 1} (19) \\ = \frac {\sigma_ {c} ^ {2}}{d} \mathbf {U V} \boldsymbol {\Lambda} \left(\frac {\sigma_ {c} ^ {2}}{d} \boldsymbol {\Lambda} ^ {2} + \left(\sigma_ {w} ^ {2} + \sigma_ {z} ^ {2} / m\right) \mathbf {I}\right) ^ {- 1} \mathbf {W} ^ {T} (20) \\ = \mathbf {U V} \operatorname {d i a g} \left(\frac {\sigma_ {c} ^ {2} \lambda_ {i}}{\sigma_ {c} ^ {2} \lambda_ {i} ^ {2} + d \sigma_ {w} ^ {2} + \sigma_ {z} ^ {2} \frac {d}{m}}\right) \mathbf {W} ^ {T}. (21) \\ \end{array}
$$
C.3 Numerical simulations supporting Conjecture 3.3
In the following we present numerical simulations for general linear inverse problems in the subspace model to support the Conjecture 3.3 further. The results show that the (empirical) optimal robust risk, obtained via adversarial training, is the same as the robust risk of the conjectured optimal estimator. Moreover, the results show that jittering can yield suboptimal robust estimators in some cases.
We consider linear inverse problems  $\mathbf{y} = \mathbf{A}\mathbf{x} + \mathbf{z}$  with the signal  $\mathbf{x}$  lying in a subspace  $\mathbf{x} = \mathbf{U}\mathbf{c}$  with  $\mathbf{c} \sim \mathcal{N}(0, \sigma_c^2 / d\mathbf{I})$  and Gaussian noise  $\mathbf{z} \sim \mathcal{N}(0, \sigma_z^2 / n\mathbf{I})$ . For the simulations, we choose  $\sigma_c / \sqrt{d} = 1$  and  $\sigma_z / \sqrt{n} = 0.1$  with dimensions  $d = 50$  and  $n = 100$ . Moreover, we consider two diagonal forward operators  $\mathbf{A} = \mathrm{diag}(\lambda_i)$ : an operator with linear decaying singular values  $(\lambda_i = \frac{i}{n})$  and one with geometrically decaying singular values  $\lambda_i = 0.7^i$ .
Optimal worst-case estimator. We first estimate the optimal robust risks by performing adversarial training and compare it with the robust risk of the conjectured optimal estimator. Adversarial training is performed as described in Section 4, where we generate data  $\{(\mathbf{y}_i,\mathbf{x}_i)\}$  using the respective forward model in the subspace. The estimator  $f(\mathbf{y}) = \mathbf{H}\mathbf{y}$  can be viewed as a neural network with one layer and without bias. Figure 5 shows that the robust risk of the conjectured estimator is essentially the same as the (empirical) optimal robust risk.
Figure 5: Robust risks of adversarial training (dots) and the conjectured optimal estimator (lines) for two forward operators within the subspace model. It can be seen that the conjectured estimators yield essentially the same robust risks as those of adversarial training.
Suboptimality of jittering. We further investigate whether optimal robust estimators can be obtained via jittering. To that end, we calculate the minimal robust risk attainable via jittering
$$
\min _ {\sigma_ {w}} R _ {\epsilon} (\operatorname * {a r g m i n} _ {f} J _ {\sigma_ {w}} (f)),
$$
where  $f(\mathbf{y}) = \mathbf{H}\mathbf{y}$  is a linear reconstruction operator as before. We make use of the analytic expression of the jittering estimator in Eq. (21) and calculate the attainable robust risk by minimizing the robust risk of jittering with respect to the jittering noise level. The results are compared to the (conjectured) optimal robust risks via Eq. (18) and the robust risk of the standard estimator. Figure 6 shows the results of the calculations for the forward operator with linear and geometrically decaying singular values described above. It can be seen that the standard estimator is noticeable less robust compared to denoising setups. Moreover, a gap can be observed between the robust risk obtained via jittering and the optimal robust risks.
Linear decay
Geometric decay
Figure 6: Pixel-wise robust risk of the optimal worst-case estimator, the jittering estimator with optimal jittering noise level, and the standard estimators. A small gap can be observed, showing that (isotrope) jittering does not yield the optimal linear worst-case estimator in general. Moreover, the standard estimator is more sensitive to adversarial perturbations as in the denoising setup.
D Details on the experimental results and further experimental results
In this section, we present details on the experimental results in Section 4, and present further experimental results on U-nets trained with robustness-enhancing methods for image denoising, deconvolution, and compressive sensing. The setup and methods considered are as described in the main body.
D.1 Optimal robust denoiser for large perturbation levels
In the main body, we presented empirical results for perturbation levels in the range  $\epsilon^2 /\mathbb{E}\left[\| \mathbf{x}\| _2^2\right]\in$ $[0,0.3]$ , since this is the practically most relevant regime. Figure 7 shows the risk of linear estimators and U-nets trained adversarially for perturbation levels in the range  $\epsilon^2 /\mathbb{E}\left[\| \mathbf{x}\| _2^2\right]\in [0,1.5]$ . From those plots, we see that the transition at  $\epsilon^2 /\mathbb{E}\left[\| \mathbf{x}\| _2^2\right] = 1$  predicted by Theorem 3.1 for the estimator to map to zero occurs for the subspace model (as predicted by the theory) as well as for the U-net for image denoising.
D.2 Convolution kernel for deconvolution experiments
In addition to experiments on denoising image data, we consider a deconvolution setup  $\mathbf{y} = \mathbf{k} \star \mathbf{x} + \mathbf{z}$  in this work. The kernel  $\mathbf{k}$  is Gaussian, applied channel-wise and visualized in Figure 8.
D.3 Quantifying the smoothing effect of Jittering
In the main text we presented visual example reconstructions and observed that neural networks trained via Jittering yield smoother reconstructions compared to adversarial and standard training.
Figure 7: The empirical robust risk for models trained with adversarial training for noise levels  $\sigma_z = 0$  (no noise),  $\sigma_z / \sqrt{n} = 0.5$  (medium noise) and  $\sigma_z / \sqrt{n} = 1.5$  (high noise). The transition predicted by Theorem 3.1 at  $\epsilon^2 \approx \mathbb{E}\left[\| \mathbf{x}\|_2^2\right]$  for the estimator to map to zero occurs for the subspace and image denoising settings.
Figure 8: Gaussian kernel for the deconvolution experiment. The  $8 \times 8$ -pixel kernel is calculated as discretization of the  $2d$  Gaussian density with standard deviation 2.
We further quantify this observation by calculating the total variation norm of reconstructed images for the deconvolution task. The results are shown in Figure 9 and confirm the observation that networks trained via Jittering yield smoother images.
D.4 Hyperparameter selection
In the experiments we treat the Jittering noise level  $\sigma_w$  as a hyperparameter, which we optimize over a validation dataset to obtain robust estimators at the desired perturbation levels. The hyperparameter search is performed by choosing a grid of jittering noise levels for each task. For each noise level neural networks (U-Nets) are trained via Jittering, and subsequently evaluated on the considered perturbation levels. Figures 10, 11 and 12 show the robust risks of jittering for the considered tasks, as well as the derived jittering choice rule. The smooth curves on the left panels represent the robust risk at a particular robustness level and are obtained by applying uniform filters on the evaluation results. It can be seen that for image denoising the empirical jittering choice is close to the prediction from theory.
Figure 9: Jittering yields smoother reconstructions for deconvolution. The plot shows the total variation norm of images, which are reconstructions using U-nets  $f_{\epsilon}$  trained via jittering, adversarially and via standard training. The TV-norm is calculated per reconstructed image  $f_{\epsilon}(\mathbf{y}_i)$  and averaged over the dataset, i.e.  $\mathrm{TV}(f_{\epsilon}) = \frac{1}{n}\sum_{i=1}^{n}\|f_{\epsilon}(\mathbf{y}_i)\|_{\mathrm{TV}}$ , with  $n = 2000$  measurements  $\mathbf{y}_i$ .