SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Joan Bruna
Loucas Pillaud-Vivien
New York University Flatiron Institute
Aaron Zweig
Sparse high-dimensional functions have arisen as a rich framework to study the behavior of gradient-descent methods using shallow neural networks, showcasing their ability to perform feature learning beyond linear models. Amongst those functions, the simplest are single-index models  $f(x) = \phi (x\cdot \theta^{*})$ , where the labels are generated by an arbitrary non-linear scalar link function  $\phi$  applied to an unknown one-dimensional projection  $\theta^{*}$  of the input data. By focusing on Gaussian data, several recent works have built a remarkable picture, where the so-called information exponent (related to the regularity of the link function) controls the required sample complexity. In essence, these tools exploit the stability and spherical symmetry of Gaussian distributions. In this work, building from the framework of Ben Arous et al. [2021], we explore extensions of this picture beyond the Gaussian setting, where both stability or symmetry might be violated. Focusing on the planted setting where  $\phi$  is known, our main results establish that Stochastic Gradient Descent can efficiently recover the unknown direction  $\theta^{*}$  in the high-dimensional regime, under assumptions that extend previous works Yehudai and Shamir [2020], Wu [2022].
Over the past years, there has been sustained effort to enlarge our mathematical understanding of high-dimensional learning, particularly when using neural networks trained with gradient-descent methods — highlighting the interplay between algorithmic, statistical and approximation questions. An essential, distinctive aspect of such models is their ability to perform feature learning, or to extract useful low-dimensional features out of high-dimensional observations.
An appealing framework to rigorously analyze this behavior are sparse functions of the form  $f(x) = \phi(\Theta_{*}^{\top}x)$ , where the labels are generated by a generic non-linear, low-dimensional function  $\phi: \mathbb{R}^{k} \to \mathbb{R}$  of linear features  $\Theta_{*}^{\top}x$ , with  $\Theta_{*} \in \mathbb{R}^{d \times k}$  with  $k \ll d$ . While the statistical and approximation aspects of such function classes are by now well-understood Barron [1993], Bach [2017], the outstanding challenge remains computational, in particular in understanding the ability of gradient-descent methods to succeed. Even in the simplest setting of single-index models ( $k = 1$ ), and assuming that  $\phi$  is known, the success of gradient-based learning depends on an intricate interaction between the data distribution  $x \sim \nu$  and the 'link' function  $\phi$ ; and in fact computational lower bounds are known for certain such choices Yehudai and Shamir [2020], Song et al. [2021], Goel et al. [2020], Diakonikolas et al. [2017], Shamir [2018].
Positive results thus require to make specific assumptions, either about the data, or about the link function, or both. On one end, there is a long literature, starting at least with Kalai and Sastry [2009], Shalev-Shwartz et al. [2010], Kakade et al. [2011], that exploits certain properties of  $\phi$ , such as invertibility or monotonicity, under generic data distributions satisfying mild anti-concentration properties Soltanolkotabi [2017], Frei et al. [2020], Yehudai and Shamir [2020], Wu [2022]. On the other end, by focusing on canonical high-dimensional measures such as the Gaussian distribution, the seminal works Ben Arous et al. [2021], Dudeja and Hsu [2018] built a harmonic analysis
framework of SGD, resulting in a fairly complete picture of the sample complexity required to learn generic link functions  $\phi$ , and revealing a rich asymptotic landscape beyond the proportional regime  $n \asymp d$ , characterized by the number of vanishing moments, or information exponent  $s$  of  $\phi$ , whereby  $n \asymp d^{s-1}$  samples are needed for recovery. Since then, several authors have built and enriched this setting to multi-index models Abbe et al. [2022, 2023], Damian et al. [2022], Arnaboldi et al. [2023], addressing the semi-parametric learning of the link function Bietti et al. [2022], as well as exploring SGD-variants Ben Arous et al. [2022], Barak et al. [2022], Berthier et al. [2023], Chen et al. [2023]. This harmonic analysis framework relies on two key properties of the Gaussian measure and their interplay with SGD: its spherical symmetry and its stability by linear projection. Together, they provide an optimization landscape that is well-behaved in the limit of infinite data, and enable SGD to escape the 'mediocrity' of initialisation, where the initial direction  $\theta_0$ , in the high-dimensional setting, has vanishingly small correlation  $|\theta_0 \cdot \theta^*| \simeq 1/\sqrt{d}$  with the planted direction  $\theta^*$ .
In this work, we study to what extent the 'Gaussian picture' is robust to perturbations, focusing on the planted setting where  $\phi$  is known. Our motivation comes from the fact that real data is rarely Gaussian, yet amenable to being approximately Gaussian via CLT-type arguments. We establish novel positive results along two main directions: (i) when spherical symmetry is preserved but stability is lost, and (ii) when spherical symmetry is lost altogether. In the former, we show that spherical harmonics can be leveraged to provide a benign optimization landscape for SGD under mild regularity assumptions, for initialisations that can be reached with constant probability with the same sample complexity as in the Gaussian case. In the latter, we quantify the lack of symmetry with robust projected Wasserstein distances, and show that for 'quasi-symmetric' measures with small distance to the Gaussian reference, SGD efficiently succeeds for link functions with information exponent  $s \leq 2$ . Finally, using Stein's method, we address substantially 'non-symmetric' distributions, demonstrating the strength and versatility of the harmonic analysis framework.
The focus of this work is to understand regression problems with input/output data  $(x,y)\in \mathbb{R}^d\times \mathbb{R}$  generated by single-index models. This is a class of problems where the data labels are produced by a non-linear map of a one-dimensional projection of the input, that is
$$
y = \phi (x \cdot \theta^ {*}), \tag {1}
$$
where  $\phi : \mathbb{R} \to \mathbb{R}$  is also known as the link function, and  $\theta^{*} \in S_{d-1}$ , the sphere of  $\mathbb{R}^d$ , is the hidden direction that the models want to learn. Quite naturally, the learning is made through the family of generalized linear predictors  $\mathcal{H} = \{\phi_\theta : x \to \phi(x \cdot \theta)$ , for  $\theta \in S_{d-1}\}$ , built upon the link function (which is assumed known) and parametrized by the sphere.
Loss function. We assume that the input data is distributed according to a probability  $\nu \in \mathcal{P}(\mathbb{R}^d)$ . Equation (1) implies that the target function that produces the labels,  $\phi_{\theta^{*}}$ , lies in this parametric class. The overall loss classically corresponds to the average over all the data of the square penalisation  $l(\theta ,x)\coloneqq (\phi_{\theta}(x) - \phi_{\theta^{*}}(x))^{2}$  so that the population loss writes
$$
L (\theta) := \mathbb {E} _ {\nu} \left[ \left(\phi (x \cdot \theta) - \phi (x \cdot \theta^ {*})\right) ^ {2} \right] = \| \phi_ {\theta} - \phi_ {\theta^ {*}} \| _ {L _ {\nu} ^ {2}} ^ {2}, \tag {2}
$$
where we used the notation  $\| f\|_{L_\nu^p}^p = \mathbb{E}_\nu [|f|^p]$ , valid for all  $p\in \mathbb{N}^*$ . Let us put emphasis on the fact that the loss  $L$  is a non-convex function of the parameter  $\theta$ , hence it is not a priori guaranteed that gradient-based method are able to retrieve the ground-truth  $\theta^{*}$ . This often requires a precise analysis of the loss landscape, and where the high-dimensionality can play a role of paramount importance: we place ourselves in this high-dimensional setting for which the dimension is fixed but considered very large  $d\gg 1$ . Finally, we assume throughout the article that  $\phi_{\theta}$  belongs to the weighted Sobolev space  $W_{\nu}^{1,4}\coloneqq \{\phi$ , such that  $\sup_{\theta \in S_{d - 1}}\left[\| \phi_{\theta}\|_{L_{\nu}^{4}} + \| \phi_{\theta}'\|_{L_{\nu}^{4}}\right] < \infty \}$ .
Stochastic gradient descent. To recover the signal given by  $\theta^{*} \in S_{d-1}$ , we run online stochastic gradient descent (SGD) on the sphere  $S_{d-1}$ . This corresponds to having at each iteration  $t \in \mathbb{N}^*$  a fresh sample  $x_{t}$  drawn from  $\nu$  and independent of the filtration  $\mathcal{F}_{t} = \sigma(x_{1}, \ldots, x_{t-1})$  and performing a spherical gradient step, with step-size  $\delta > 0$ , with respect to  $\theta \to l(\theta, x_{t})$ :
$$
\theta_ {t + 1} = \frac {\theta_ {t} - \delta \nabla_ {\theta} ^ {S} l \left(\theta_ {t} , x _ {t}\right)}{\left| \theta_ {t} - \delta \nabla_ {\theta} ^ {S} l \left(\theta_ {t} , x _ {t}\right) \right|}, \quad \text {w i t h i n i t i a l i z a t i o n} \theta_ {0} \sim \operatorname {U n i f} \left(S _ {d - 1}\right), \tag {3}
$$
An important scalar function that enables to track the progress of the SGD iterates is the correlation with the signal  $m_{\theta} \coloneqq \theta \cdot \theta^{*} \in [-1, 1]$ . We will drop the subscript in case there is no ambiguity. Note that, due to the high-dimensionality of the setup, we have the following lemma:
Lemma 2.1. For all  $a > 0$ , we have  $\mathbb{P}_{\theta_0}(m_{\theta_0} \geq a / \sqrt{d}) \leq a^{-1} e^{-a^2 / 4}$ . Additionally, for any  $\delta > 0$  such that  $\max \{a, \delta\} \leq \sqrt{d} / 4$ , we have the lower bound:  $\mathbb{P}_{\theta_0}(m_{\theta_0} \geq a / \sqrt{d}) \geq \frac{\delta}{4} e^{-(a + \delta)^2}$ .
This fact implies that, when running the algorithm in practice, it is initialized with high probability near the equator of  $S_{d-1}$ , or at least in a band of typical size  $1 / \sqrt{d}$  (see Figure 1 for a schematic illustration of this fact). Finally, we use the notation  $\nabla_{\theta}^S$  to denote the spherical gradient, that is  $\nabla_{\theta}^S l(\theta, x) = \nabla_{\theta} l(\theta, x) - (\nabla_{\theta} l(\theta, x) \cdot \theta)\theta$ . As  $\nabla_{\theta}^S l(\cdot, x_t)$  is an unbiased estimate of  $\nabla_{\theta}^S L$ , it is expected that the latter gradient field rules how the SGD iterates travel across the loss landscape.
Loss landscape in the Gaussian case. As stressed in the introduction, this set-up has been studied by Dudeja and Hsu [2018], Ben Arous et al. [2021] in the case where  $\nu$  is the standard Gaussian, noted as  $\gamma$  here to avoid any confusion for later. Let us comment a bit this case to understand what can be the typical landscape of this single-index problem. Thanks to the spherical symmetry, the loss admits a scalar summary statistic, given precisely by the correlation  $m_{\theta}$ . Moreover, the loss admits an explicit representation in terms of the Hermite decomposition of the link function  $\phi$ : if  $\{h_j\}_j$  denotes the orthonormal basis of Hermite polynomials of  $L_{\gamma}^{2}$ , then  $L(\theta) = 2\sum_{j}|\langle \phi ,h_j\rangle |^2 (1 - m^j)\coloneqq \bar{\ell} (m)$ . As a result, the gradient field projected along the signal is a (locally simple) positive function of the correlation that behaves similarly to
$$
- \nabla_ {\theta} ^ {\mathcal {S}} L (\theta) \cdot \theta^ {*} \simeq C m ^ {s - 1} (1 - m), \tag {4}
$$
where  $s \in \mathbb{N}^*$  is the index of the first non-zero of the Hermite coefficients  $\{\langle \phi, h_j \rangle\}_j$ . This has at least three important consequences for the gradient flow: (i) if initialized positively, the correlation is an increasing function along the dynamics and there is no bad local minima in the loss landscape, (ii) the parameter  $s \in \mathbb{N}^*$  controls the flatness of the loss landscape near the origin and therefore controls the optimization speed of SGD in this region (iii) as soon as  $m$  is large enough, the contractive term  $1 - m$  makes the dynamics converge exponentially fast.
Loss landscape in general cases. Obviously for general distributions  $\nu$ , the calculation presented in Eq.(4) is no-longer valid. However, the crux of the present paper is that properties (i)-(ii)-(iii) are robust to the change of distribution and can be shown to be preserved under small adaptations. More precisely, we have the following definition.
Definition 2.2 (Local Polynomial Growth). We say that  $L$  has the local polynomial growth of order  $k \in \mathbb{N}^*$  and scale  $b \geq 0$ , if there exists  $C > 0$  such that for all  $m_{\theta} \geq b$ ,
$$
- \nabla_ {\theta} ^ {S} L (\theta) \cdot \theta^ {*} \geq C (1 - m _ {\theta}) \left(m _ {\theta} - b\right) ^ {k - 1}. \tag {5}
$$
In such a case we say that  $L$  satisfies  $\mathsf{LPG}(k,b)$
In this definition, and as showed later in specific examples given in Section 4, the scale parameter  $b$  should be thought as a small parameter proportional to  $1 / \sqrt{d}$ . If  $\nu$  is Gaussian, we can rewrite Eq.(4) and show that  $L$  verifies  $\mathrm{LPG}(s,0)$  for  $s\in \mathbb{N}^*$ , referred to as the information exponent of the problem in Ben Arous et al. [2021]. An important consequence of satisfying  $\mathrm{LPG}(k,b)$  is that the population landscape is free of bad local minima outside the equatorial band  $\Sigma_{b}\coloneqq \{\theta \in S_{d - 1},m_{\theta}\leq b\}$ . Therefore, when  $b$  is of scale  $1 / \sqrt{d}$ , Lemma 2.1 indicates that one can efficiently produce initializations that avoid it. Hence, this property is the fundamental ingredient that enables the description of the path taken by SGD that we derive it in the next Section. Section 4 is devoted to showcasing generic examples when this property is satisfied.
3 Stochastic Gradient Descent under LPG
In this section, we derive the main results on the trajectory of the stochastic gradient descent. They state that the property  $\mathsf{LPG}(s,b / \sqrt{d})$  is in fact sufficient to recover the same quantitative guarantees as the one depicted in Ben Arous et al. [2021], despite the lack of Gaussianity of the distribution  $\nu$ . Recall that the recursion satisfied by the SGD iterates is given by Eq.(3). To describe their movement, let us introduce the following notations: for all  $t\in \mathbb{N}^*$ , we denote the normalization by  $r_t\coloneqq \left|\theta_t - \delta \nabla_\theta^S\ell (\theta_t,x_t)\right|$  and the martingale induced by the stochastic gradient descent as  $M_{t}\coloneqq l(\theta_{t},x_{t}) - \mathbb{E}_{\nu}[l(\theta_{t},x)]$ .
Moment growth assumptions. To be able to analyse the SGD dynamics, we make the following assumptions on the moments of the martingale increments induced by the random sampling. Let us denote for all  $\theta \in \mathbb{R}^d$ ,  $x_\theta = x \cdot \theta \in \mathbb{R}$  and  $C(u,v) = \phi'(u)\phi(v)$ , for  $u,v \in \mathbb{R}$ .
Assumption 3.1 (Moment Growth Assumption). There exists a constant  $K > 0$ , independent of the dimension  $d$ , such that
$$
\sup  _ {\theta \in \mathcal {S} _ {d - 1}} \mathbb {E} _ {x} \left[ x _ {\theta_ {*}} ^ {2} C ^ {2} \left(x _ {\theta}, x _ {\theta_ {*}}\right) \right] \vee \mathbb {E} _ {x} \left[ x _ {\theta} ^ {2} C ^ {2} \left(x _ {\theta}, x _ {\theta_ {*}}\right) \right] \leq K, \tag {6}
$$
$$
\sup  _ {\theta \in \mathcal {S} _ {d - 1}} \mathbb {E} _ {x} \left[ | x | ^ {2 k} C ^ {2} \left(x _ {\theta}, x _ {\theta_ {*}}\right) \right] \leq K d ^ {k}, \quad \text {f o r} k = 1, 2. \tag {7}
$$
A precise care is given to the dependency in the dimension in the upper bound to match the practical cases that we later discuss in Section 4. Note that these assumptions are typically true for sub-gaussian random variables if  $\phi$  belongs to a Sobolev regularity class. These assumptions are similar to the one given in Eqs. (1.3)-(1.4) in Arous et al. [2021] for the Gaussian case. In all the remainder of the section we assume that Assumption 3.1 is satisfied.
Tracking the correlation. Recall that the relevant signature of the dynamics is the one-dimensional correlation  $m_t = \theta_t \cdot \theta^*$ . For infinitesimal step-sizes  $\delta \to 0$ , it is expected that  $\theta_t$  follow the spherical gradient flow  $\dot{\theta}_t = -\nabla_\theta^S L(\theta_t)$ , that translates to the evolution on the summary statistics  $m_t$  whereby
$$
\dot {m} _ {t} = - \nabla_ {\theta} ^ {S} L \left(\theta_ {t}\right) \cdot \theta^ {*}. \tag {8}
$$
The main idea behind the result of this section is to show that, even if the energy landscape near  $m = m_0$  is rough at scale  $1 / \sqrt{d}$ , the noise induced by SGD does not prevent  $m$  to grow as the idealized dynamics described by the ODE (8). Let us write the iterative recursion followed by  $(m_t)_{t\geq 0}$ : with the notation recalled above, for  $t\in \mathbb{N}^*$ , we have
$$
m _ {t + 1} = \frac {1}{r _ {t}} \left(m _ {t} - \delta \nabla_ {\theta} ^ {S} L \left(\theta_ {t}\right) \cdot \theta^ {*} - \delta \nabla_ {\theta} ^ {S} M _ {t} \cdot \theta^ {*}\right). \tag {9}
$$
With this dynamics at hand, the proof consists in controlling both the discretization error through  $r_t$  and the directional martingale induced by the term  $\delta \nabla_\theta^S M_t \cdot \theta^*$ .
Weak recovery. As it is the case for the gradient flow, most of the time spent by the SGD dynamics is near the equator, or more precisely in a band of the type  $\Sigma_{b,c} = \{\theta \in S_{d - 1}, b / \sqrt{d} \leq m_{\theta} \leq c / \sqrt{d}\}$ , where  $b < c$  are constants independent of the dimension. Hence, the real first step of the dynamics is to go out any of these bands. This is the reason why it is natural to define  $S_{a} := \{\theta \in S_{d - 1}, m_{\theta} \geq a\}$ , the spherical cap of level  $a \in (0,1)$  as well as the hitting time
$$
\tau_ {a} ^ {+} := \inf  \{t \geq 0, m _ {\theta_ {t}} \geq a \}, \tag {10}
$$
which corresponds to the first time  $(\theta_t)_{t \geq 0}$  enters in  $S_a$ . We fix a dimension-independent constant  $a = 1/2$ , and refer to the related hitting time  $\tau_{1/2}^+$  as the weak recovery time of the algorithm.
Theorem 3.2 (Weak Recovery). Let  $(\theta_t)_{t \geq 0}$  follow the SGD dynamics of Eq.(3) and let  $L$  satisfy  $\mathsf{LPG}(s, b / \sqrt{d})$ , with  $b > 0$  and  $s \in \mathbb{N}^*$ , then, conditionally on the fact that  $m_0 \geq 5b / \sqrt{d}$ , for any  $0 < \varepsilon \leq \varepsilon_*$ , we have
$$
\tau_ {1 / 2} ^ {+} \leq \left\{ \begin{array}{l l l l} d \cdot \mathrm {K} / \varepsilon & \text {w h e n} s = 1, & \text {a n d w i t h t h e c h o i c e} & \delta = \varepsilon / d \\ d \log^ {2} (d) \cdot \mathrm {K} / \varepsilon & \text {w h e n} s = 2, & \text {a n d w i t h t h e c h o i c e} & \delta = \varepsilon / (d \log d) \\ d ^ {s - 1} \cdot \mathrm {K} / \varepsilon & \text {w h e n} s \geq 3, & \text {a n d w i t h t h e c h o i c e} & \delta = \varepsilon d ^ {- s / 2} \end{array} \right. \tag {11}
$$
with probability at least  $1 - \mathsf{K}\varepsilon$ , for generic constants  $\mathsf{K}, \varepsilon_* > 0$  that depend solely on the link function  $\phi$  and the distribution  $\nu$ .
Let us comment on this result. It says that the integer  $s$  coming from the growth condition controls the hardness of exiting the equator of the sphere. Indeed, as can be seen in  $\mathsf{LPG}(s,b / \sqrt{d})$ , the larger the  $s$ , the smaller the gradient projection is and hence the less information the SGD dynamics has to move from the initialization. This result can be seen as an extension of [Ben Arous et al., 2021, Theorem 1.3] valid only in the Gaussian case ( $b = 0$ ). Furthermore, the Gaussian case shows that Theorem 3.2 is tight up to  $\log (d)$  factors. Finally, note that the result is conditional to the fact that the
Figure 1: Sketch of the SGD dynamics. After a long time spent in a band of typical size  $1 / \sqrt{d}$ , the dynamics escapes at weak recovery point  $\theta_{\mathrm{w}}$  and then goes rapidly to the ground-truth  $\theta^{*}$ .
initialization is larger that some constant factor of  $1 / \sqrt{d}$ , which has at least constant probability to happen in virtue of Lemma 2.1. This probability can be lowered by any constant factor by sampling offline a constant factor (independent of  $d$ ) of i.i.d. initializations and keeping the one that maximizes its correlation with  $\theta^{*}$ . Finally, note that in all the cases covered by the analysis, it is possible to keep track of the constant  $C$ , and show that overall it depends only (i) on the property of  $\nu$  w.r.t. the Gaussian on the one hand and (ii) on the Sobolev norm of the link function  $\| \phi \|_{W_\nu^{1,4}}$  on the other.
Strong recovery. We place ourselves after the weak recovery time described in the previous question and want to understand if the  $(\theta_{t})_{t\geq 0}$  dynamics goes to  $\theta^{*}$  and if yes, how fast it does so. This is what we refer to as the strong recovery question, captured by the fact that the one-dimensional summary statistics  $m$  go towards 1. Thanks to the Markovian property of the SGD dynamics, we have the equality between all time  $s > 0$  marginal laws of
$$
\left(\theta_ {\tau_ {1 / 2} ^ {+} + s} \mid \tau_ {1 / 2} ^ {+}, \theta_ {\tau_ {1 / 2} ^ {+}}\right) \stackrel {{\text {L a w}}} {{=}} \left(\theta_ {s} \mid \theta_ {s} = \theta_ {\tau_ {1 / 2} ^ {+}}\right),
$$
and hence the strong recovery question is equivalent to study the dynamics with initialization that has already weakly recovered the signal, i.e. such that  $m_{\theta} = 1 / 2$ . We show that this part of the loss landscape is very different that the equator band in which the dynamics spends most of its times: in all the cases, we can choose stepsizes independent of the dimension and show that the time to reach the vicinity of  $\theta^{*}$  will be independent of  $d$ .
Theorem 3.3 (Strong Recovery). Let  $(\theta_t)_{t \geq 0}$  follow the SGD dynamics of Eq.(3) and let  $L$  satisfy  $\mathsf{LPG}(s, b / \sqrt{d})$ , with  $b > 0$  and  $s \in \mathbb{N}^*$ , then, for any  $\varepsilon > 0$ , taking  $\delta = \varepsilon / d$ , we have that there exists a time  $T > 0$ , such that
$$
\left| 1 - m _ {T} \right| \leq \varepsilon , a n d \quad \left| T - \tau_ {1 / 2} ^ {+} \right| \leq K d \log (1 / \varepsilon) \varepsilon^ {- 1} \tag {12}
$$
with probability at least  $1 - \mathsf{K}\varepsilon$ , for some generic  $\mathsf{K} > 0$  that depends solely of the link function  $\phi$ .
As introduced above, the important messages conveyed by this theorem are that (i) there is no difference between the different parameters setups captured by the information exponent  $s$ , and (ii) the time it takes to reach an  $\varepsilon$ -vicinity of  $\theta^*$  is always strictly smaller than the one needed to exit the weak recovery phase (e.g.  $d$  compared to  $d^{s-1}$  when  $s \geq 3$ ). This means that the dynamics spends most of its time escaping the mediocrity. Remark that we decided to present Theorem 3.3 resetting the step-size  $\delta$  to put emphasis on the intrinsic difference between the two phases. Yet, we could have kept the same stepsize at the expense of slowing the second phase.
4 Typical cases of loss landscape with LPG property
In this section, we showcase two prototypical cases where the LPG holds true: the section 4.1 deals with the spherically symmetric setting, whereas the section 4.2 describes a perturbative regime where the distribution is approximately Gaussian is a quantitative sense.
4.1 The symmetric case
We start our analysis with the spherically symmetric setting. We show that a spherical harmonic decomposition provides a valid extension of the Hermite decomposition in the Gaussian case, leading to essentially the same quantitative performance up to constant (in dimension) factors.