Title: Minimax-Optimal Univariate Function Selection in Sparse Additive Models: Rates, Adaptation, and the Estimation-Selection Gap
Venue: NeurIPS 2025
Accepted: yes
Keywords: sparse additive model

variable selection

minimax separation rate

FDR + FNR control

Hamming loss control
Abstract: The sparse additive model (SpAM) offers a trade-off between interpretability and flexibility, and hence is a powerful model for high-dimensional research.
This paper focuses on the variable selection, i.e., the univariate function selection problem in SpAM.
We establish the minimax separation rates from both the perspectives of sparse multiple testing (FDR + FNR control) and support recovery (wrong recovery probability control).
We further study how adaptation to unknown smoothness affects the minimax separation rate, and propose an adaptive selection procedure.
Finally, we discuss the difference between estimation and selection in SpAM: Procedures achieving optimal function estimation may fail to achieve optimal univariate function selection.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study variable selection in sparse additive models and establish minimax separation rates under both Hamming loss and a multiple testing loss (FDR + FNR). They obtain sharp results and show the problem is fundamentally different from the problem of estimating the model. Specializing to the case of $\\alpha$-smooth component functions, the authors also address adaptation to $\\alpha$ and demonstrate an unavoidable cost in the rate.\n\nSTRENGTHS AND WEAKNESSES:\n__Strengths__\nSparse additive models are a popular and important topic in machine learning, so these results are quite welcome. The variable selection problem is quite important, and the authors obtain sharp theoretical results of a very fundamental nature. Furthermore, the setting is fairly general, allowing the component functions to live in a generic RKHS (not just e.g. Sobolev space).\n\n__Weaknesses__\nThere are no major weaknesses. One might nitpick and say the white noise model is too abstract, but the paper's focus is theoretical and there is a long tradition in the statistics literature of relying on asymptotic equivalence to focus on the essence.\n\nQUESTIONS:\n- Can the authors address adaptation to sparsity? Currently, the variable selection procedure under the FDR + FNR loss requires choosing the truncation $K$ depending on $s$. The sparsity level $s$ may not typically be known. Can adaptation be done here? Is there some unavoidable cost that must also be paid?\n\n- Can the authors address adaptation more generally besides just Sobolev space? A major finding of the work by Kotekal and Gao cited in the paper was that, in the signal detection problem, there are different adaptation costs depending on the underlying function space. Some spaces impose an unavoidable cost, whereas others do not. Is there a similar phenomenon in the variable selection problem?\n\n- The selectors the authors study are essentially component-wise. Can the authors handle the more general setting where the component functions might live in different spaces? For example, $f_j \\in \\mathcal{H}_j$ where the $\\mathcal{H}_j$ might be different across $j$? This is the setting considered by Raskutti, Wainwright, and Yu.\n\n- Line 769: \"taht\" should be \"that\"\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the problem of variable selection in the nonparametric Gaussian white noise model, where the unknown signal is assumed to be a smooth multivariate function. This function is modeled as a sum of a small number of univariate components, each belonging to an ellipsoid defined via a quadratic constraint on its Fourier coefficients. The proposed selection procedure relies on (group) thresholding the Fourier coefficients of the noisy observed signal. The authors establish the minimax rates of separation for detecting the relevant variables. Additionally, they explore the case of unknown smoothness and propose an adaptive detection strategy.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. The problem of variable selection is one of the central challenges in nonparametric statistics and learning theory.\n2. The approach based on establishing minimax rates of separation is a highly relevant and principled way of quantifying the difficulty of the task.\n3. The paper establishes matching upper and lower bounds, which significantly strengthens the theoretical contribution.\n4. The writing is clear and the paper is pleasant to read.\n\n**Weaknesses**\n\n1. While this paper would be a perfect fit for a statistics journal, I am concerned that only a small portion of the NeurIPS audience will find these results directly relevant.\n2. The noise level is assumed to be known and fixed at $1/\\sqrt{n}$. A more general and practically relevant setting would allow the noise level to be an unknown parameter $\\sigma$.\n\n**Minor Remarks**\n\n1. Some important references are missing:\n   - [Comminges and Dalalyan, AoS 2012] and [Min Xu, Minhua Chen, John Lafferty, AoS 2016] studied variable selection in the same problem under different conditions.\n   - [Dalalyan, Ingster, Tsybakov, PTRF 2014] and [Tyagi et al., AISTATS 2016] considered a closely related model where the components of the additive model are not necessarily univariate. These works derived minimax rates for estimation.\n\n2. In the main theorems, the phrasing ''for every $\\delta$'' should be revised. As currently written, it may suggest that the assumptions on $n$ and other parameters must hold uniformly for all $\\delta$. A clearer formulation would be to begin the theorem with: \"Let $\\delta$ be an arbitrary number in...\" and then state the result. This makes it explicit that $\\delta$ is fixed within the statement.\n\nQUESTIONS:\nCan the proposed detection methods be easily adapted to deal with the case of unknown $n$ ?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

comments: SUMMARY:
This work establishes minimax separation rates for sparse additive models with Gaussian white noise, from the perspectives of sparse multiple testing and support recovery.

STRENGTHS AND WEAKNESSES:
Strengths: 1. Establishes minimax rates that integrate smoothness conditions, providing theoretical bounds for sparse additive models under Gaussian noise.  2. Proposes adaptive schemes to accommodate varying smoothness levels, enhancing practical applicability.   3. Clarifies key distinctions between optimal estimation and variable selection to avoid conceptual confusion.

Weaknesses:  While variable selection in additive models is a well-studied, the study lacks experimental comparisons between the proposed truncation method and existing selection approaches.

QUESTIONS:
1. Since variable selection in additive models is well-studied, could you elaborate on the proposed truncation method's strengths over existing approaches, such as its ease of tuning?  

2. In Corollary 1 for the finite-dimensional case, the setting resembles the detection boundary in signal testing. Could you derive a more specific bound on the relationship between $\epsilon_{\text{test}}$ and the number of non-zero components $m$, together with $\mu_1>0$?  

3. For the selector's truncation set  $\mathcal{K}_{rec}$  that adapts to unknown smoothness parameters, can bases other than 2 be chosen (e.g., base 3)? What is the essential impact of using base 2?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-24

comments: SUMMARY:
This paper studies the model selection problem in sparse additive model (SpAM), under a gaussian white noise (GWN) observation model. More specifically, it assumes that the dependence of the response on each of the relevant coordinates is modeled by a function belonging to an ellipsoid, and studies the problem of determining which coordinates of the input vector affect the output. 
Under a separation assumption between relevant and irrelevant coordinates, it determines minimax rates for this model selection problem.

STRENGTHS AND WEAKNESSES:
Strengths. The problem is quite canonical and the gives a complete answer for what concern rates. 
The paper is also clear and easy to read.

Weaknesses: Technically, the treatment is quite standard. 
Also, such exact support  recovery questions require to assume a separation condition (either y is independent of coordinate x_i, or the dependence is above a minimum value). This separation is typically considered unrealistic, so other forms of recoveries are of greater relevance.

QUESTIONS:
General question:
- Can you describe some concrete examples in which this exact recovery criterion (and the associated minimum strength assumption) are appropriate, and the work presented here provides some insights?

A couple of minor points:
-Above the (speculation) formula in Section 2.2. Here you did not yet define formally the problem, so it is unclear what does this speculation refer to.
-Above Theorem 1: Please avoid "sophisticated analysis", this is a judgement that you are expressing about your own work.

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 4
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers variable selection in a sparse additive model. Assuming the existence of an orthonormal basis, the  problem is reformulated as follows: For each $1\\leq j\\leq p$, we observe an infinite sequence $X_{ij}\\sim N(\\theta_{ij}, 1/n)$, where $\\theta_{ij}$ is the true coefficient of the $j$th function on the $i$th basis, satisfying that $\\sum_i \\frac{\\theta^2_{ij}}{\\mu_i}<1$. It is assumed that for most $j$, the infinite sequence {$\\theta_{ij}$}$_{i\\geq 1}$ is a zero sequence (i.e., the $j$th function $f_j$ is a zero function). The variable selection goal is to identify those $j$ such that $f_j$ is nonzero. \n\nThe authors consider two loss metrics, the sum of FDR and FNR, and the wrong recovery probability. For loss metric, the minimax order of signal strength for success has been derived. Compared to the previous work on estimation, the authors found that the minimax order has a factor of $\\log(p/s)$ or $\\log(p)$, which is considered as the ``interplay\" between high-dimensional selection error and estimation error of each individual function $f_j$.\n\nSTRENGTHS AND WEAKNESSES:\n1. There is a gap here transiting from the sparse additive model to the model of $X_{ij}=\\theta_{ij}+N(0,1/n)$ for infinite $j$. For people who are familiar with this problem, it may be straightforward. But for general readers, it is better to explain it. For example, there is no sample size $n$ in the current setting. The readers may get confused why $n$ appears here (it is actually a re-parametrization, right?). \n\n2. The \"interplay\" discovered here is a result of the loss function for variable selection. I don't really understand why this is a \"subtle and critical\" phenomenon. For example, in Gaussian sequence model, we need to set the threshold at $\\log(p/s)$ or $\\log(p)$ to achieve successful selection. What is special here? \n\n3. In a remark, the authors mention that the non-trivial part is due to the truncated chi-square statistics, in which we need to choose the truncation parameter $K$. However, is there any particular technical difficulty here? The novelty of the theory compared to those for Gaussian sequence model needs to be clarified. \n\n4. A major limitation of this work is that there is completely no numerical work and applications.\n\nQUESTIONS:\nWhen $\\mu_k$'s are unknown, how to implement the selector?\n\nLIMITATIONS:\nNo numerical work and real applications.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-09"}