Title: Learning from Interval Targets
Venue: NeurIPS 2025
Accepted: yes
Keywords: Interval regression

Weak supervision

Learning theory
Abstract: We study the problem of regression with interval targets, where only upper and lower bounds on target values are available in the form of intervals. This problem arises when the exact target label is expensive or impossible to obtain, due to inherent uncertainties. In the absence of exact targets, traditional regression loss functions cannot be used. First, we study the methodology of using a loss functions compatible with interval targets, for which we establish non-asymptotic generalization bounds based on smoothness of the hypothesis class that significantly relaxing prior assumptions of realizability and small ambiguity degree. Second, 
  we propose a novel min-max learning formulation: minimize against the worst-case (maximized) target labels within the provided intervals. The maximization problem in the latter is non-convex, but we show that good performance can be achieved with the incorporation of smoothness constraints. Finally, we perform extensive experiments on real-world datasets and show that our methods achieve state-of-the-art performance.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses a weakly supervised regression problem where, instead of exact labels, only upper and lower bounds of the target values are provided. This problem is easier to collect labels than the traditional supervised setting. While this setting has been explored in prior work, existing approaches typically rely on strong theoretical assumptions. In contrast, this paper adopts more relaxed assumptions and offers several theoretical results that shed light on the problem, particularly regarding the influence of function class smoothness. The authors also explore a variety of objective functions suitable for learning from interval-based supervision and empirically evaluate their performance through comprehensive experiments.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. This paper addresses an important and compelling problem setting, proposing a practical algorithm that is flexible with respect to model choice, easy to implement, and empirically effective.\n2. The theoretical analysis is thorough and well-presented. The authors provide clear interpretations that help readers grasp the core insights of the theoretical results.\n3. For a natural baseline in this setting is to regress the midpoint of the lower and upper bounds; this paper demonstrates that such an approach can be overly pessimistic in terms of the worst-case (min-max) loss. Building on this insight, the authors propose a more optimistic and effective alternative. This result is, in my view, a significant contribution to the field.\n\nWeaknesses\n1. While I appreciate the overall contribution of the paper, I found that it is not fully self-contained, and substantial reorganization may be needed to meet the standards of a top-tier conference paper. Although the theoretical analysis is clearly presented in the main text, the algorithmic and experimental sections are difficult to follow without frequent reference to the appendix. Notably, the best-performing objective function used in the experiments is only described in the appendix (Equation 134), which makes it challenging for readers to fully understand or assess the empirical results based on the main paper alone.\n2. The paper reports results on only five UCI datasets, whereas Cheng et al. (2023a)—which the paper claims to follow—uses six datasets, including AgeDB, IMDB-WIKI, and STS-B. Including a more diverse and challenging set of datasets would help better demonstrate the generalizability of the proposed method.\n3. The experimental comparison does not include strong baselines from related prior work, such as the LM loss introduced in Cheng et al. (2023a). Including such methods would strengthen the empirical evaluation and clarify whether the proposed approach offers a clear improvement over existing techniques.\n\nQUESTIONS:\n1. Clarity and Self-Containment:\nPlease consider addressing the weaknesses I noted regarding the organization and self-containment of the paper. In particular, key experimental details and the best-performing objective function (Equation 134) are only described in the appendix, which hinders comprehension of the main results. If I have misunderstood or overlooked any explanations already present in the main text, I would appreciate clarification.\n2. Comparison with Noisy Label Regression:\nI believe it would be quite interesting to compare this interval-based framework with the standard setting of regression from noisy ground truth labels. Specifically, how would the proposed method perform relative to learning from noisy scalar labels, and are there scenarios where interval targets offer a clear advantage? I would be interested in hearing your perspective on this.\n3. Fair Comparison with PL (Mean) Approach:\nWhile the PL (mean) approach is shown to be highly effective, it requires training K+1 regressors, which introduces a significant computational cost. For a fairer comparison in terms of training time, it would be helpful to evaluate the performance of an ensemble of K+1 models using the minmax or projection-based objectives. Have you explored this comparison, or do you have any expectations about how such ensembles might perform?\n\nLIMITATIONS:\nyes (but limitation was discussed in Appendix B)", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper extends regression with interval-valued supervision by dropping the restrictive “ambiguity-degree” assumption and replacing it with a global Lipschitz-smoothness condition on the hypothesis class. It derives new generalization bounds for the standard projection loss and introduces a complementary min-max surrogate, then shows that enforcing an appropriate Lipschitz constant can reduce test MAE on a suite of tabular datasets.  \n\nPlease note that the following represents my initial impressions of the paper, and I am open to discussion. I welcome any corrections to potential misunderstandings.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths. \n- Theoretical advance over prior work. Extends the analysis to agnostic settings and relaxed ambiguity assumptions of Cheng et al. (2023) while still proving $O(n^{-1/2})$ convergence rates.\n- Clear link between smoothness and effective interval width. The “reduced-interval lemma” (Thm 3.6) formalizes how a global Lipschitz constant $m$ shrinks the set of plausible labels, providing an intuitive knob for bias–variance trade-off.\n- Two complementary risk surrogates. By analyzing both projection and min-max objectives under the same smoothness framework, the paper offers upper- and lower-risk envelopes that bracket the unknown true loss.\n- Simple, architecture-agnostic implementation. Enforcing global Lipschitzness via spectral normalisation can be applied to any feed-forward network without architectural redesign.\n- Comprehensive supplementary material. The appendix provides full proofs, additional datasets, hyperparameter grids, and ablation studies, facilitating in-depth verification.  \n\n## Weaknesses. \nThe paper was an enjoyable read, but I found the following potential weaknesses and questions. I am willing to reconsider my score once these issues are addressed.  \n\n### Potentially unclear link between the theory and experiments.  \nFig. 7 is confusing to me in sense that the claimed smoothness-realizability tradeoff (p.8, l.268-272) appears only on one of the three datasets (abalone) shown; the center (airfoil) and right (concrete) plots show monotonic decrease, without showing a minima due to the tradeoff between the term (b) and Rademacher complexity. Please clarify and discuss whether it is expected from the theory, or limitation of it.\n\n### Global Lipschitz constraint may be unrealistic. \nThe assumption that a single scalar $m$ upper bounds $|\\nabla f|$ for all $x$ might be difficult to be satisfied in high-dimensional or heterogeneous data: it may have risks of e.g., underfitting sharp local variations or exploding the output scale when the data manifold is low-dimensional inside $\\mathbb{R}^d$.\n\n### Reproducibility concern.\nCode is currently unavailable (though promised upon acceptance), preventing reviewers from assessing reproducibility.  \n\n### (Minor) Synthetic-only supervision.  \nIt seems that all interval targets are generated by an artificial random procedure; no evaluation on naturally occurring interval or censored data is provided.\n\nQUESTIONS:\nMy actionable requests already appear in the Weaknesses section; please address them point-by-point. In addition, I highlight a few items here:  \n\n- Could you extend Fig.7's $m$ sweep until MAE stats rising? If MAE does not raise, please add discussion why it is still compatible with what the theory expects.  \n- Could you possibly release an anonymous sample code so that I can evaluate parameter sensitivity of $m$, learning rate, weight-decay, overfitting/early-stopping etc. by myself?\n- In a practitioner's point of view, $m$ must be found via grid-search on training/validation data. How susceptible are conventional models to overfitting? \n\nAddressing these points would improve my assessment of the paper further.\n\nLIMITATIONS:\nThe authors provided Limitations section at Appendix B, but highlighting several points e.g., Global Lipschitz constant assumption and non-convexity of minmax optimization would be beneficial for readers.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies regression with interval-valued labels under an m-Lipschitz hypothesis class, where each observed interval captures uncertainty about the true target.\nWhen the true target is not available, it adopts two practical surrogates, an (optimistic) projection loss and a closed-form min–max loss, plus an auxiliary network for the inner maximization. This paper also provides a decomposed generalization bound on the loss.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. In general, the ideas are clearly conveyed.\n\n2. The generalization bound decomposition indicates the usefulness of the proposed method.\n\n3. The setting is interesting and practical in real-world data collection, where the labels are often noisy.\n\n**Weaknesses**\n\n1. While I appreciate the theoretical results, the technical contribution is unclear to me. The projection loss is very similar to prior works (such as [1]), while the minmax formulation seems to be a special or simplified case of robust ML (such as [2, 3]), where the feature variations/uncertainty do not exist.\n\n2. The approach for minmax formulation that utilizes an auxiliary worst-case learner is not connected to interval targets.\n\n3. Since the ultimate goal of this work is to reduce the overall loss of the classifier given interval targets, there are other baselines that could be considered in the evaluation. For instance, the evaluation could be strengthened by comparing with works that learn robust models from noisy labels, such as [4]. To construct their setting, the noisy labels could be sampled from the interval ranges.\n\n[1] Cheng, Xin, Yuzhou Cao, Ximing Li, Bo An, and Lei Feng. \"Weakly supervised regression with interval targets.\" In International Conference on Machine Learning, pp. 5428-5448. PMLR, 2023.\n[2] Mirman, Matthew, Timon Gehr, and Martin Vechev. \"Differentiable abstract interpretation for provably robust neural networks.\" In International Conference on Machine Learning, pp. 3578-3586. PMLR, 2018.\n[3] Wang, Shiqi, Huan Zhang, Kaidi Xu, Xue Lin, Suman Jana, Cho-Jui Hsieh, and J. Zico Kolter. \"Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification.\" Advances in neural information processing systems 34 (2021): 29909-29921.\n[4] Kim, Chris Dongjoo, Sangwoo Moon, Jihwan Moon, Dongyeon Woo, and Gunhee Kim. \"Sample selection via contrastive fragmentation for noisy label regression.\" Advances in Neural Information Processing Systems 37 (2024): 127561-127609.\n\nQUESTIONS:\n1. What does PL Mean mean in the experiment?\n\n2. Could this approach be applied beyond tabular data? I think the application on the unstructured data, such as images, is very important and practical.\n\n3. Is there a better way of approximating the Lipschitz constant? It seems the approximated ones in the experiment does not lead to optimal results.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies regression with interval targets, where the true labels are only known to lie within given intervals rather than being exact values. To address this setting, the authors propose two learning strategies: (1) minimizing a projection loss that penalizes predictions falling outside the intervals, and (2) a min-max objective that minimizes the worst-case loss over possible target values within each interval. They develop novel generalization bounds for the projection-based method under both realizable and agnostic settings, removing restrictive assumptions from prior work by leveraging the smoothness (e.g., Lipschitz continuity) of the hypothesis class. Theoretical insights reveal that smoother models implicitly reduce the effective size of interval constraints, improving predictive performance. Empirical results on real-world datasets support the theory and demonstrate that both proposed methods achieve state-of-the-art accuracy in interval-based regression tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n1. The paper is clearly written, and the theoretical results appear to be rigorous and well-justified. \n2. The work significantly improves upon prior results by removing the restrictive realizability and small ambiguity degree assumptions, instead establishing generalization guarantees under a smoothness condition on the hypothesis class. The empirical results are comprehensive and demonstrate that the proposed methods achieve strong performance across multiple real-world datasets.\n3. The empirical results are comprehensive and demonstrate that the proposed methods achieve strong performance across a few real-world datasets. \n\nWeakness\nIt would strengthen the paper to include a discussion on the tightness of the provided generalization error bounds. In particular, can lower bounds be established to show whether the derived rates are optimal or near-optimal?\n\nQUESTIONS:\nCan a similar generalization error bound be established for the min-max objective method? If so, under what conditions would it match or differ from the projection-based analysis?\n\nLIMITATIONS:\nSee weakness", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}