Title: What Data Enables Optimal Decisions? An Exact Characterization for Linear Optimization
Venue: NeurIPS 2025
Accepted: yes
Keywords: data informativeness

linear optimization

Blackwell's informativeness theory

data value

data-driven decision-making
Abstract: We study the fundamental question of how informative a dataset is for solving a given decision-making task. In our setting, the dataset provides partial information about unknown parameters that influence task outcomes. Focusing on linear programs, we characterize when a dataset is sufficient to recover an optimal decision, given an uncertainty set on the cost vector. Our main contribution is a sharp geometric characterization that identifies the directions of the cost vector that matter for optimality, relative to the task constraints and uncertainty set.
We further develop a practical algorithm that, for a given task, constructs a minimal or least-costly sufficient dataset. 
Our results reveal that small, well-chosen datasets can often fully determine optimal decisions---offering a principled foundation for task-aware data selection.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe goal is to choose  $x$ to minimize a loss function $L(x,\\theta)$, where $\\theta$ is a fixed but unknown parameter. Since $\\theta$. is not known, they evaluate the loss at N points to estimate $\\theta$. This paper considers the setting where the function $L$ is linear and the decision variable $x$ has to be selected from a polyhedral set. It is also assumed that the parameter $\\theta$ comes from a given uncertainty set. The authors introduce the notion of a sufficient dataset, defined as a collection of $N$ objective functions that allow exact recovery of the true optimal solution. The paper proposes an algorithm for finding minimal sufficient dataset for this problem.\n\nSTRENGTHS AND WEAKNESSES:\n- The problem setting is difficult to understand and not clearly explained. Initially, the paper states that the parameter $\\theta$ is not known. But later the concept of prior contextual knowledge is introduced.   \n\n- The example based on the Secretary Problem is difficult to follow. If $\\phi$ represents resumes of the candidates, what does $\\theta$ represent?   Also, in the Secretary Problem, the decision space increases as more candidates are interviewed, whereas the formal problem setup in Section 2 assumes a fixed-dimensional decision space $\\chi \\in \\mathcal{R}^d$. \n\n-  Algorithm 1 proposes a meta-algorithm that solves the LP optimization problem for every possible parameter value and stores all the unique solutions. This seems to be a trivial way to implement and I do not see the novelty. Indeed, if one could enumerate all possible solutions, the LP problem can be solved. Moreover, in most LP problems, the parameter space is continuous, making it unrealsistic to solve the LP for every possible parameter value $c$. \n\n- In general, the paper lacks consistent focus. It begins with the motivation of defining the informativeness of a dataset for solving an optimization problem, but this idea is not mentioned in Sections 3 and 4. As a result, the narrative loses a clear goal  and seems disconnected from the motivation in the Introduction. \n\n- I feel that the theoretical framework introduced in the earlier sections does not align well with the Secretary Problem use case. In the Secretary Problem, it is unclear what the basis vector of a solution would be. Additionally, LP solutions can be characterized by the extreme points (vertices) of the feasible region. So, if one stores all the vertices, the LP solution is guaranteed to come from this set. However, one cannot make a similar claim for the Secretary Problem. There is no guarantee that the best candidate will come from a predefined set of candidates.  Therefore, in my opinion, the Secretary Problem does not fit well with the problem formulation. \n\n- There is only one experiment, not sufficient for empirical evaluation. Moreover, the setup is not clearly explained. it is not immediately clear what the authors want to highlight in Figure 2.\n\nQUESTIONS:\n- Can you provide another use case, which actually solves an LP optimization problem?\n\n- What you want to show in Figure 2?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 2, "CLARITY": 1, "IMPACT": 1, "ORIGINALITY": 4, "DATE": "2025-07-01"}

comments: SUMMARY:
This paper investigates when a dataset contains sufficient information to recover the optimal solution to a linear program under cost uncertainty. The authors provide a sharp geometric characterization of task-relevant directions and propose an algorithm to construct minimal sufficient datasets. The work offers a principled perspective on task-aware data selection.

STRENGTHS AND WEAKNESSES:
**Strengths:**
- The paper addresses a fundamental and important question at the core of data-driven decision-making.
- The theoretical contributions are a key strength. The authors establish a clean formalism and derive their results with rigorous proofs. The connection established in Thm. 2 is particularly deep and non-trivial, creating an elegant bridge from pure theory to practical application.
- The paper is well-written, and its arguments are clear.

**Weaknesses:**

- The proposed algorithm relies on iteratively solving an MILP. While this is a standard and powerful approach, MILPs can be computationally intensive, and the paper lacks a detailed discussion of the algorithm's practical scalability.
- The focus on a noiseless setting is a notable limitation. While Prop. 3 offers a valuable local robustness guarantee; the concept of "sufficiently small" noise would be more tangible with concrete and practical examples to illustrate its scope.

QUESTIONS:
See Weaknesses.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the fundamental question of how informative a dataset is for solving decision-making tasks, with a focus on linear optimization under uncertainty. It introduces a geometric framework to determine when a dataset is sufficient to recover the optimal decision, given partial information about the cost vector. The authors provide exact characterizations using relevant extreme directions and develop an efficient algorithm to construct minimal sufficient datasets. Their results demonstrate that small, carefully chosen datasets can fully determine optimal solutions, providing a principled foundation for task-aware data selection and efficient data acquisition in settings such as hiring or resource allocation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The paper considers an important question of ML\n\n- The paper provides conditions for when a dataset is sufficient to recover optimal decisions in linear programs. This sharp geometric formulation, based on relevant extreme directions.\n\n- The fact that the span of relevant directions for dataset sufficiency can equivalently be expressed as the span of differences between optimal solutions under different cost vectors in the uncertainty set is interesting. From this, the authors develop a practical and efficient algorithm to construct minimal sufficient datasets. \n\nWeaknesses:\n\n-  The theory assumes that the decision-maker can query arbitrary directions and observe the exact inner product. This may be a strong assumption for making decision problems I have known.\n\n-  The paper’s results apply to linear optimization where the constraint set $X$ is fully known and fixed. This can limit the generality and applicability of the framework to nonlinear programming.\n\nQUESTIONS:\n- I am wondering whether the framework can be extended to making-decision problems in RKHS space, Bayesian optimization or nonlinear programming? What are the challenges?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a framework to identify the minimal sufficient dataset required to solve a linear program (LP) where the cost vector is uncertain. The authors provide a geometric characterization for data sufficiency based on the problem's constraints and the uncertainty set. They further develop an algorithm based on this theory to construct such a dataset for offline data collection settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper is well-written and easy to read.  The ''data sufficiency'' question in the offline setting is novel to me. The theoretical results are written clearly with a good pace.\n\n2. The theoretical support is rigorous. The paper provides necessary and sufficient conditions for data sufficiency, grounding the problem in the polyhedral geometry of the LP's decision space and uncertainty set.\n\n\nWeaknesses:\n\n1. $\\textbf{Overlap with sensitivity analysis.}$ The paper's main theoretical contribution appears to have overlap with established concepts in the literature on sensitivity analysis for robust LP, which is not sufficiently discussed or cited. The core theoretical mechanism of identifying \"relevant directions\" ($\\Delta(\\mathcal{X}, \\mathcal{C})$) by checking whether the uncertainty set $\\mathcal{C}$ intersects with the faces of the LP's optimality cones is a fundamental tool to post-optimal analysis. The paper's framework seems to be a re-application of these well-known principles to answer a \"data sufficiency\" question, rather than a fundamentally new theoretical discovery. \n\n2. $\\textbf{Uncertainty sets $\\mathcal{C}$.}$  The main weakness of the proposed methodology is the framework's dependence on a pre-defined uncertainty set $\\mathcal{C}$. The method's usefulness depends on the ability to specify this set, or equivalently, knowing the noise level before data collection. This is challenging in practice, as the magnitude of unobserved, \"Game-Changing\" factors is often what data collection aims to discover (see point 3 for an example). If $\\mathcal{C}$ is defined too conservatively to be safe, the method's recommendations can become trivial (i.e., collect all data).\n\n3. $\\textbf{Oversimplfied data example.}$ The paper's data example (hiring) is based on unrealistic assumptions. It implies that unobserved factors (e.g., personality, collaboration skills) have a bounded, non-decisive impact on extreme candidates (which corresponds to a high noise level regime in the paper). As stated in 2, if the unobserved factors are decisive, the method will be trivial to recommend collecting all the data.\n\n4. $\\textbf{Experimental setup inconsistent with proposed methodology.}$  There is a mismatch between the paper's theoretical treatment of noise and its experimental setup. Proposition 3 shows that with a fixed uncertainty set $\\mathcal{C}$, using data sufficiency is robust to small measurement noise $\\varepsilon$. However, the experiment takes a different approach: it constructs the uncertainty set $\\mathcal{C}$ itself by using a known, underlying \"noise level\" parameter $\\eta$. This experimental design avoids the main practical issue of how a user would define $\\mathcal{C}$ without prior knowledge of $\\eta$, thus failing to demonstrate how the framework would be applied in reality.\n\nQUESTIONS:\n1. How does the proposed methodology relate to sensitivity analysis for parametric and robust linear programming? What key theoretical contributions are novel beyond applying post-optimal analysis tools to a data sufficiency problem?\n\n2. How should a practitioner realistically define the uncertainty set $\\mathcal{C}$ for problems where the magnitude of unobserved factors is unknown beforehand? \n\n3. How to revise the experiments to better demonstrate the usefulness of the proposed methodology by addressing weaknesses points 3 and 4?\n\nLIMITATIONS:\nNA.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-17"}