Elicitation is the study of statistics or properties which are computable via empirical risk minimization. While several recent papers have approached the general question of which properties are elicitable, we suggest that this is the wrong question---all properties are elicitable by first eliciting the entire distribution or data set, and thus the important questionis how elicitable. Specifically, what is the minimum number of regression parameters needed to compute the property? Building on previous work, we introduce a new notion of elicitation complexity and lay the foundations for a calculus of elicitation. We establish several general results and techniques for proving upper and lower bounds on elicitation complexity. These results provide tight bounds for eliciting the Bayes risk of any loss, a large class of propertieswhich includes spectral risk measuresand several new properties of interest. Finally, we extend our calculus to conditionally elicitable properties, which are elicitable conditioned on knowing the value of another property, giving a necessary condition for the elicitability of both properties together .
A property, or statistical functional, is said to be elicitable if it minimizes expected loss for some loss function. The study of which properties are elicitable sheds light on the capabilities and limits of empirical risk minimization. While several recent papers have asked which properties are elicitable, we instead advocate for a more nuanced question: how many dimensions are required to indirectly elicit a given property? This number is called the elicitation complexity of the property. We lay the foundation for a general theory of elicitation complexity , including several basic results about how elicitation complexity behaves, and the complexity of standard properties of interest. Building on this foundation, we establish several upper and lower bounds for the broad class of Bayes risks. We apply these results by proving tight complexity bounds, with respect to identifiable properties, for variance, financial risk measures, entropy, norms, and new properties of interest. We then show how some of these bounds can extend to other practical classes of properties, and conclude with a discussion of open directions .
R
Elicitation is the study of statistics or properties which are computable via
A property, or statistical functional, is said to be elicitable if it minimizes expected loss for some loss function. The study of which properties are elicitable sheds light on the capabilities and limits of
R
approached the general question of
asked
R
suggest that this is the wrong question---all properties are elicitable by first eliciting the entire distribution or data set, and thus the important questionis how elicitable. Specifically, what is
instead advocate for a more nuanced question: how many dimensions are required to indirectly elicit a given property? This number is called the elicitation complexity of
R
minimum number of regression parameters needed to compute the property? Building on previous work, we introduce a new notion
property. We lay the foundation for a general theory
R
and lay the foundations for a calculus of elicitation. We establish several general results and techniques for proving
, including several basic results about how elicitation complexity behaves, and the complexity of standard properties of interest. Building on this foundation, we establish several
R
on elicitation complexity. These results provide tight bounds for eliciting the Bayes risk of any loss, a large class of propertieswhich includes spectral risk measuresand several
for the broad class of Bayes risks. We apply these results by proving tight complexity bounds, with respect to identifiable properties, for variance, financial risk measures, entropy, norms, and
R
Finally, we extend our calculus to conditionally elicitable properties, which are elicitable conditioned on knowing the value of another property, giving a necessary condition for the elicitability of both properties together
We then show how some of these bounds can extend to other practical classes of properties, and conclude with a discussion of open directions