  Modern statistics provides an ever-expanding toolkit for estimating unknown
parameters. Consequently, applied statisticians frequently face a difficult
decision: retain a parameter estimate from a familiar method or replace it with
an estimate from a newer or complex one. While it is traditional to compare
estimators using risk, such comparisons are rarely conclusive in realistic
settings. In response, we propose the "c-value" as a measure of confidence that
a new estimate achieves smaller loss than an old estimate on a given dataset.
We show that it is unlikely that a computed c-value is large and that the new
estimate has larger loss than the old. Therefore, just as a small p-value
provides evidence to reject a null hypothesis, a large c-value provides
evidence to use a new estimate in place of the old. For a wide class of
problems and estimators, we show how to compute a c-value by first constructing
a data-dependent high-probability lower bound on the difference in loss. The
c-value is frequentist in nature, but we show that it can provide a validation
of Bayesian estimates in real data applications involving hierarchical models
and Gaussian processes.
