We investigate proper scoring rules for continuous distributions on the real line. It is known that the log score is the only such rule that depends on the quoted density only through its value at the outcome that materializes. Here we allow further dependence on a finite number m of derivatives of the density at the outcome, and describe a large class of such m-local proper scoring rules: these exist for all even m but no odd m. We further show that for m ≥ 2 all such mlocal rules can be computed without knowledge of the normalizing constant of the distribution. 1. Introduction. A scoring rule S(x, Q) is a loss function measuring the quality of a quoted distribution Q, for an uncertain quantity X, when the realized value of X is x. It is proper if it encourages honesty in the sense that the expected score E X∼P S(X, Q), where X has distribution P , is minimized by the choice Q = P . Traditionally, a scoring rule has been termed local if it depends on the density function q(·) of Q only through its value, q(x), at x. With this definition, any proper local scoring rule is equivalent to the log score, S(x, Q) = − ln q(x). However, we can weaken the locality condition by allowing further dependence on a finite number m of derivatives of q(·) at x, and this introduces many further possibilities. We term m the order of the rule. In this paper we describe a large class of such order-m proper local scoring rules for densities on the real line. These turn out to depend on the density q(·) in a way that is insensitive to a multiplicative constant, and hence can be computed without knowledge of the normalizing constant of q. Hyvärinen (2005) proposed a method for approximating a distribution P on X = R k by a distribution Q in a specified family P of distributions by
Among these, the log-loss is special: it is the only proper loss that is local [16] , REF .
Proper local scoring rules
Annals of Statistics 2012, Vol. 40, No. 1, 561-592
Mathematics