This paper is concerned with a two-fold objective. Firstly, we establish elicitability and identifiability results for systemic risk measures introduced in Feinstein, Rudloff and Weber (2017 ). Specifying the entire set of capital allocations adequate to render a financial system acceptable, these systemic risk measures are examples of set-valued functionals.A functional is elicitable (identifiable ) if it is the unique minimiser (zero) of an expected scoring function(identification function). Elicitability and identifiability are essential for forecast ranking and validation, M- and Z-estimation, both possibly in a regression framework. To account for the set-valued nature of the systemic risk measures mentioned above, we secondly introduce a theoretical framework of elicitability and identifiability of set-valued functionals. It distinguishes between exhaustive forecasts, being set-valued and aiming at correctly specifying the entire functional, and selective forecasts, content with solely specifying a single point in the correct functional. Uncovering the structural relation between the two corresponding notions of elicitability and identifiability, we establish that a set-valued functional can be either selectively elicitable or exhaustively elicitable. Notably, selections of quantiles such as the lower quantile turn out not to be elicitable in general. Applying these structural results to systemic risk measures, we construct oriented selective identification functions, which induce a family of strictly consistent exhaustive elementary scoring functions. We discuss equivariance properties of these scores. We demonstrate their applicability in a simulation study considering comparative backtests of Diebold-Mariano type with a pointwise traffic-light illustration of Murphy diagrams .
Identification and scoring functions are statistical tools to assess the calibration and the relative performance of risk measure estimates, e.g., in backtesting. A risk measures is called identifiable (elicitable) it it admits a strict identification function (strictly consistent scoring function). We consider measures of systemic risk introduced in Feinstein, Rudloff and Weber (2017 ). Since these are set-valued , we work within the theoretical framework of Fissler, Hlavinov\'a and Rudloff (2019) for forecast evaluation of set-valued functionals. We construct oriented selective identification functions, which induce a mixture representation of (strictly) consistent scoring functions. Their applicability is demonstrated with a comprehensive simulation study .
R
This paper is concerned with a two-fold objective. Firstly, we establish elicitability and identifiability results for systemic risk measures introduced in Feinstein, Rudloff and
Identification and scoring functions are statistical tools to assess the calibration and the relative performance of risk measure estimates, e.g., in backtesting. A risk measures is called identifiable (elicitable) it it admits a strict identification function (strictly consistent scoring function). We consider measures of systemic risk introduced in Feinstein, Rudloff and
R
). Specifying the entire set of capital allocations adequate to render a financial system acceptable, these systemic risk measures are examples of
). Since these are
R
functionals.A functional is elicitable (identifiable ) if it is the unique minimiser (zero) of an expected scoring function(identification function). Elicitability and identifiability are essential for forecast ranking and validation, M- and Z-estimation, both possibly in a regression framework. To account for the set-valued nature of the systemic risk measures mentioned above, we secondly introduce a
, we work within the
R
elicitability and identifiability
Fissler, Hlavinov\'a and Rudloff (2019) for forecast evaluation
R
It distinguishes between exhaustive forecasts, being set-valued and aiming at correctly specifying the entire functional, and selective forecasts, content with solely specifying a single point in the correct functional. Uncovering the structural relation between the two corresponding notions of elicitability and identifiability, we establish that a set-valued functional can be either selectively elicitable or exhaustively elicitable. Notably, selections of quantiles such as the lower quantile turn out not to be elicitable in general. Applying these structural results to systemic risk measures, we
We
R
family of strictly consistent exhaustive elementary
mixture representation of (strictly) consistent
R
We discuss equivariance properties of these scores. We demonstrate their applicability in a simulation study considering comparative backtests of Diebold-Mariano type with a pointwise traffic-light illustration of Murphy diagrams
Their applicability is demonstrated with a comprehensive simulation study