  Surrender poses one of the major risks to life insurance and a sound modeling
of its true probability has direct implication on the risk capital demanded by
the Solvency II directive. We add to the existing literature by performing
extensive experiments that present highly practical results for various
modeling approaches, including XGBoost and neural networks. Further, we detect
shortcomings of prevalent model assessments, which are in essence based on a
confusion matrix. Our results indicate that accurate label predictions and a
sound modeling of the true probability can be opposing objectives. We
illustrate this with the example of resampling. While resampling is capable of
improving label prediction in rare event settings, such as surrender, and thus
is commonly applied, we show theoretically and numerically that models trained
on resampled data predict significantly biased event probabilities. Following a
probabilistic perspective on surrender, we further propose time-dependent
confidence bands on predicted mean surrender rates as a complementary
assessment and demonstrate its benefit. This evaluation takes a very practical,
going concern perspective, which respects that the composition of a portfolio
might change over time.
