  Since the advent of U-Net, fully convolutional deep neural networks and its
many variants have completely changed the modern landscape of deep learning
based medical image segmentation. However, the over dependence of these methods
on pixel level classification and regression has been identified early on as a
problem. Especially when trained on medical databases with sparse available
annotation, these methods are prone to generate segmentation artifacts such as
fragmented structures, topological inconsistencies and islands of pixel. These
artefacts are especially problematic in medical imaging since segmentation is
almost always a pre-processing step for some downstream evaluation. The range
of possible downstream evaluations is rather big, for example surgical
planning, visualization, shape analysis, prognosis, treatment planning etc.
However, one common thread across all these downstream tasks is the demand of
anatomical consistency. To ensure the segmentation result is anatomically
consistent, approaches based on Markov/ Conditional Random Fields, Statistical
Shape Models are becoming increasingly popular over the past 5 years. In this
review paper, a broad overview of recent literature on bringing anatomical
constraints for medical image segmentation is given, the shortcomings and
opportunities of the proposed methods are thoroughly discussed and potential
future work is elaborated. We review the most relevant papers published until
the submission date. For quick access, important details such as the underlying
method, datasets and performance are tabulated.
