  Batch Normalization (BN) is one of the key components for accelerating
network training, and has been widely adopted in the medical image analysis
field. However, BN only calculates the global statistics at the batch level,
and applies the same affine transformation uniformly across all spatial
coordinates, which would suppress the image contrast of different semantic
structures. In this paper, we propose to incorporate the semantic class
information into normalization layers, so that the activations corresponding to
different regions (i.e., classes) can be modulated differently. We thus develop
a novel DualNorm-UNet, to concurrently incorporate both global image-level
statistics and local region-wise statistics for network normalization.
Specifically, the local statistics are integrated by adaptively modulating the
activations along different class regions via the learned semantic masks in the
normalization layer. Compared with existing methods, our approach exploits
semantic knowledge at normalization and yields more discriminative features for
robust segmentation results. More importantly, our network demonstrates
superior abilities in capturing domain-invariant information from multiple
domains (institutions) of medical data. Extensive experiments show that our
proposed DualNorm-UNet consistently improves the performance on various
segmentation tasks, even in the face of more complex and variable data
distributions. Code is available at https://github.com/lambert-x/DualNorm-Unet.
