# Comparison of Image Normalization Methods for Multi-Site Deep Learning
###### Abstract  
In this study, we evaluate the influence of normalization on the performance of deep learning networks for tumor segmentation and the prediction of the pathological response of locally advanced rectal cancer to neoadjuvant chemoradiotherapy. The techniques were applied to a multicenter and multimodal magnet resonance imaging data set consisting of 201 patients recorded at six centers. We implemented and investigated six different normalization methods (setting the mean and standard deviation, histogram matching, percentiles, combining percentiles and histogram matching, fixed window and an auto-encoder with adversarial loss using the imaging parameters) and evaluated their impact on four deep learning tasks: tumor segmentation, prediction of treatment outcome, and prediction of sex and age. The latter two tasks were implemented as a reference test. We trained a modified U-Net with different normalization methods in multiple configurations: on all images, images from all centers except one, and images from a single center. Our results show that normalization only plays a minor role in segmentation, with a difference in Dice of less than 0.02 between the best and worst performing networks. For the prediction of sex and treatment outcomes, the percentile method combined with histogram matching works best for all scenarios. The biggest difference in performance, depending on the normalization method, occurs for classification. In conclusion, normalization is especially important for small data sets or for generalizing to different data distributions. The deep learning method was superior to the classical methods only in a minority of cases, probably due to the limited amount of training data. n  
Footnote †: journal: Medical  
For staging, magnetic resonance imaging (MRI) plays a key role [3]; however, T2-weighted (T2w) morphological imaging has low sensitivity while diffusion-weighted imaging (DWI) shows controversial results. Therefore, emerging techniques from machine learning are currently being investigated to overcome these limitations in predicting the pathologic treatment response. Thus, tumor segmentation and classification approaches have been studied. Tumor segmentation in rectal cancer is intrinsically challenging [4], even for state-of-the-art deep neural networks, due to hard-to-delineate tumors. Similar problems arise for the prediction of treatment outcomes [5]. One problem is that data sets are relatively small and often include only one or two centers [6]. To apply such models in a broader clinical setting, they must show good generalization, which means that the network should be trained on a set of images from different centers. For this, it is important to implement a standardized and harmonized protocol. Nevertheless, certain difference arising from, e.g., vendor-specific differences between scanners and sequence implementations, exist [7]. To mitigate such effects to a certain extent, image normalization methods could be employed as a preprocessing method prior to training a machine learning algorithm. There are different normalization methods, which could help reduce such data inhomogeneity. Statistical methods [8; 9] can also be challenging because the intensity distribution can vary due to variations in the anatomy of the patient, the chosen field of view, and the content of the bladder and bowels. In addition to classical approaches, deep learning methods for normalization have emerged [10; 11]. In the literature, there are various methods for domain adaptation [12], which go beyond simply changing the mean and the scale. A basic technique is histogram matching [13]. Many newer approaches are based on deep learning. Cycle GANs can be used to translate images from one acquisition protocol and scanner to another [14]. This can be extended to include more than two sites by using a StarGAN, which uses multiple encoders and decoders [15]. Other methods, such as Combat [16; 17] try to estimate and remove batch-dependent effects.