  In the field of multimodal segmentation, the correlation between different
modalities can be considered for improving the segmentation results. In this
paper, we propose a multi-modality segmentation network with a correlation
constraint. Our network includes N model-independent encoding paths with N
image sources, a correlation constraint block, a feature fusion block, and a
decoding path. The model independent encoding path can capture
modality-specific features from the N modalities. Since there exists a strong
correlation between different modalities, we first propose a linear correlation
block to learn the correlation between modalities, then a loss function is used
to guide the network to learn the correlated features based on the linear
correlation block. This block forces the network to learn the latent correlated
features which are more relevant for segmentation. Considering that not all the
features extracted from the encoders are useful for segmentation, we propose to
use dual attention based fusion block to recalibrate the features along the
modality and spatial paths, which can suppress less informative features and
emphasize the useful ones. The fused feature representation is finally
projected by the decoder to obtain the segmentation result. Our experiment
results tested on BraTS-2018 dataset for brain tumor segmentation demonstrate
the effectiveness of our proposed method.
