  Deep segmentation models that generalize to images with unknown appearance
are important for real-world medical image analysis. Retraining models leads to
high latency and complex pipelines, which are impractical in clinical settings.
The situation becomes more severe for ultrasound image analysis because of
their large appearance shifts. In this paper, we propose a novel method for
robust segmentation under unknown appearance shifts. Our contribution is
three-fold. First, we advance a one-stage plug-and-play solution by embedding
hierarchical style transfer units into a segmentation architecture. Our
solution can remove appearance shifts and perform segmentation simultaneously.
Second, we adopt Dynamic Instance Normalization to conduct precise and dynamic
style transfer in a learnable manner, rather than previously fixed style
normalization. Third, our solution is fast and lightweight for routine clinical
adoption. Given 400*400 image input, our solution only needs an additional
0.2ms and 1.92M FLOPs to handle appearance shifts compared to the baseline
pipeline. Extensive experiments are conducted on a large dataset from three
vendors demonstrate our proposed method enhances the robustness of deep
segmentation models.
