A system for fitting a deformable model to an anatomical structure in a medical image to obtain a segmentation of the anatomical structure, comprising: - an image interface for obtaining a first medical image of a patient and a second medical image of the patient, both medical images showing the anatomical structure and having been acquired by different medical imaging modalities or different medical imaging protocols, thereby establishing a different visual representation of the anatomical structure in both medical images; - a processing subsystem configured for: i) fitting a first deformable model to the anatomical structure in the first medical image, thereby obtaining a first fitted model; ii) fitting a second deformable model to the anatomical structure in the second medical image, thereby obtaining a second fitted model; wherein the second deformable model differs from the first deformable model for accommodating said different visual representation of the anatomical structure in both medical images; iii) mutually aligning the first fitted model and the second fitted model, and iv) after said mutual alignment, fusing the first fitted model and the second fitted model to obtain a fused model, wherein the processing subsystem is configured for mutually aligning the first fitted model and the second fitted model by: j) roughly aligning the first fitted model and the second fitted model; and jj) refining the alignment based on matching of pre-defined anchor structure; and wherein the processing subsystem is configured for fusing the first fitted model and the second fitted model by: adjusting or replacing a part of the first fitted model based on a corresponding part of the second fitted model having obtained a better fit. - an image interface for obtaining a first medical image of a patient and a second medical image of the patient, both medical images showing the anatomical structure and having been acquired by different medical imaging modalities or different medical imaging protocols, thereby establishing a different visual representation of the anatomical structure in both medical images; - a processing subsystem configured for: i) fitting a first deformable model to the anatomical structure in the first medical image, thereby obtaining a first fitted model; ii) fitting a second deformable model to the anatomical structure in the second medical image, thereby obtaining a second fitted model; wherein the second deformable model differs from the first deformable model for accommodating said different visual representation of the anatomical structure in both medical images; iii) mutually aligning the first fitted model and the second fitted model, and iv) after said mutual alignment, fusing the first fitted model and the second fitted model to obtain a fused model, wherein the processing subsystem is configured for mutually aligning the first fitted model and the second fitted model by: j) roughly aligning the first fitted model and the second fitted model; and jj) refining the alignment based on matching of pre-defined anchor structure; and wherein the processing subsystem is configured for fusing the first fitted model and the second fitted model by: adjusting or replacing a part of the first fitted model based on a corresponding part of the second fitted model having obtained a better fit. i) fitting a first deformable model to the anatomical structure in the first medical image, thereby obtaining a first fitted model; ii) fitting a second deformable model to the anatomical structure in the second medical image, thereby obtaining a second fitted model; wherein the second deformable model differs from the first deformable model for accommodating said different visual representation of the anatomical structure in both medical images; iii) mutually aligning the first fitted model and the second fitted model, and iv) after said mutual alignment, fusing the first fitted model and the second fitted model to obtain a fused model, j) roughly aligning the first fitted model and the second fitted model; and jj) refining the alignment based on matching of pre-defined anchor structure; and