NeurIPS 2025

GauSAM: Contour-Guided 2D Gaussian Fields for
Multi-Scale Medical Image Segmentation with
Segment Anything
Jinxuan Wu1∗
Jiange Wang1∗
Dongdong Zhang1†
1School of Computer Science and Technology
Tongji University
{2154146, 2252699, ddzhang}@tongji.edu.cn
Abstract
Effective multiscale medical image segmentation requires simultaneously preserv-
ing smooth spatial continuity and accurately delineating high-frequency bound-
aries, yet pixel-wise decoders often fail to maintain this balance consistently
across varying resolutions. We introduce GauSAM, which seamlessly integrates
contour-guided 2D Gaussian probability fields into the Segment Anything Model to
address these challenges. In our framework, segmentation masks are parameterized
as continuous probability fields of learnable 2D Gaussian primitives, enforcing
spatially smooth and structurally consistent. Contourlet transforms extract rich
multidirectional frequency information, notably edges and fine textures, which
dynamically guide the spatial distribution of Gaussian primitives to substantially
improve boundary fidelity in complex structures. The incorporation of these high-
frequency contour priors also enriches the expressive capacity of the SAM image
encoder. Extensive experiments on diverse 2D medical segmentation tasks con-
firm that GauSAM consistently delivers robust generalization and state-of-the-art
performance with only 1.2M trainable parameters. The official implementation of
GauSAM is publicly available at https://github.com/Quinten-Wu504/GauSAM.
1
Introduction
Medical image segmentation demands high precision and smooth boundary delineation, as anatom-
ical structures often exhibit ambiguous edges and segmentation errors can lead to serious clinical
consequences [1]. To address this, a wide range of methods have been proposed, including classical
architectures such as U-Net [2] and several fine-tuned variants of the Segment Anything Model
(SAM) [3]. These approaches typically rely on fixed-resoluti...