Title: FALSA: Fairness-Aware Latent Space Alignment in Vision-Language Models for Medical Image Segmentation
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Fairness, Vision Language Model, Medical Image Segmentation, Latent Space Alignment, Bias Mitigation

Abstract: Vision-language models (VLMs) achieve impressive generalization in medical image segmentation but often propagate demographic biases from large-scale pretraining data. These biases manifest not only across single attributes (e.g., gender, race) but also through intersectional bias, where overlapping identities amplify disparities, undermining equitable clinical reliability. To address these problems, we introduce FALSA, a Fairness-Aware Latent Space Alignment framework that mitigates representational and predictive disparities without sacrificing accuracy. FALSA integrates three components: (1) Demographic-Invariant Contrastive Learning (DICL) to align multimodal embeddings while suppressing group-specific leakage, (2) Adaptive Fairness Calibration (AFC) to adversarially remove demographic cues from latent features, and (3) Unified Cross-Attribute Fairness Loss (UniFairLoss) to jointly reduce intra- and inter-attribute disparities, including intersectional ones. Applied to SAM and SAMed on the Harvard-FairSeg benchmark, FALSA achieves state-of-the-art equity-scaled Dice and IoU scores, reducing disparity indices and performance gaps by up to 70-80\% while improving overall segmentation accuracy. Unlike prior models that compromise segmentation accuracy, FALSA advances both fairness and performance, providing a scalable framework for equitable multimodal healthcare AI. Code and Data will be available at https://github.com/.../FALSA.

TL;DR: We introduce a fairness-aware latent space alignment (FALSA) framework for vision-language models that reduces demographic and intersectional biases in medical image segmentation while improving accuracy and equity across populations.