PAMN : Multi-phase Correlation Modeling for Contrast-Enhanced 3 D Medical Image Retrieval (2025)

Abstract: Contrast-enhanced 3D Medical imaging (e.g., CT, MRI) leverages phase sequences to uncover temporal dynamics vital for diagnosing tumors, lesions, and vascular issues. However, current retrieval models primarily focus on spatial features, neglecting phase-specific progression detailed in clinical reports. We present the **Phase-aware Memory Network (PAMN)**, a novel framework enhancing 3D medical image retrieval by fusing imaging phases with diagnostic text. PAMN creates rich radiological representations that enhance diagnostic accuracy by combining image details with clinical report context, rigorously tested on a novel phase-series dataset of 12,230 hospital CT scans. PAMN achieves an effective balance of performance and scalability in 3D radiology retrieval, outperforming state-of-the-art baselines through the robust fusion of spatial, temporal, and textual information.

## introduction multi-view image analysis primarily addresses spatial and geometric relationships between images captured from different perspectives @xcite . these methods-applied in tasks such as video understanding @xcite @xcite , 3d rendering @xcite , and segmentation @xcite @xcite -rely on synthesizing geometric features like depth, shape, or texture to enhance scene understanding and object classification @xcite . multi-scale feature aggregation leverages spatial information across multiple resolutions to extract both fine details @xcite @xcite and broader contextual cues @xcite , while sequential architectures-such as long short-term memory (lstm) networks @xcite and transformer-based models @xcite )-capture sequential @xcite or temporal @xcite dependencies within image sequences. despite these advancements, adapting these techniques to medical imaging, especially for aligning spatial and phase information with textual clinical insights in 3d contrast-enhanced imaging, remains a challenging open problem. diagnostic reports offer rich clinical insights from multi-phase medical images by integrating spatial, temporal, and pathological correlations. while vision-language models like biomedclip @xcite and llava-med @xcite excel at joint visual-textual representation learning for tasks like classification and retrieval, and unimedi @xcite and fvlm @xcite have unified multi-modal images using reports or direct 3d image-text pairing, most current methods overlook crucial temporal information. in multi-phase radiology, temporal dynamics, such as contrast agent progression, are vital for clinical decisions and often detailed in reports. aligning these reports with specific imaging phases can significantly enhance feature representations in 3d contrast-enhanced imaging. to address this, we introduce the phase-aware memory network (pamn), a novel framework that combines phase-specific spatial cues with temporal progression modeling of contrast enhancement, utilizing diagnostic reports to refine feature representations (figure 2 ). pamn comprises three core components: pamn uses diagnostic reports as supervisory cues to model clinically relevant features that align with diagnostic patterns, boosting interpretability and robustness. ## method in this section, we present the phase-aware memory network (pamn), which aligns multi-phase medical images with text by organizing images into phase sequences and extracting multi-scale spatial features (figure 2 ). ## conclusion this study introduces the phase-aware memory network (pamn), a novel framework enhancing 3d medical image retrieval by integrating multiview spatial alignment, multi-phase temporal dynamics, and diagnostic text context. unlike conventional methods, pamn captures contrast-phase progression, aligning it with clinical reports to produce richer, clinically relevant representations. experiments on a 12,230-sample ct dataset show pamn outperforming 2d and 3d baselines, with up to 45% recall@10 improvement and robust scalability. ablations confirm the value of temporal cues and dynamic feature fusion. pamn's adaptability to single-phase datasets highlights its versatility, paving the way for intelligent, context-driven radiological analysis, with potential for automated diagnostics and broader clinical applications.