pwc-archive/papers-with-abstracts huggingface dataset:
Understanding Stain Separation Improves Cross-Scanner Adenocarcinoma Segmentation with Joint Multi-Task Learning
Authors: Ho Heon Kim, Won Chan Jeong, Young Shin Ko, Young Jin Park
Date: 2024-09-20 00:00:00
Tasks: Decoder, Diagnostic, Multi-Task Learning, Segmentation, Task 2

Digital pathology has made significant advances in tumor diagnosis and segmentation, but image variability due to differences in organs, tissue preparation, and acquisition - known as domain shift - limits the effectiveness of current algorithms. The COSAS (Cross-Organ and Cross-Scanner Adenocarcinoma Segmentation) challenge addresses this issue by improving the resilience of segmentation algorithms to domain shift, with Task 2 focusing on adenocarcinoma segmentation using a diverse dataset from six scanners, pushing the boundaries of clinical diagnostics. Our approach employs unsupervised learning through stain separation within a multi-task learning framework using a multi-decoder autoencoder. This model isolates stain matrix and stain density, allowing it to handle color variation and improve generalization across scanners. We further enhanced the robustness of the model with a mixture of stain augmentation techniques and used a U-net architecture for segmentation. The novelty of our method lies in the use of stain separation within a multi-task learning framework, which effectively disentangles histological structures from color variations. This approach shows promise for improving segmentation accuracy and generalization across different histopathological stains, paving the way for more reliable diagnostic tools in digital pathology.
Multiscale Encoder and Omni-Dimensional Dynamic Convolution Enrichment in nnU-Net for Brain Tumor Segmentation
Authors: Sahaj K. Mistry, Sourav Saini, Aashray Gupta, Aayush Gupta, Sunny Rai, Vinit Jakhetiya, Ujjwal Baid, Sharath Chandra Guntuku
Date: 2024-09-20 00:00:00
Tasks: Brain Tumor Segmentation, Segmentation, Tumor Segmentation

Brain tumor segmentation plays a crucial role in computer-aided diagnosis. This study introduces a novel segmentation algorithm utilizing a modified nnU-Net architecture. Within the nnU-Net architecture's encoder section, we enhance conventional convolution layers by incorporating omni-dimensional dynamic convolution layers, resulting in improved feature representation. Simultaneously, we propose a multi-scale attention strategy that harnesses contemporary insights from various scales. Our model's efficacy is demonstrated on diverse datasets from the BraTS-2023 challenge. Integrating omni-dimensional dynamic convolution (ODConv) layers and multi-scale features yields substantial improvement in the nnU-Net architecture's performance across multiple tumor segmentation datasets. Remarkably, our proposed model attains good accuracy during validation for the BraTS Africa dataset. The ODconv source code along with full training code is available on GitHub.
A solution for co-locating 2D histology images in 3D for histology-to-CT and MR image registration: closing the loop for bone sarcoma treatment planning
Authors: Robert Phillips, Constantine Zakkaroff, Keren Dittmer, Nicholas Robillard, Kenzie Baer, Anthony Butler
Date: 2024-09-20 00:00:00
Tasks: Image Registration

This work presents a proof-of-concept solution designed to improve the accuracy of radiographic feature characterisation in pre-surgical CT/MR volumes. The solution involves 3D co-location of 2D digital histology slides within ex-vivo, tumour tissue CT volumes. In the initial step, laboratory measurements obtained during histology dissection were used to seed the placement of the individual histology slices in corresponding tumour tissue CT volumes. The process was completed by aligning corresponding bone in histology images to bone in the CT using in-plane point-based registration. Six bisected canine humerus datasets of ex-vivo CT and corresponding measurements were used to validate dissection placements. Digital seeding exhibited a plane misalignment of 0.19 +- 1.8 mm. User input sensitivity caused 0.08 +- 0.2 mm in plane translation and between 0 and 1.6 degrees deviation. These are of similar magnitude to reported misalignment of 0.9-1.3 mm and 1.1-1.9 degrees in related prostate histology co-location [1]. Although this work only reported on animal femur sarcoma CT images and histology slide images, the solution can be generalised to various sarcoma geometries and presentation sites. Furthermore, the solution co-locates high-fidelity data to advance tissue characterisation with minimal disruption to existing clinical workflows. Improved tissue characterisation accuracy, supported by the resolution of histology images, can enhance surgical planning accuracy and patient outcomes by bringing the insights offered by histology closer to the start of the presentation-diagnosis-planning-surgery-recovery loop.
Utility of Multimodal Large Language Models in Analyzing Chest X-ray with Incomplete Contextual Information
Authors: Choonghan Kim, Seonhee Cho, Joo Heung Yoon
Date: 2024-09-20 00:00:00
Tasks: Large Language Model

Background: Large language models (LLMs) are gaining use in clinical settings, but their performance can suffer with incomplete radiology reports. We tested whether multimodal LLMs (using text and images) could improve accuracy and understanding in chest radiography reports, making them more effective for clinical decision support. Purpose: To assess the robustness of LLMs in generating accurate impressions from chest radiography reports using both incomplete data and multimodal data. Material and Methods: We used 300 radiology image-report pairs from the MIMIC-CXR database. Three LLMs (OpenFlamingo, MedFlamingo, IDEFICS) were tested in both text-only and multimodal formats. Impressions were first generated from the full text, then tested by removing 20%, 50%, and 80% of the text. The impact of adding images was evaluated using chest x-rays, and model performance was compared using three metrics with statistical analysis. Results: The text-only models (OpenFlamingo, MedFlamingo, IDEFICS) had similar performance (ROUGE-L: 0.39 vs. 0.21 vs. 0.21; F1RadGraph: 0.34 vs. 0.17 vs. 0.17; F1CheXbert: 0.53 vs. 0.40 vs. 0.40), with OpenFlamingo performing best on complete text (p<0.001). Performance declined with incomplete data across all models. However, adding images significantly boosted the performance of MedFlamingo and IDEFICS (p<0.001), equaling or surpassing OpenFlamingo, even with incomplete text. Conclusion: LLMs may produce low-quality outputs with incomplete radiology data, but multimodal LLMs can improve reliability and support clinical decision-making. Keywords: Large language model; multimodal; semantic analysis; Chest Radiography; Clinical Decision Support;
GASA-UNet: Global Axial Self-Attention U-Net for 3D Medical Image Segmentation
Authors: Chengkun Sun, Russell Stevens Terry, Jiang Bian, Jie Xu
Date: 2024-09-20 00:00:00
Tasks: Image Segmentation, Medical Image Segmentation, Semantic Segmentation

Accurate segmentation of multiple organs and the differentiation of pathological tissues in medical imaging are crucial but challenging, especially for nuanced classifications and ambiguous organ boundaries. To tackle these challenges, we introduce GASA-UNet, a refined U-Net-like model featuring a novel Global Axial Self-Attention (GASA) block. This block processes image data as a 3D entity, with each 2D plane representing a different anatomical cross-section. Voxel features are defined within this spatial context, and a Multi-Head Self-Attention (MHSA) mechanism is utilized on extracted 1D patches to facilitate connections across these planes. Positional embeddings (PE) are incorporated into our attention framework, enriching voxel features with spatial context and enhancing tissue classification and organ edge delineation. Our model has demonstrated promising improvements in segmentation performance, particularly for smaller anatomical structures, as evidenced by enhanced Dice scores and Normalized Surface Dice (NSD) on three benchmark datasets, i.e., BTCV, AMOS, and KiTS23.
Comparison and calibration of MP2RAGE quantitative T1 values to multi-TI inversion recovery T1 values
Authors: Adam M. Saunders, Michael E. Kim, Chenyu Gao, Lucas W. Remedios, Aravind R. Krishnan, Kurt G. Schilling, Kristin P. O'Grady, Seth A. Smith, Bennett A. Landman
Date: 2024-09-20 00:00:00

While typical qualitative T1-weighted magnetic resonance images reflect scanner and protocol differences, quantitative T1 mapping aims to measure T1 independent of these effects. Changes in T1 in the brain reflect structural changes in brain tissue. Magnetization-prepared two rapid acquisition gradient echo (MP2RAGE) is an acquisition protocol that allows for efficient T1 mapping with a much lower scan time per slab compared to multi-TI inversion recovery (IR) protocols. We collect and register B1-corrected MP2RAGE acquisitions with an additional inversion time (MP3RAGE) alongside multi-TI selective inversion recovery acquisitions for four subjects. We use a maximum a posteriori (MAP) T1 estimation method for both MP2RAGE and compare to typical point estimate MP2RAGE T1 mapping, finding no bias from MAP MP2RAGE but a sensitivity to B1 inhomogeneities with MAP MP3RAGE. We demonstrate a tissue-dependent bias between MAP MP2RAGE T1 estimates and the multi-TI inversion recovery T1 values. To correct this bias, we train a patch-based ResNet-18 to calibrate the MAP MP2RAGE T1 estimates to the multi-TI IR T1 values. Across four folds, our network reduces the RMSE significantly (white matter: from 0.30 +/- 0.01 seconds to 0.11 +/- 0.02 seconds, subcortical gray matter: from 0.26 +/- 0.02 seconds to 0.10 +/- 0.02 seconds, cortical gray matter: from 0.36 +/- 0.02 seconds to 0.17 +/- 0.03 seconds). Using limited paired training data from both sequences, we can reduce the error between quantitative imaging methods and calibrate to one of the protocols with a neural network.
Explainable AI for Autism Diagnosis: Identifying Critical Brain Regions Using fMRI Data
Authors: Suryansh Vidya, Kush Gupta, Amir Aly, Andy Wills, Emmanuel Ifeachor, Rohit Shankar
Date: 2024-09-19 00:00:00
Tasks: Diagnostic

Early diagnosis and intervention for Autism Spectrum Disorder (ASD) has been shown to significantly improve the quality of life of autistic individuals. However, diagnostics methods for ASD rely on assessments based on clinical presentation that are prone to bias and can be challenging to arrive at an early diagnosis. There is a need for objective biomarkers of ASD which can help improve diagnostic accuracy. Deep learning (DL) has achieved outstanding performance in diagnosing diseases and conditions from medical imaging data. Extensive research has been conducted on creating models that classify ASD using resting-state functional Magnetic Resonance Imaging (fMRI) data. However, existing models lack interpretability. This research aims to improve the accuracy and interpretability of ASD diagnosis by creating a DL model that can not only accurately classify ASD but also provide explainable insights into its working. The dataset used is a preprocessed version of the Autism Brain Imaging Data Exchange (ABIDE) with 884 samples. Our findings show a model that can accurately classify ASD and highlight critical brain regions differing between ASD and typical controls, with potential implications for early diagnosis and understanding of the neural basis of ASD. These findings are validated by studies in the literature that use different datasets and modalities, confirming that the model actually learned characteristics of ASD and not just the dataset. This study advances the field of explainable AI in medical imaging by providing a robust and interpretable model, thereby contributing to a future with objective and reliable ASD diagnostics.
Breaking the Barriers of One-to-One Usage of Implicit Neural Representation in Image Compression: A Linear Combination Approach with Performance Guarantees
Authors: Sai Sanjeet, Seyyedali Hosseinalipour, JinJun Xiong, Masahiro Fujita, Bibhu Datta Sahoo
Date: 2024-09-19 00:00:00
Tasks: Image Compression

In an era where the exponential growth of image data driven by the Internet of Things (IoT) is outpacing traditional storage solutions, this work explores and advances the potential of Implicit Neural Representation (INR) as a transformative approach to image compression. INR leverages the function approximation capabilities of neural networks to represent various types of data. While previous research has employed INR to achieve compression by training small networks to reconstruct large images, this work proposes a novel advancement: representing multiple images with a single network. By modifying the loss function during training, the proposed approach allows a small number of weights to represent a large number of images, even those significantly different from each other. A thorough analytical study of the convergence of this new training method is also carried out, establishing upper bounds that not only confirm the validity of the method but also offer insights into optimal hyperparameter design. The proposed method is evaluated on the Kodak, ImageNet, and CIFAR-10 datasets. Experimental results demonstrate that all 24 images in the Kodak dataset can be represented by linear combinations of two sets of weights, achieving a peak signal-to-noise ratio (PSNR) of 26.5 dB with as low as 0.2 bits per pixel (BPP). The proposed method matches the rate-distortion performance of state-of-the-art image codecs, such as BPG, on the CIFAR-10 dataset. Additionally, the proposed method maintains the fundamental properties of INR, such as arbitrary resolution reconstruction of images.
Personalized 2D Binary Patient Codes of Tissue Images and Immunogenomic Data Through Multimodal Self-Supervised Fusion
Authors: Areej Alsaafin, Abubakr Shafique, Saghir Alfasly, H. R. Tizhoosh
Date: 2024-09-19 00:00:00
Tasks: Multimodal Association, whole slide images

The field of medical diagnostics has witnessed a transformative convergence of artificial intelligence (AI) and healthcare data, offering promising avenues for enhancing patient care and disease comprehension. However, this integration of multimodal data, specifically histopathology whole slide images (WSIs) and genetic sequencing data, presents unique challenges due to modality disparities and the need for scalable computational solutions. This paper addresses the scarcity of multimodal solutions, primarily centered around unimodal data solutions, thus limiting the realization of the rich insights that can be derived from integrating images and genomic data. Here, we introduce MarbliX ``Multimodal Association and Retrieval with Binary Latent Indexed matriX,'' an innovative multimodal framework that integrates histopathology images with immunogenomic sequencing data, encapsulating them into a concise binary patient code, referred to as ``monogram.'' This binary representation facilitates the establishment of a comprehensive archive, enabling clinicians to match similar cases. The experimental results demonstrate the potential of MarbliX to empower healthcare professionals with in-depth insights, leading to more precise diagnoses, reduced variability, and expanded personalized treatment options, particularly in the context of cancer.
AutoPET III Challenge: PET/CT Semantic Segmentation
Authors: Reza Safdari, Mohammad Koohi-Moghaddam, Kyongtae Tyler Bae
Date: 2024-09-19 00:00:00
Tasks: Data Augmentation, Lesion Segmentation, Segmentation, Semantic Segmentation

In this study, we implemented a two-stage deep learning-based approach to segment lesions in PET/CT images for the AutoPET III challenge. The first stage utilized a DynUNet model for coarse segmentation, identifying broad regions of interest. The second stage refined this segmentation using an ensemble of SwinUNETR, SegResNet, and UNet models. Preprocessing involved resampling images to a common resolution and normalization, while data augmentation techniques such as affine transformations and intensity adjustments were applied to enhance model generalization. The dataset was split into 80% training and 20% validation, excluding healthy cases. This method leverages multi-stage segmentation and model ensembling to achieve precise lesion segmentation, aiming to improve robustness and overall performance.
Semi-overcomplete convolutional auto-encoder embedding as shape priors for deep vessel segmentation
Authors: Amine Sadikine, Bogdan Badic, Jean-Pierre Tasu, Vincent Noblet, Dimitris Visvikis, Pierre-Henri Conze
Date: 2024-09-19 00:00:00
Tasks: Medical Image Analysis

The extraction of blood vessels has recently experienced a widespread interest in medical image analysis. Automatic vessel segmentation is highly desirable to guide clinicians in computer-assisted diagnosis, therapy or surgical planning. Despite a good ability to extract large anatomical structures, the capacity of U-Net inspired architectures to automatically delineate vascular systems remains a major issue, especially given the scarcity of existing datasets. In this paper, we present a novel approach that integrates into deep segmentation shape priors from a Semi-Overcomplete Convolutional Auto-Encoder (S-OCAE) embedding. Compared to standard Convolutional Auto-Encoders (CAE), it exploits an over-complete branch that projects data onto higher dimensions to better characterize tiny structures. Experiments on retinal and liver vessel extraction, respectively performed on publicly-available DRIVE and 3D-IRCADb datasets, highlight the effectiveness of our method compared to U-Net trained without and with shape priors from a traditional CAE.
KLDD: Kalman Filter based Linear Deformable Diffusion Model in Retinal Image Segmentation
Authors: Zhihao Zhao, Yinzheng Zhao, Junjie Yang, Kai Huang, Nassir Navab, M. Ali Nasseri
Date: 2024-09-19 00:00:00
Tasks: Image Segmentation, Retinal Vessel Segmentation, Segmentation, Semantic Segmentation

AI-based vascular segmentation is becoming increasingly common in enhancing the screening and treatment of ophthalmic diseases. Deep learning structures based on U-Net have achieved relatively good performance in vascular segmentation. However, small blood vessels and capillaries tend to be lost during segmentation when passed through the traditional U-Net downsampling module. To address this gap, this paper proposes a novel Kalman filter based Linear Deformable Diffusion (KLDD) model for retinal vessel segmentation. Our model employs a diffusion process that iteratively refines the segmentation, leveraging the flexible receptive fields of deformable convolutions in feature extraction modules to adapt to the detailed tubular vascular structures. More specifically, we first employ a feature extractor with linear deformable convolution to capture vascular structure information form the input images. To better optimize the coordinate positions of deformable convolution, we employ the Kalman filter to enhance the perception of vascular structures in linear deformable convolution. Subsequently, the features of the vascular structures extracted are utilized as a conditioning element within a diffusion model by the Cross-Attention Aggregation module (CAAM) and the Channel-wise Soft Attention module (CSAM). These aggregations are designed to enhance the diffusion model's capability to generate vascular structures. Experiments are evaluated on retinal fundus image datasets (DRIVE, CHASE_DB1) as well as the 3mm and 6mm of the OCTA-500 dataset, and the results show that the diffusion model proposed in this paper outperforms other methods.
AutoPETIII: The Tracer Frontier. What Frontier?
Authors: Zacharia Mesbah, Léo Mottay, Romain Modzelewski, Pierre Decazes, Sébastien Hapdey, Su Ruan, Sébastien Thureau
Date: 2024-09-19 00:00:00
Tasks: Lesion Segmentation, Segmentation

For the last three years, the AutoPET competition gathered the medical imaging community around a hot topic: lesion segmentation on Positron Emitting Tomography (PET) scans. Each year a different aspect of the problem is presented; in 2024 the multiplicity of existing and used tracers was at the core of the challenge. Specifically, this year's edition aims to develop a fully automatic algorithm capable of performing lesion segmentation on a PET/CT scan, without knowing the tracer, which can either be a FDG or PSMA-based tracer. In this paper we describe how we used the nnUNetv2 framework to train two sets of 6 fold ensembles of models to perform fully automatic PET/CT lesion segmentation as well as a MIP-CNN to choose which set of models to use for segmentation.
A Systematic Post-Processing Approach for Quantitative $T_{1ρ}$ Imaging of Knee Articular Cartilage
Authors: Junru Zhong, Yongcheng Yao, Fan Xiao, Tim-Yun Michael Ong, Ki-Wai Kevin Ho, Siyue Li, Chaoxing Huang, Queenie Chan, James F. Griffith, Weitian Chen
Date: 2024-09-19 00:00:00

Objective: To establish an automated pipeline for post-processing of quantitative spin-lattice relaxation time constant in the rotating frame ($T_{1\rho}$) imaging of knee articular cartilage. Design: The proposed post-processing pipeline commences with an image standardisation procedure, followed by deep learning-based segmentation to generate cartilage masks. The articular cartilage is then automatically parcellated into 20 subregions, where $T_{1\rho}$ quantification is performed. The proposed pipeline was retrospectively validated on a dataset comprising knee $T_{1\rho}$ images of 10 healthy volunteers and 30 patients with knee osteoarthritis. Three experiments were conducted, namely an assessment of segmentation model performance (using Dice similarity coefficients, DSCs); an evaluation of the impact of standardisation; and a test of $T_{1\rho}$ quantification accuracy (using paired t-tests; root-mean-square deviations, RMSDs; and coefficients of variance of RMSDs, $CV_{RMSD}$). Statistical significance was set as p<0.05. Results: There was a substantial agreement between the subregional $T_{1\rho}$ quantification from the model-predicted masks and those from the manual segmentation labels. In patients, 17 of 20 subregions, and in healthy volunteers, 18 out of 20 subregions, demonstrated no significant difference between predicted and reference $T_{1\rho}$ quantifications. Average RMSDs were 0.79 ms for patients and 0.56 ms for healthy volunteers, while average $CV_{RMSD}$ were 1.97% and 1.38% for patients and healthy volunteers. Bland-Altman plots showed negligible bias across all subregions for patients and healthy volunteers. Conclusion: The proposed pipeline can perform automatic and reliable post-processing of quantitative $T_{1\rho}$ images of knee articular cartilage.