Denoising Diffusion Probabilistic Models for High-Fidelity fMRI Intrinsic Connectivity Network Data Generation

Authors: meenu  ajith
vince d calhoun
Venue: IEEE_org_EMBS_BHI_2024_Conference
Published: 2024-06-14 22:41:23

Abstract: The emergence of diffusion models such as Glide, Dalle-2, Imagen, and Stable Diffusion marks a significant breakthrough in generative AI-based image generation. This paper introduces an rs-fMRI image synthesis framework that leverages the nonlinear capabilities of denoising diffusion probabilistic models (DDPMs) to overcome the limitations of linear methods like independent component analysis (ICA) in neuroimaging analysis. Unlike ICA, which assumes linearity, DDPMs capture the intricate and complex patterns inherent in neuroimaging data. Our approach advances from 2D to 3D representations, providing a comprehensive visualization of intrinsic connectivity networks (ICNs). This framework also addresses the challenge of sparse training datasets commonly encountered in deep learning applications for neuroimaging. Trained on a large database, our model captures the intricate variability of different ICNs, generating realistic connectivity patterns. The proposed method is evaluated quantitatively to compare the synthesized ICNs against ground truth data. Results demonstrate that the proposed DDPM-based framework showcases competitive performance accuracy in reflecting the true complexity of neural connectivity patterns.

Main paper: 
 i. introduction: 
 In recent years, deep learning (DL) based methods for medical image synthesis, such as generative adversarial networks (GANs) [1] , have shown remarkable success across various tasks, including image denoising [2] , super-resolution [3] , data augmentation [4] , cross-modality image synthesis [5] , and image-to-image translation [6] . However, GANs encounter challenges such as mode collapse, training instability, and a lack of interpretability. To tackle this issue, diffusionbased generative models [7] were introduced in 2015. These models gained popularity recently with the emergence of denoising diffusion probabilistic models (DDPMs) [8] and latent diffusion models (LDMs) [9] . These alternatives to GANs provide higher quality and more diverse synthetic images by employing U-nets [10] to learn denoising for image generation. Furthermore, by avoiding adversarial training, diffusion models also enhance training stability and produce more realistic images.In a recent study, DDPM was used to generate 3D brain structural MRI images, and in a quantitative comparison, it outperformed a 3D-α-Wasserstein GAN [11] . Another study employed a conditional latent DDPM for medical image *This work was supported by the GSU RISE program and NSF grants 2316421 and 2316421.1 Meenu Ajith and 2 Vince Calhoun are with the Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS) @ Georgia State, Georgia Tech, and Emory University, Atlanta, GA 30303, USA, USA (e-mail: majith@gsu.edu; vcalhoun@gsu.edu). generation, comparing it with GAN-based models using images from ophthalmology, radiology, and histopathology. They showed that DDPMs excelled over GANs in both precision and diversity [12] . These studies have demonstrated that DDPMs offer a promising alternative to GANs in the field of neuroimaging. In this work, we use DDPMs for highquality intrinsic connectivity networks (ICNs) generation. In our work, ICNs are specific independent components (ICs) obtained through independent component analysis (ICA) that have been identified as representing meaningful and reproducible patterns of functional connectivity within the brain. Here, we generate 2D and 3D synthetic ICNs using the proposed model by training on real ICNs obtained from a pipeline called NeuroMark [13] , which leverages an a spatially constrained ICA model [14] . While ICA is a powerful method for extracting ICs from neuroimaging data, using DDPMs offers several distinct advantages. DDPMs can generate many synthetic ICNs, which is valuable for augmenting limited datasets. This is particularly useful in neuroimaging, where acquiring new data can be expensive and time-consuming. DDPMs can be used to generate ICNs with controlled variability, allowing researchers to create datasets with specific characteristics or test hypotheses under various conditions. Moreover, DDPMs can effectively denoise data, potentially improving the quality of the ICNs generated compared to direct ICA results, which may still contain residual noise and artifacts. Finally, DDPMs are nonlinear models, whereas ICA assumes linearity. This nonlinearity allows DDPMs to capture more complex patterns in neuroimaging data that linear methods like ICA might miss.The neuroimaging training dataset for this analysis was acquired from the UK Biobank database. It comprised 30,000 participants who had undergone rs-fMRI scanning using 3 Tesla Siemens Skyra scanners with 32-channel head coils. Preprocessing steps for the rs-fMRI data included motion correction with MCFLIRT, grand-mean intensity normalization, high-pass temporal filtering, and geometric corrections using FSL's Topup tool. Additionally, EPI unwarping and gradient distortion correction were applied. Artifacts were eliminated using ICA and FMRIB's ICA-based Xnoiseifier, and the data were standardized to an MNI EPI template, followed by Gaussian smoothing with a 6mm FWHM. Subsequently, the NeuroMark ICA process was applied to the 4D-preprocessed rs-fMRI data, utilizing the DDPMs form the foundation of current diffusion models in the field of generative AI. These models operate through two main steps: the forward noising process and the backward denoising process. In the forward noising process, DDPMs employ a Markov chain model with a predefined number of time steps, denoted as T . During each step, noise is incrementally introduced to the input image, starting at t = 0 and proceeding until t = T . The objective of this process is to transform the image into pure Gaussian noise by the final time step T . This transformation is governed by a noising function, represented by the conditional distribution q(x t |x t-1 ), which determines the noise addition at each step based on the image from the preceding time step. Noise schedulers are utilized to regulate the amount of noise added at each step.The goal is to develop a generative model that can predict the noise added to an image at a specific timestamp. This is achieved through the concept of the backward process, utilizing the conditional distribution function p(x t-1 |x t ). Directly modeling this function is impractical due to the vast number of possible images x t-1 for a given x t . Consequently, neural networks are employed for estimation, modifying the function to p θ (x t-1 |x t , t) where θ represents the network parameters. For this purpose, a U-Net architecture is employed, as illustrated in Fig. 1 . The modified U-Net architecture comprises three downsampling parts, where each part typically includes a convolutional layer followed by a max-pooling operation, reducing the feature maps' spatial dimensions. The central bottleneck consists of multiple convolutional layers, reducing the number of feature maps. Following the bottleneck, there are three up-sampling steps, each consisting of a transposed convolutional layer, followed by concatenation with feature maps from the corresponding down-sampling step. Thus, the model takes as input the image at time t and the timestep t and outputs the noise present in the image, as illustrated in Fig. 2 . The model effectively captures spatial dependencies in the data and learns to predict the noise distribution based on both the current image and the timestep.The proposed model was initially utilized to generate 2D representations of ICNs. Subsequently, the model was adapted and extended to produce 3D ICNs. This progression from 2D to 3D generation allowed for a more comprehensive and detailed visualization of ICNs, capturing their spatial complexity and differences in their functional characteristics. The single-subject rs-fMRI data generates 53 ICNs through ICA each of them belonging to different functional domains. The dataset used for training the model was sourced from the UK Biobank and comprised data from 10,000 subjects. Before training, we performed preprocessing steps such as normalization and background removal. Leveraging this large and diverse dataset, the model effectively captured variability across different networks, learning to generate realistic images that reflect the complex patterns of brain activity observed in each ICN. The ICN images generated using DDPMs are compared with ICNs obtained from ICA, as illustrated in Fig. 3 . While ICA is a linear method primarily designed for blind source separation, DDPMs are advanced nonlinear generative models. The comparison between them was done to highlight the specific strengths of DDPMs in capturing complex, nonlinear relationships within the data, which ICA may not fully address due to its linear nature. This also demonstrates the ability of DDPMs to generate more realistic and representative ICNs that better reflect underlying brain dynamics. Additionally, the model was compared with Progressive GANs [15] , which, despite also being a nonlinear approach, resulted in inconsistent structural patterns in the generated ICNs. These likely occurred due to the challenges Progressive GANs face with mode collapse and instability during training. Unlike DDPMs, which employ a gradual noise addition and removal process that enhances stability and coverage of the data distribution, Progressive GANs struggle to accurately capture the complex variability inherent in ICNs. Furthermore, we extended our approach by training 3D-DDPMs on ICNs specifically from the Sensorimotor and Visual domains as shown in Fig. 4 . These 3D models were designed to capture the intricate three-dimensional spatial patterns of brain activity, providing a more detailed and comprehensive visualization of the connectivity networks. An assessment of the reconstruction quality of the 3D-DDPMs was performed through a numerical evaluation, utilizing metrics such as MS-SSIM (Multiscale Structural Similarity Index Measure), MSE (Mean Square Error), and PSNR (Peak Signal-to-Noise Ratio). They are standard metrics used in literature for assessing image quality, measuring structural similarity, and quantifying reconstruction error, respectively. MS-SSIM measures image similarity across various scales, encompassing luminance, contrast, and structure, with values ranging from -1 to 1. A score of 1 represents perfect similarity, while -1 indicates perfect dissimilarity. Hence it is crucial to ensure that the generated ICNs maintain the intricate structures of neuroimaging data. MSE and PSNR provide insights into the generated images' pixel-wise accuracy and signal fidelity. Together, these metrics offer a comprehensive evaluation of both the visual quality and quantitative accuracy of the generated ICNs. The study evaluates the model using both the UK Biobank and an independent dataset, specifically the Human Connectome Project (HCP) dataset [16] , across all 53 ICNs. This approach helps mitigate the risk of overfitting associated with the complexity of DDPMs and ensures the generalizability of the findings to other datasets. After cross-validation with the UK Biobank dataset, the mean MS-SSIM was found to be 0.74, the mean MSE was 0.001, and the mean PSNR was 26.37. For the HCP dataset, the mean MS-SSIM was 0.72, the mean MSE remained 0.001, and the mean PSNR was 28.74. The results show that the model performs competitively on the HCP dataset, similar to its performance on the UK Biobank data, indicating that the model generalizes effectively.Moreover, Fig. 5 In conclusion, DDPMs offer a promising approach in the field of generative modeling in neuroimaging. By modeling the underlying probability distribution of data, DDPMs provide a powerful framework for generating high-quality samples. One of the key advantages of DDPMs is their versatility across dimensions, making them suitable for various types of data. This flexibility allows DDPMs to capture complex nonlinear dependencies in the data and generate realistic samples in different domains. The quantitative evaluation using various metrics also demonstrated the reconstruction capability of the DDPMs. However, scaling from 2D to 3D image generation using DDPMs presents significant computational challenges due to the increased data volume and complexity. Handling 3D data requires more memory, processing power, and sophisticated operations, leading to higher computational overhead. To address these challenges, strategies such as employing efficient 3D convolutional operations and utilizing distributed computing across multiple GPUs were implemented. These approaches collectively reduced the computational burden, making 3D generation more feasible and efficient. In future work, we will focus on subject-specific ICN generation using conditional DDPMs, allowing for a more tailored and individualized understanding of brain connectivity. 
 
 Appendix:  



Reviews: review: title: title: an innovative application of DDPMs but needs improvements in fair comparison, evaluation metrics clarification and rigorous validation
paper_summary: paper_summary: The paper presents a framework for generating high-fidelity intrinsic connectivity network (ICN) data from resting-state functional MRI (rs-fMRI) using Denoising Diffusion Probabilistic Models (DDPMs). The authors propose that DDPMs, which capture complex nonlinear patterns, offer significant advantages over traditional linear methods like Independent Component Analysis (ICA). The framework progresses from generating 2D to 3D ICN representations, providing more detailed visualizations of brain connectivity. Trained on a large dataset from the UK Biobank, the model generates realistic ICNs, which are quantitatively evaluated against ground truth data, showing competitive performance in terms of accuracy and fidelity. The results suggest that DDPMs are effective for neuroimaging applications, especially in generating synthetic data to augment limited datasets and enhance research capabilities in brain connectivity studies.
strength_weakness: strength_weakness: 1. The paper introduces DDPMs as a superior alternative to traditional methods like ICA for generating ICNs. However, the comparison might not be entirely fair. ICA is a well-established method for a specific purpose, and comparing it with DDPMs, which are fundamentally different and more complex, may not provide a balanced evaluation. It would be beneficial to include a comparison with other advanced nonlinear methods or generative models specifically designed for similar tasks. This could provide a more meaningful benchmark for evaluating the performance of the proposed framework.
2. While the paper uses quantitative metrics like MS-SSIM, MSE, and PSNR to evaluate the generated ICNs, the explanation of why these specific metrics were chosen and how they relate to the quality and fidelity of neuroimaging data is lacking. The authors could strengthen their argument by providing a clearer rationale for the selection of these metrics and discussing how they reflect the accuracy and utility of the generated ICNs in practical neuroimaging applications.
3. The study evaluates the model on a specific dataset from the UK Biobank and focuses on a limited number of ICNs. While the results are promising, the generalizability of the findings to other datasets and a broader range of ICNs is not fully addressed. The authors should consider validating their model on additional datasets from different sources and testing it across all 53 ICNs to ensure that the results are robust and applicable in various contexts.
4. Given the complexity of the DDPMs and the extensive training on a large dataset, there is a risk of overfitting, particularly if the generated ICNs closely resemble the training data. The authors should address how they mitigate this risk and provide evidence that the model can generalize well to unseen data. Techniques like cross-validation or testing on an independent dataset could help in demonstrating the model's generalization capabilities.
questions: questions: 1. I interpreted the comparison between DDPMs and ICA as possibly unfair because ICA is a linear method with a different primary purpose. It would be better to compare DDPMs with other advanced nonlinear methods or generative models specifically designed for similar tasks. However, if the authors intended to highlight specific strengths of DDPMs over ICA in this context, clarifying the rationale behind this comparison would be helpful. 
2. It would be better to exchange the order of Figure 4 and 5.
title: title: Denoising Diffusion Probabilistic Models for High-Fidelity fMRI Intrinsic Connectivity Network Data Generation
paper_summary: paper_summary: This paper introduces an rs-fMRI image synthesis framework leveraging denoising diffusion probabilistic models (DDPMs) to synthesize 3D intrinsic connectivity networks (ICNs).  The framework is trained on a large database to handle the sparsity of training datasets in neuroimaging, offering an advanced method for generating realistic connectivity patterns, which are evaluated against ground truth data.
strength_weakness: strength_weakness: 1. While DDPMs provide advantages over traditional methods, they also introduce complexity in model training and interpretation, which could be a barrier in clinical applications.

2. The paper does not mention validation of the proposed method on external datasets, which is crucial for generalizability.

3. The paper lacks a detailed comparison with state-of-the-art methods, particularly other non-linear models that might offer competitive or superior performance.
questions: questions: 1. Can you provide more details on the challenges faced during the training of the DDPMs, particularly concerning the stability of the learning process?

2. Could you discuss any specific computational requirements or limitations encountered while scaling from 2D to 3D ICN generation?

3. How does the model handle the variability in ICNs across different subjects, and could this impact the generalizability of the synthesized ICNs?