Communication-efficient Quantum Federated Learning Optimization for Multi-Center Healthcare Data

Authors: amandeep  singh bhatia
kaur  saggi
sabre  kais
cleveland  hospital
Venue: IEEE_org_EMBS_BHI_2024_Conference
Published: 2024-07-05 15:35:15

Abstract: In the healthcare sector, the scarcity of data and privacy concerns present formidable challenges to the widespread adoption of machine learning. In the present day scenario, Federated Learning (FL) emerges as a pivotal solution, fostering the rapid evolution of distributed
machine learning paradigms while adeptly addressing the problem of data governance and privacy. It allows distributed clients to collaboratively train a global model by synchronizing their local updates without sharing private data. In recent years, federated learning and quantum computing have individually shown great promise to revolutionize various sectors, including healthcare, finance, and manufacturing, where privacy protection is paramount. In this article, we propose a communication-efficient Quantum Federated Learning (QFL) framework based on a variational circuit that enables clients to efficiently train and transmit quantum model parameters, thereby reducing communication rounds significantly and enhancing QFL performance using quantum natural gradient descent (QNGD)
optimization. This paper demonstrates the feasibility of a QFL framework for predicting the presence of coronary heart disease, diagnosing whether a patient is suffering from diabetes or not, and differentiating malignant and benign cancer by distributing the data unbalanced among healthcare institutions. The proposed framework has the potential to incorporate privacy, security, and the expedited
processing of distributed data. QNGD outperformed classical GD by reducing communication rounds by a range of 5% to 60%. In addition to reducing the communication rounds by optimizing the QFL training algorithm and achieving quicker convergence, it also determines the important features regardless of the data imbalance among the clients.

Main paper: 
 i. introduction: 
 I N today's context, Federated Learning (FL) [1] has emerged as a remedy for the healthcare sector, enabling training across multiple hospitals without sharing private data. It addresses the critical need for collaborative model training while respecting patient privacy and data security concerns. In a federated learning framework, the central/global server initializes the machine learning model and distributes it to multiple hospitals, each acting as a client. Subsequently, each client conducts training using its local medical datasets and transmits the refined model updates back to the central hub. Upon receiving updates from all clients, the central server Amandeep Singh Bhatia, Mandeep Kaur Saggi, and Sabre Kais are with the Department of Electrical and Computer Engineering, North Carolina State University, Raleigh, North Carolina 27606, USA and Department of Chemistry, Purdue University, West Lafayette, Indiana 47907, USA (email: drasinghbhatia@gmail.com, drmandeepsaggi@gmail.com, skais@ncsu.edu) Fig. 1 . An overview of FedVQC Initially, the global server broadcasts weight parameters to the hospitals/clients. Subsequently, clients conduct training on the received quantum model, utilizing their local medical repository, and then transmit the parameters to the server. In FedVQC, each client employs a quantum natural gradient descent optimization with its local data distribution to refine the parameters of the quantum circuit instead of classical optimization.undertakes an aggregation process, thereby updating the status of the global model. The updated model is then redistributed to all clients for subsequent rounds of training. This iterative process continues until the global model converges to a state, where it accurately represents the knowledge learned from the diverse datasets across all hospitals. However, the heterogeneous nature of client datasets in federated learning settings often results in sluggish and erratic convergence, impeding the efficiency of the learning process [2] .In parallel, Quantum Machine Learning (QML) [3] has emerged as an enticing application of quantum technology and received significant attention from academic institutions and research communities alike. It uses the superposition, entanglement, parallelism and other characteristics of quantum computing to improve the performance of machine learning tasks. Till now, several approaches have been proposed and shown great potential to revolutionize various sectors, including healthcare, finance, chemistry, cybersecurity, optimization, and many more [4] , [5] . However, the current limitations of quantum computers, including noise and limited scalability, hinder the feasibility of many prominent quantum algorithms for handling practical problems [6] , [7] .Currently, Variational Quantum Algorithms (VQAs) are primarily designed for implementation on quantum computers during the noisy intermediate-scale quantum (NISQ) era [8] , [9] . To address these constraints, VQAs use a classical optimizer to refine the parameters of a parametrized quantum The iterative optimization process involves adjusting the parameters (θ) of a variational quantum circuit V(θ) using a classical or quantum optimization method, to minimize the associated cost function. Variational parameterized circuit architectures are recognized for their capability to produce feature map encodings that lead to readily trainable loss functions [8] , [9] . The choice of optimizer plays a crucial role in achieving effective generalization. Although, the trainability of variational quantum circuits suffers from the curse of "barren" plateaus, extensively discussed in recent literature [10] . It is the landscape where the gradient becomes exponentially small in the number of qubits. Nonetheless, even with the current con-straints of quantum hardware capabilities, variational quantum algorithms remain at the forefront as the preferred approach for gaining an advantage.In today's context, researchers have only produced a few related works on quantum federated machine learning. Some notable examples of quantum machine learning algorithms in federated settings include quantum neural networks using pre-trained classical models [11] , quantum neural networks [12] , variational quantum circuits [13] , quanvolutional neural networks [14] , quantum tensor networks [19] , variational quantum circuits in manufacturing [20] , and the application of quantum federated machine learning to handle privacy sensitive clinical data [15] .In recent years, there's been a growing reluctance among healthcare organizations to share data or participate in big data communities. This hesitance stems from various factors, including concerns about healthcare data breaches, regulatory compliance, and the potential misuse of sensitive information [16] . Another significant challenge to leveraging data analytics is the sheer volume of healthcare data required to train and validate machine learning models effectively. Particularly deep learning architectures, with millions or even billions of parameters, require vast amounts of high-quality data, which can be challenging to obtain and maintain, especially while adhering to strict privacy regulations.A challenge in FL is the high communication cost of exchanging weight updates between the global server and the clients. Heterogeneous client datasets lead to slow and unstable convergence, hindering the effectiveness of the FL process. However, while implementing multiple local updates before global aggregation can significantly reduce communication costs, it can increase the computational load on the client side.Motivated by the versatility and effectiveness of variational quantum circuits, a quantum federated model is introduced to address the reluctance of healthcare entities to share data directly. This model not only prioritizes data privacy but also fosters collaborative analysis and insights, thereby presenting a holistic solution to the challenges faced in the healthcare sector. The primary aim is twofold: to harness the untapped advantages of quantum machine learning and federated learning within the healthcare sector and to alleviate the high communication costs incurred by the frequent transmissions between the FL global server and clients using quantum optimization.The main contributions of this paper can be summarized as follows:• We propose a quantum federated learning framework (QFL) based on variational quantum algorithms and demonstrate its learning capability on popular UCI medical machine learning datasets, and validate our approach and its adaptability to heterogeneous data. Furthermore, we conduct validation of the quantum global model (FedVQC) using both local and external datasets. • We opt for a quantum variant of gradient descent optimizer instead of classical optimization to tune the parameters of a variational quantum circuit. Through experiments, we demonstrate the efficiency of our QFL framework utilizing quantum optimization in significantly reducing communication costs compared to the classical optimization method.In this section, we will explore how to harness the power of quantum federated learning through a combination of quantum and classical optimization techniques. Our aim is to enable collaborative training with improved accuracy and enhanced privacy within healthcare institutions. To achieve this, we concentrate on optimizing Variational Quantum Circuits (VQCs) for diagnosing heart disease, diabetes, and breast cancer. This involves leveraging pure quantum AI-driven analysis within the federated learning environment.We present a QFL algorithm that is an effective solution for collaborative training at specific hospitals, leveraging quantum computing to achieve superior model performance while preserving data privacy. The first step is to encode classical data into a quantum state. To efficiently simulate quantum circuits, we implemented both angle/qubit and amplitude encoding strategies for quantum state preparation. In angle encoding, the n qubits are utilized to depict n-dimensional data, with the input data features encoded into the rotation angles of the qubits. Any n-dimensional feature vector x = [x 1 , x 2 , ..., x n ] is encoded into a quantum state as:EQUATIONAmplitude encoding stores the normalized classical Ndimensional input data (N = 2 n ) in the amplitudes of an n-qubit quantum state |ψ⟩ as |ψ⟩ = 1 ∥x∥ N j=1 x j |j⟩. In amplitude encoding, the number of qubits needed to represent n classical bits is logarithmic, typically O(log n). After encoding, the next step involves applying a quantum circuit V(θ) with limited depth (l) to the feature state |ψ⟩, depending on the chosen parameterization for the gates and the number of layers. After angle encoding, each layer of VQC consists of single-qubit gates (R z , R y , R z ) on each qubit, followed by a linear arrangement of control Z gates. After performing amplitude encoding, we utilize a dense quantum circuit, where each layer consists of single qubit rotations and entangle them with control Not gates.The goal is to determine a sequence of gates that collectively yield the final state |ψ o ⟩. The quantum/classical optimizer manages these parameters throughout training, aiming to minimize a specified loss function. Following this, postprocessing is employed to calculate the circuit's expectation value, ultimately yielding the conclusive outcome of the classifier. The objective is to identify the optimal classifying circuit V(θ) that effectively distinguishes the dataset with distinct labels.Suppose there exists clients/hospitals (H) and each hospital (h ∈ H) has its own medical data containing n d samples, denoted state {|ψ d i ⟩} n d i=1 . Thus, the medical data at each hospital consists of unique ids, represented asEQUATIONInitially, a variational quantum circuit (VQC) is employed for local training. For client h ∈ H, the vector of trainable parameters (θ) is represented by- → θ h = (θ 1 , θ 2 , ..., θ N -1 , θ N ) ⊺ . A cost function (C f ) isdefined as the square of trace distance (T ) between final |ψ o o h ⟩, and initial state |ψ⟩, which is calculated asEQUATIONwhereO f = 1-|0⟩ ⟨0|. It is identical to C f = T (|ψ h o ⟩ ⟨ψ|) 2 .Our aim in client-side optimization is twofold: to minimize the divergence of clients from the global model and to reduce communication costs. To achieve this, we employed both classical gradient descent (CGD) and quantum natural gradient descent (QNGD) optimization techniques to refine the parameters of the local VQCs at each communication round. Classical optimizer aims to minimize a cost function V (θ) by iteratively updating the parameters θ in the opposite direction of the gradient of the cost function with respect to θ. However, this approach often encounters challenges when dealing with complex, high-dimensional optimization landscapes. The update rule for classical gradient descent is represented as θ r+1 = θ r -η∇V (θ r ), where θ r is the parameter vector at round rm η is the learning rate, determining the step size for each update, and ∇V (θ t ) is the gradient of the cost function V (θ) with respect to θ evaluated at θ r . While CGD guarantees convergence to a local minimum for convex functions, its performance in non-convex optimization landscapes can be sensitive to the choice of η and the initialization of parameters.Quantum Natural Gradient Descent (QNGD) [18] is an optimization technique that operates on the complex projective space and utilizes the Quantum Fisher Information or Fubini-Study metric to improve convergence speed. The quantum state space features an invariant metric tensor referred to as the Fubini-Study metric tensor, which can be used to develop a quantum version of a natural gradient descent. In QNGD, the cost function and the gradient are transformed in a way that preserves the geometry of the parameter space. The update rule for quantum natural gradient descent is represented as:EQUATIONwhere θ r represents the parameter vector at round r, η is the learning rate, F (θ t ) denotes the quantum Fisher information matrix evaluated at θ r , ∇V (θ r ) is the gradient of the cost function V (θ) with respect to θ at θ r , and F -1 (θ r ) signifies the inverse of the quantum Fisher information matrix. QNGD optimizer calculates the block-diagonal metric tensor during each optimization step, requiring n quantum evaluations. The analytic gradient of the objective function is determined ∇V (θ) using the parameter shift rule by introducing a small parameter shift s in θ values, its gradients can be calculated as ∇V (θ r ) = V (θ + s) -V (θ -s).After performing the local training on the client side, the vector of trainable parameters ( -→ θ h ) is uploaded to the global server, while the local data stays at each hospital. Finally, an aggregation is performed of all local updates (θ h r ) and a global server uses a cost function to minimize the gap between the predictions and actual values.EQUATIONwhere n d is the number of samples available with the client and η s is the learning rate of a global server. For the next round of communication (r), a global model sends the updated model parameters (θ s r ) to all clients.To demonstrate the robustness and generalizability of a proposed quantum federated learning algorithm, we studied the non-iid data partitioning strategy in the experiments. We first define the problem studied in this paper and then delve into the details of our results.In this paper, we studied three different datasets. 1. Heart disease dataset (HDD) [17] : It comprises four well-known heart disease databases: Cleveland, Hungarian, Switzerland, and Long Beach VA. Originally, this dataset contained 76 attributes, but only a subset of 14 attributes was utilized in the analysis, (13 as predictors and 1 as outcome) [17] . The outcome is an integer valued from 0 (no presence) to 4. For experiments, we have considered class 0 (no heart disease (NHD)) versus class 1, 2, 3, 4 together (heart disease (HD)). 2. Pima Indian diabetes dataset (PIDD) [17] : The National Institute of Diabetes and Digestive and Kidney Diseases released the PIDD dataset. It consists of 9 attributes (8 predictors and 1 class label). It is the representation of 8 characteristics of 768 women having more than 21 years [17] . The objective of the dataset is to diagnostically predict whether or not a patient has diabetes, based on certain diagnostic measurements included in the dataset. 3. We employed the Wisconsin Breast Cancer Dataset (WBCD) from the UCI Machine Learning repository [17] . It has 699 instances that are classified as benign and malignant. There are 569 data points in the dataset: 212 as Malignant, and 357 as Benign. Table I . Summary of datasets detailing the train-test-split and subject counts for three distinct datasets: Heart Disease, Diabetes, and Breast Cancer Wisconsin. Each dataset is categorized by its respective target classes, such as Heart Disease (with and without heart disease), Diabetes (diabetic and non-diabetic), and Breast Cancer (malignant and benign). Table II provides a summary of variational quantum circuits.We first tested the diagnosis capacity of VQC to detect the presence of heart disease in patients across three different hospital databases (Cleveland: 304 cases, Hungary: 293 cases, and Switzerland: 123 cases). Table III provides a summary of demographics for the heart disease dataset sourced from various hospital databases [17] . The table presents information on the distribution of sex (male and female) and age statistics (mean and standard deviation) for each hospital database included in the study.A total of 720 samples were utilized for classification evaluation. This comprised 15% of each database for internal validation of the global FedVQC model, while the remaining 85% was retained as local training data. The generalizability of FedVQC was validated on the external database. Each set of local training data comprises 13 features, each of which is encoded into a quantum state using angel encoding, utilizing 13 qubits respectively. The data distribution of each hospital is illustrated in Fig 4(a) , with the Cleveland hospital having the maximum number of samples and Switzerland having the fewest number of samples (specifically, 6 cases of no heart disease). The internal test dataset contains 109 patient cases (heart disease=50; no heart disease=59). As testing accuracy and loss are widely used to measure the quality of training capability, we consider this metric versus training communication rounds. To efficiently simulate all circuits, we used a QNGD optimizer to refine the VQC parameters of each hospital. We intend to measure the influence of client in We compare our proposed QFL framework consisting VQC algorithm with a classical GD and quantum NGD optimizer. In all three medical datasets, our experimental results indicate that FedVQC with QNGD outperformed FedVQC with a CGD optimizer and significantly reduced the number of communication rounds, as shown in Fig 7(a-c ). The box plots illustrate the performance metrics, including testing accuracy, precision, recall, and f-score of federated VQC across all datasets using CGD and QNGD optimizers. The blue line indicates the target testing set accuracy achieved by classical SVM. We address the communication rounds required to reach the target testing set accuracy. FedVQC-QNGD improves accuracy and convergence speed significantly and consistently compared with the FedVQC-CGD approach.FedVQC-QNGD achieved better convergence than the FedVQC-CGD method, resulting in higher precision, recall, and F-score, as depicted in Fig 7(d-f ), clearly indicating a significant reduction in communication rounds. FedVQC with quantum optimizer can reduce the communication overhead cost and maintain the baseline performance of binary classifications on the three medical datasets. Therefore, the optimizer selection plays a critical role in optimizing the local client models in federated settings. In Table IV , we compare the mean performance of CGD and QNGD with FedVQC.Our FedVQC model with quantum optimizer successfully discovered useful mechanism insights to guarantee sorting accuracies. Fig 8(a-c ) presents the top-hit feature importance of the Heart disease, diabetes, and breast cancer dataset. These visualizations elucidate the significant features identified through comprehensive analysis, offering insights into their collective relevance and potential impact on classification and prediction tasks. Next, we visualize the loss landscape of FedVQC-QNGD and FedVQC-CGD in Fig 9(a, b ), showcasing a local loss function for the classification task of distinguishing between diabetic and non-diabetic instances. The 3D landscape plot showcases the optimization landscape of the FedVQC loss function for two parameters (θ 1 and θ 2 ), highlighting QNGD's superior performance, nearing perfection, compared to the classical optimizer in federated settings. Classical GD represents the traditional approach, while QNGD leverages quantum techniques for optimization. The bulk of zero eigen- values shows a flat direction of the θ in the loss landscape of CGD, whereas QNGD optimizer generalizes well.In this work, we have demonstrated: (a) variational quantum circuits can successfully diagnose diseases within a federated environment. and (b) The choice of optimizer plays a crucial role in achieving effective generalization across clients or hospitals. FedVQC would provide robust/scalable classification for detecting various diseases, validated internally and externally. Their ability to leverage quantum computing capabilities alongside distributed data sources enables accurate and privacy-preserving disease diagnosis across multiple healthcare institutions. We demonstrated the convergence of QNGD in federated settings under different data distributions and presented extensive empirical results. Our findings show the superiority of QNGD over classical CGD in various scenarios, without the need for any tuning. This benefit is crucial, particularly as the increase in communication costs poses challenges for many practical FL applications, especially those involving clients with limited network bandwidth. 
 
 Appendix:  



Reviews: review: title: title: Review of the paper "Communication-efficient Quantum Federated Learning Optimization for Multi-Center Healthcare Data"
paper_summary: paper_summary: The authors propose a communication-efficient Quantum Federated Learning (QFL) framework based on a variational circuit that enables clients to efficiently train and transmit quantum model parameters, so reducing communication rounds significantly and enhancing QFL performance using quantum natural gradient descent optimization.
strength_weakness: strength_weakness: Only UCI datasets have been used.
No validation on real-world data is provided.
questions: questions: It should be specified in the abstract that the results were only obtained on UCI datasets.
title: title: This paper proposes a Quantum Federated Learning (QFL) framework that integrates variational quantum circuits (VQCs) with federated learning to enhance the efficiency of multi-center healthcare data analysis.
paper_summary: paper_summary: The paper presents a Quantum Federated Learning (QFL) framework designed to address the challenges of privacy and data sharing in multi-center healthcare studies. 
The framework leverages Variational Quantum Circuits (VQCs) and introduces Quantum Natural Gradient Descent (QNGD) to optimize the training process, reducing the number of communication rounds between the central server and distributed clients. Specifically, QNGD outperformed classical GD by reducing communication rounds by a range of 5% to 60%.
The study validates this approach using three healthcare datasets—heart disease, diabetes, and breast cancer—demonstrating that the quantum optimization method outperforms classical gradient descent in terms of accuracy and convergence speed.
strength_weakness: strength_weakness: The integration of quantum computing into federated learning, while innovative, adds a layer of complexity that may be challenging to implement in practice, particularly for institutions without access to quantum computing resources. The paper could benefit from a discussion on the practical deployment of this technology in real-world healthcare settings.
questions: questions: 1. How do you plan to validate the generalizability of your approach on more diverse and larger healthcare datasets, especially those representing varied patient populations?
2. What are the specific hardware and computational requirements for implementing your QFL framework, and how do these compare with the resources typically available in healthcare institutions?
title: title: Quantum Federated Learning Model for Multi-Center Healthcare Data
paper_summary: paper_summary: The paper presents a Quantum Federated Learning (QFL) framework that trains models using quantum computers in a federated learning approach. This allows different medical centers to collaboratively train a global model without sharing their data. In the proposed QFL framework, the paper introduces the quantum natural gradient descent (QNGD) technique, which enhances the performance of the quantum machine learning model compared to the classical gradient descent approach. Overall, the paper demonstrates the feasibility of using the QFL framework for various classification tasks on clinical datasets without privacy concerns and excessive costly communication rounds across distributed medical centers. The paper also shows improved performance using the QNGD technique proposed in the work.
strength_weakness: strength_weakness: - Quantum machine learning is a very novel and experimental area. While the motivation behind federated learning in this paper was clear, the motivation for quantum machine learning was not as clear. The paper describes methods to handle quantum machine learning algorithms in detail, but it should include more discussion on why quantum computing could be beneficial in this domain, given the challenges in quantum machine learning. The motivation behind quantum computing (reduced communication rounds and privacy protection) wasn’t clear to me. The work should emphasize more how this could be more beneficial than traditional federated learning algorithms.
- In the results section, there are some missing figures, missing descriptions (e.g., what are the samples, are they the measurements of each subject in different time point?), and some typos that need to be confirmed and clarified. (See Questions in below section for more details)
- In the comparison, is there any performance data for classical federated machine learning models (not on quantum computers)? How does the performance of the quantum machine learning model in this work compare to classical models? Is there any cost gain from the reduced number of communications or more efficient computing?
questions: questions: - Page 1: “Till now, several approaches have been proposed and shown great potential to revolutionize various sectors, including healthcare, finance, chemistry, cybersecurity, optimization, and many more [4], [5]”
    
    → How could Quantum Machine Learning revolutionize these areas? For example, can it provide much more powerful model accuracy? Or can it offer a more cost-efficient (energy-friendly) solution?
    
- Page 2: “application of quantum federated machine learning to handle privacy-sensitive clinical data [15].”
    
    → How was quantum machine learning utilized here? How was the performance in the previous work, or how is it different from this work?
    
- Page 3: “a quantum federated model is introduced to address the reluctance of healthcare entities to share data directly. This model not only prioritizes data privacy but also fosters collaborative analysis and insights, thereby presenting a holistic solution to the challenges faced in the healthcare sector.”
    
    → Similar comment here. How and why can a quantum model help with data privacy issues? Why can’t optimized federated learning with classical machine learning be used given the challenges in quantum computation?
    
- Page 3: “Section 4 provides details on the dataset,” → This may be referring to Section 3
- Page 4 Section A: “where θr is the parameter vector at round rm η is the learning rate” → round rm might be typo?
- Page 5 Table 1: It would be helpful if the table included how many instances came from each source within the train or test split for each dataset
- Page 5 Section B: “A total of 720 samples were utilized for classification evaluation”
    
    → How many samples were there for each subject? What were the samples? Were they measurements of each subject across different time points?
    
- Figure 4(e) may be confusing to read: Describe or put a legend indicating which trace represents accuracy vs. loss
- In Figure 4, there is no figure (d) showing global FedVQC model performance
- In Figures 4, 5, 6 (b), what is the blue dotted horizontal line?
- Page 7 Section E: The text says figure 8, but it seems to be referring to figure 7