E Pluribus Unum (Out of Many, One): Leveraging Federated Learning to Promote Deep Learning Model Development

The term federated is an adjective which traditionally refers to a country or organization set up as a single centralized unit within which each state or division keeps some internal autonomy [1Oxford LanguagesGoogle search for federated.https://google.comDate accessed: June 24, 2022Google Scholar]. Federated learning (FL) (also known as collaborative learning) is a machine learning technique that trains an algorithm across multiple decentralized facilities using local data, without directly exchanging that data. In essence, FL enables multiple, disparate groups to build a common machine learning model or algorithm without sharing data which ameliorates critical issues related to data privacy, data security, data access rights and access to heterogeneous data. One of the key differences between FL and more traditional distributed learning methods in deep learning (DL) research pertains to how the local datasets are leveraged. FL aims at training on heterogeneous datasets located at multiple sites while distributed learning aims at maximizing compute power on local datasets. In this issue, two articles by Darzidehkalani et al. [2Darzidehkalani E. Ghasemi-rad M. van Ooijen P.M.A. Federated learning in medical imaging: part I: toward Multicentral Health Care Ecosystems.J Am Coll Rad. 2022; 19: 967-972Google Scholar,3Darzidehkalani E. Ghasemi-rad M. van Ooijen P.M.A. Federated learning in medical imaging: part II: methods, challenges, and Considerations.J Am Coll Rad. 2022; 19: 973-980Google Scholar] describe the advantages and challenges of applying FL techniques to machine learning research and model/algorithm development. Neural networks trained on a single dataset from a single institution may result in a biased model/algorithm due to overfitting to that particular institutional dataset thereby leading to poor generalization. FL is touted as a potential solution to lessen these inherent biases by providing larger and more heterogeneous datasets by which the model can be derived. This patient cohort, image parameter/technique, and modality/vendor heterogeneity promotes a more robust and generalizable model. This is especially helpful in instances in which diseases are rare and that the only way to create a generalizable model is to have access to data from multiple sites. The additional bonus of leveraging FL is the preservation of privacy given that the data stays within the institution’s servers and only the model, which does not contain any protected health information (PHI), moves between sites for purposes of “learning” from that local data. Certain infrastructure and expertise at the various sites is essential for creating and maintaining a federated learning platform across multiple sites. A file transmission schema in addition to standardized software platforms (e.g. Jupyter notebook) are often utilized to allow for the transmission of the model/algorithm at the various sites. The authors highlight the FAIR principle for the data standard which consists of Findable, Accessible, Interoperable, and Reusable data collection. Also, there are different types of FL methods (e.g. centralized, decentralized, and heterogeneous) that serve different purposes depending on the model being developed. A federated round occurs when a model is distributed between the various sites and is typically made up of different phases including initialization, client selection, configuration, reporting, and termination. Single and cyclic weight transfers (SWT and CWT) are also utilized depending on the number of clients and the transfer time between clients. A SWT model is trained at a single client up to a particular time and then transferred to the next client. A CWT model is trained at each site for a certain number of iterations or epochs and then transferred to the next client (therefore the model visits each client more than once). These FL and machine learning standards and hyperparameters are continuing to evolve and will be essential for the successful creation, implementation, and validation of the model in the clinical setting. There are many scenarios in which DL and FL can help reshape and enhance radiology practice as evidenced by a huge investment in imaging-based research and industry efforts. A few examples include:1.The Medical Imaging and Data Resource Center (MIDRC), funded through the National Institute of Biomedical Imaging and Bioengineering (NIBIB) under the National Institutes of Health (NIH) special emergency COVID-19 process, which is an open-access platform for COVID-19- related medical images and associated data [4RSNAMedical imaging data resource center (MIDRC).https://www.rsna.org/covid-19/covid-19-ricordDate accessed: June 24, 2022Google Scholar].2.The Cancer Imaging Archive (TCIA) is a service which de-identifies and hosts a large publicly available archive of medical images of cancer. TCIA is funded by the Cancer Imaging Program (CIP), a part of the United States National Cancer Institute (NCI), and is managed by the Frederick National Laboratory for Cancer Research (FNLCR) [5National Cancer InstituteCancer imaging archive.https://www.cancerimagingarchive.net/about-the-cancer-imaging-archive-tcia/Date accessed: June 24, 2022Google Scholar].3.Brain segmentation datasets and machine learning challenges such as The 2021 Brain Tumor Segmentation (BraTS) challenge jointly organized by the Radiological Society of North America (RSNA), the American Society of Neuroradiology (ASNR), and the Medical Image Computing and Computer Assisted Interventions (MICCAI) society [6Center for Biomedical Image Computing & AnalyticsRSNA-ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge 2021.http://braintumorsegmentation.org/Date accessed: June 24, 2022Google Scholar]. These dataset repositories have the potential to act as the springboard for current and future DL and FL initiatives. There is also a movement to convert EHR data into repositories in which Natural Language Processing (NLP) techniques can be utilized in FL instances to achieve better predictive models and patient outcome determinants (e.g. hospitalizations due to cardiac events, mortality and ICU stay time). There are numerous challenges and considerations that have to be considered before embarking on a FL endeavor. Many institutions are not equipped with either on premises or cloud servers that can manage this volume of data and the processing power required to develop and implement the model. FL systems should also be set up not to fail if one or more clients are compromised. There are potential risks that an adversary may reconstruct or reverse-engineer private data from the local model updates. The model could potentially be “tainted” or “hacked” between clients and therefore lead to models that could potentially cause harm in various clinical settings. Data heterogeneity can be both a blessing and a curse. It is often difficult to fully vet the fidelity and completeness of data in a decentralized environment thereby potentially diminishing the veracity or quality of the data. Also, heterogeneous data can hinder model convergence or generalizability. Bias can be introduced depending on how the neural network interfaces with various clients in addition to the neural network applying “preference” to a certain client’s data based on data size or the timing in which the data interfaces with the model. The lack of standardized data and labels has plagued healthcare practice and research which are amplified in the setting of machine learning and FL. For FL to fulfill its full potential, a multidisciplinary team of data scientists, information technologists, and clinicians, just to name a few, need to collaborate to ensure the proper execution of deep learning methodologies in the FL setting. The power of FL and ensembling of a model has been shown to be very important when considering heterogeneous data and applying the model in disparate settings. The radiology, and overall healthcare, communities will undoubtedly benefit from this type of model development and validation. While FL is not a panacea for all model development, if used judiciously and effectively, it is a very powerful tool that will continue to bear fruit by enhancing diagnostics, prediction, and delivery of care.