*HomeU.S. Government Accountability Office*
Menu

      close


          
          
                          	
				
				
	
	
					Reports & Testimonies
									
																				
				
				
	
	
								Reports & Testimonies
													
																												
																						
							
																Recommendations Database
							
																												
																						
							
																Month in Review
							
																												
																						
							
																Order Copies
							
																												
																						
							
																Restricted Reports
							
																												
																						
							
																View Topics
							
																												
																						
							
																View Agencies
							
																												
																						
							
																Bid Protests & Appropriations Law
									
																				
				
				
	
	
								Bid Protests & Appropriations Law
													
																												
																						
							
																Bid Protests
							
																												
																						
							
																Appropriations Law
							
																												
																						
							
																Federal Vacancies Reform Act
							
																												
																						
							
																Other Legal
							
																												
																						
							
																About
									
																				
				
				
	
	
								About
													
																												
																						
							
																What GAO Does
							
																												
																						
							
																U.S. Comptroller General
							
																												
																						
							
																Careers
							
																												
																						
							
																Contact UsFor Congress
                                
                          
        
                Press Center
                                
                          
        
                Careers
                                
                          
        
                BlogSearch Breadcrumb
        Home
                            Reports & Testimonies
                            Artificial Intelligence In Health Care: Benefits and Challenges of Machine Learning Technologies For Medical Diagnostics
    Artificial Intelligence in Health Care:
                          Benefits and Challenges of Machine Learning Technologies for Medical Diagnostics **GAO-22-104629**
        Published: Sep 29, 2022. Publicly Released: Sep 29, 2022.Jump To:
    Jump ToElder Abuse
          Services for Older Adults
          State and Local-Level Drivers and Trends
          State and Local Fiscal Projections
          Federal Borrowing
          Managing the Debt
          Summary of Funding Provisions
          Auditing the Government's Books
          Unpacking the Financial Report
          Highlights
          Recommendations
          Background
          About This Work
          About This Work
          Trends
          About the Center
          Why It's High Risk
          What Remains To Be Done
          Key Reports
          Newest COVID-Related Reports
          AI Use Cases
          Flood Insurance
          Past Pandemic-Related Reports
          CARES Act Oversight Reports
          All COVID-19 Reports
          Report Suspected Fraud
          Subjects
          How to Get Notified
          Past Reports
          Public Health Threats
          Explore the FY 2022 and 2023 Funds
          Lessons Learned
          Agency Reports
          Recent Reports
          Recommendations
          Supplemental Material
          Full Report
          Additional Data
          View Decision
          Downloads
          Previously Identified Actions
          Related Pages
          Associated Agencies
          Tough Choices and Opportunities Ahead
          Americas Fiscal Future Key Areas
          Retirement Security Key Areas
          Current List
          About The High Risk List
          Area Ratings
          Previous High Risk Products
          Search Federal Vacancies
          Past Federal Vacancies
          Violation Letters
          About the Yellow Book
          Advisory Council
          Related Resources
          Comment Letters
          Related Publications
          Contacts and Resources
          Major Issues Facing the Nation
          Meeting Strategic Challenges
          Additional Resources
          Resources
          GAO Contacts
          Mission Teams
          Operations and Staff Offices
          Organization Chart
          Why a Career at GAO?
          Am I Qualified?
          Company Culture
          Reasonable Accommodations
          Video Gallery
          Paid Internships
          Student Volunteers
          Professional Development Program
          Advancement
          Executive Candidate Program
          Summer Associates
          Externships
          Benefits
          Contacts
                    Fast Facts
              Each year, medical diagnosis errors affect the health of millions of Americans and cost billions of dollars. Machine learning technologies can help identify hidden or complex patterns in diagnostic data to detect diseases earlier and improve treatments.

We identified such technologies in use and development, including some that improve their own accuracy by learning from new data. But developing and adopting these technologies has challenges, such as the need to demonstrate real-world performance in diverse clinical settings.

Our policy options—like improving data access and collaboration—may help address the challenges.

An illustration of a doctor's office, diagnostic tools (like x-rays), and a doctor and patient.
Skip to Highlights
Highlights
  What GAO Found

Several machine learning (ML) technologies are available in the U.S. to assist with the diagnostic process. The resulting benefits include earlier detection of diseases; more consistent analysis of medical data; and increased access to care, particularly for underserved populations. GAO identified a variety of ML-based technologies for five selected diseases — certain cancers, diabetic retinopathy, Alzheimer’s disease, heart disease, and COVID-19 —with most technologies relying on data from imaging such as x-rays or magnetic resonance imaging (MRI). However, these ML technologies have generally not been widely adopted.

Academic, government, and private sector researchers are working to expand the capabilities of ML-based medical diagnostic technologies. In addition, GAO identified three broader emerging approaches—autonomous, adaptive, and consumer-oriented ML-diagnostics—that can be applied to diagnose a variety of diseases. These advances could enhance medical professionals’ capabilities and improve patient treatments but also have certain limitations. For example, adaptive technologies may improve accuracy by incorporating additional data to update themselves, but automatic incorporation of low-quality data may lead to inconsistent or poorer algorithmic performance.

**Spectrum of adaptive algorithms**



We identified several challenges affecting the development and adoption of ML in medical diagnostics:

Demonstrating real-world performance across diverse clinical settings and in rigorous studies.
	Meeting clinical needs, such as developing technologies that integrate into clinical workflows.
	Addressing regulatory gaps, such as providing clear guidance for the development of adaptive algorithms.These challenges affect various stakeholders including technology developers, medical providers, and patients, and may slow the development and adoption of these technologies.

GAO developed three policy options that could help address these challenges or enhance the benefits of ML diagnostic technologies. These policy options identify possible actions by policymakers, which include Congress, federal agencies, state and local governments, academic and research institutions, and industry. See below for a summary of the policy options and relevant opportunities and considerations.

**Policy Options to Help Address Challenges or Enhance Benefits of ML Diagnostic Technologies**

**Opportunities**
			**Considerations****Evaluation**(report
			page 28)

			Policymakers could create incentives, guidance, or policies to encourage or require the evaluation of ML diagnostic technologies across a range of deployment conditions and demographics representative of the intended use.

			*This policy option could help address the challenge of demonstrating real world performance.*
			Stakeholders could better understand the performance of these technologies across diverse conditions and help to identify biases, limitations, and opportunities for improvement.
				Could inform providers' adoption decisions, potentially leading to increased adoption by enhancing trust.
				Information from evaluations can help inform the decisions of policymakers, such as decisions about regulatory requirements.
			May be time-intensive, which could delay the movement of these technologies into the marketplace, potentially affecting patients and professionals who could benefit from these technologies.
				More rigorous evaluation will likely lead to extra costs, such as direct costs for funding the studies. Developers may not be incentivized to conduct these evaluations if it could show their products in a negative light, so policymakers could consider whether evaluations should be conducted or reviewed by independent parties, according to industry officials.**Data Access**(report
			page 29)

			Policymakers could develop or expand access to high-quality medical data to develop and test ML medical diagnostic technologies. Examples include standards for collecting and sharing data, creating data commons, or using incentives to encourage data sharing.

			*This policy option could help address the challenge of demonstrating real world performance.*
			Developing or expanding access to high-quality datasets could help facilitate training and testing ML technologies across diverse and representative conditions. This could improve the technologies' performance and generalizability, help developers understand their performance and areas for improvement, and help to build trust and adoption in these technologies.
				Expanding access could enable developers to save time in the development process, which could shorten the time it takes for these technologies to be available for adoption.
			Entities that own data may be reluctant to share them for a number of reasons. For example, these entities may consider their data valuable or proprietary. Some entities may also be concerned about the privacy of their patients and the intended use and security of their data.
				Data sharing mechanisms may be of limited use to researchers and developers depending on the quality and interoperability of these data, and curating and storing data could be expensive and may require public and private resources.**Collaboration**(report
			page 30)

			Policymakers could promote collaboration among developers, providers, and regulators in the development and adoption of ML diagnostic technologies. For example, policymakers could convene multidisciplinary experts together in the design and development of these technologies through workshops and conferences.

			*This policy option could help address the challenges of meeting medical needs and addressing regulatory gaps.*
			Collaboration between ML developers and providers could help ensure that the technologies address clinical needs. For example, collaboration between developers and medical professionals could help developers create ML technologies that integrate into medical professionals' workflows, and minimize time, effort, and disruption.
				Collaboration among developers and medical providers could help in the creation and access of ML ready data, according to NIH officials.
			As previously reported, providers may not have time to both collaborate with developers and treat patients; however, organizations can provide protected time for employees to engage in innovation activities such as collaboration.
				If developers only collaborate with providers in specific settings, their technologies may not be usable across a range of conditions and settings, such as across different patient types or technology systems.Source: GAO. | GAO-22-104629

Why GAO Did This Study

Diagnostic errors affect more than 12 million Americans each year, with aggregate costs likely in excess of $100 billion, according to a report by the Society to Improve Diagnosis in Medicine. ML, a subfield of artificial intelligence, has emerged as a powerful tool for solving complex problems in diverse domains, including medical diagnostics. However, challenges to the development and use of machine learning technologies in medical diagnostics raise technological, economic, and regulatory questions.

GAO was asked to conduct a technology assessment on the current and emerging uses of machine learning in medical diagnostics, as well as the challenges and policy implications of these technologies. This report discusses (1) currently available ML medical diagnostic technologies for five selected diseases, (2) emerging ML medical diagnostic technologies, (3) challenges affecting the development and adoption of ML technologies for medical diagnosis, and (4) policy options to help address these challenges.

GAO assessed available and emerging ML technologies; interviewed stakeholders from government, industry, and academia; convened a meeting of experts in collaboration with the National Academy of Medicine; and reviewed reports and scientific literature. GAO is identifying policy options in this report.

For more information, contact Karen L. Howard at (202) 512-6888 or howardk@gao.gov.

Full Report
    Highlights Page (3 pages)
              Full Report (103 pages)
              Accessible PDF (107 pages)
GAO Contacts
    
      Karen Howard
            
            Director
      
            howardk@gao.gov
      
            (202) 512-6888
Office of Public Affairs
    
      Chuck Young
            
            Managing Director
      
            youngc1@gao.gov
      
            (202) 512-4800
Topics
    
      Science and TechnologyArtificial intelligenceDisease detection or diagnosisDiseasesEmerging technologiesHealth careMedical technologyMedicineSoftwareTechnology assessmentPatient care
                                                      MultimediaIllustration showing how machine learning works in diagnosing patients --a scan or test is run, which is fed into a computer, which is then used to diagnose the problem, and then a patient is notified by a doctor
          Blog Post
              Machine Learning’s Potential to Improve Medical Diagnosis
            Thursday, November 10, 2022
      
            Diagnostic errors are some of the most common, catastrophic, and costly of medical errors, according...
          Graphic illustrating Artificial Intelligence--a persons hands are typing on a keyboard. Above that image are small icons for different applications. The center one says AI.
          Blog Post
              Artificial Intelligence’s Use and Rapid Growth Highlight Its Possibilities and Perils
            Wednesday, September 6, 2023
      
            The rise of artificial intelligence has created growing excitement and much debate about its...
 Receive GAO Updates
  Stay informed as we add new reports & testimonies.

    GAO Logo
      U.S. Government Accountability Office


                                   Column 1
                            Press Center
                              Contact Us
                              Inspector General
                              Restricted Reports
                              Column 2
                            Copyright & Terms of Use
                              Privacy Policy
                              Accessibility
                              Sitemap
                              Column 3
                            FOIA Requests
                              Scam Alerts
                              No FEAR Act Data
                              Health Care Advisory Committees