*HomeU.S. Government Accountability Office*
Menu

      close


          
          
                          	
				
				
	
	
					Reports & Testimonies
									
																				
				
				
	
	
								Reports & Testimonies
													
																												
																						
							
																Recommendations Database
							
																												
																						
							
																Month in Review
							
																												
																						
							
																Order Copies
							
																												
																						
							
																Restricted Reports
							
																												
																						
							
																View Topics
							
																												
																						
							
																View Agencies
							
																												
																						
							
																Bid Protests & Appropriations Law
									
																				
				
				
	
	
								Bid Protests & Appropriations Law
													
																												
																						
							
																Bid Protests
							
																												
																						
							
																Appropriations Law
							
																												
																						
							
																Federal Vacancies Reform Act
							
																												
																						
							
																Other Legal
							
																												
																						
							
																About
									
																				
				
				
	
	
								About
													
																												
																						
							
																What GAO Does
							
																												
																						
							
																U.S. Comptroller General
							
																												
																						
							
																Careers
							
																												
																						
							
																Contact UsFor Congress
                                
                          
        
                Press Center
                                
                          
        
                Careers
                                
                          
        
                BlogSearch Breadcrumb
        Home
                            Reports & Testimonies
                            Artificial Intelligence In Natural Hazard Modeling: Severe Storms, Hurricanes, Floods, and Wildfires
    Artificial Intelligence in Natural Hazard Modeling:
                          Severe Storms, Hurricanes, Floods, and Wildfires **GAO-24-106213**
        Published: Dec 14, 2023. Publicly Released: Dec 14, 2023.Jump To:
    Jump ToElder Abuse
          Services for Older Adults
          State and Local-Level Drivers and Trends
          State and Local Fiscal Projections
          Federal Borrowing
          Managing the Debt
          Summary of Funding Provisions
          Auditing the Government's Books
          Unpacking the Financial Report
          Highlights
          Recommendations
          Background
          About This Work
          About This Work
          Trends
          About the Center
          Why It's High Risk
          What Remains To Be Done
          Key Reports
          Newest COVID-Related Reports
          AI Use Cases
          Flood Insurance
          Past Pandemic-Related Reports
          CARES Act Oversight Reports
          All COVID-19 Reports
          Report Suspected Fraud
          Subjects
          How to Get Notified
          Past Reports
          Public Health Threats
          Explore the FY 2022 and 2023 Funds
          Lessons Learned
          Agency Reports
          Recent Reports
          Recommendations
          Supplemental Material
          Full Report
          Additional Data
          View Decision
          Downloads
          Previously Identified Actions
          Related Pages
          Associated Agencies
          Tough Choices and Opportunities Ahead
          Americas Fiscal Future Key Areas
          Retirement Security Key Areas
          Current List
          About The High Risk List
          Area Ratings
          Previous High Risk Products
          Search Federal Vacancies
          Past Federal Vacancies
          Violation Letters
          About the Yellow Book
          Advisory Council
          Related Resources
          Comment Letters
          Related Publications
          Contacts and Resources
          Major Issues Facing the Nation
          Meeting Strategic Challenges
          Additional Resources
          Resources
          GAO Contacts
          Mission Teams
          Operations and Staff Offices
          Organization Chart
          Why a Career at GAO?
          Am I Qualified?
          Company Culture
          Reasonable Accommodations
          Video Gallery
          Paid Internships
          Student Volunteers
          Professional Development Program
          Advancement
          Executive Candidate Program
          Summer Associates
          Externships
          Benefits
          Contacts
                    Fast Facts
              Severe storms, hurricanes, floods, and wildfires kill hundreds of people and cause billions of dollars in damage in the U.S. each year. Forecasting models can help predict these hazards. Machine learning—a type of artificial intelligence—can help improve model accuracy, increase warning times, and reduce uncertainty.

Some machine learning forecasting models are in use, but there are challenges to developing and adopting them more widely. For example, there may not be enough data from some rural areas to train machine learning models.

Our policy options—such as improving how data is collected, shared, and used—may help address the challenges.

Four side-by-side images of natural disasters in a circle with a clear circular globe overlay in the center of them.
Skip to Highlights
Highlights
  What GAO Found

GAO found that machine learning, a type of artificial intelligence (AI) that uses algorithms to identify patterns in information, is being applied to forecasting models for natural hazards such as severe storms, hurricanes, floods, and wildfires, which can lead to natural disasters. A few machine learning models are used operationally—in routine forecasting—such as one that may improve the warning time for severe storms. Some uses of machine learning are considered close to operational, while others require years of development and testing.

GAO identified potential benefits of applying machine learning to this field, including:

Reducing the time required to make forecasts by replacing components of models that are slow and that increase the cost of modeling.
	Increasing model accuracy by more fully exploiting available data, using other data that traditional models cannot, and creating synthetic data to fill gaps.
	Reducing the uncertainty of model output by improving ensemble modeling—the processes of generating combined predictions from numerous models—and making better use of historical data.**Forecasting natural disasters using machine learning**



GAO also identified challenges to the use of machine learning. For example:

Data limitations hamper the training of machine learning models and can reduce accuracy in some regions, such as rural areas where weather observations are sparse.
	A lack of trust and understanding of the algorithms as well as concerns about bias can make forecasters and other users hesitant to use machine learning models.
	Limited coordination and collaboration create challenges for fully developing some machine learning models. For example, some forecasters told us they lack opportunities to interact with researchers and convey their needs.
	Workforce and resource gaps also create challenges. For example, the upfront costs to develop and run machine learning models are high, and some companies working on these models do not fully understand the data and phenomena they are modeling, according to academic researchers.GAO identified five policy options that could help address these challenges. These options are intended to inform policymakers, including Congress, federal and state agencies, academic and research institutions, and industry of potential policy implementations. The status quo option illustrates a scenario in which government policymakers take no additional actions beyond current ongoing efforts.

**Policy Options to Help Address Challenges to the Use of Machine Learning in Natural Hazard Modeling**

Policy Option
			Opportunities
			Considerations**Facilitate improved data collection, sharing, and use (report p. 37).**
			Government policymakers could expand use of existing observational data and infrastructure to close gaps, expand access to certain data, and (in conjunction with other policymakers) establish guidelines for making data AI-ready.
			Efforts to address gaps in data sets can improve machine learning model performance.
				Expanded access to existing data would improve the ability of researchers and groups to develop and test machine learning technologies.
				Adopting standards for AI-ready data could reduce resources needed to curate data and facilitate more efficient modeling.
			Expanding observational infrastructure can be expensive and could divert limited resources from other efforts.
				Agencies need to weigh the benefits of greater data sharing against any increase in risks related to data security and privacy.
				Strict data standards may slow research and innovation if they burden or constrain machine learning researchers.**Expand education and training (report p. 38).**
			Government policymakers could update education requirements to include machine learning-related coursework and expand learning and support centers, while academic policymakers could adjust physical science curricula to include more machine learning coursework.
			Updating education requirements would better prepare students to use machine learning in government.
				More robust education can better prepare both researchers and end users in fields like meteorology and climatology to develop and use machine learning.
			Education and training reforms may need to be repeatedly adjusted, as technological change in this space can be rapid and unpredictable.
				Establishing and expanding professional development and training opportunities throughout government may require substantial investment.**Address hiring and retention barriers and certain resource shortfalls (report p. 39).**
			Government policymakers could address pay scale limitations for positions that include machine learning expertise and work with private sector policymakers to expand the use of public-private partnerships (PPP).
			Providing workforce incentives to government employees for machine learning development could allow agencies to bolster workforce capacities.
				Expanding PPPs might help agencies overcome computational resource shortfalls and help industry draw on government expertise.
			Increasing salary limits for some employees would require agency budget increases or cuts to other budget items.
				Expanding the use of PPPs could magnify resource disparities between government and private industry.
				PPPs that entail hosting government data on collaborator systems may pose security risks that would need to be considered and addressed.**Take steps to mitigate bias and foster trust in data and machine learning models (report p. 40).**
			Policymakers could establish efforts to better understand and mitigate various forms of bias, support inclusion of diverse stakeholders for machine learning models, and develop guidelines or best practices for reporting methodological choices.
			Sustained efforts to address bias in data sets can reduce the likelihood of models negatively affecting certain communities.
				Acquiring diverse stakeholder perspectives throughout machine learning models’ life cycles can help reduce certain types of bias in data and models.
				Fostering machine learning model transparency could improve end-user and decision-maker trust.
			Embedding efforts to address bias throughout the model life cycle may increase model costs and slow model development.**Maintain status quo efforts (report p. 41).**
			Government policymakers could maintain existing policy efforts and organizational structures, along with existing strategic plans and agency commitments.
			Some agency efforts are underway to address the challenges described.
			The extent to which agencies will meet their commitments under status quo efforts is unclear. Status quo efforts may not address challenges specific to the use of AI in natural hazard modeling.Source: GAO. | GAO-24-106213

Why GAO Did This Study

Natural disasters cause on average hundreds of deaths and billions of dollars in damage in the U.S. each year. Forecasting natural disasters relies on computer modeling and is important for preparedness and response, which can in turn save lives and protect property. AI is a powerful tool that can automate processes, rapidly analyze massive data sets, enable modelers to gain new insights, and boost efficiency.

This report on the use of machine learning in natural hazard modeling discusses (1) the emerging and current use of machine learning for modeling severe storms, hurricanes, floods, and wildfires, and the potential benefits of this use; (2) challenges surrounding the use of machine learning; and (3) policy options to address challenges or enhance benefits of the use of machine learning.

GAO reviewed the use of machine learning to model severe storms, hurricanes, floods, and wildfires across development and operational stages; interviewed a range of stakeholder groups, including government, industry, academia, and professional organizations; convened a meeting of experts in conjunction with the National Academies; and reviewed key reports and scientific literature. GAO is identifying policy options in this report.

For more information, contact Brian Bothwell at (202) 512-6888 or bothwellb@gao.gov.

Full Report
    Highlights Page (2 pages)
              Full Report (61 pages)
              Accessible PDF (61 pages)
GAO Contacts
    
      Brian Bothwell
            
            Director
      
            bothwellb@gao.gov
      
            (202) 512-6888
Office of Public Affairs
    
      Chuck Young
            
            Managing Director
      
            youngc1@gao.gov
      
            (202) 512-4800
Topics
    
      Homeland SecurityArtificial intelligenceFloodsHurricanesNatural disastersNatural hazardsPublic and private partnershipsResearch and developmentStormsWeather forecastingWildfires
                                                      MultimediaPodcasts
    
      Artificial Intelligence in Severe Weather and Wildfire Modeling
    Thursday, December 14, 2023
      
            Transcript
 Receive GAO Updates
  Stay informed as we add new reports & testimonies.

    GAO Logo
      U.S. Government Accountability Office


                                   Column 1
                            Press Center
                              Contact Us
                              Inspector General
                              Restricted Reports
                              Column 2
                            Copyright & Terms of Use
                              Privacy Policy
                              Accessibility
                              Sitemap
                              Column 3
                            FOIA Requests
                              Scam Alerts
                              No FEAR Act Data
                              Health Care Advisory Committees