# Recent Reports

Reports News from the GAO

- Federal Research: Key Practices for Scientific Program Managers What GAO FoundTo oversee basic and applied research at federal agencies, scientific program managers are typically responsible for managing award selection, monitoring ongoing awards, and coordinating with awardees and the research community. Program managers GAO interviewed from selected agencies identified key practices they used to carry out these responsibilities. They said these practices helped advance their agencies' goals, further science, and avoid unnecessary duplication. Further, the practices may help program managers, agencies, and others assess and improve management of basic and applied research.As outlined in the figure below, the key practices fall into three areas.Strengthening and building expertise—Practices that help program managers maintain scientific and management expertise.	Developing connections—Practices that help program managers enhance collaboration with the scientific community and the public, as well as within their own agencies and in other agencies.	Building a strong research portfolio—Practices that help program managers advance their agencies' research mission and scientific knowledge in general, while ensuring their own accountability and that of federally funded researchers.Key Practices for Federal Program Managers to Select, Coordinate, and Monitor Scientific ResearchWhy GAO Did This StudyThe federal government invests in basic and applied scientific research to drive innovation, promote economic competitiveness, and enhance national security. The National Science Foundation estimates that 32 federal agencies funded over $85 billion in basic and applied research in fiscal year 2021.Scientific program managers at federal agencies that sponsor basic and applied research play a critical role in guiding and shaping the research funded by their agencies.In this report, GAO describes key practices that federal program managers use to manage their research.GAO held 14 group discussions with 79 program managers from seven selected agencies that funded over 90 percent of basic and applied research obligations in fiscal year 2021. GAO asked the program managers to describe the practices they use when managing projects in their basic and applied research portfolios. GAO conducted qualitative analysis to identify common themes and distilled them into 10 key practices. These key practices were cited by multiple program managers or agencies and could be used by program managers across the federal government when managing projects in their basic and applied research portfolios.GAO also conducted a literature review to help corroborate the key practices. GAO sought and incorporated feedback on these practices from the selected agencies as well as experts identified by the National Academies of Sciences, Engineering, and Medicine.For more information, contact Candice N. Wright at (202) 512-6888 or wrightc@gao.gov.
- Science & Tech Spotlight: At-Home Tools to Diagnose Alzheimer's, Parkinson's, and Related Diseases Why This MattersNeurodegenerative diseases such as Alzheimer’s and Parkinson’s can make day-to-day tasks challenging because they may affect cognitive, motor, and social function. Neurodegenerative diseases affect about 7 million people in the U.S. Researchers expect cases to increase as the population ages. Earlier detection of these diseases may help health care providers treat patients before the onset of severe symptoms and slow disease progression.Key TakeawaysAt-home tools may facilitate earlier detection of neurodegenerative diseases.	These tools can be a cost-effective way to collect information that is difficult to gather in a clinical setting.	Health care providers can use information from at-home tools to help diagnose neurodegenerative disease, but such tools are not a replacement for clinical care.The TechnologyWhat is it?&nbsp;At-home tools collect and analyze patient data on certain indicators that relate to cognitive ability—memory, reasoning, and language—and motor function. This includes walking, writing, stability, and the presence of tremors. Tools such as digital cognitive assessments and wearable sensors have the potential to help health care providers detect and diagnose neurodegenerative diseases earlier and more accurately. The tools help assess patients in an environment more reflective of daily life, detect indicators that are challenging to observe in a clinical setting, and collect data on patients’ health over time. Traditional diagnostic approaches—which occur in a clinical setting such as a physician’s office—can be more invasive, less convenient, and may not identify these symptoms until the disease is more advanced.Figure 1: Patient and health care provider use of at-home toolsHow does it work?&nbsp;Digital cognitive assessments use web- or application-based technologies to gather data on a patient’s cognitive abilities. For example, one activity asks patients to memorize objects and their position in a pictured room.Wearable sensors worn on or near the body measure a range of indicators. Sensors on the hands, wrists, or ankles can collect data on the speed and direction of movements—such as hand tremors, sway, or postural adjustments—that may be indicative of neurodegenerative disease. Smartphones equipped with movement sensors can collect similar data when worn near the body. Researchers found that smartphone sensors can identify whether a person exhibits symptoms associated with Parkinson’s disease based on the way they move.How mature is it?&nbsp;Multiple manufacturers of digital cognitive assessments—which could be administered at-home—have registered with the Food and Drug Administration. However, such at-home assessments are not widely commercially available, and more research is needed on their accuracy.Wearable sensors vary in their maturity and use. Specifically, sensors worn on the body are an established tool to assess mobility. Researchers call for further studies to investigate the accuracy of smartphone-based sensors prior to applying them more broadly in diagnosis.OpportunitiesProvide data to support early diagnosis.&nbsp;Providers can use data from at-home tools to inform earlier or more appropriate interventions to slow cognitive decline and improve symptom management. As technologies mature, patients could also potentially use insights from at-home tools to decide whether they need to consult a provider.	Reduce time and cost.&nbsp;At-home tools can be a practical and cost-effective way to reduce the number of clinic visits in the early stages of a disease when symptoms are still ambiguous. Because the tools can be used remotely, patients can reduce travel time and cost, especially if they live far from a clinic or have limited mobility.	Provide easier access to diagnosis.&nbsp;At-home tools could help improve access to diagnostic information if they are self-administered, which can be particularly helpful in instances where a patient has health care access issues.ChallengesLimited testing.&nbsp;Some tools were designed for use at the clinic and researchers have conducted limited testing for at-home use. For example, quality control testing of at-home digital cognitive assessments lags behind those delivered in a clinical setting, but testing is critical to understand a tool’s reliability.	Digital divide.&nbsp;At-home tools may be less useful and less widely adopted if patients do not understand how they work or do not have devices to access them. This challenge could contribute to disparities in diagnosis.	Data privacy and security.&nbsp;Data could be sold, shared without consent, or accessed via data breaches, although data covered by the Health Insurance Portability and Accountability Act of 1996 are provided some additional safeguards.Policy Context and QuestionsWhat are the possible policy implications for existing federal programs such as Medicare, Medicaid, and Social Security Insurance?	What strategies, if any, might help enhance the benefits and use of at-home tools?	What oversight or resources could help ensure the tools do not exacerbate health disparities and the digital divide?	What safeguards could protect data privacy and security?Selected GAO WorkScience &amp; Tech Spotlight: Wearables,&nbsp;GAO-24-107303Technology Assessment: Artificial Intelligence in Health Care,&nbsp;GAO-22-104629Selected ReferencesOhman et al., “Current Advances in Digital Cognitive Assessment for Preclinical Alzheimer’s Disease,”Alzheimer’s and Dementia Diagnosis,Assessment and Disease Monitoring,&nbsp;(2021) doi: 10.1002/dad2.12217.Adams et al., “Digital Technology in Movement Disorders: Updates, Applications, and Challenges,”Current Neurology and Neuroscience Reports,vol. 21, no. 4(2021).For more information, contact Karen L. Howard, PhD at (202) 512-6888 or howardk@gao.gov.
- Clinical Research: FDA Should Evaluate Its Efforts to Recruit and Retain Its Inspection Workforce What GAO FoundThe Food and Drug Administration (FDA) conducts inspections to, among other things, help ensure the quality and integrity of clinical research used to support drugs seeking marketing approval in the U.S. FDA's clinical research inspections peaked in fiscal year 2017 but have since declined. FDA officials attributed this decrease to the COVID-19 pandemic and not having enough investigators.Number of FDA Clinical Research Inspections Related to Drugs, Fiscal Year 2014 through March 1, 2023From fiscal years 2012 through 2020, FDA classified 3 percent of clinical research inspections as having serious deficiencies that would warrant regulatory actions. Investigators GAO spoke with were frustrated that problems they identified (e.g., failure to follow research protocols) did not result in more serious classifications. FDA is limited in its ability to cite serious deficiencies for a common type of study supporting generic drugs. Specifically, the regulations for these studies do not include certain requirements for basic study conduct, such as record retention and following study protocols. FDA has started the process of revising these regulations. Having effective requirements will be important to help ensure high-quality research.FDA has faced challenges recruiting and retaining investigators, resulting in fewer inspections and a less experienced workforce. For example, FDA was unable to complete about 30 percent of one type of common inspection within the requested time frames from fiscal year 2018 through July 2023, according to agency information. FDA officials and the investigators GAO spoke with identified low compensation and high amounts of travel as contributing to these challenges. FDA has taken steps to increase recruitment and retain investigators, such as increased compensation and student loan repayment. The agency recently made progress recruiting new investigators, but attrition has been a persistent problem and it can take new investigators up to a year to independently conduct inspections. Although FDA made progress, the agency has not formally evaluated its efforts to determine their effectiveness. Such an evaluation could help FDA determine whether it is using the most appropriate tools to maintain its workforce. GAO has cited workforce as a concern across multiple FDA programs and sustained attention in this area will be critical.Why GAO Did This StudyClinical research—clinical trials and other studies involving human subjects—for drugs seeking FDA approval can occur in the U.S. or overseas. During inspections, FDA goes on-site, such as to hospitals or other health care settings, to examine research protocols and records as well as the entity and facility involved in the research. Challenges in other FDA inspection programs contributed to GAO placing FDA medical product oversight on its High-Risk List in 2009.GAO was asked to review FDA's inspections of clinical research. This report, among other objectives, describes inspections FDA conducted from fiscal years 2012 through 2023; describes the frequency with which FDA identified serious deficiencies during inspections; and examines FDA's efforts to maintain its investigator workforce. For this work, GAO examined FDA data and documents and interviewed FDA officials. GAO also interviewed 15 out of about 100 investigators, selected to represent diversity among the different investigator positions and tenure with FDA.
- Surface Transportation: DOT Considers Multiple Factors when Choosing the Volpe Center to Conduct Research What GAO FoundThe U.S. Department of Transportation's (DOT) operating administrations enter into agreements with DOT's John A. Volpe National Transportation Systems Center (Volpe Center) for a variety of services, including research. DOT officials told GAO that they consider multiple factors when choosing the Volpe Center to conduct research or perform other services. These factors include:Expertise. DOT officials told GAO that they consider the Volpe Center's expertise and institutional knowledge on a topic. For example, the Federal Highway Administration has leveraged the Volpe Center's technical expertise in noise reduction and air quality to create tools for state and regional officials to demonstrate reduced congestion and improved air quality.	Nature of the work. DOT officials told GAO that, because the Volpe Center is a part of the agency, certain issues (e.g., inherently governmental functions) are best addressed by Volpe Center staff. For example, Volpe Center staff have long served as the primary technical resource informing the National Highway Traffic Safety Administration's fuel economy standards rulemaking.	Response time. DOT officials told GAO that the Volpe Center's ability to rapidly respond to agency needs is also a consideration. Volpe Center officials cited the center's recent work to help form the new Joint Office of Energy and Transportation, which involved performing technical reviews of grant applications, as an example of its ability to respond rapidly.The proportion of research funding committed to the Volpe Center compared to other entities varied across five selected DOT operating administrations in fiscal year 2022 (see figure). Combined, these operating administrations committed a total of $50.5 million—13 percent of their total fiscal year 2022 research funding—to the Volpe Center. The remaining 87 percent went to other entities such as universities, businesses, and non-profit organizations.Research Funding Committed to the Volpe Center and Other Entities by Five Department of Transportation Operating Administrations, Fiscal Year 2022DOT operating administrations manage their respective research portfolios. According to DOT officials, each operating administration's distribution of research funding is driven by its mission and role, the scope of individual projects, and the factors described above.Why GAO Did This StudyDOT's research activities are critical to its mission of making the nation's transportation system safer and more efficient. In 2020, GAO found that this research may be conducted by the agency's operating administrations, nonfederal research entities, or DOT's Volpe Center. DOT established what is now the Volpe Center in 1970. Its mission is to improve the U.S. transportation system by anticipating emerging issues and advancing technical, operational, and institutional innovations for the public good.The Infrastructure Investment and Jobs Act includes a provision for GAO to review the surface transportation activities at the Volpe Center. This report describes (1) factors that selected DOT operating administrations consider when choosing the Volpe Center for research or other services, and (2) how much research funding selected DOT operating administrations committed to the Volpe Center compared to other entities in fiscal year 2022, among other objectives.GAO selected five DOT operating administrations for analysis that have a surface transportation-focused mission, are responsible for duties and initiatives intended to improve the safety of the traveling public, and entered into agreements with the Volpe Center in fiscal year 2022. GAO also reviewed relevant laws and DOT policies; analyzed DOT data; and interviewed officials from DOT, the Volpe Center, and the five selected operating administrations.For more information, contact Elizabeth Repko at (202) 512-2834 or repkoe@gao.gov.
- Science & Tech Spotlight: Combating Deepfakes Why This MattersMalicious use of deepfakes could erode trust in elections, spread disinformation, undermine national security, and empower harassers.Key TakeawaysCurrent deepfake detection technologies have limited effectiveness in real-world scenarios.	Watermarking and other authentication technologies may slow the spread of disinformation but present challenges.	Identifying deepfakes is not by itself sufficient to prevent abuses. It may not stop the spread of disinformation, even after the media is identified as a deepfake.The TechnologyWhat is it?Deepfakes are videos, audio, or images that have been manipulated using artificial intelligence (AI), often to create, replace, or alter faces or synthesize speech. They can seem authentic to the human eye and ear. They have been maliciously used, for example, to try to influence elections and to create non-consensual pornography. To combat such abuses, technologies can be used to detect deepfakes or enable authentication of genuine media.Detection technologies&nbsp;aim to identify fake media without needing to compare it to the original, unaltered media. These technologies typically use a form of AI known as machine learning. The models are trained on data from known real and fake media. Methods include looking for (1) facial or vocal inconsistencies, (2) evidence of the deepfake generation process, or (3) color abnormalities.Authentication technologies&nbsp;are designed to be embeddedduring the creation of a piece of media. These technologies aim to either prove authenticity or prove that a specific original piece of media has been altered. They include:Digital watermarks&nbsp;can be embedded in a piece of media, which can help detect subsequent deepfakes. One form of watermarking adds pixel or audio patterns that are detectable by a computer but are imperceptible to humans.The patterns disappear in any areas that are modified, enabling the owner to prove that the media is an altered version of the original. Another form of watermarking adds features that cause any deepfake made using the media to look or sound unrealistic.	Metadata—which describe the characteristics of data in a piece of media—can be embedded in a way that is cryptographically secure. Missing or incomplete metadata may indicate that a piece of media has been altered.	Blockchain.&nbsp;Uploading media and metadata to a public blockchain creates a relatively secure version that cannot be altered without the change being obvious to other users. Anyone could then compare a file and its metadata to the blockchain version to prove or disprove authenticity.Figure 1. Examples of Deepfake Detection and AuthenticationHow mature is it?Detection technologies.&nbsp;According to recent studies, existing detection methods and models may not accurately identify deepfakes in real-world scenarios. For example, accuracy may be reduced if lighting conditions, facial expressions, or video or audio quality are different from the data used to train the detection model, or if the deepfake was created using a different method than that used in the training data. Further, future advances in deepfake generation are expected to eliminate hallmarks of current deepfakes, such as abnormal eye blinking.Authentication technologies.&nbsp;These technologies are not new, but their use in combating deepfakes is an emerging area. Several companies offer authentication services, including using digital watermarks, metadata, and blockchain technologies. Some claim to let website visitors authenticate media found on the internet, provided the original is in the company’s database. Prominent social media companies are also beginning to label AI-generated content.OpportunitiesCombined defenses.&nbsp;Using multiple detection and authentication methods may help to identify deepfakes.	Updated training datasets.&nbsp;Including diverse and recent media in training data could help detection models keep up with the latest deepfake generation techniques.	Competitions.&nbsp;Deepfake detection competitions could encourage the development of more accurate detection tools and models. One 2019 competition included over 2,000 participants and generated over 35,000 models.ChallengesDisinformation and public trust.&nbsp;Disinformation can spread from the moment a deepfake is viewed, even if it is identified as fraudulent. Further, trust in real media may be undermined by false claims that real media is a deepfake or if people do not trust a detection model’s results.	Adaptation to detection.&nbsp;Techniques and models used to identify deepfakes tend to lead developers to create more sophisticated deepfake generation techniques.Policy Context and QuestionsAre current laws and regulations adequate to address evolving concerns about the malicious use of deepfakes? How do they address data security, privacy concerns, and First Amendment considerations, such as a deepfake creator’s freedom of speech and expression?	What entities (e.g., government, nonprofit, private company) should make decisions about identifying and blocking deepfakes, or about when and how to sanction those who produce or disseminate them?	How can organizations across society coordinate on the development and improvement of deepfake detection and authentication technologies? What standards could be used or developed to evaluate these technologies?Selected GAO WorkScience &amp; Tech Spotlight: Deepfakes,&nbsp;GAO-20-379SPTechnology Assessment: Blockchain,&nbsp;GAO-22-104625Selected ReferencesGourav Gupta, Kiran Raja, Manish Gupta, Tony Jan, Scott Thompson Whiteside, and Mukesh Prasad, “A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods,”Electronics, vol. 13, no. 95 (2024) https://doi.org/10.3390/electronics13010095.National Security Agency, Federal Bureau of Investigation, and Cybersecurity and Infrastructure Security Agency,Contextualizing Deepfake Threats to Organizations, Sept. 2023.For more information, contact Brian Bothwell at (202) 512-6888 or bothwellb@gao.gov.
- Science & Tech Spotlight: Wearable Technologies in the Workplace Why This MattersIn 2022, the warehousing, manufacturing, and construction industries experienced over 700,000 nonfatal injuries and over 2,000 fatal accidents. Meanwhile, consumer demand on these industries grows, creating pressure for increased productivity. To enhance and monitor worker safety and productivity, companies have begun deploying wearable technologies, from ergonomic sensors to exoskeletons.Key TakeawaysRecent innovations in sensor and networking technologies have increased the feasibility of and interest in the use of wearables in the workplace.	Companies have already deployed some wearables, but there are limited published data on the efficacy of these technologies to increase safety in the workplace.	Concerns about data privacy, cost, and ease of use may hinder widespread workplace adoption of wearables.The TechnologyWhat is it?&nbsp;Wearable technologies, or wearables, are devices worn on the body and can vary in size, shape, and function. Some employers have an increasing interest in using wearables to improve worker safety and productivity. Industrial uses fall into four general categories:&nbsp;(1) supporting devices&nbsp;physically assist workers with tasks like lifting (e.g., exoskeletons and powered gloves);&nbsp;(2) monitoring&nbsp;devices&nbsp;alert workers to specific changes in vital signs or the workplace environment (e.g., smart helmets);&nbsp;(3) training&nbsp;devices&nbsp;provide feedback on movements (e.g., ergonomic sensors) or help improve worker performance (e.g., augmented reality (AR) glasses); and&nbsp;(4) tracking devices&nbsp;observe the location of employees on a worksite (e.g., GPS trackers). See figure 1 for examples.Figure 1. Illustration of some wearable technologies of interest to modern industrial workplaces.How does it work?&nbsp;The way wearables work depends on the type of technology.Supporting devices&nbsp;like&nbsp;exoskeletons&nbsp;or&nbsp;powered gloves&nbsp;provide physicalsupport to the user’s shoulders, hands, or back during repetitive overhead work, gripping, or lifting.Most&nbsp;monitoring, training, and tracking&nbsp;technologies take advantage of innovations in networking by connecting many types of sensors to collect, exchange, and analyze data—sometimes referred to as the industrial internet of things (IIOT).&nbsp;Smart helmets, for example,incorporate physiological and environmental sensors and GPS trackers into protective headgear. These sensors can alert employees and workplace medical teams to accidents, such as falls, and to potential hazards, like heat and humidity. Similarly,&nbsp;ergonomic sensors, worn on the hip, back, or arm, can alert a user when they perform potentially unsafe movements (e.g., lifting with improper form). Through real-time alerts and movement data analytics, these sensors may help train workers and reduce injuries.How mature is it?&nbsp;Advancements in sensors, data analytics, and networking technologies have increased the feasibility of wearables, although their maturity varies. Several companies use exoskeletons or ergonomic sensors to some extent. Other wearables, such as smart helmets, have had pilot studies but still need further testing. Battery life and accuracy of underlying location tracking technologies pose challenges for scale-up.Recent studies have drawn limited conclusions in assessing the net effect of wearables on employee health and productivity. For example, one peer reviewed study showed that ergonomic sensors may reduce time in risky postures, but the study had limitations including a small sample size and short duration. A different review of wearables points out that the pressure of being constantly monitored can lead to increased stress in some workers and, consequently, increased probability of injury.OpportunitiesImprove employee safety.&nbsp;Wearables could reduce the risk of injuries from strenuous work or worker-equipment collisions and may improve response time to emergencies.	Increase employee productivity.&nbsp;Some companies have used AR glasses to aid repair technicians or to reduce error rates when item picking to fill orders.ChallengesPrivacy. Monitoring devices can store data on employee physiology and movements, which may create privacy concerns. For example, employees surveyed as a part of a wearable pilot test cited concerns about being tracked.;	Data Security.&nbsp;Data stored on wearables may be vulnerable to hackers because updating software can be difficult and many devices lack strong encryptions.	Cost.&nbsp;Companies can face high up-front costs to acquire and deploy wearables.	Ease of use.&nbsp;Employees may find some technologies cumbersome, complex, or uncomfortable to use. For example, one study cited employee concerns about the additional weight of the wearable.Policy Context and QuestionsThe National Institute for Occupational Safety and Health (NIOSH) conducts and coordinates research on wearables. Officials from the Occupational Safety and Health Administration (OSHA) told us that the agency oversees workplace safety but does not have any specific standards related to wearables.What types of additional studies would determine whether wearables achieve benefits such as improving employee safety?	What is the federal role, if any, in overseeing these technologies and their use in the workplace?	What safeguards or standards, if any, could help ensure that the development and adoption of wearables achieves positive outcomes and responds to employee concerns?Selected GAO WorkWorkplace Safety and Health: Actions Needed to Improve Reporting of Summary Injury and Illness Data,&nbsp;GAO-21-122.Internet of Things: Status and implications of an increasingly connected world,&nbsp;GAO-17-75.Selected References"Wearable Technologies" NIOSH Science Blog,&nbsp;https://blogs.cdc.gov/niosh-science-blog/category/wearable-technologies/.&nbsp;Accessed January 29, 2024.Ekaterina Svertoka, et al., “Wearables for Industrial Work Safety: A Survey,”&nbsp;Sensors,vol. 21 (2021) doi:10.3390/s21113844.For more information, contact Karen L. Howard, PhD at (202) 512-6888 or howardk@gao.gov.
- Small Business Innovation Research: Better Data and Clarity on Eligibility of Venture Capital-Owned Businesses Are Needed What GAO FoundFor decades, the Small Business Innovation Research (SBIR) program has provided federal funding to small businesses for technology development and commercialization. In fiscal year (FY) 2022, the 11 participating federal agencies made over 5,000 SBIR awards to small businesses valued at nearly $4 billion. In 2011, Congress amended the Small Business Act to give participating agencies the authority to make SBIR awards to certain small businesses that are majority owned by multiple venture capital operating companies (VCOC), hedge funds, or private equity firms (i.e., "qualified small businesses"). Agencies that use this authority must first submit a written determination to the Small Business Administration (SBA), which oversees the SBIR program, and certain congressional committees. Four agencies have used this authority to include qualified small businesses in their SBIR programs.Only two of those agencies—the Departments of Defense (DOD) and Health and Human Services—made SBIR awards to qualified small businesses in FY 2021 through 2023. Despite this, the number of awards to such businesses and the dollars awarded to qualified small businesses appear to have increased substantially compared to the prior 6 years. The increase is largely due to Air Force's use of the authority beginning in FY 2021. However, the full extent of the increase is unknown; as a result of GAO's review, DOD uncovered reliability issues with the data used to identify its SBIR awards to qualified small businesses. DOD is taking steps to address data reliability, but these steps did not include processes to monitor the effectiveness of its efforts. Without quality data, DOD may report inaccurate information to SBA and may have difficulty meeting requirements to track its use of the authority to make awards to qualified small businesses.Most agencies in the SBIR program have not used the authority to make SBIR awards to qualified small businesses. However, GAO found that some of these agencies may be inappropriately excluding some small businesses owned by VCOCs, hedge funds, or private equity firms that are eligible to participate in their SBIR programs without the agencies submitting a written determination to SBA and Congress. SBA updated its regulations in June 2016 and made other efforts to clarify eligibility rules for such businesses. Nonetheless, not all agencies were aware of the information SBA provided or fully understood VCOC-owned companies' eligibility, and officials from some of the agencies said eligibility issues related to small business ownership are potentially confusing. Without fully understanding the eligibility of companies owned by some VCOCs, hedge funds, or private equity firms, agencies may continue to inappropriately exclude certain companies from applying for and potentially receiving awards—contrary to SBIR program regulations and SBA guidance.Why GAO Did This StudySmall businesses have been a major source of technology development in the U.S. economy but can face challenges obtaining the funding and investment needed to fully develop and commercialize their technologies. The SBIR program was established to increase the participation of small innovative companies in federally funded research and development. Eleven participating federal agencies make SBIR awards to small businesses. To qualify for awards, businesses must meet certain ownership and other eligibility criteria. The Small Business Act, as amended, authorizes agencies to allow participation in their SBIR programs by qualified small businesses.The Small Business Act includes a provision for GAO to conduct a study of agencies' use of this authority and submit a report to Congress every 3 years. GAO's previous reports covered FY 2013 through 2020. This Q&amp;A report, the fourth report in the series, provides information from FY 2021 through 2023 on the extent to which the 11 agencies in the SBIR program have elected to include qualified small businesses and the factors that have influenced these decisions.GAO reviewed agencies' data on SBIR awards to qualified small businesses from FY 2021 through 2023, analyzed SBIR solicitations and other documents, and interviewed SBIR program officials from SBA and the 11 participating agencies.
- Precision Agriculture: Benefits and Challenges for Technology Adoption and Use What GAO FoundPrecision agriculture technologies can improve resource management through the precise application of inputs, such as water, fertilizer, and feed, leading to more efficient agricultural production. Precision agriculture can be implemented through a suite of technologies that can be used in isolation or in conjunction with other technologies. Examples of emerging precision agriculture technologies are in the table below.Examples of emerging precision agriculture technologiesTechnology			Description		Remote sensing platforms			Drones and ground robots can provide new ways to provide measurements on crop conditions.		In-ground sensors			Provide farmers near-real-time information on soil and plant properties such as temperature, moisture, and nutrients.		Targeted spray systems			Use machine learning to precisely spray in a specific spot.		Automated mechanical weeders			Use machine learning to start and stop weeding blades to avoid damaging the growing crops.		Source: GAO summary of literature, interviews, and agency documentation. &nbsp;| &nbsp;GAO-24-105962While precision agriculture technologies, such as variable rate fertilizer applications and yield monitoring, have been available since the 1990s, only 27 percent of U.S. farms or ranches used precision agriculture practices to manage crops or livestock, based on 2023 U.S. Department of Agriculture (USDA) reporting.Use of precision agriculture practices by U.S. farms, June 2022–June 2023Federal agencies support precision agriculture adoption, research and development, education, and training. USDA supports precision agriculture technology adoption with financial assistance and loan programs, such as through payments for implementing practices that provide a conservation benefit. USDA and the National Science Foundation (NSF) have provided almost $200 million for precision agriculture research and development funding in fiscal years 2017—2021. This funding includes partnerships between the two agencies to support artificial intelligence (AI) research institutes.Benefits to using precision agriculture technologies include:&nbsp;Increased profits. Farmers can increase yields and thus profits with the same amount of inputs or achieve an equivalent yield with fewer inputs.	Reduced application of crop inputs. Technologies can reduce the application of crop inputs such as fertilizer, herbicide, fuel, and water. They can also address water scarcity by promoting the efficient use of water in agriculture.	Environmental benefits. Technologies can prevent excessive use of chemicals and nutrients in a field, potentially reducing runoff into soil and waterways.Challenges limiting the broader adoption and use of precision agriculture include:High up-front acquisition costs. Acquisition costs for the latest technologies can be prohibitive for farmers with limited resources or access to capital.	Farm data sharing and ownership issues. Concerns regarding farm data sharing and ownership can pose obstacles to the widespread use of AI in agriculture.	Lack of standards. An absence of uniform standards can hamper interoperability between different precision agriculture technologies.GAO examined three policy goals and associated options that could help address adoption challenges or enhance the benefits of precision agriculture technologies. These policy options identify possible actions by policymakers, which include Congress, federal agencies, state and local governments, academic and research institutions, and industry. In addition, for each policy goal, policymakers may choose no additional policy interventions, maintaining the status quo by continuing existing activities.Policy Goals and Options That Could Address Challenges or Help Enhance Benefits of Adoption and Use of Precision Agriculture TechnologiesPolicy goals and options			Implementation approaches			Opportunities and considerations		Encourage greater adoption and use&nbsp;(report p. 43)		Provide additional incentives or other financial support						Consider modifying eligibility criteria for existing governmental financial assistance programs.				Consider expanding levels of financial assistance through new or existing programs.									Consider what types of programs to provide to farmers, such as loan guarantees or grants.				Overall program cost, the extent to which equipment acquisition and operating costs are covered, and the potential trade-off with other agricultural programs could be factors.				Programs could be devised to realize goals, such as environmental protection and food insecurity.					Better understand and quantify benefits and costs						Support development of analytical tools and models to quantify benefits and costs.									Analytical tools could be configured for farm geography, soil type, and other factors.				Farmer confidence could increase if estimates of benefits are data-driven and based on real-world experiences.					Promotion and outreach to farmers						Consider expanding Extension services to enable development of more expertise and technical support to farmers.				Increase on-field demonstrations.									The extent to which federal agencies could provide more training, such as through USDA service centers.				Technology companies and dealerships could assume more of a role in educating or helping farmers.					Encourage further innovation&nbsp;(report p. 48)		Conduct research and development to improve on-farm data gathering and analysis						Support research for in-ground sensors for measuring soil conditions.				Examine ways that remote imagery, such as from drones, can be more effectively used to gather data.									In-ground sensors can provide more detailed information on soil conditions, which can enable farmers to further optimize the application of inputs to increase yields and minimize costs.				New remote imagery sensors from drones or ground robots promise to provide greater resolution, more frequent data, and quicker data delivery than traditional satellite sources.&nbsp;					Promote the development and use of standards						Promote data standards that could improve equipment interoperability.				Financial assistance programs could specify that precision agriculture equipment comply with certain standards.									Standards can improve interoperability and compatibility among different devices and platforms.				A lack of standards could result in challenges assessing the quality of the data derived from farm measurements.				Consider the level of federal involvement in standards development and use.					Manage greater amounts of data&nbsp;(report p. 52)		Enhance data analysis						Support the development of software to help farmers better manage their farms.				Examine how AI and machine learning could help facilitate analysis and interpretation of data.									Data analytics and high-performance computing approaches promise to generate valuable information for farmers but are dependent on the availability of large amounts of data.				Farmers may have tools, such as yield monitors, to help identify outcomes of decisions, but few analytical tools and software are available to enhance analysis and translate farm data into actionable decisions.					Encourage data sharing						Develop a governance framework to manage and store agricultural data and its access.				Establish easy-to-understand data license agreements and codes of conduct to enable better flow of data.									Farmers often do not trust the ways farm data are being collected and managed; thus, terms and conditions regarding data use should be simple, transparent, and accountable.				Farmer concerns about data sharing include a potential loss of competitive advantage, data security, and additional regulatory scrutiny may increase as AI is increasingly used for data analysis.					Source: GAO. &nbsp;| &nbsp;GAO-24-105962Why GAO Did This StudyPrecision agriculture involves collecting, analyzing, and taking actions based on data. It can help the agricultural sector meet increasing demand for food products, while also helping farmers improve efficiencies such as through reduced input costs.The Advancing IoT for Precision Agriculture Act of 2021, contained in what is commonly referred to as the CHIPS and Science Act of 2022, included provisions for GAO to conduct a technology assessment and review federal programs.This report examines (1) emerging precision agriculture technologies and precision agriculture technology adoption; (2) federal programs providing support for precision agriculture; (3) benefits and challenges of adopting and using precision agriculture technologies; and (4) policy options that could address challenges or help enhance benefits of adopting and using precision agriculture technologies.To conduct this technology assessment and review, GAO reviewed scientific literature and other key reports; interviewed officials and representatives from government, industry, academia, and associations; conducted two site visits to observe technology operations and obtain stakeholder perspectives; and convened a 3-day virtual expert meeting in collaboration with the National Academies of Sciences, Engineering, and Medicine. GAO is identifying policy options in this report.For more information, contact Brian Bothwell at (202) 512-6888 or BothwellB@gao.gov, or Steve D. Morris at (202) 512-3841, morriss@gao.gov.
- Artificial Intelligence: GAO's Work to Leverage Technology and Ensure Responsible Use What GAO FoundGAO is exploring internal use of artificial intelligence (AI) to make its work for Congress and taxpayers more efficient, in-depth, and effective. For example, GAO recently began deployment of a large language model to explore generative AI capabilities similar to those found in industry, supplemented with GAO-specific information and security. Among other capabilities, this prototype will also support other use cases to help efficiently synthesize past reports, assist with editorial review, and scan congressional documents for mandated work for GAO. By developing these tools, GAO is also gaining insight into the benefits and risks of AI, which will help GAO evaluate other agencies' use and better provide technical assistance to Congress. See the table for details on eight AI use cases GAO is currently exploring.List of Artificial Intelligence Use Cases GAO Is Exploring as of January 2024			Use case									Potential benefits								Organizes large volumes of text, such as public comments from Regulations.gov&nbsp;									Groups contents by similar themes			Prioritizes reviews based on relevant hierarchical topics								Summarizes draft GAO legislative mandates&nbsp;&nbsp;									Increases efficiency and reduces manual processes&nbsp;&nbsp;&nbsp;			Highlights potentially fragmented and duplicative mandates&nbsp;&nbsp;								Assists with copyediting according to GAO's style guide&nbsp;									Automates select copyediting tasks&nbsp;			Enables staff to focus on narrative structure and clarity&nbsp;								Provides automated responses to chat questions on published GAO work&nbsp;									Summarizes published GAO contents to enhance knowledge management and increase efficiency &nbsp;			Enhances specificity and accuracy of results&nbsp;								Summarizes qualitative responses from annual GAO Employee Experience Survey&nbsp;									Identifies trends, patterns, and sentiments quickly			Improves survey interpretation with less manual intervention			Assists with root-cause analyses								Monitors information about congressional committee calendars, press releases, and web contents&nbsp;&nbsp;									Matches congressional interests with relevant GAO work&nbsp;&nbsp;			Enhances timeliness of outreach and technical assistance&nbsp;&nbsp;								Enhances GAO auditing through use of extended reality glasses&nbsp;&nbsp;									Improves data collection and collaboration across locations&nbsp;			Reduces costs and risks&nbsp;			Enables new data capturing and analysis opportunities&nbsp;&nbsp;								Triages IT help desk requests and answers internal GAO policy questions									Provides 24/7 self-service assistance to GAO employees			Allows support staff to focus on more complex requests					Source: GAO. | GAO-24-107237Note: For the full list of use cases, updated periodically, see www.gao.gov/science-technology/artificial-intelligence-use-cases.AI also remains a top priority for many congressional committees and offices. GAO therefore has a robust body of work to meet Congress's need for information on AI's promise and risks. Since 2018, GAO has issued nearly 50 products on AI. It currently has 20 ongoing projects on AI with many more planned, including two technology assessments and one performance audit on generative AI. For example, in a landmark December 2023 report, GAO reviewed the implementation of AI at major federal agencies and made 35 recommendations to 19 agencies to fully implement federal AI requirements. In addition, in 2021, GAO published  Artificial Intelligence: An Accountability Framework for Federal Agencies and Other Entities  , identifying 31 key practices to help ensure accountability and responsible use of AI. GAO's AI use cases are built upon this framework. The following is a selection of GAO AI-related products:Cybersecurity and Privacy Facial Recognition Services: Federal Law Enforcement Agencies Should Take Actions to Implement Training, and Policies for Civil Liberties.  GAO-23-105607. September 5, 2023.Facial Recognition: CBP and TSA are Taking Steps to Implement Programs, but CBP Should Address Privacy and System Performance Issues.  GAO-20-568. September 2, 2020.Facial Recognition Technology: Privacy and Accuracy Issues Related to Commercial Uses.  GAO-20-522. July 13, 2020.AI Opportunities and Challenges Artificial Intelligence in Natural Hazard Modeling: Severe Storms, Hurricanes, Floods, and Wildfires.  GAO-24-106213. December 14, 2023.Generative AI  (Science &amp; Tech Spotlight). GAO-23-106782. June 13, 2023.Digital Twins: Virtual Models of People and Objects.  (Science &amp; Tech Spotlight). GAO-23-106453. February 14, 2023.Federal Research and Management Artificial Intelligence: Agencies Have Begun Implementation but Need to Complete Key Requirements.  GAO-24-105980. December 12, 2023.Artificial Intelligence: DOD Needs Department-Wide Guidance to Inform Acquisitions.  GAO-23-105850. June 29, 2023.Artificial Intelligence: DOD Should Improve Strategies, Inventory Process, and Collaboration Guidance.  GAO-22-105834. March 30, 2022.Federal Workforce Artificial Intelligence: Actions Needed to Improve DOD's Workforce Management  GAO-24-105645. December 14, 2023.FAA Workforce: Better Assessing Employees' Skill Gaps Could Help FAA Prepare for Changes in Technology.  GAO-21-310. May 13, 2021.Automated Technologies: DOT Should Take Steps to Ensure Its Workforce Has Skills Needed to Oversee Safety.  GAO-21-197. December 18, 2020.Defense and Law Enforcement Information Environment: Opportunities and Threats to DOD's National Security Mission.  GAO-22-104714. September 21, 2022.Artificial Intelligence: Status of Developing and Acquiring Capabilities for Weapon Systems.  GAO-22-104765. February 17, 2022.Forensic Technology: Algorithms Strengthen Forensic Analysis, but Several Factors Can Affect Outcomes  . GAO-21-435SP. July 06, 2021.Why GAO Did This StudyAI, in general, refers to computer systems that can solve problems and perform tasks that have traditionally required human intelligence. Many federal agencies are using or planning to use AI, through applications such as facial recognition, automated email responses, and enhanced user experience on websites. At the same time, AI capabilities continue to expand in areas such as medical diagnostics, image generation, and emulating human writing styles.This testimony describes how GAO (1) plans to use AI internally to enhance effectiveness and efficiency across its mission and operations and (2) supports Congress through oversight of federal use of AI and foresight on its wider implementation.GAO performed the work on which this testimony is based in accordance with all applicable sections of GAO's Quality Assurance Framework.For more information, contact Taka Ariga at (202) 512-6888 or ArigaT@gao.gov.
- Federal Regulation: Selected Emerging Technologies Highlight the Need for Legislative Analysis and Enhanced Coordination What GAO FoundSelected federal agencies—the Department of Transportation (DOT), the Federal Communications Commission (FCC), and the Food and Drug Administration (FDA)—reported using a variety of practices and approaches to prepare for and address regulatory challenges and opportunities posed by emerging technologies. However, FDA officials said updated authorities would help it regulate medical devices enabled with artificial intelligence (AI), in particular. Members of Congress are currently considering enhancing oversight of AI, including in medical devices, and congressional members have discussed barriers with FDA. However, FDA has not clearly identified, documented, and communicated to Congress the specific legislative changes that would help it address these challenges. Without this information, Congress may not be able to appropriately update FDA's authorities, and FDA may miss opportunities to fully realize the public health benefits of this technology.Examples of Emerging Technologies: 3D Printing of Biological Materials, Next Generation Wireless, and Drones Selected federal agencies have reported coordinating with other domestic and foreign agencies to support their efforts to regulate some emerging technologies. Agencies' interagency collaboration activities include efforts to share information and pursue goals for these technologies. However, DOT has not communicated progress made on a department-wide council to resolve jurisdictional and regulatory gaps associated with emerging transportation technologies. By assessing and publicly communicating the council's plans and progress, DOT could provide important information to stakeholders and help ensure the council is accountable for achieving its goals.Knowledge-building and outreach efforts used by some selected foreign regulators were more extensive than those used by selected domestic regulators, particularly with industry and academia. For example, foreign regulators have clear channels for communicating with industry, the public, and regulated entities. The Federal Aviation Administration (FAA) recently established a group to engage with industry on drone regulation. But FAA could improve its outreach and communication with industry by publicizing this initiative, called the Emerging Technologies Coordination section. FAA currently does not include information online about how to contact this group. Industry can only participate in it by FAA's invitation, which may prevent some entities from getting the full benefit from its assistance.Why GAO Did This StudyEmerging technologies have the potential to unlock immense societal benefits. Developing effective regulations to tackle complex emerging problems takes time. As a result, these technologies present a pacing problem for regulators. Ensuring regulations keep pace with the rapid development of emerging technologies is critical to protecting public interests and facilitating innovation.GAO was asked to review how federal agencies regulate emerging technologies. This report examines, for selected agencies, (1) challenges and opportunities they report facing in regulating emerging technologies; (2) their collaboration and cooperation activities; and (3) lessons they can learn from other governments' experiences.GAO reviewed documentation and interviewed officials from DOT, FCC, FDA, and other knowledgeable agencies and compared agencies' coordination efforts to selected leading practices. GAO also interviewed government officials in the European Union, the United Kingdom, and Japan, as well as 10 stakeholders spanning industry groups, academia, and other experts about practices in regulating emerging technologies.
- Higher Education: VA Should Provide Additional Information to Its Staff and Schools on the Rogers STEM Scholarship What GAO FoundMore than 7,000 veterans have used the Edith Nourse Rogers STEM Scholarship since the program began in August 2019. The scholarship provides financial assistance to eligible veterans pursuing degrees in science, technology, engineering, and mathematics (STEM). The vast majority of veterans (99 percent) used the scholarship towards an undergraduate STEM degree, and about 2,000 veterans graduated while using the scholarship.GAO found that the Department of Veterans Affairs has tools—such as staff training and guidance—to help ensure consistent Rogers STEM scholarship application processing, and more information for its staff and schools would bolster these efforts. Some STEM majors are easy to identify (e.g., engineering or mathematics). VA staff judgement becomes part of the application award process in the small number of applications where a veteran's major is not easily identifiable as STEM.Using VA data, GAO identified more than 100 instances where VA potentially made erroneous scholarship award decisions because the major approved for the scholarship did not appear to be STEM. VA confirmed that in 56 instances it appeared the agency approved scholarships for veterans pursuing ineligible majors, such as applied management and construction management. These 56 instances reflect VA's approval of up to $1.7 million in scholarship funds for veterans to use toward majors not in STEM fields.Through data analysis, review of VA documents, and interviews with VA staff, GAO identified two areas where VA could provide additional information to help ensure consistent scholarship award decisions. Specifically:Information for VA staff.  VA does not have a complete list of eligible majors for its staff to use when processing applications and deciding whether to award the scholarship. VA also does not have formal procedures to ensure its staff raise questions or further examine applications with majors that appear to be non-STEM.	Information for schools. School officials provide key information during the application process, however, the resources that VA provides to schools do not fully describe the school's role. As a result, schools do not always provide the information needed and VA may deny the scholarship to an otherwise eligible veteran, according to VA staff.Providing this additional information to VA staff and schools can help ensure consistent scholarship award decisions.Why GAO Did This StudyVeterans who received technical training in the military may be well suited to pursue degrees in STEM. The Edith Nourse Rogers STEM Scholarship provides up to $30,000 to assist veterans with continuing qualifying STEM programs after they exhaust their Post-9/11 GI Bill benefits.The Harry W. Colmery Veterans Educational Assistance Act of 2017 includes a provision for GAO to provide an interim and final assessment of the Rogers STEM scholarship program. This final assessment (1) reports the number of veterans that pursued and obtained STEM degrees using the Rogers STEM scholarship, and (2) evaluates how consistently VA processes Rogers STEM scholarship applications. GAO analyzed VA administrative data from August 2019 to June 2023, the most recently available data at the time of GAO's analysis. GAO also held four discussion groups with VA staff who process scholarship applications and reviewed relevant VA documents.
- Research Security: Strengthening Interagency Collaboration Could Help Agencies Safeguard Federal Funding from Foreign Threats What GAO FoundGAO found that determining whether federal research and development (R&amp;D) funds were provided to a foreign entity of concern is challenging. Such entities include foreign terrorist organizations and specially designated nationals, among others. Awarding agencies are generally prohibited from doing business with foreign terrorist organizations and specially designated nationals.GAO found that government-wide databases which report on some of these entities lack common identifying information such as a unique identifier or personally identifiable information. For example, a physical address, date of birth, or other identifying information for entities on the foreign terrorist organization list is often unavailable as entities often try to conceal their identity or location, according to the Treasury Department. When personally identifiable information such as date of birth is available, similar information is not available in the public federal funding data source, USAspending.gov. Specifically, although awarding agencies collect other identifying information, such as phone numbers of entities seeking an award, not all such information is reported in USAspending.gov. Together, these challenges limit the ability to match foreign entities on certain lists with those receiving government-wide funding.Example of Gaps in Crossmatching Identifying Information for Restricted Foreign Entities With Federal Spending DataGAO found that some requirements exist specific to R&amp;D awards to foreign entities of concern. For example, agencies are generally required to vet potential awardees against the federal exclusions list, which includes foreign terrorist organizations and specially designated nationals. Agencies that GAO interviewed reported challenges in identifying foreign ownership, control, or influence. Officials from two agencies we reviewed said they use analytical tools and other available information to try to identify foreign ownership. Such efforts, however, are not shared across agencies. Recent guidance from the Office of Science and Technology Policy (OSTP)—the lead federal agency for interagency coordination on science and technology policies—does not address the issue of foreign ownership, control, or influence. OSTP officials told GAO that they handle this issue on a case-by-case basis and acknowledged the challenges that agencies face in addressing such risks. Information sharing on such complex matters would better position agencies to more fully safeguard R&amp;D funds.Why GAO Did This StudyAccording to the National Science Foundation, federal agencies obligated about $1.4 billion for R&amp;D with foreign entities in fiscal year 2020 (the most recent data available). However, some foreign entities may try to exploit U.S. openness in sharing R&amp;D for nefarious purposes.The Research and Development, Competition, and Innovation Act includes a provision for GAO to review research funding provided to foreign entities of concern. The act's definition of such entities includes foreign terrorist organizations and foreign entities subject to the control, ownership, and jurisdiction of Russia, China, North Korea, and Iran.This report examines challenges in identifying R&amp;D funds awarded to foreign entities of concern and requirements for awarding and monitoring such funding.GAO searched USAspending.gov and reviewed documents from federal funding and law enforcement agencies. GAO also interviewed federal agency officials, including those from the five agencies that obligated the most R&amp;D funding in fiscal year 2020.
- Artificial Intelligence: Actions Needed to Improve DOD's Workforce Management What GAO FoundThe Department of Defense (DOD) typically establishes standard definitions of its workforces to make decisions about which personnel are to be included in that workforce, and identifies its workforces by coding them in its data systems. DOD has taken steps to begin to identify its artificial intelligence (AI) workforce, but has not assigned responsibility and does not have a timeline for completing additional steps to fully define and identify this workforce. DOD developed AI work roles—the specialized sets of tasks and functions requiring specific knowledge, skills, and abilities. DOD also identified some military and civilian occupations, such as computer scientists, that conduct AI work. However, DOD has not assigned responsibility to the organizations necessary to complete the additional steps required to define and identify its AI workforce, such as coding the work roles in various workforce data systems, developing a qualification program, and updating workforce guidance. DOD also does not have a timeline for completing these additional steps. Assigning responsibility and establishing a timeline for completion of the additional steps would enable DOD to more effectively assess the state of its AI workforce and be better prepared to forecast future workforce requirements (see figure).Questions DOD Cannot Answer Until It Fully Defines and Identifies Its AI WorkforceDOD's plans and strategies address some AI workforce issues, but are not fully consistent with each other. Federal regulation and guidance state that an agency's Human Capital Operating Plan should support the execution of its Strategic Plan. However, DOD's Human Capital Operating Plan does not consistently address the human capital implementation actions for AI workforce issues described in DOD's Strategic Plan. DOD also uses inconsistent terms when addressing AI workforce issues, which could hinder a shared understanding within DOD. The military services are also developing component-level human capital plans that encompass AI and will cascade from the higher-level plans. Updating DOD's Human Capital Operating Plan to be consistent with other strategic documents would better guide DOD components' planning efforts and support actions necessary for achieving the department's strategic goals and objectives related to its AI workforce.Why GAO Did This StudyDOD has invested billions of dollars to integrate AI into its warfighting operations. This includes analyzing intelligence, surveillance, and reconnaissance data, and operating lethal autonomous weapon systems. DOD identified cultivating a workforce with AI expertise as a strategic focus area in 2018. However, in 2021 the National Security Commission on Artificial Intelligence concluded that DOD's AI talent deficit is one of the greatest impediments to the U.S. being AI-ready by the Commission's target date of 2025.House Report 117-118, accompanying a bill for the National Defense Authorization Act for Fiscal Year 2022, includes a provision for GAO to review DOD's AI workforce. This report evaluates the extent to which DOD has (1) defined and identified its AI workforce and (2) established plans and strategies to address AI workforce issues, among other objectives. GAO assessed DOD strategies and plans, reviewed laws and guidance that outline requirements for managing an AI workforce, and interviewed officials.
- National Institute of Standards and Technology: Strengthening Disclosure Requirements and Assessing Training Could Improve Research Security What GAO FoundResearchers employed at the National Institute of Standards and Technology (NIST) collaborate on research projects with about 2,500 domestic and foreign national researchers (known as “associates”) each year. The agency also awards grants and cooperative agreements under which extramural (i.e., external) researchers carry out research. While such collaborations are intended to benefit NIST, they may pose security risks. NIST has taken steps to help ensure research security by requiring researchers to disclose information that can help it determine whether they have potential conflicts of interest or commitment.However, at the time of our review, NIST had not fully implemented federal disclosure requirements as the agency was waiting for the Office of Science and Technology Policy (OSTP) to issue government-wide guidance in two areas:uniform disclosure forms for extramural researchers, and	guidelines on foreign talent recruitment programs, which seek to recruit researchers—sometimes with malign intent.According to NIST officials, OSTP's delays in issuing the forms and guidelines have delayed NIST's collection of certain disclosures. Without these disclosures, NIST is missing key information—such as domestic researchers' participation in foreign talent recruitment programs—that could help it address research security risks.Separately, NIST requires fewer disclosures from domestic associates than from foreign national associates. Officials said the agency primarily focuses on risks posed by foreign national associates and by certain countries of concern. However, domestic researchers can also have concerning affiliations with foreign entities. By not requiring domestic associates to disclose the same information as foreign national associates, NIST is missing opportunities to assess and mitigate risks.Information That NIST Requires Associates to Disclose			Type of researcher									Organizational affiliations/ employment									Positions/ appointments									Participation in foreign talent recruitment programs									Current and pending research support								Foreign national associate									✓									✓									✓									✓								Domestic associate									✓									-									-									-					Source: GAO analysis of the National Institute of Standards and Technology (NIST) information. | GAO-24-106074NIST and Commerce also help ensure research security by training researchers. The training program generally aligns with most selected leading training practices. However, because they do not evaluate the program's effectiveness, the agencies are limited in their ability to identify opportunities for improvement. For example, NIST employees told GAO that NIST could provide more examples of risks that employees may encounter. Collecting and analyzing such feedback could help strengthen the agency's training and improve research security.Why GAO Did This StudyCountries of concern pose security risks to U.S. research and innovation. Such countries have sought to access information through collaborative research efforts. NIST employees regularly collaborate with outside researchers from academia or private-sector companies. The Research and Development, Competition, and Innovation Act includes a provision for GAO to review NIST's research security program.This report examines, among other things, NIST's efforts to (1) meet federal disclosure requirements for intramural and extramural researchers, (2) collect and review disclosures from foreign national associates and domestic associates, and (3) align its security training with selected leading training practices.GAO reviewed NIST's information and available data on identified risks, research security policies, and procedures, and interviewed agency officials. GAO also compared NIST's policies and practices against selected federal requirements and leading practices on training.
- Artificial Intelligence in Natural Hazard Modeling: Severe Storms, Hurricanes, Floods, and Wildfires What GAO FoundGAO found that machine learning, a type of artificial intelligence (AI) that uses algorithms to identify patterns in information, is being applied to forecasting models for natural hazards such as severe storms, hurricanes, floods, and wildfires, which can lead to natural disasters. A few machine learning models are used operationally—in routine forecasting—such as one that may improve the warning time for severe storms. Some uses of machine learning are considered close to operational, while others require years of development and testing.GAO identified potential benefits of applying machine learning to this field, including:Reducing the time required to make forecasts by replacing components of models that are slow and that increase the cost of modeling.	Increasing model accuracy by more fully exploiting available data, using other data that traditional models cannot, and creating synthetic data to fill gaps.	Reducing the uncertainty of model output by improving ensemble modeling—the processes of generating combined predictions from numerous models—and making better use of historical data.Forecasting natural disasters using machine learningGAO also identified challenges to the use of machine learning. For example:Data limitations hamper the training of machine learning models and can reduce accuracy in some regions, such as rural areas where weather observations are sparse.	A lack of trust and understanding of the algorithms as well as concerns about bias can make forecasters and other users hesitant to use machine learning models.	Limited coordination and collaboration create challenges for fully developing some machine learning models. For example, some forecasters told us they lack opportunities to interact with researchers and convey their needs.	Workforce and resource gaps also create challenges. For example, the upfront costs to develop and run machine learning models are high, and some companies working on these models do not fully understand the data and phenomena they are modeling, according to academic researchers.GAO identified five policy options that could help address these challenges. These options are intended to inform policymakers, including Congress, federal and state agencies, academic and research institutions, and industry of potential policy implementations. The status quo option illustrates a scenario in which government policymakers take no additional actions beyond current ongoing efforts.Policy Options to Help Address Challenges to the Use of Machine Learning in Natural Hazard ModelingPolicy Option			Opportunities			Considerations		Facilitate improved data collection, sharing, and use (report p. 37).			Government policymakers could expand use of existing observational data and infrastructure to close gaps, expand access to certain data, and (in conjunction with other policymakers) establish guidelines for making data AI-ready.						Efforts to address gaps in data sets can improve machine learning model performance.				&nbsp;Expanded access to existing data would improve the ability of researchers and groups to develop and test machine learning technologies.				Adopting standards for AI-ready data could reduce resources needed to curate data and facilitate more efficient modeling.									Expanding observational infrastructure can be expensive and could divert limited resources from other efforts.				Agencies need to weigh the benefits of greater data sharing against any increase in risks related to data security and privacy.				Strict data standards may slow research and innovation if they burden or constrain machine learning researchers.					Expand education and training (report p. 38).			Government policymakers could update education requirements to include machine learning-related coursework and expand learning and support centers, while academic policymakers could adjust physical science curricula to include more machine learning coursework.						Updating education requirements would better prepare students to use machine learning in government.				More robust education can better prepare both researchers and end users in fields like meteorology and climatology to develop and use machine learning.									Education and training reforms may need to be repeatedly adjusted, as technological change in this space can be rapid and unpredictable.				Establishing and expanding professional development and training opportunities throughout government may require substantial investment.					Address hiring and retention barriers and certain resource shortfalls (report p. 39).			Government policymakers could address pay scale limitations for positions that include machine learning expertise and work with private sector policymakers to expand the use of public-private partnerships (PPP).						Providing workforce incentives to government employees for machine learning development could allow agencies to bolster workforce capacities.				Expanding PPPs might help agencies overcome computational resource shortfalls and help industry draw on government expertise.									Increasing salary limits for some employees would require agency budget increases or cuts to other budget items.				Expanding the use of PPPs could magnify resource disparities between government and private industry.				PPPs that entail hosting government data on collaborator systems may pose security risks that would need to be considered and addressed.					Take steps to mitigate bias and foster trust in data and machine learning models (report p. 40).			Policymakers could establish efforts to better understand and mitigate various forms of bias, support inclusion of diverse stakeholders for machine learning models, and develop guidelines or best practices for reporting methodological choices.						Sustained efforts to address bias in data sets can reduce the likelihood of models negatively affecting certain communities.				Acquiring diverse stakeholder perspectives throughout machine learning models’ life cycles can help reduce certain types of bias in data and models.				Fostering machine learning model transparency could improve end-user and decision-maker trust.									Embedding efforts to address bias throughout the model life cycle may increase model costs and slow model development.					Maintain status quo efforts (report p. 41).			Government policymakers could maintain existing policy efforts and organizational structures, along with existing strategic plans and agency commitments.						Some agency efforts are underway to address the challenges described.									The extent to which agencies will meet their commitments under status quo efforts is unclear. Status quo efforts may not address challenges specific to the use of AI in natural hazard modeling.					Source: GAO. | GAO-24-106213Why GAO Did This StudyNatural disasters cause on average hundreds of deaths and billions of dollars in damage in the U.S. each year. Forecasting natural disasters relies on computer modeling and is important for preparedness and response, which can in turn save lives and protect property. AI is a powerful tool that can automate processes, rapidly analyze massive data sets, enable modelers to gain new insights, and boost efficiency.This report on the use of machine learning in natural hazard modeling discusses (1) the emerging and current use of machine learning for modeling severe storms, hurricanes, floods, and wildfires, and the potential benefits of this use; (2) challenges surrounding the use of machine learning; and (3) policy options to address challenges or enhance benefits of the use of machine learning.GAO reviewed the use of machine learning to model severe storms, hurricanes, floods, and wildfires across development and operational stages; interviewed a range of stakeholder groups, including government, industry, academia, and professional organizations; convened a meeting of experts in conjunction with the National Academies; and reviewed key reports and scientific literature. GAO is identifying policy options in this report.For more information, contact Brian Bothwell at (202) 512-6888 or bothwellb@gao.gov.
- Artificial Intelligence: Agencies Have Begun Implementation but Need to Complete Key Requirements What GAO FoundTwenty of 23 agencies reported about 1,200 current and planned artificial intelligence (AI) use cases—specific challenges or opportunities that AI may solve. Three agencies reported not having uses for AI. Agency reported uses included analyzing data from cameras and radar to identify border activities, analyzing photographs from drones, and targeting of scientific specimens for planetary rovers. The National Aeronautics and Space Administration (NASA) and the Department of Commerce (Commerce) reported the highest number of AI use cases (see figure).Agency Reported AI Use Cases in Fiscal Year 2022Most of the reported AI use cases were in the planning phase and not yet in production (i.e., currently used) (see figure). In about 200 instances, agencies reported that they were currently using AI.Agency Reported Artificial Intelligence (AI) Use Case Lifecycle Stage, as of Fiscal Year 2022a“Other” includes life cycle stage responses not included in the  2021 Guidance for Creating Agency Inventories of Artificial Intelligence Use Cases,  such as “research and design” and “exploratory.”GAO's analysis of agencies' inventories of use cases identified instances of incomplete and inaccurate data. Specifically, five agencies provided comprehensive information for each of their reported use cases while the other 15 had instances of incomplete and inaccurate data. For example, some inventories did not include required data elements, such as the AI life cycle stage or an indication of whether an AI use case was releasable or not. In addition, two inventories included AI uses that were later determined by the agencies to not be AI. Without accurate inventories, the government's management of its use of AI will be hindered by incomplete and inaccurate data.Federal agencies have taken initial steps to comply with AI requirements in executive orders and federal law; however, more work remains to fully implement these.Commerce and the General Services Administration (GSA) fully implemented selected requirements. Specifically, Commerce created a plan to develop AI technical standards and GSA established the AI Center of Excellence.	The Offices of Management and Budget (OMB) and Personnel Management (OPM) did not fully implement selected requirements. OMB has not yet developed guidance for the acquisition and use of AI. OPM has not yet established or updated an occupational category for those employees performing AI work.	The Office of Science and Technology Policy (OSTP) did not communicate its designation of which agencies were expected to fulfill specific AI requirements.	Ten of 23 agencies implemented all AI requirements specific to their agencies, 12 implemented some but not all, and one was exempt from the requirements. These requirements included preparing an inventory on the use of AI, planning for inventory updates, and planning for AI regulatory authorities.Addressing these requirements will improve agency identification, development, implementation, and oversight of AI.Why GAO Did This StudyWhile there are varying definitions of AI, they generally refer to computing systems that “learn” how to improve their performance. AI has the potential to rapidly change the world and holds substantial promise for improving government operations. However, AI poses risks that can negatively impact individuals, groups, organizations, communities, and society.The President's fiscal year 2023 budget request included $1.8 billion for nondefense research and development investment in AI. In addition, some agencies are using AI operationally to identify information security threats and facilitate the review of large datasets, among other uses. Given the rapid growth in capabilities and widespread adoption of AI, the federal government should have safeguards to manage AI's complexities, risks, and societal consequences.In this report, GAO reviewed the implementation of AI at major federal agencies. This report examines (1) federal agency reported current and planned uses of AI, (2) the extent to which federal agencies' AI reporting was comprehensive and accurate, and (3) the extent to which federal agencies have complied with selected federal policy and guidance on AI.This review focused on agencies with government-wide roles in AI implementation (including OMB and OPM) as well as agencies with individual responsibilities for AI implementation (including 23 of the 24 Chief Financial Officers (CFO) Act of 1990 agencies). The Department of Defense was excluded because GAO had issued recent AI reports on that department.GAO reviewed the 23 civilian CFO Act   agencies' AI inventories and analyzed reported use cases and supporting data. GAO also compared agency reported data to the requirements for developing an AI inventory outlined in federal guidance.In addition, GAO identified requirements from executive orders, OMB guidance, and a law regarding the implementation of AI. GAO then assessed agencies' implementation of these requirements. GAO also interviewed relevant officials.
- Low-Dose Radiation: Agency Leadership and Collaboration Are Needed to Set the Direction of Future Research What GAO FoundIn 2018, Congress directed the Department of Energy (DOE) to carry out a research program on low-dose radiation and in 2020, directed DOE to enter into an agreement with the National Academies of Sciences, Engineering, and Medicine (National Academies) to develop a research agenda for that program. The National Academies issued a report in June 2022 with recommendations and priorities for a research program to better assess health effects of low-dose radiation (see figure).DOE and three selected agencies involved in low-dose radiation research have some ongoing and planned efforts that can help address the findings and recommendations of the June 2022 report. DOE's Office of Environment, Health, Safety, and Security has long supported research on the health of people exposed to low-dose radiation. In addition, DOE's Office of Science resumed some low-dose radiation research activities in 2020 after ending its previous research program in 2016. The new research is focused on using artificial intelligence (AI) to better understand low-dose radiation biology. DOE invested $22.3 million in this research in fiscal years 2020 through 2023. One of the funded projects uses AI to build a framework for estimating cancer risk based on long-term exposure to radiation. Other selected agencies continue to support and conduct research that generally aligns with agency specific missions, such as understanding cancer risk in people exposed to radiation and promoting occupational safety and health.Figure: Human Health Effects of Low-Dose Exposure to Ionizing RadiationNote: Low-dose radiation is a dose of less than 100 millisieverts (mSv). For details see fig. 1 in GAO-24-106317.DOE's Office of Science—the sponsor of the National Academies report—has taken some steps in response to the report. According to DOE, its efforts—including plans to apply AI—are relevant to three of the 11 research priorities from the National Academies report. Currently, DOE does not have plans to expand its research to address the other priorities. However, DOE is seeking input from its advisory committee by spring 2024 on the potential scope of a program. Nevertheless, Office of Science officials told GAO that a low-dose radiation research program is not a priority because, among other reasons, the research might not reduce knowledge gaps enough to lead to changes in radiation protection regulations.In 2020, Congress directed DOE to engage with other federal agencies as part of its low-dose radiation research program. Additionally, the National Academies called for coordination across federal agencies that carry out low-dose radiation research. However, DOE's Office of Science has not led a coordinated response to the National Academies report or worked with the Office of Science and Technology Policy (OSTP) or another agency to do so. By taking a leadership role for an interagency low-dose radiation research program or taking steps to initiate efforts with another agency, DOE could help to resolve uncertainty with low-dose radiation research in the United States.Why GAO Did This StudyIonizing radiation comes from both natural sources and medical, commercial, and industrial activities. Prior research has not clearly identified how low doses of such radiation affect the health of millions of people exposed in various settings. The 2022 National Academies report found that a coordinated, multidisciplinary low-dose radiation research program could enable a better assessment of current risk estimates.Congress included a provision in statute for GAO to review the activities of DOE's low-dose radiation research program. This report addresses the extent to which (1) DOE and selected agencies have efforts that can help address the findings and recommendations of the National Academies report and (2) DOE has developed a plan for addressing the findings and implementing the recommendations of the report.GAO analyzed documents from DOE and three other agencies that fund or conduct low-dose radiation research and interviewed agency officials involved in low-dose radiation research, as well as officials from OSTP. GAO compared DOE's efforts to leading practices on interagency collaboration.
- National Science Foundation: Additional Steps Would Improve Cost Estimate for Antarctic Research Infrastructure Project What GAO FoundSince GAO's July 2022 report, the National Science Foundation (NSF) has finalized the cost and schedule increases that were due to the COVID-19 pandemic and other factors for all five major facilities projects in construction. NSF anticipates additional increases for two of its major facilities projects—the Vera C. Rubin Observatory and the Antarctic Infrastructure Modernization for Science. NSF also advanced the design of two projects and approved the advancement of two new projects to the design stage—the Giant Magellan Telescope, and the Thirty Meter Telescope.Examples of National Science Foundation Major Facilities ProjectsNSF's Antarctic Infrastructure Modernization for Science (AIMS) project substantially met three of the four characteristics of a reliable cost estimate and met all the characteristics of a reliable schedule. The AIMS cost estimate was unreliable because it did not substantially or fully meet all four characteristics of a reliable cost estimate, as described in GAO's cost guide. Specifically, the AIMS project partially met the “well-documented” characteristic associated with reliable cost estimates. This was, in part, because the estimate did not specify the source data used and it lacked details to trace technical baselines to cost and other key information for management review. Without good documentation, senior management and others providing oversight will not have confidence that the estimate is reliable. In addition, a reliable cost estimate may help the project to prevent any unnecessary tradeoffs or loss of research capabilities that may result from unexpected cost increases in the future.NSF guidance requires that project teams identify all known risks and opportunities that may affect the supply chain for their projects in construction. Several projects have experienced unforeseen supply chain related risks due in part to the pandemic and other external factors. For example, the Rubin Observatory reported supply chain issues stemming from the war in Ukraine and the project's inability to receive shipments from that region to support construction. Because of the unforeseen nature of these risks, NSF determined that the agency would provide management reserve funds in response. To manage known supply chain risks, the award recipients may identify specific supplier performance and component availability for projects prior to construction and include in the project's budget contingency to respond to those risks should they occur.Why GAO Did This StudyNSF supports the design, construction, and operations of science and engineering research infrastructure such as telescopes and research vessels. These projects include major facilities projects that cost over $100 million. Currently, NSF has five major facilities projects under construction at a combined authorized cost of $1.4 billion and four additional projects in design. Building these on time and within budget helps support the scientific community's ability to conduct research and advance U.S. scientific goals.The Consolidated Appropriations Act, 2022, includes provisions for GAO to review projects funded from NSF's Major Research Equipment and Facilities Construction account. This report, the sixth, (1) describes the cost and schedule performance of NSF's research infrastructure projects, (2) examines the AIMS project's adoption of cost estimating and schedule development best practices and (3) examines supply chain risk management for NSF's major facilities projects in construction. GAO reviewed NSF and award recipient documents, examined policies and procedures to manage and oversee projects, and interviewed NSF officials.
- Quantum Technologies: Defense Laboratories Should Take Steps to Improve Workforce Planning What GAO FoundIn fiscal year 2023, seven defense laboratories reported 255 staff working all or part of their time on quantum projects. Most of these quantum workers had PhDs, were either physicists or engineers, and were a mix of mostly federal civilian employees and some contractors. Consistent with DOD's priority to expand the quantum workforce beyond the field of physics, these laboratories also had specialists from fields such as chemistry, computer science, and mathematics.The four defense laboratories conducting most of DOD's quantum work included quantum information science as part of their broader strategic workforce planning but varied in their adoption of five leading practices (see figure). One laboratory fully adopted all five, while three laboratories conducted workforce planning that was aligned with most but not all leading practices. Three laboratories did not fully adopt the leading practice of monitoring and evaluating progress because their strategic workforce plans either did not have performance measures or inconsistently applied them. One laboratory also did not fully adopt the leading practice of involving key stakeholders in workforce planning because it did not have a means to communicate its workforce goals, initiatives, and metrics for success. Better workforce planning could help these laboratories build and retain the workforce needed to maintain global leadership in quantum technologies.Defense Laboratories' Adoption of Leading Practices for Strategic Workforce PlanningAccording to GAO's questionnaire results, 41 DOD science, technology, engineering, and mathematics (STEM) education programs contributed to quantum workforce development from fiscal years 2019 through 2023. Through these programs, at least 423 students and postdoctoral researchers gained quantum work experience at defense laboratories. These programs also reported developing students through exposure to DOD quantum research, postdoctoral research funding, and training in a variety of STEM disciplines that can feed into the quantum workforce, such as physics, engineering, and computer science.Why GAO Did This StudyDOD considers quantum technologies as critical to protecting national security. Nations that lead in quantum technologies can gain a competitive edge in areas such as cryptography, sensing, and computing. To remain competitive, DOD is developing its quantum workforce.A Senate report included a provision for GAO to review DOD's quantum workforce planning and workforce development activities. This report examines (1) the size and composition of the quantum workforce at defense laboratories that conduct quantum research and development, (2) the extent to which the primary defense laboratories conducting this research and development followed leading practices for strategic workforce planning, and (3) how DOD's STEM education programs have contributed to quantum workforce development and the extent to which DOD has monitored these contributions.GAO analyzed defense laboratories' workforce data and workforce planning activities, compared workforce planning activities to GAO leading practices for strategic workforce planning, and administered a questionnaire to 86 DOD STEM education programs. GAO received responses from 59 programs.
- Science & Tech Spotlight: Ocean Warming Why This MattersThe ocean is critical to human society and life on Earth. Its currents help shape the weather and regulate the climate. Its fisheries feed billions of people and accounted for 1.7 million jobs and $253 billion in sales in the U.S. in 2020. But the ocean is warming, which could lead to flooding, declining fisheries, more severe weather, and other negative consequences.The ScienceWhat is it?  According to NASA, 2022 was the ocean's warmest year since modern record keeping began in 1955. Estimates indicate that the oceans account for more than 90 percent of the Earth's warming since at least 1971, mostly near the surface between 0 and 700 meters deep. To measure how much the oceans have warmed, researchers use various temperature sensing instruments in the water as well as satellites that measure sea level. Researchers then use these measurements to calculate the average heat gained in the oceans worldwide.When sunlight reaches the Earth’s surface, the oceans absorb most of this energy and store it as heat. Water has a much higher heat capacity than air, so the oceans absorb larger amounts of heat energy with only a slight increase in temperature. The oceans’ ability to store heat is a crucial factor in moderating the pace of atmospheric warming. This heat is initially absorbed at the surface, but some of it eventually spreads to deeper waters. Increased heat absorption can change ocean currents that are driven by differences in temperature. Currents move heat around the world, influencing climate patterns and ecosystems.What are some concerns?  Ocean warming causes sea level rise through thermal expansion and by melting glaciers and ice sheets. Sea level rise can damage infrastructure and make coastal communities more vulnerable to storm surges and flooding. Global average sea level has risen 8-9 inches since 1880. In places such as Norfolk, Virginia; Miami Beach, Florida; and the San Francisco Bay Area, California, substantial investments are being made to cope with frequent coastal flooding that disrupts daily life and business operations.Warmer ocean water also provides more energy for storms to develop and intensify. From 2000-2021, tropical cyclones in the U.S. caused $1 trillion in damages and 6,200 deaths. Ocean warming could result in even greater damage to property, infrastructure, and human health and safety.Marine heatwaves—periods of abnormally warm ocean temperatures—can be caused by natural variability, but studies report that ocean warming is making them more frequent, intense, and prolonged. Marine heatwaves can stress and kill marine life and disrupt ecosystems. For example, they can cause coral bleaching, which can lead to the death of coral reefs and the ecosystems that depend on them.As the ocean warms, some marine species move to cooler waters because they can only tolerate certain temperatures. Such changes can disrupt food chains and ecosystems. This can lead to a decline in fish populations and have cascading impacts on other marine life and the industries and communities that rely on them. For example, higher temperatures in the Bering Sea have been linked to a population collapse of snow crabs, with 10 billion crabs disappearing between 2018 and 2021.Ocean warming can lead to more harmful algal blooms, which can produce toxins that can sicken people who eat seafood. These toxins can cause a range of health problems, including nausea, vomiting, diarrhea, and neurological problems.Figure 1. Process of ocean warmingWhat are the knowledge gaps? Global ocean heat inventory.  Due to insufficient reliable measurements, there is limited knowledge about current and historical temperatures and warming across different oceans and depths. This leads to large uncertainties related to weather forecasts and prediction of extreme weather events, long-term impacts of ocean warming and adaptation capacity on ecosystems and species, and related management and governance issues.Marine ecosystem impacts.  The impact of ocean warming on marine ecosystems is not fully understood. This includes the effects of warming on marine biodiversity and food webs.Ocean currents and circulation.  The effects of ocean warming on ocean currents and circulation patterns are not well understood. Changes in these patterns could have significant impacts on global climate and weather patterns. Ocean currents play a critical role in regulating Earth's climate by transporting heat from one part of the world to another.ChallengesUnderstanding, predicting, and communicating complex systems.   Scientists attempting to understand ocean warming are faced with complex systems with many interdependent parts, making it challenging to understand and predict how these systems will change over time. In addition, natural variability in the global climate, limitations of climate models, and measurement errors from imprecise instruments add to the complexity and limitations of predicting ocean warming.   Communicating these complexities and uncertainties   is also difficult.	Tipping points.  Some   researchers predict that there are several potential tipping points, or thresholds, beyond which changes could become irreversible. Examples include the collapse of the Greenland ice sheet, the collapse of the West Antarctic ice sheet, tropical coral reef die-off, and permafrost thawing. Studies of Earth's history, using tree rings, ocean sediments, and ice cores, show tipping points occurring through large, rapid changes over the course of decades or as little as a few years. One such tipping point may have been reached about 12,000 years ago, abruptly ending a 1,000 year long, dry, cold period, and is associated with the extinction of over 70 percent of large mammals in North America. Understanding, predicting, and preventing these tipping points is complex.	Effects on federal and military facilities and operations.  Sea level rise and extreme weather events threaten some government property. These events, as well as changes to shipping routes due to changes in ice sheet formation, also affect military and rescue operations and could pose new security risks.OpportunitiesResearch.  Scientists may be able to improve our understanding of these complex systems by improving monitoring and strengthening research to address knowledge gaps.	Coordination.  Governmental and nongovernmental entities could coordinate to better understand and adapt to ocean warming and to mitigate its effects.	Nature-based solutions.  Protecting and restoring marine and coastal ecosystems,   such as planting kelp and seaweed forests, could help enhance the ocean's capacity to absorb greenhouse gases that contribute to atmospheric and ocean warming. Restoring mangroves and other coastal habitats can also sequester carbon and act as a barrier against storm surges and coastal erosion.Figure 2. Nature-based solutions to adapt to ocean warming Policy Context and QuestionsHow can research initiatives improve data collection, modeling, and simulation related to ocean warming?	What strategies could be employed to improve intergovernmental and international coordination to address the causes and effects of ocean warming?	What policies, programs, or strategies could be used to help communities adapt to the effects of ocean warming?For more information, contact Brian Bothwell at (202) 512-6888 or bothwellb@gao.gov.
- Small Business Research Programs: Information Regarding Subaward Use and Data Quality What GAO FoundSeveral factors determine which first-tier subawards must be reported by recipients of Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) awards under the Federal Funding Accountability and Transparency Act. These factors include the value of the subawards, applicable exemptions, and varying definitions of first-tier subawards. According to small business representatives that GAO interviewed, a lack of awareness and understanding by small businesses may also affect reporting rates. Based on GAO's review of USAspending.gov, few SBIR/STTR awards (about 10 percent) display subawards. Limitations on first-tier subaward reporting are a government-wide issue and not unique to SBIR/STTR awards (see GAO-24-106237).SBIR/STTR award recipients commonly partner with other entities, including subcontractors or consultants, to assist with a portion of the work for contracts, grants, and cooperative agreements. Based on a sample of fiscal year 2019 awards, GAO estimates that approximately 70 percent of SBIR/STTR Phase II awards—which are used to advance concepts after feasibility is assessed—included the use of subcontractors or consultants. In addition, GAO estimates that about 16 percent of the total award value across all Phase II awards went to subcontractors and about 2 percent went to consultants.Estimated Percentage of Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) Awards That Used Subcontractors or Consultants, Fiscal Year 2019SBIR/STTR award recipients use private sector entities, academic institutions, and nonprofits for a variety of support. GAO estimates, for example, that 29 percent of awards included one or more subcontracts to private sector entities and 24 percent included one or more subcontracts to academic institutions. Award recipients use subcontractors and consultants for a wide variety of purposes, such as providing technical insights and expertise; assisting with design activities; developing systems, products, or software; and performing testing and evaluation of technological innovations.Why GAO Did This StudySmall businesses are important drivers of economic growth, but they can face challenges in accessing capital to fund research and development (R&amp;D). Congress established the SBIR/STTR programs to enable small businesses to undertake and obtain the benefits of R&amp;D. Agencies issue SBIR/STTR awards as contracts, grants, and cooperative agreements. The award recipients may enter into agreements—subawards—with other entities to carry out part of the work or to provide needed materials, supplies, or services.The SBIR and STTR Extension Act of 2022 includes provisions for GAO to evaluate subcontracting practices of SBIR/STTR award recipients. This report (1)&nbsp;examines factors that affect the reporting of SBIR/STTR first-tier subaward data on USAspending.gov; and (2)&nbsp;estimates the extent to which SBIR/STTR award recipients used subawards, the general purposes for these subawards, and which types of entities received them.GAO reviewed a generalizable sample of 198 fiscal year 2019 SBIR/STTR awards and interviewed agency officials from the Small Business Administration and 11 agencies that participate in the SBIR/STTR programs. GAO also interviewed a non-generalizable sample of 10 SBIR/STTR award recipients to obtain their perspectives on subaward reporting requirements.For more information, contact Candice N. Wright at (202) 512-6888 or WrightC@gao.gov.
- Small Business Research Programs: Agencies Are Implementing Programs to Manage Foreign Risks and Plan Further Refinement What GAO FoundThe Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs face risks of foreign governments seeking to illicitly acquire U.S.-funded research and technologies. To help address such risks, the SBIR and STTR Extension Act of 2022 (Extension Act) required participating agencies to develop a due diligence program to assess aspects of small businesses seeking a federally funded award. In March 2023, the Small Business Administration (SBA) issued a set of 12 best practices that provide general guidance on what agencies should consider in developing their programs. The best practices, which were developed in collaboration with other agencies, cover different types of risks, such as foreign ownership, employee affiliations, and cybersecurity practices. SBA also developed a set of standardized disclosure questions about foreign affiliations or relationships to foreign countries that SBIR/STTR applicants must answer to help participating agencies assess foreign influence.Examples of Best Practices SBA Issued for SBIR/STTR Participating Agencies' Due Diligence Programs to Address Foreign RisksAll 11 agencies participating in the SBIR and STTR programs established their due diligence programs by June 27, 2023, as required by the Extension Act. GAO found that agencies plan to assess all new proposals or applications that lead to SBIR/STTR awards using a documented risk-based approach—as called for by SBA best practices. These approaches vary. Some agencies plan to adapt existing due diligence review processes. Others plan to apply reviews based on different phases of technology development or require disclosures from a wider range of individuals who can significantly influence the research on a particular project. Regardless of these differences, each participating agency plans to assess, among other things, the cybersecurity practices and foreign ownership (e.g., financial ties and obligations) of small business award applicants.All 11 participating agencies plan to refine their due diligence programs through implementation experience. Agency officials identified several potential areas of refinement, such as hiring additional staff, supporting additional training, acquiring vetting tools, and addressing any effects of due diligence reviews on the timeliness of award issuance.Why GAO Did This StudySmall businesses are important drivers of economic growth and innovation, but they can face challenges accessing capital to fund research and development. Through the SBIR and STTR programs, participating agencies award funding to small businesses that helps bring important technologies to market. However, certain foreign governments are actively working to illicitly acquire such technologies.The federal government has taken a number of recent actions to help agencies, universities, and businesses counter foreign influence on federally funded research by implementing due diligence activities to identify and mitigate risks. The Extension Act builds on these actions by requiring each relevant federal agency to establish and implement a due diligence program to manage foreign risks.The Extension Act also includes a provision for GAO to issue a series of reports on the implementation of and best practices for these programs. This report, the first in the series, examines (1) SBA's efforts to develop best practices for participating agencies' due diligence programs, in collaboration with others; and (2) participating agencies' efforts to design and implement their due diligence programs.GAO interviewed officials from SBA, the 11 participating agencies, and other relevant federal entities. GAO also analyzed information related to the development of SBA's SBIR/STTR due diligence best practices and individual agencies' due diligence programs.For more information, contact Candice N. Wright at (202) 512-6888 or WrightC@gao.gov.
- U.S. Army Corps of Engineers: Use of Other Transaction Agreements in Water Development Projects What GAO FoundThe Water Resource Development Act of 2022 (Act) authorized the U.S. Army Corps of Engineers (Corps) to use other transaction agreements to carry out certain projects to support research activities for its Civil Works program. Other transaction agreements are those other than procurement contracts, cooperative agreements, and grants. Other transaction agreements are generally not subject to the same federal laws and regulations applicable to federal contracts or financial assistance. This allows agencies to customize their other transaction agreements to help meet project requirements and mission needs.Corps officials told GAO that they have drafted guidance for the use of other transaction agreements and that this guidance is under review by the Assistant Secretary of the Army for Civil Works. According to these officials, the draft guidance relevant to Civil Works projects is similar to existing guidance that pertains to the Corps' military program. These officials further said that an Army-wide review of the department's authorities for other transaction agreements is in progress and must be completed before the Corps can finalize its guidance.According to an official from the Office of the Deputy Assistant Secretary of the Army for Procurement, this Army-wide review is expected to be completed in November 2023. Corps and Assistant Secretary of the Army for Civil Works officials said they plan to issue their guidance for use of other transaction agreements for Civil Works projects as early as the end of 2023, subject to the completion of the Army-wide review.Once guidance is in place, Corps officials said that the U.S. Army Engineer Research and Development Center will oversee the implementation of other transaction agreements. According to Corps officials, such agreements could be used to, for example, study how automation could allow one operator to remotely operate multiple canal locks, presenting potentially significant cost savings. In another example, these agreements could be used to study different concrete mixes that produce fewer adverse environmental effects than existing materials.Why GAO Did This StudyLocated within the Department of Defense, the Corps has both military and civilian responsibilities. The Act authorized the Corps to use other transaction agreements to carry out prototype projects and follow-on production transactions to support the basic, applied, and advanced research activities of its civilian Civil Works mission. These research efforts can aid the Corps' management of water resources infrastructure including dams, levees, hurricane barriers, and floodgates across the U.S.The Act also includes a provision for GAO to report on the Corps' use of its other transaction agreement authority for its Civil Works program. This report examines the status of the Corps' efforts to implement and use its other transaction agreement authority for its Civil Works program. To determine this information, GAO reviewed relevant Department of Defense documents and GAO reports and conducted interviews with Corps and Department of Defense officials about the status of guidance and implementation efforts.For more information, contact Brian Bothwell at (202) 512-6888 or BothwellB@gao.gov.
- Small Business Research Programs: Most Agencies Allow Applicants to Define Needs and Propose Solutions What GAO FoundSeven of 11 federal agencies participating in the Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs used open topics from fiscal years 2019 through 2021. In response to solicitations with open topics, small businesses submit proposals that both define research needs and propose solutions to address them. The remaining four agencies only used conventional topics. For these topics, agencies define needs and small businesses propose solutions. Officials from the 11 agencies said they expect to continue their existing use of topic types in future years.Over the 3-year period, open topic awards accounted for about 40 percent of all awards and $4.1 billion. The percentage of awards originating from open topics increased from 36 percent in 2019 to 40 percent in 2020 and 46 percent in 2021.Open and conventional topic awards differed in terms of small business participation from fiscal years 2019 through 2021, according to GAO analysis of award data. Based on several GAO analyses, businesses receiving conventional topic awards tended to receive multiple awards, which could indicate that open topics promote a more competitive environment.Number of Businesses Receiving Open and Conventional Topic Awards in Small Business Research Programs, Fiscal Years 2019–2021Open topic awards also went to a higher percentage of small businesses owned by veterans and individuals from socially and economically disadvantaged groups, compared to conventional topic awards, but additional factors could explain this difference. For example, one agency stated that nontraditional businesses are less prevalent in certain fields, such as physics. This could contribute to lower proportions of nontraditional businesses receiving awards from agencies working in physics, regardless of the agencies' use of open topics.In deciding whether to use open or conventional topics, agencies reported considering their goals for using the SBIR/STTR programs and available resources. For example, agencies aiming to meet the goal of increasing applicant competition may consider using open topics, according to officials. Additionally, agencies consider the availability and expertise of proposal reviewers, which is especially important given the greater variation in the types of technologies covered in research proposals for open topics.Why GAO Did This StudySmall businesses are important drivers of economic growth, but they can face challenges accessing capital to fund research and development. Through the SBIR and STTR programs, agencies provide awards to small businesses to spur technological innovation, among other goals. For these awards, agencies release solicitations that include topics. Small businesses submit proposals with technical solutions to problems. Under open topics, the small businesses define the problems, whereas under conventional topics, agencies define the problems.The SBIR and STTR Extension Act of 2022 includes provisions for GAO to review agencies' use of open topics. This report examines, among other things, (1) the extent to which agencies have used open topics, (2) how open and conventional topic awards compare, and (3) agency considerations about which topic type to use.GAO compiled and analyzed data from multiple sources, including the 11 participating agencies and the Small Business Administration (SBA). The scope of the review covers fiscal years 2019 through 2021, because SBA award data for fiscal year 2022 were not yet available at the time of GAO's review. GAO also reviewed agency documentation and interviewed officials from the 11 participating agencies and SBA.For more information, contact Candice N. Wright at (202) 512-6888 or wrightc@gao.gov.
- Antiviral Drugs: Economic Incentives and Strategies for Pandemic Preparedness [Reissued with revisions on Sept. 29, 2023] What GAO FoundAntiviral drugs help the body fight off harmful viruses and can ease symptoms and shorten the length of an infection. Some of these drugs can be broadly acting—that is, they can treat infections from multiple viruses. This makes them especially valuable in a future pandemic where the viral threat is unknown in advance. Some scientists predict that viruses are most likely to be the source of the next pandemic because they are able to spread rapidly and because there are fewer treatment options for viruses.The National Institutes of Health (NIH) has identified several viral families that have potential to cause future pandemics. However, as of May 2023, no known drugs for a number of these viral families had been approved or were in clinical trials funded by the Department of Health and Human Services (HHS), according to HHS. GAO identified a number of technologies to speed antiviral drug development, such as using artificial intelligence to identify drug candidates.State of Drug Development for Viral Families with High Potential to Cause Future PandemicsNote: Monoclonal antibodies are lab-produced proteins that can be used to treat or prevent viral infections. Graphic refers to clinical trials funded by either the Biomedical Advanced Research Development Authority or the National Institute of Allergy and Infectious Disease within the Department of Health and Human Services (HHS).Experts GAO spoke with noted that a number of factors complicating the market’s efficient functioning make it unlikely that market forces alone will induce the development of antiviral drugs at levels that would maximally benefit society and aid future pandemic preparedness. However, policymakers could use several mechanisms to incentivize investment in antiviral drugs. “Push” mechanisms, such as research grants, can incentivize early research, while “pull” mechanisms, such as purchase commitments, can incentivize the production of completed antivirals.In 2022, the White House issued the National Biodefense Strategy and Implementation Plan, which called for the development of two novel antiviral drugs. Experts GAO spoke with also identified a number of approaches to guide the use of economic incentives. These approaches include developing (1) a number of antiviral drugs to the point of phase 1 clinical trials, which could be less costly than developing fully approved drugs; (2) a wide range of antiviral drugs that act against different parts of the viral lifecycle; and (3) broadly acting antiviral drugs for a wide range of pathogens.GAO identified three policy options that may help to spur antiviral drug development. These policy options are provided to inform policymakers—including Congress, federal agencies, state and local governments, academia, industry, and international organizations—about potential approaches and actions to address gaps highlighted in this technology assessment. Policymakers would need to consider how to align these potential actions with existing federal programs and initiatives. See below for a summary of policy options as well as related opportunities and considerations.Policy Options to Facilitate the Development of Antiviral Drugs for Future PandemicsPolicy option			Opportunities			Considerations					Create a strategy to focus on developing diverse antiviral drugs to respond to pandemics caused by the most dangerous pathogens (report p. 33).			&nbsp;			Policymakers could ensure focus on pathogens and pathogen families, such as flaviviruses and bunyaviruses, that are most likely to cause severe future pandemics, with an emphasis on pathogen families for which no drug candidates are currently in clinical trials.									Designing hypothetical pandemic scenarios could help identify the relative likelihoods that an outbreak could quickly become severe and widespread, indicating which antiviral drugs may need to be stockpiled and which need to be developed only to the point of phase 1 clinical trials.				Focusing attention on viral families with no drug candidates in clinical trials could spur research and development.				Developing drug candidates through phase 1 clinical trials would provide information on the initial safety profile and side effects of different doses in advance of potential pandemics.									Not every possible pandemic scenario can be considered, and a pathogen that may cause a future pandemic might not appear on current lists of potential pandemic pathogens.				It is difficult to predict how a novel pathogen may travel across populations. Therefore, assessing the stage to which each drug candidate should be developed and how much manufacturing might be needed may also be difficult.								Assign to a new or existing entity the authority to lead, implement, and be accountable for identifying and developing antiviral drugs for pathogens or pathogen families of greatest risk (report p. 34).			&nbsp;			Policymakers could ensure that this entity has authority and accountability for making decisions and allocating resources to implement a strategy for addressing pathogens and pathogen families informed by pandemic scenarios and the risks they present.									A single, centralized entity could direct strategic investments for pandemics, increase communication across federal agencies and with drug developers, and facilitate greater accountability for more effective allocation of resources and response to the spectrum of pandemic threats.				A single entity that is explicitly accountable for pandemic preparedness and response could effectively mitigate pandemic risks and enhance government efforts to deal with uncertain but likely pandemic risks. When multiple entities share fragmented responsibilities, no one entity is accountable for the ultimate success of the effort.									Because authorities among HHS entities vary, designating an entity with responsibility and authority may require new directives or legislation.				To be effective, any entity with such authority for identifying and developing pandemic antiviral drugs would need to coordinate or align its efforts with related efforts, such as the development of vaccines, diagnostics, and surveillance.								Implement economic incentives to develop antiviral drug candidates and spur new drug-development technologies (report p. 35).			&nbsp;			Policymakers could use various push and pull economic incentives to encourage the development of pandemic antiviral drugs, based on the economic costs of scenarios, scientific opportunities, and pathogen priorities. They could also consider other policy priorities, such as drug accessibility.			&nbsp;			Policymakers could use economic incentives to stockpile certain drugs in advance as well to ensure sufficient manufacturing capacity.									Economic incentives could address the identified challenges related to critical market failures that have limited investment in pandemic antiviral drugs. Through such actions, economic and societal costs from viral pandemics could be meaningfully reduced.				Manufacturing capacity for pandemic antiviral drugs may be useable for similar antiviral drugs with an existing commercial market.				New drug development technologies spurred by investment in pandemic antiviral drugs may allow for the treatment of nonpandemic infectious diseases, especially diseases caused by pathogens in the same family as those targeted for antiviral development.									Periodic updates to evaluations of pandemic risks would be needed to guide the deployment of economic incentives for new antivirals and drug development technologies and the extent of investment needed as new pandemic pathogens arise.				Evidence of the full range of potential pandemic costs may be unavailable or insufficient for precise determinations of the level of investment needed to achieve certain levels of preparedness.					Source: GAO. | GAO-23-105847Why GAO Did This StudyPandemics impose large human and economic costs on society. According to the World Health Organization, almost 7 million people died worldwide during the COVID-19 pandemic. The International Monetary Fund estimates that by 2024, COVID-19 will have reduced global economic output by $13.8 trillion relative to prepandemic forecasts. Scientists have predicted that another pandemic is highly likely.The CARES Act included a provision for GAO to report on the federal response to the COVID-19 pandemic. This report discusses incentives and strategies to enhance antiviral drug investment.GAO reviewed economic and scientific literature relevant to these areas. GAO also interviewed stakeholders and experts with perspectives on the science and economics of antiviral drug development, including those at HHS. In addition, GAO convened, with assistance from the National Academies of Sciences, Engineering, and Medicine, a 2-day meeting of 16 experts.GAO is identifying policy options in this report.For more information, contact Michael Hoffman at (202) 512-2700, HoffmanME@gao.gov or Candice Wright at (202) 512-6888, WrightC@gao.gov.
- Science and Engineering Statistics: Improved Communication Needed with Stakeholders on Data Needs What GAO FoundTo address its responsibilities as a federal statistical agency, the National Center for Science and Engineering Statistics (NCSES) evaluates its data quality and coordinates with agencies about current data needs. NCSES evaluates data accuracy, relevance, and timeliness, and it reports the results of these evaluations. To supplement its internal evaluation efforts, NCSES contracts with the research and evaluation community. These external evaluations can identify priority focus areas to ensure data remain relevant and can result in recommendations to improve processes. NCSES also coordinates with federal agencies, primarily through interagency groups, to communicate and plan new data collections and standardize data collection measures. Through its evaluation and coordination activities, NCSES interacts with a range of stakeholders to address science and engineering statistical needs (see figure).Examples of National Center for Science and Engineering Statistics (NCSES) StakeholdersTo identify emerging data needs, NCSES conducts research activities, but it has not communicated with stakeholders about agency priorities. In 2019, NCSES developed a strategic communication plan to guide engagement with stakeholders, but it has not updated this plan to account for new priorities. NCSES has also developed priority-setting documents, including a learning agenda, which identifies priority questions for evidence building. But NCSES has not engaged with external stakeholders about these priorities, potentially missing opportunities to better address needs, such as enabling academics to align their research to relevant policy questions. Communications strategies designed to ensure consistent, broad outreach could expand users' awareness of NCSES planned data collection, which could, in turn, more effectively enlist opportunities for stakeholders to provide input to ensure the relevance of these data.Why GAO Did This StudyGovernments at all levels, businesses, and the public rely on federal statistics to inform evidence-based decisions and enable the development of sound policies. Housed within the National Science Foundation, NCSES is the nation's source for science and engineering information and statistics. NCSES produces data and analysis on how the United States compares to other countries on key topics such as producing science and engineering graduates and spending on research and development in the context of rising global economic competition.The Research and Development, Competition, and Innovation Act   includes a provision for GAO to examine to what extent NSF has processes to ensure NCSES data and analyses meet current and future needs. This report (1) describes how NCSES ensures its data and analyses meet data quality standards including relevance, and (2) evaluates the extent to which NCSES has processes to identify emerging needs.GAO reviewed NCSES planning documents, and interviewed NCSES officials and stakeholders of its statistical products. GAO compared NCSES processes to relevant standards and guidelines published by Office of Management and Budget and National Academies of Sciences, Engineering, and Medicine.
- Science & Tech Spotlight: Drone Swarm Technologies Why This MattersDrone swarm technologies allow groups of drones to coordinate with each other, often without direct human control. Potential civilian applications include fighting wildfires and finding missing persons. But advances are needed in computing and communication to realize these applications, and the technology may raise safety, cybersecurity, and privacy concerns.The TechnologyWhat is it?  Drones are uncrewed aerial vehicles ranging in size from an inch to a wingspan of over 130 feet. Drone swarm technologies coordinate at least three and up to thousands of drones to perform missions cooperatively with limited need for human attention and control. For example, an aerial drone swarm could potentially assist with controlling a wildfire, assessing damages, finding access points, and suppressing the fire by raining firefighting liquids on it—all with minimal human direction. Drone swarms may be more efficient and robust for certain applications than single drones because swarms can complete a variety of tasks in parallel without human supervision. And they can continue operating if individual drones become inoperable.How does it work?  Drone swarms integrate advanced computer algorithms with local sensing and communication technologies to synchronize multiple drones to achieve a goal.Drone swarms can use various methods of command and control, including preprogrammed missions with specific predefined flight paths, centralized control by a ground station or a single control drone, or distributed control where the drones communicate and collaborate based on shared information. More advanced methods of control include swarm intelligence, inspired by the collective behaviors of insect colonies and flocks of birds, as well as artificial intelligence techniques to teach drone swarms to respond to new or unexpected situations.Figure 1. Methods of drone swarm command and control How mature is it?  Drone swarm technologies and algorithms have become more mature in recent years. Advancements in artificial intelligence and machine learning have improved decision-making and obstacle avoidance. High-speed communications technologies such as 5G and 6G networks have improved real-time data sharing among devices. Other advancements include energy efficient components, such as lighter materials and energy efficient motors, as well as advanced sensing technologies for environment mapping. In addition, there are now high-resolution cameras and infrared sensors for surveillance, reconnaissance, and search and rescue.Despite these advances, drone swarm use remains limited due to a number of challenges. Most current drone swarm applications are still relatively simple. For example, aerial light displays are conducted with preplanned motions. Tasks such as tracking and determining the positions of multiple drones in uncontrolled environments still pose a significant challenge for drone swarm technologies. Weather conditions in emergency management situations like hurricanes or wildfires could exacerbate these challenges.OpportunitiesCurrent and potential civilian applications of drone swarm technologies include:Agriculture:  Drone swarm technologies could plant seeds, identify disease outbreaks by surveilling large areas, and deploy treatments such as fertilizers to crops.	Emergency management:  Responders could use drone swarms to find missing persons and deliver emergency care and supplies during natural disasters.	Drone swarms could also help firefighters track and control the spread of wildfires and collect information about damages, access points, and more.	Entertainment:  Event organizers have used   drone swarms for entertainment as an alternative to fireworks. Doing so can mitigate debris, pollution, fires, and disturbances to animals and humans.Figure 2. Current and potential civilian applications of drone swarms ChallengesSafety and security:  Drone swarms can operate with minimal human intervention, but human intervention with control systems may be necessary for sensitive missions, such as those that could put humans in danger. Federal law prohibits drone swarm operation in restricted areas or for illegal or nefarious activities like spying, cyber-attacks, or deployment of improvised explosive devices. Operators must obtain a waiver to operate a drone swarm, as current regulations do not permit a person to operate more than one drone at the same time.	Privacy and cybersecurity:  Drone swarms collect information about their surroundings, so protocols need to be in place to protect against the collection and storage of certain information, such as photographs, videos, or sound recordings of individuals. Cybersecurity measures could help ensure drones are not hijacked or hacked by bad actors and used for malicious purposes.	Technical advancements:  Some applications will require miniaturization of hardware, such as sensors, as well as improved computing power. Advancements in algorithms could better simulate swarm behavior and improve connectivity, communications, and decision-making among drones.Policy Context and QuestionsWhat actions could be taken to ensure drone swarm technologies operate safely?	What threats may arise from the use of drone swarm technologies for illegal activities, and how may they be mitigated?	What standards may be used or developed to ensure (1) the privacy of information that could be collected by drones and (2) appropriate cybersecurity protections?	How could research initiatives address key technical challenges and advance high-value, secure, and ethical applications for drone swarms?For more information, contact Brian Bothwell at (202) 512-6888 or bothwellb@gao.gov.
- Chemical Weapons: Status of Forensic Technologies and Challenges to Source Attribution What GAO FoundWhen a chemical weapon is used, investigators may use forensic technologies to detect that it was used, identify the chemical, and help attribute it to a likely source. Most technologies for chemical identification are mature. Some technologies for attributing chemical agents based on chemical analyses are under development. For example, a wide variety of laboratory-based and fieldable instruments for chemical identification are in commercial use. Investigators also use data from these instruments to help attribute chemicals to their potential sources, but there are limits to the information existing technologies can provide.Key technologies to identify and attribute chemical weapons to a likely sourceTechnology			Information it provides		Chromatography (Gas or Liquid)			Separates and identifies chemicals in a sample mixture. Mature for identification of chemical agents, can inform attribution.		Nuclear Magnetic Resonance Spectroscopy			Determines arrangement of atoms in a molecule. Mature for identification of chemical agents, developing for complex mixtures. Can inform attribution.		Mass Spectrometry			Measures mass-to-charge ratios to identify chemicals in a mixture. Mature for identification of chemical agents, can inform attribution.		Impurity Profiling			Identifies key impurities to link samples to production process or precursor. In development for attribution.		Isotope Ratio Methods			Measures ratio of stable isotopes in a sample to link to a precursor. In development for attribution.		Source: GAO. | GAO-23-105439GAO found several challenges that can hinder identification and attribution of chemical agents, including the following:Poor samples: In some instances, investigators cannot obtain useful chemical information because samples are too small, dilute, or degraded.	Limited reference data: Generally, chemical identification methods rely on comparison to data from known chemicals, called reference data. However, reference data can be limited because they are resource intensive to collect, analyze, and archive. Additionally, reference data may be challenging to use in some instances, such as if they were developed using different experimental methods or stored in an incompatible format.	Lack of information sharing: Controls on information sharing are needed for national security concerns but can hinder collaboration among researchers in developing technologies and improving understanding of chemicals and their sources.	Limited coordination: Entities may not be aware of individual and laboratory expertise that could assist with identifying a chemical agent or its source. Researchers may unknowingly duplicate work, and opportunities to strengthen capabilities may be missed.GAO identified six high-level policy options in response to these challenges. These policy options are provided to inform policymakers of potential actions to address the challenges identified in this technology assessment. They identify possible actions by policymakers, which include Congress, federal agencies, state and local governments, academic and research institutions, and industry.Policy options to address challenges that hinder identification and attribution of chemical agentsPolicy Option			Opportunities			Considerations		Develop technology to aid with sampling			(report p. 35)Policymakers could encourage development of technologies that allow more rapid sample analysis or that slow sample degradation.						Could reduce likelihood of degradation by shortening time between sampling and analysis.				May reduce cost of analysis by minimizing the number of chemical signatures generated by sample degradation.									Small market size for certain instrumentation may limit industry interest in developing new technologies.				Testing of new technologies can be challenging due to the toxicity and reactivity of chemical agents.					Study known threats			(report p. 36)Policymakers could advance scientific knowledge on known chemical agents and threats.						Could support a faster response to an event and faster attribution to a source.				Could ensure that the U.S. has the appropriate workforce, capabilities, knowledge, and facilities to respond effectively to chemical incidents.									In some cases, additional knowledge is unlikely to help identify a chemical (e.g., due to highly degraded samples) or attribute it to its source (e.g., when there are many possible production processes for a given chemical agent).				Building the necessary body of knowledge could be resource intensive.					Further anticipate emerging threats			(report p. 37)Federal policymakers could foster development of technologies and approaches that help anticipate emerging chemical threats.						Could deter use of chemical agents and support a faster response in the event of use.				Emerging chemical threat assessment and characterization technologies could better prepare U.S. government entities and international partners for the unexpected.									A vast number of chemicals are potential emerging threats, and efforts would require careful prioritization.				Computational approaches would require security considerations to protect potentially classified chemicals.					Standardize data			(report p. 37)Policymakers could encourage standardization of future data collection and support efforts to modernize legacy data on chemical threats.						Could ensure that computational methods work on data from different sources, maximizing efficiency and potential insights.				Modernized legacy data may provide insights that are otherwise expensive or difficult to acquire.									Consensus on standard operating procedures may be difficult to achieve.				Legacy data may be challenging to use in certain circumstances, such as due to changes in experimental design, instrumentation, and analytical techniques over time.					Share information			(report p. 38)Federal and international policymakers could facilitate information sharing to increase collaboration on chemical agent forensic analyses.						Quickly making classification decisions about new and emerging information could improve appropriate research collaboration and information exchange.				Increased understanding of classification guidance could improve handling of classified information and allow researchers to more confidently know what information can be shared.									Classification systems are complex, and creating tools to assist in navigating them could be challenging.				Agencies need to balance facilitating additional information sharing with protecting information critical to national security.					Coordinate on chemical attribution			(report p. 40)Federal and international policymakers could encourage increased coordination between entities involved in chemical attribution.						Clear roles and responsibilities could enable each entity to quickly execute its mission in response to a chemical incident.				Collective development of robust methods and capabilities could increase international confidence in findings.									Federal agencies often have different goals and requirements, so efficiencies may be limited, even with increased coordination.				Coordination activities with multi agency participation could divert resources away from other tasks.					Source: GAO. | GAO-23-105439Why GAO Did This StudyDespite the Chemical Weapons Convention's ban on their use, chemical weapons have been used in the past decade in assassinations and on civilian populations. To identify the use of a chemical weapon and then attribute that weapon back to its source, researchers rely on several technologies for chemical analysis. Chemical analysis is one piece of an overall chemical weapon investigation.This report discusses (1) the status of key technologies available to identify a chemical agent or its source, including their strengths and limitations; (2) challenges researchers and investigators face in trying to identify a chemical agent or its source; and (3) policy options that may help address the challenges of using key technologies to identify a chemical agent and its source.To conduct this technology assessment, GAO reviewed key reports and scientific literature; interviewed government, intergovernmental, and academic representatives; conducted site visits; and convened two meetings of experts with the assistance of the National Academies of Sciences, Engineering, and Medicine. GAO is identifying policy options in this report.For more information, contact Karen Howard at (202) 512-6888 or HowardK@gao.gov.
- Leading Practices: Iterative Cycles Enable Rapid Delivery of Complex, Innovative Products Why This MattersAgencies are increasingly acquiring complex products, such as combined networks of hardware and software, which require new processes to design, produce, and deliver. GAO has found that to consistently deliver products with speed to users, acquisition programs for these networks—known as cyber-physical systems, such as aircraft and uncrewed vehicles—must adopt new approaches to evaluate performance and assess execution risks. Solutions, though, are unlikely to originate exclusively within government. Rather, identifying the practices that leading companies rely on to create cyber-physical products can provide crucial, cutting-edge information to acquisition leaders in government and, in turn, ultimately help frame changes to agencies’ acquisition processes.For over 20 years, GAO has made numerous recommendations to the Departments of Defense (DOD) and Homeland Security (DHS) and the National Aeronautics and Space Administration (NASA) to implement best practices for their major acquisition programs that underpin successful product development within leading companies. Over this time, agency implementation of these practices saved taxpayers tens of billions of dollars.Key TakeawaysLeading companies use iterative processes to design, validate, and deliver complex cyber-physical products with speed. Activities in these iterative cycles often overlap as the design undergoes continuous user engagement and testing. Knowledge about the product’s design is progressively refined and stored in a digital thread—a common source of information that helps stakeholders make decisions, like determining product requirements, throughout the product’s life.As they proceed, product teams refine the design to achieve a minimum viable product (MVP)—one with the initial set of capabilities needed for customers to recognize value. They use modern manufacturing tools and processes to produce and deliver the product in time to meet their customers’ needs.Leading Companies Progress through Iterative Cycles to Develop a Minimum Viable ProductSimilar to Agile software development, the iterative structure leading companies employ when developing complex, cyber-physical products revolves around companies rapidly designing, validating, and delivering products. These cycles are:Design modeling and simulation:&nbsp;During design modeling and simulation, product teams feed technical data from fast, iterative design cycles into the digital thread. Stakeholders—including users, engineers, and manufacturers and suppliers—use this information to confirm that the team has captured the right requirements and is on track to meet them.	&nbsp;Digital Thread Captures Information throughout the Product Life CycleFor example, one leading company establishes models based on high-quality data from physical engineering, and captures input from users and manufacturers to feed into the models. Another leading company collaborates with users to identify the one thing that will differentiate the product from others, which then functions as an indicator of product performance. The outcome of design modeling and simulation is a solution—in the form of an MVP—that can then be validated through testing.Validation: Leading companies validate the design using prototypes—including combinations of physical and entirely digital prototypes. This prototyping incorporates all hardware and software components to test the product’s integrated functionality in its operating environment. Sometimes companies do this by developing virtual representations of physical products—a process known as digital twinning—or by using 3D-printed parts to test performance.	For instance, one leading company uses a digital twin and virtual reality—immersion in a virtual environment using head-mounted displays—to enable users to step into a virtual vehicle and validate its design. Another leading company used a digital twin of an industrial motor drive to simulate its overload to the point of explosion. Compared with a physical test, which would have destroyed the prototype, developers could observe the specific point of explosion, locate defects, and fix them in the digital twin.	Production and Delivery: Leading companies develop the design to the point that it satisfies user requirements for a MVP. Product teams then stop designing hardware for the given MVP and prepare parts for production, recognizing that they can add functionality through software updates later. Companies use digital twinning to understand optimal factory design and manufacturing processes before the design enters production.	For example, one leading company uses digital twins to align parts assembly processes to ensure the robot handling the part can do so efficiently. The digital thread documents all the steps throughout production, from the design of the machinery and toolset, to the processes for manufacturing and assuring the product meets the company’s quality standards. Post-production user feedback informs further development of the next product.How GAO Did This StudyThree congressional mandates provide for GAO to annually assess selected DOD, DHS, and NASA acquisition programs, projects, and activities. In order to respond effectively to these mandates, GAO conducted this work to understand (1) how leading companies structure their development of complex, cyber-physical products; and (2) the specific practices that enable this structure to properly function.GAO identified 14 leading product development companies based on rankings in well-recognized lists; interviewed company representatives; reviewed supplementary documentation; and synthesized information to determine key product development structures and activities.
- Regenerative Medicine: Therapeutic Applications, Challenges, and Policy Options What GAO FoundRegenerative medicine offers the hope of being able to restore or replace cell, tissue, and organ functions affected by disease, injury, or aging. This may eventually help manage or cure many conditions that are currently considered chronic, untreatable, or terminal.Examples of Diseases and Regenerative Medicine Therapies That Might Address ThemGAO identified many challenges that may affect the development and use of regenerative medicine technologies and therapies including:Challenges related to standardization. Standards are rules, conditions, guidelines, or agreed-upon practices that are adopted within an industry to provide developers with a common framework and promote consistency. Developing regenerative medicine standards is challenging because these technologies and therapies are complex and rapidly evolving. In addition, standards require consensus from stakeholders, which may be difficult to obtain.Challenges related to regulation. The Food and Drug Administration (FDA) ensures the safety, efficacy, and security of human medical products in the U.S. through regulation. Regenerative medicine faces challenges related to regulation, including difficulty navigating a complex regulatory framework, uncertainty over which regulatory pathway is most appropriate for certain emerging technologies and therapies, and staffing shortages at FDA and collaborating agencies.Challenges related to manufacturing. Manufacturing is the creation of products from starting materials, in a way that is generally consistent and reproducible. It is a key step for many emerging technologies and therapies, but the cells, tissues, and organs used for regenerative medicine are complex and difficult to manufacture at scale. Other challenges to manufacturing include a lack of infrastructure and difficulty ensuring quality and consistency.GAO developed 11 policy options that could help address the challenges or enhance the benefits of regenerative medicine. These policy options are provided to inform policymakers of potential actions to address the policy challenges identified in this technology assessment. They identify possible actions by policymakers, which include Congress, federal agencies, state and local governments, academic and research institutions, and industry. Policymakers would need to consider the impacts these new technologies will have on existing federal programs that are already strained. We suggested possible federal components for the policy options.Selected Policy Options to Mitigate Challenges Associated with Regenerative Medicine Technologies and TherapiesSelected policy Option			Opportunities			Considerations					Invest in standards development.			(report p. 26)						This policy option could help address the challenge that standards require consensus.									Could streamline standards development, which may, in turn, accelerate innovation, increase product safety and reliability, accelerate regulatory review, and decrease costs of regenerative medicine therapies.									Existing organizations may not include all stakeholders, and stakeholders may hesitate to accept standards created without their input.				Industry stakeholders may hesitate to adopt standards if they perceive it will cost them a controlling position in the market.				Standards should be appropriately flexible to allow for innovation, while still being detailed and specific enough to support manufacturing of consistent, quality products.								Provide opportunities for increased interactions between regulatory experts (at FDA or in industry) and smaller companies, especially early in the development process (report p. 32)						This policy option could help address the lack of access to regulatory expertise.			&nbsp;									May provide more timely advice and avoid unnecessary delays or uncertainty by pursuing the wrong regulatory pathways or submitting data that do not meet regulatory requirements.									May require additional resources to bolster the workforce of regulatory scientists at FDA or public-private partnerships.				FDA may be limited in its ability to advise companies early in the process so as not to create a conflict of interest.								Consider whether changes to the framework for evaluating combination products and medical devices to accommodate emerging technologies and therapies may be necessary. (report p. 33)						This policy option could help address whether current regulatory pathways are sufficient for emerging technologies and therapies.									May encourage innovators, researchers, and developers of new products to provide valuable feedback to regulators.									Coordinating among stakeholders to consider changes to regulatory pathways may be time- and resource-intensive.				If such consideration leads to recommended changes to the framework, statutory and regulatory changes may be necessary.								Provide more oversight and feedback to suppliers to increase consistency in starting materials (report p. 40)						This policy option could help address inconsistency in starting materials for manufacturing.									May accelerate manufacturing by reducing variation in input materials.				May reduce the risk of failure during product development.									Starting material suppliers may lack incentives to follow standards if they lead to higher costs					Source: GAO. &nbsp;| &nbsp;GAO-22-105274Why GAO Did This StudyRegenerative medicine represents a paradigm shift in the medical field because it aims to restore or supplement function, rather than just treating symptoms, and opens the door for personalized therapies. GAO conducted an assessment of current and emerging regenerative medicine technologies and therapeutic applications. This report examines (1) current and emerging regenerative medicine technologies and therapies and their potential benefits, (2) challenges that hinder their development and use, and (3) policy options that could help enhance benefits and mitigate challenges associated with these technologies and therapies. GAO reviewed scientific and policy literature and other key reports; convened a 3-day expert meeting; interviewed subject matter experts and stakeholder groups including government agencies, such as the Department of Health and Human Services, non-government organizations, industry, academia, end user groups such as patient groups; and conducted site visits. GAO is identifying policy options in this report.For more information, contact Karen L. Howard at (202) 512-6888 or howardk@gao.gov.