Strengthening U.S.
AI Innovation Through anAmbitious Investment in NIST
Progress in artificial intelligencethat is central to promoting U.S. innovation. We argue that a strongly(AI) has rapidly advanced such that

resourced AI program at NIST — on the order of 22 additional

contemporary AI systems are poised

positions and an increase of $15 million over FY 2023 levels — canto transform a number of areas ofadvance research and development (R&D) through the creationsociety. Large language models

of a robust ecosystem of AI assurance. This would involve doubling(LLMs) in particular have become

the staff at NIST working on AI programs and would create a robust

increasingly powerful, and we stand

improvement in policy making capacity.

at an inflection point for widespreadintegration of these models into
everyday life. America’s ability toNIST’s work on standards development,1 the AI Risk Management
Framework,2 and the Face Recognition Vendor Test (FRVT)3 has set

harness their potential depends
on whether we can systematically

Positions Requested (Following Year) vs. Actual Number of Positions

evaluate the capabilities, limitations,5040sensible AI policy. Given this, weneed to significantly enhance thegovernment’s ability to build andNumber of positions

Effective measurement tools arealso a necessary precursor for

Anthropic Recommendationand potential harms of LLMs.

302010

use these means of quantitativeassessment.
The National Institute of Standards
and Technology (NIST), the nation’s
leader in measurement science andtechnical standards, is uniquely
positioned to lead this critical work

02020

2021

2022

2023

2024

YearIncrease requested but not received

Actual increase

BaseFig 1. Number of staff positions working on AI-related programs at NIST from 2020-2024. Red bars
represent the base number of positions each year, orange bars combined with yellow bars represent
requested increase in AI program positions, and yellow bars represent the actual increase in numberof positions over the previous year. Base positions for 2024 TBD and are presented as constantfrom 2023 staffing levels. Dashed black lines represent Anthropic recommendation of 22 additional
positions over FY 2023 (inclusive of the estimated 11 positions already requested by NIST).

1 National Institute of Standards and Technology. (2022). Towards a Standard for Identifying and Managing Bias in Artificial Intelligence.
https://www.nist.gov/publications/towards-standard-identifying-and-managing-bias-artificial-intelligence
2 National Institute of Standards and Technology. (2023). AI Risk Management Framework. https://www.nist.gov/itl/ai-risk-management-framework
3 National Institute of Standards and Technology. (2010). Face Recognition Vendor Test (FRVT). https://www.nist.gov/programs-projects/face-recognition-vendor-test-frvt

APRIL 2023

ecosystem. Congress can further

50

increase the impact of this work by40

making an ambitious investment inso, Congress will enhance publictrust in AI, provide the government
with confidence in the safety of AI
systems, and encourage industrial
competitiveness.

($ millions)

NIST in 2024 and beyond. In doing

3020

100

During the time NIST has been activein AI measurement efforts, we have

2020

2021

Funding increase requested but not received

2022

2023

Actual funding increase

2024
Base fundingFig 2. Amount of funding (in millions) for AI-related programs at NIST from 2020-2024. Red bars
represent the base budget each year, orange bars combined with yellow bars represent funding
increase requested by NIST for AI programs over previous year, and yellow bars represent actual
increase in funding awarded. 2024 amounts reflect the current FY 2024 budget request. Dashedblack lines represent Anthropic recommendation of a $15 million increase over FY 2023 (inclusiveof NIST’s $5 million request).

witnessed rapid and tremendous
technological progress. AI systems
now score in the 90th percentile ofa simulated bar exam,4 accelerate
biology research by predicting theSubsequent years exhibit unmet staffing requests and the data show

3D models of protein structures,5a concerning plateau in staffing levels from the start of 2022 onward

and generate photo-realistic images(Figure 1). Funding levels have been similarly modest (Figure 2).

from plain text descriptions.

The FY 2020 budget submission was the last time NIST’s funding

However, the government’s ability torequest was met in full, and financial support stagnated from FY

meaningfully assess these systems

2022 to FY 2023. The FY 2024 budget request represents only afor potential capabilities and risks

modest increase of $5 million dollars for AI-related programs9

hasn’t kept pace.7 In our review(down from a requested $25 million for FY 2021).10 This reducedof past budget submissions andfinancial support has happened against the backdrop of publicly

omnibus spending bills, we founddeployed, increasingly capable AI systems.

6a general under-resourcing andconcerning stagnation in funding for
AI programs at NIST.

While ambitious in scope, our recommendation is not
unprecedented — it represents approximately the same
number of positions as the request for FY 2021 and theOn personnel, the FY 2020 budget(partially met) funding requests for FY 2022 and FY 2023. We

submission appears to have beenencourage the Subcommittee on Commerce, Justice, Science, andthe last time NIST’s AI-related

Related Agencies, of both the U.S. House and Senate Appropriations
Committees, to consider this an investment in the development ofheadcount request was met (12 newpositions added at the start of 2021).

8safe and innovative AI systems that can benefit all Americans.

4 OpenAI. (2023). GPT-4. https://openai.com/research/gpt-4
5 DeepMind. (2021). AlphaFold. https://www.deepmind.com/research/highlighted-research/alphafold
6 Google Research, Brain Team. (2022). Imagen. https://imagen.research.google/
7 Whittlestone, J., & Clark, J. (2021). Why and How Governments Should Monitor AI Development. arXiv. https://arxiv.org/abs/2108.12427
8 National Institute of Standards and Technology. Budget Submission to Congress for Fiscal Years 2020, 2021, 2022, 2023, and 2024. Note that in some years,
AI-related programs are combined with other programs, so personnel and funding requests are approximate measures.
9 National Institute of Standards and Technology. (2023). Fiscal Year 2024 Budget Submission to Congress.
https://www.commerce.gov/sites/default/files/2023-03/NIST-NTIS-FY2024-Congressional-Budget-Submission.pdf
10 National Institute of Standards and Technology. (2020). Fiscal Year 2021 Budget Submission to Congress.
https://www.commerce.gov/sites/default/files/2020-02/fy2021_nist_ntis_congressional_budget_justification.pdf

APRIL 2023

Anthropic Recommendation

Funding Requested (Following Year) vs. Actual Funding

the foundation for an AI assuranceWith this additional resourcing, NIST could continue

productivity tools and chatbots. Recent large-scale AIand expand its work on AI assurance efforts like:

deployments have highlighted a unique characteristicof these systems, which is that they contain both•C ataloging existing AI evaluations anduseful capabilities and harmful behaviors that canbenchmarks used in industry and academia

not always be anticipated during the development• I nvestigating the scientific validity of existingprocess. The lack of available measurement

evaluations (e.g., adherence to quality control

techniques to identify and measure these behaviors

practices, effects of technical implementation

exacerbates this problem — harmful behaviors canchoices on evaluation results, etc.)

emerge unpredictably and may only become apparent•D esigning novel evaluations that address
limitations of existing evaluations

once the AI system is deployed in real-world settings.11•D eveloping technical standards for how to identify

To address this, NIST could develop a testbed

vulnerabilities in open-ended systems

similar to FRVT, but one focused on LLMs instead of•D eveloping disclosure standards to enhancefacial recognition systems. FRVT serves as a prooftransparency around complex AI systems•P artnering with allies on international standards
to promote multilateral interoperability

point in how the development of testbeds canboth enhance safety and promote innovation as
developers compete to build better technology.

•F urther developing and updating the AI Risk

With additional personnel and financial support,

Management Framework

NIST could create a voluntary, centrally run testbed

More resourcing will allow NIST to build out much-that helps developers catch risks prior to publicneeded testing environments for today’s generative

deployment and benchmark their AI systems against

AI systems, including LLMs that power various

the current state of the art.

Number of AI Incidents and Controversies, 2012–21
Source: AIAAIC Repository, 2022 | Chart: 2023 AI Index Report

260

Number of AI Incidents and Controversies

250

200

150

100500

2012

2013

2014

2015

2016

2017

2018

2019

2020

This figure does not consider AI incidents reported in 2022, as the incidents submitted to the AIAAIC database undergo a lengthy vetting process before they are fully added.

11 Ganguli, D., et al. (2022). Predictability and Surprise in Large Generative Models. ACM Digital Library. https://dl.acm.org/doi/abs/10.1145/3531146.3533229

2021

There is an urgent need for this work. AI systems are
rapidly moving from controlled lab settings into theABOUT USworld. Alongside this, we’ve seen a concerning rise

Anthropic is a public benefit corporationin the number of incidents where AI systems don’t

and AI safety research company that is

perform as intended or perform in harmful ways.

working to build reliable, interpretable,

With increased funding, NIST can build on its workand steerable AI systems.

12on responsible development standards and design
new guardrails to help avoid future incidents.
Ambitiously investing in NIST will be key to building
a robust ecosystem for AI assurance. In doing so,Congress can increase public confidence in AI
technology, enable a competitive R&D environment,
and foster a more productive U.S. economy. Anthropic
would be delighted to talk further about theimportance of a robust measurement capacity withpolicymakers both in the United States and abroad.

12 Maslej, N., et al. (2023) The AI Index 2023 Annual Report. AI Index Steering Committee, Institute for Human-Centered AI, Stanford University.
https://aiindex.stanford.edu/report/CONTACT: POLICY@ANTHROPIC.COM

APRIL 2023