Artificial Intelligence: A NIST strategic priority

1

Chuck Romine

Director, Information Technology Laboratory

February 12, 2020
Major advances in artificial intelligence

Raise productivity, enable 

more efficient use of 

resources, change the 

way we live and work, 

and increase creativity.

Negative impact on job, 

exacerbate the trend of 

rising inequality, and 

(even) threat to 

humanity. 

2
NIST will become an authoritative source of measurement tools, 

capabilities, and data necessary to define, develop, and evaluate 

Trustworthy AI. 

Applied

Revolutionizing metrology at NIST from 

experiment design to research results. 

Fundamental

Measure and enhance the 

trustworthiness of AI systems.

3
From innovation to application

Fundamental 

Research

Applied 

Research

Standards + Best 

Practice Guides Adoption

Image Credit: wsj.com

4
Coordination Activities

5 5
Interagency coordination and leadership

6

MLAI Subcommittee

Chaired by NIST, OSTP, DoE

AI Select Committee

Chaired by OSTP, NSF, DARPA

USG AI Standards Coordinator

Networking and Information 

Technology R+D (NITRD) 

AI Interagency Working Group 

National Security Commission on AI

Chief Technical Advisor
Collaboration with other agencies

Joint AI Center
Federal Engagement in Artificial 

Intelligence Standards

8
>32,000 SRM units sold per year ($20M)

~13,000 calibrations and tests per year

EO on Maintaining American Leadership in AI

Within 180 days...

www.whitehouse.gov/presidential-actions/executive-order-maintaining-americanleadership-artificial-intelligence/

Secretary of Commerce, through Director 

of NIST, shall issue a plan for Federal 

engagement in the development of 

technical standards and related tools in 

support of reliable, robust, and 

trustworthy systems that use AI 

technologies. 
By the Numbers

10

>400

WORKSHOP 

ATTENDEES

97

RFI 

RESPONSES

6

BREAKOUT 

SESSIONS

43

PUBLIC 

COMMENTS

2

DOCUMENTS

10

AUGUST

10
Recommended Actions

Bolster AI standards-related 

knowledge, leadership, and 

coordination among Federal 

agencies to maximize 

effectiveness and efficiency.

Coordination

Promote focused research to 

accelerate broader exploration and 

understanding of how aspects of 

trustworthiness can be practically 

incorporated within standards.

Research

Strategically engage with 

international parties to 

advance AI standards for 

U.S. economic and national 

security needs.

Engagement

Support and expand publicprivate partnerships to develop 

and use AI standards and 

related tools to advance 

trustworthy AI.

Partnership
Trustworthy AI

12 12
Foundational research for trustworthy AI
Technical requirements for trustworthy AI

accurate secure

explainable

robust

objective

14

reliable

and more ...
Secure AI

Terminology and Taxonomy of 

attacks and defenses for 

Adversarial Machine Learning. 

Testbed for secure AI.

Collaboration with MITRE.

IARPA, DARPA, and DoD 

JAIC.

Draft NIST IR 8269: 

Terminology and Taxonomy

15

Testbed to evaluate AI 

vulnerabilities.
Explainable AI

Forms the bases of addressing 

bias, transparency, security, 

safety and ultimately trust in AI 

systems. 

Draft for public comment 

planned for Spring 2020.

Developed principles of 

explainable AI.

Socializing with experts in 

the community.

16
Bias in AI

Define, identify, measure and 

mitigate bias in AI systems.

Publication for public comment 

Fall 2020.

Touch points of bias throughout AI 

development life cycle.

Workshop in Summer 2020.

17
Novel computational paradigms for AI

Establishing metrics and benchmarks 

for AI hardware. 

Foundational analysis of the 

computational capacity of a physical 

system.

Analysis and development of 

algorithms for spike-based 

computation.

Schneider, M.L., Donnelly, C.A., Haygood, I.W. et al. Synaptic weighting in 

single flux quantum neuromorphic computing. Sci Rep 10, 934 (2020)
AI at NIST: Spectrum sharing

Outcome

• 10 standards for service providers (Bronze Medal)

• NIST developed AI algorithms for more efficient detection 

of offshore radar signals relying on NASCTN spectrograms

• AI methods significantly more effective than energy 

detection (ESC)

Citizens Broadband Radio Service (3.55 – 3.7 GHz) for mobile 

broadband use.

• Current incumbents include Navy, satellite service providers and utilities.

• Incumbent signals are detected using automated detectors that look for 

energy rises in the electromagnetic spectrum, but these detectors (ESC) 

are not consistently discriminating enough, sometimes confusing other RF 

signals as radar or missing the radar signals altogether

Impact

• Revised occupancy statistics  greater accessibility for 

commercial providers (2 hrs  20 mins)

• May allow for more users and increase valuation of 

spectrum licenses.

• NIST developing curated data sets for industry use. Top 6 NIST News Story in 2019

https://www.nist.gov/programs-projects/35-ghz-spectrum-sharing
AI at NIST: Advanced materials discovery

Creating a high-fidelity database, Joint 

Automated Repository for Various Integrated 

Simulations, density functional theory (JARVISDFT), with more than 30,000 materials and 

500,000 properties to be used as training data. 

https://jarvis.nist.gov/
AI at NIST: Small angle neutron scattering

Creation of reference datasets for Small Angle Neutron 

Scattering (SANS) for machine learning applications as 

well as Neutron Instrument Automation through AI and 

Automatic Phase Diagram Mapping for Liquid 

Formations.

https://www.nist.gov/ncnr/neutron-instruments/small-angle-neutron-scattering-sans
AI at NIST: Manufacturing

Agile Robotics for Industrial Automation 

Competition (ARIAC)

“Nestor” – Natural language processing toolkit for 

manufacturing documents

Item(s) Problem(s)

Leak Replaced; 

Repaired

Broken; 

Low Pressure

High Pressure Coolant; 

Gauge; Low Pressure Line Cleaned

Leak Repaired

Hydraulic; 

Saw attachment; Seal

Hydraulic; SP#6 

Horseshoe;

Seal 

Cleaned and Annotated Text for Analysis

Solution(s)

“Major hydraulic 

leak at Sp#6 

horseshoe”

“Hyd leak at saw 

atachment” 

“HP coolant 

pressure at

75 psi”

“Replaced seal in saw 

attachment but still leaking –

Reapirs pending with ML”

“Bad Gauge / Low 

pressure lines 

cleaned ou”

“Repaired horseshoe 

seals”

Raw Text from Maintenance Documents
AI at NIST: Neuromorphic computing 

Simulation of a self-training single-flux-quantum 

neural network

Successful simulation of a fanout of 1-to-1,024 and fan-in of 1-to-256 at NIST. 

Although short of the human brain, it brings the technology in line with silicon 

neuromorphic computing. The fan-in of 256-to-1 is the same level that was used 

in the IBM True North chips.

https://www.nist.gov/publications/energy-efficient-single-flux-quantum-based-neuromorphic-computing
Planned Efforts

24
Our plan

To support the 

responsible design, 

development, and use of 

trustworthy AI 

Foundational AI research

Revolutionizing metrology 

across NIST

Launching an AI resource center 

for use by NIST researchers and 

industry
In the coming year

Foundational Research AI for Metrology AI Hardware

AI Standards Data and Tools AI workshops
QUESTIONS?

27