Fundamental and Applied 

Research and Standards for AI 

(FARSAI)

Briefing on NIST AI Research Program to VCAT 

Chuck Romine

5 June 2018

1
1956 Summer Research at Dartmouth

A PROPOSAL FOR THE DARTMOUTH SUMMER RESEARCH PROJECT ON ARTIFICIAL 

INTELLIGENCE

J. McCarthy, Dartmouth College

M. L. Minsky, Harvard University

N. Rochester, I.B.M. Corporation

C.E. Shannon, Bell Telephone Laboratories

August 31, 1955

We propose that a 2 month, 10 man study of artificial intelligence be carries out during the summer of 

1956 at Dartmouth College in Hanover, New Hampshire. The study is to proceed on the basis of the 

conjecture that every aspect of learning or any other feature of intelligence can in principle be so 

precisely describes that a machine can be made to simulate it. An attempt will be made to find how to 

make machines use language, form abstractions and concepts, solve kind of problems of now reserved 

for humans, and improve themselves. We think that a significant advance can be made in one or more 

of these problems if a carefully selected group of scientists work on it together for a summer.

www-formal.stanford.edu/jmc/history/dartmouth/dartmouth.html 2
62 years later ...

3
Major advances in artificial intelligence - especially in 

the subset of machine learning

raise productivity, enable 

more efficient use of 

resources, change the 

way we live and work, 

and increase creativity.

Negative impact on job, 

exacerbate the trend of 

rising inequality, and 

(even) threat to 

humanity. 

4
• Investing in skills and training. There is going 

to be churn in labor markets as a result of AI 

is widely accepted.

• require constant monitoring and close 

cooperation between industry, 

educators, and policymakers. 

• Dealing with data. The uses and misuses of 

data are going to be among the defining 

issues of the 21st century.

• Need for interoperable anonymized data

5
Fundamental and Applied Research and 

Standards for AI (FARSAI)

Short Term

Strengthening machine learning and 

artificial intelligence expertise at NIST.

Long Term

Measure and enhance the security 

and trustworthiness of AI systems.

6
The now

Strengthening machine learning and artificial intelligence expertise at NIST
Strengthening machine learning and

artificial intelligence expertise at NIST

AI Training AI Computing 

Platform

AI Community of 

Interest

AI Visiting 

Fellows

8
NIST AI Training

Speaker 

Series

coursera.org

9
NIST AI Community of Interest

10

Catalyzing the use of machine learning and AI within 

NIST scientific program
NIST AI Computing Platform

GoogLeNet, 2014

Many of deep learning architectures require GPU or special hardware support. 

We are building an AI lab to address this need.

11
NIST AI visiting fellows will provide technical support in ML/AI methods, and 

contribute through participation in workshops, meetings, and assist with 

development activities for NIST AI research projects.

12

Visiting 

Fellows
The Future - Coming soon

Measure and enhance the security and trustworthiness of AI systems
Measure and enhance the security and 

trustworthiness of AI systems

Foundational 

research 

Challenge 

problems

Security of AI 

systems

Standards and 

best practices

14
Open and public data

“Yet another advice: don’t get fooled by people who 

claim to have a solution to Artificial General 

Intelligence. Ask them what error rate they get on 

MNIST or ImageNet.”

Yann LeCun, 2014. 

Vice President, Chief AI Scientist at Facebook 

and prof. at NYU

NIST Special Database 19 NIST Handprinted Forms 

and Characters Database

www.nist.gov/srd/nist-special-database-19 www.reddit.com/r/MachineLearning/comments/25lnbt/ama_yann_lecun/

15
Advance scientific disciplines via evaluations

“Our prior QA work took shape in the form of a QA 

system called PIQUANT. PIQUANT development started 

in 1999, predating our work in UIMA, and was funded 

by government research grants and tested against 

NIST evaluation data in the Text REtrieval

Conference (TREC) QA track between 1999 

and 2005.”

Introduction to “This is Watson”. IBM Journal of Research and 

Development (Volume: 56, Issue: 3.4, May-June 2012)

16
Mathematics of metrology for foundational 

validity and reliability

17
Trusted convener to lead the development of 

standards and best practices

(Photo: Fabrice Coffrini—AFP/Getty Images)

▪ Opportunity for voluntary standards to 

help with development, evaluation and 

adoption of AI applications.

▪ Working with ISO, we provide technical 

support to development of international 

standards for artificial intelligence.

18
Discussion

19