1 

Building the NIST AI Risk Management Framework: Workshop #3 

Day 1: Tuesday October 18, 2022

Topic Speaker(s) Suggested Resources

Welcome, Goals, and AI RMF 

Update

Elham Tabassi, National Institute of Standards and Technology

Panel 1: How Does the AI RMF Fit 

In with Other AI Risk Management 

Approaches?

This panel will explore how the AI 

RMF fits in relation to other 

approaches to AI risk management 

including international standards, 

industry best practices, and national 

strategies.

Moderator: 

Mark Latonero, National Institute of Standards and Technology

Panelists: 

Nozha Boujemaa, IKEA Retail (Ingka Group) 

Greg Cannon, Amazon Web Services 

Sri Krishnamurthy, QuantUniversity

Lee Wan Sie, Infocomm Media Development Authority (IMDA), 

Singapore

Anneke Olvera, Standards Council of Canada

ISO 31000 - Risk Management

ISO/IEC FDIS 23894 - Information technology — Artificial 

intelligence — Guidance on risk management

Staffing for Equitable AI: Roles & Responsibilities 

ISO/IEC DIS 22989(en), Information technology — Artificial 

intelligence — Artificial intelligence concepts and terminology

Risk Management Body of Knowledge

ISO - ISO/IEC TS 5723:2022 - Trustworthiness — Vocabulary

Panel 2: Exactly What Is an AI 

RMF Profile?

Draft 2 of the AI RMF and the 

companion Playbook place heavy 

emphasis on the value of AI RMF 

Profiles. This session will illustrate 

and discuss several potential 

approaches to profiles that will help 

those who choose to use the 

Framework. 

Moderator: 

Cheri Pascoe, National Institute of Standards and Technology 

Panelists: 

Tony Barrett, UC Berkeley

Phillip Collett, American Express

Carlos Ignacio Gutierrez, Future of Life Institute

Snigdha Sharma, National Fair Housing Alliance

Actionable Guidance for High-Consequence AI Risk Management: 

Towards Standards Addressing AI Catastrophic Risks (Section 4)

Panel 3: Moderated Discussion: 

What We Heard and What We Hope 

to Hear

Conversation with NIST AI RMF 

team.

Moderator: 

Courtney Lang, Information Technology Industry Council

Panelists:

Elham Tabassi, National Institute of Standards and Technology

Mark Latonero, National Institute of Standards and Technology

Cheri Pascoe, National Institute of Standards and Technology

Lori Perine, National Institute of Standards and Technology

Mark Przybocki, National Institute of Standards and Technology

Reva Schwartz, National Institute of Standards and Technology

AI Risk Management Framework Draft 2
2 

Day 2: Wednesday October 19, 2022

Topic Speaker(s) Suggested Resources

Welcome Elham Tabassi, National Institute of Standards and Technology

Panel 4: Mapping the Path 

from Current to Future AI Risk 

Management 

Practice

In this panel we will learn how 

to get started with the AI RMF, 

and discuss how organizations 

can effectively implement the 

AI RMF map function through 

socio-technical approaches.

Moderator: 

Reva Schwartz, National Institute of Standards and Technology

Panelists:

Stevie Bergman, DeepMind

Sina Fazelpour, Northeastern University and NIST AI Visiting Fellow

Bogdana Rakova, Mozilla Foundation

Harini Suresh, Massachusetts Institute of Technology (MIT)

Diversity in sociotechnical machine learning systems

Fairness and Abstraction in Sociotechnical Systems

NIST Special Publication 1270 Towards a Standard for Identifying and 

Managing Bias in Artificial Intelligence

Hard Choices in Artificial Intelligence

Data and its (dis)contents: A survey of dataset development and use in 

machine learning research

Measurement and Fairness

Data Feminism (Chapter 6): The numbers don't speak for themselves

Introduction to the Theory of Change

Challenges for Responsible AI Practitioners and the Importance of 

Solidarity

Improving transparency in AI by exploring new avenues for human 

feedback, robustness, and documentation

Enabling Fairness in Healthcare Through Machine Learning

The Fallacy of Functionality

AI and the Everything in the Whole Wide World Benchmark

Actionable Auditing: Investigating the Impact of Publicly Naming 

Biased Performance Results of Commercial AI Products

Problem Formulation and Fairness
3 

Participatory Problem Formulation for Fairer Machine Learning 

Through Community Based System Dynamics

Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction 

(pre-Print)

Representativeness in Statistics, Politics, and Machine Learning

A Framework for Understanding Sources of Harm throughout the 

Machine Learning Life Cycle

Where Responsible AI meets Reality: Practitioner Perspectives on 

Enablers for Shifting Organizational Practices (arxiv.org)

Participation is not a Design Fix for Machine Learning

MLOps: What It Is, Why It Matters, and How to Implement It

Gig Workers Gather Their Own Data to Check the Algorithm’s Math

Measuring bias in self-reported data

Bias Mitigation in Data Sets

Where Responsible AI meets Reality: Practitioner Perspectives on 

Enablers for Shifting Organizational Practices

AI Fairness for People with Disabilities: Point of View

Towards Intersectional Feminist and Participatory ML: A Case Study 

in Supporting Feminicide Counterdata Collection

Fairness in Machine Learning

Collective problem solving: Functionality beyond the individual

Bias Mitigation in Data Sets

Staffing for Equitable AI: Roles & Responsibilities
4 

What I learned from talking to 20 diversity, equity, and inclusion 

experts

Participation is not a Design Fix for Machine Learning

Panel 5: What Does Good 

Governance for AI Risk 

Management Look Like?

This panel will discuss how 

organizations develop culture 

and practice to govern AI 

internally. There is much we 

can learn from how other fields 

and industries track, manage 

and avoid risks through 

organizational approaches for 

improving governance. 

Moderator: 

Kathy Baxter, Salesforce and NIST AI Visiting Fellow

Panelists:

Rumman Chowdhury, Twitter

Jen Gennai, Google

Sakshi Jain, LinkedIn

David Marcos, Microsoft

Google's AI Principles Governance page - Reviews & Processes

Annual Google AI Principles progress update, end of year 2021

AI Principles Review Process - Google AI

A developer’s guide to responsible AI review processes

Trusted AI at Salesforce

Responsible Creation of AI

Building Ethical and Inclusive Products Trailmix

Model Cards for AI Model Transparency

Ethical AI Maturity Model

Build With Intention Toolkit

Trusted AI | Salesforce Research

Responsible Creation of Artificial Intelligence

Model Cards for AI Model Transparency

Mutiny on the BiasBounty.ai

Responsible AI

ForHumanity University
5 

Value Sensitive Design Lab

Consequence Scanning – an agile practice for responsible innovators

Responsible Innovation: A Best Practices Toolkit

Building responsible AI for everyone

IEEE Ethics In Action in Autonomous and Intelligent Systems

Panel 6: How to Measure AI 

Risk across the AI lifecycle

This panel will discuss 

approaches for measuring 

trustworthiness and AI 

risks from a socio-technical 

perspective.

Moderator:

Jeanna Matthews, Clarkson University and NIST Faculty Fellow

Panelists:

Jack Clark, Anthropic

David Danks, UC San Diego

Ian Eisenberg, Credo AI

Nazneen Rajani, HuggingFace

ImportAI.net

Anthropic.com

AI Index

From AI compliance to competitive advantage

TRUSTWORTHY EVIDENCE FOR TRUSTWORTHY 

TECHNOLOGY

Resources from Jeanna Matthews

A Hierarchy of Limitations in Machine Learning

Reliance on metrics is a fundamental challenge for AI

Introducing a Practice-based Compliance Framework for Addressing 

New Regulatory Challenges in the AI Field

ISO 31000 - Risk Management

Joint AI Statement Update - Statement on Principles for Responsible 

Algorithmic AI Systems

"Why do so?" - A Practical Perspective on Machine Learning Security

“We’re Seeking Relevance”: Qualitative Perspectives on the Impact of 

Learning Analytics on Teaching and Learning
6 

Panel 7: Managing AI Risks 

This panel will discuss practical 

methods and processes that 

practitioners are engaging 

(actual or prospective) to 

manage AI risks along the 

lifecycle, and where there are 

opportunities for the AI RMF to 

enhance or fill-in the gaps.

Moderator:

Daniel Castro, Information Technology and Innovative Foundation 

(ITIF)

Panelists:

Veena Calambur, Workday

Trey Causey, Indeed

Ali Shah, Accenture

Mona Sloane, New York University (NYU)

A Silicon Valley love triangle: Hiring algorithms, pseudo-science, and 

the quest for auditability

Introducing a Practice-based Compliance Framework for Addressing 

New Regulatory Challenges in the AI Field

AI and Procurement