1

COMMENTS OF SUSAN VON STRUENSEE, JD, MPH

to the

Request for Information and Comment on the National Institute of Standards and Technology’s 

Artificial Intelligence Risk Management Framework

86 FR 40810 (July 29, 2021)

Agency/Docket Numbers:

Docket No. Docket Number: 210726-0151

____________________________________________________ 

Thank you for the opportunity to comment on the National Institute of Standards and Technology’s

development of a framework that can be used to improve the management of risks to individuals, 

organizations, and society associated with artificial intelligence (AI).

The World Economic Forum’s Centre for the Fourth Industrial Revolution launched a project,

Unlocking Public Sector in AI, which offered AI procurement guidelines for government and publicsector organizations in November 2018. The Forum’s goal was to help officials better understand AI 

and mitigate potential risks. Acquisition processes are a critical way to mitigate the risks of AI.

Federal procurement law needs to be harnessed in service of ethical algorithmic governance. Professor 

David S. Rubenstein writes how federal procurement law can and must be retrofitted to meet the 

unique challenges of algorithmic governance. His work provides a principled and pragmatic approach 

for doing so. Professor Rubenstein also writes (footnotes omitted) “A recent article by Mulligan and 

Bamberger warrants special mention here: not only for its important contributions, but also for its 

analytical route. Descriptively, they argue that “[g]overnment responsibility for policymaking is 

abdicated” when the adoption of AI systems is governed by procurement, because the embedded 

policies escape administrative law requirements of public participation and reasoned deliberation. 

Moreover, Mulligan and Bamberger urge federal and state agencies to shift from a “procurement 

mindset” to a “policymaking mindset” when acquiring AI systems.” He focuses critical attention on 

the need for AI ethics in federal procurement law. His work steers the conversation in a new 

direction—namely, toward procurement law’s positive potential. More than a marketplace, the AI 

acquisition gateway must be reimagined as a policymaking space. His articles begin the difficult work 

of integrating ethical AI into federal procurement law. Rubenstein, David S., Acquiring Ethical AI 

(Oct. 1, 2020). Florida Law Review, Vol. 73, 2021, Available at SSRN: 

https://ssrn.com/abstract=3731106

The references cited below and attached to this RFI, show the major thought given to government 

procurement and AI. But those of us charged with implementation need more than top level 

frameworks that highlight principles and provide general guidance. Have you seen any real-life 

solicitations, RFPs, or contracts for responsible AI that can serve as models for integrating AI ethics 

into the acquisition process, any model RFPs or solicitations for governments to use, even as an annex 

section? Is anyone developing templates, model contract clauses, model RFPs? Can this work be 

developed via governments? There is some excellent work out there advising what should be in the AI 

RFPs to integrate AI Ethics, but no models, examples, or templates for government procurement 

officials to use.

Can the Artificial Intelligence Risk Management Framework provide model contract clauses, model 

content, model RFPS and solicitations with AI ethics language and templates for practical 
2

implementation and customization? The Framework would be most helpful to focus on Ethical AI in 

Procurement and provide model RFPs, requirements, contractual language, to ensure we are integrating 

AI ethics into the acquisition process. Providing model language and templates speeds up 

implementation. Please include templates and model language as part of the framework.

The World Economic Forum, AI Government Procurement Guidelines, and its AI Procurement in a 

Box alludes to developing model solicitations, and RFPs and model contract clauses but none are 

accessible on-line and the few I found are very rudimentary and I have attached them. Do you agree 

they need to be developed to get procurement officials to use them-to hasten the Responsible AI 

adoption process? 

Enstrom and Ho describe that the US Federal government is using AI in its Social Security 

Administration decisions and Security and Exchange Commission (SEC) prosecution decisions, but I 

could not find the RFPS or the contract agreements to look at the AI Ethics language and how AI ethics 

is or should be put into the contracts via the RFPs. 

Intellectual Property model clauses also needed to be developed. Governments need to be much more 

explicit about the risks inherent in AI methodologies without AI ethics, and then foster market 

competition around this challenge. They can do this via RFPs, solicitations, and contracts. 

The way to improvement begins with what government asks of its AI vendors. To set the right 

conditions, the government’s market solicitations should contain prompts that explicitly tie ethical AI 

to methodologies, and committing to those things in the contract. We need to see templates and model 

solicitations and contract clauses showing what that would look like.

I found very few government solicitations for AI that have an AI ethics component. I attach one I did 

find at the Crown Commercial Services of the UK. 

https://www.crowncommercial.gov.uk/agreements/search?q=artificial+intelligence

As you likely know, Canada requires algorithmic impact assessments for government uses of AI, and is 

integrated into the government’s AI acquisition practice. Algorithmic Impact Assessment Tool -

Canada.ca

Canada has a pre-certified list of AI vendors, which are on what is the equivalent of a GSA GWAC (or 

Multiple Award Schedule)? Artificial intelligence source list - Software Acquisition Reference Centre –

Buying and Selling – PSPC (tpsgc-pwgsc.gc.ca). Attached is a Canadian RFP for the AI pre-certified 

vendor list. They mention AI ethics (at 2.4.1 see below). I do not see questions on how they have 

applied AI ethics in past work, lessons learned etc.

A view of the Canadian system, the paper, Artificial Intelligence Policy and Funding in Canada: Public

Investments, Private Interests Main Findings: 

1. Public investments in AI technologies primarily benefit the private sector 

2. Even though Canada has federal AI policy, there is no national government AI strategy 

Companies linked to human rights abuses can pre-qualify as government AI suppliers 

3. Concentrations of power provide advantages to a handful of entities 
3

The below is the only AI Ethics language I saw in a RFP for AI by the Canadian government, it is not 

sufficient and better templates with model language needs to be developed:

Mandatory Criteria 2.4 in Canadian RFP

Canada intends to pre-qualify suppliers based on the following mandatory criteria:

2.4.1 AI Ethics: Supplier must describe how they address ethical considerations when delivering AI. 

This could include experience in applying frameworks, methods, guidelines or assessment tools to test 

datasets and outcomes.

The implementation of process-based governance frameworks is suggested in the UK Guidance for 

Understanding AI ethics and safety. This provides a basis to integrate norms, values, and principles 

informing procedures and protocols that define the project workflow. The Alan Turing Institute calls it 

a ‘PBG (Process-Based Governance) Framework’. It is helpful, but contains no templates or model 

language for AI ethics in RFPs or contracts. What is needed are concrete examples showing how the

guidance looks in action, when applied.

As Professor Rubenstein states, “the governments in-house capacity challenges are a major concern. It 

is one thing if agencies must rely on vendors to satisfy the governments demand for AI tools. It is quite 

another if the government does not have the resources and capacity to responsibly manage AI 

acquisitions.” Hopefully, the framework will include the types of templates, model clauses, and model 

solicitations to assist procurement officials integrate AI Ethics into the Responsible AI acquisition 

process.

Respectfully Submitted,

Susan von Struensee, JD, MPH

Sources:

DataEthics.eu, White Paper on Data Ethics in Public Procurement of AI-based Services and Solution, 

(April 2020) https://www.dataethics.eu/wp-content/uploads/dataethics-whitepaper-april-2020.pdf

Rubenstein, David S., Acquiring Ethical AI (Oct. 1, 2020). Florida Law Review, Vol. 73, 2021, 

Available at SSRN: https://ssrn.com/abstract=3731106 (PRE_PRINT COPY-IS BEING UPDATED)

Abstract-Artificial intelligence (AI) is transforming how government operates. Federal agencies use the 

technology for law enforcement, adjudication, rulemaking, inhouse management, and the delivery of 

public services. Algorithmic governance brims with promise and peril. Under the right conditions, AI 

systems can solve complex problems, reduce administrative burdens, and optimize resource allocations. 

Under the wrong conditions, AI systems can lead to widespread discrimination, invasions of privacy, 

and the erosion of democratic norms. The United States has pledged its commitment to principles of 

“ethical AI,” including transparency, accountability, fairness, and human rights. But proselytizing is not 

actualizing. A burgeoning literature has emerged to square algorithmic governance with the precepts of 

constitutional and administrative law. Federal procurement law, however, remains a dangerous blind 

spot in the reformist agenda. The government’s pent up demand for AI systems far exceeds its inhouse 

capacity to develop, field, and monitor this powerful technology. Accordingly, many if not most of the 

tools of algorithmic governance will be procured by contract from the technology industry. This Article 

intervenes with a principled and pragmatic agenda for acquiring ethical AI. First, it provides an original 
4

account that aligns the ambition of algorithmic governance, the imperative of ethical AI, and the 

complexities of procurement law. Second, the Article argues that procurement law is not only uniquely 

situated, but also well suited, to serve as a checkpoint and catalyst for ethical algorithmic governance. 

Third, the Article prescribes a set of concrete regulatory reforms to center ethical AI throughout the 

procurement process: from acquisition planning through market solicitation, negotiation, and 

contractual award. The outsourcing of algorithmic governance raises a host of challenges that 

constitutional law and administrative law are ill equipped to handle. Procurement law will not solve all 

the challenges of algorithmic governance. Just as surely, the challenges of algorithmic governance 

cannot be solved without procurement law.

David S. Rubenstein, FEDERAL PROCUREMENT OF ARTIFICIAL INTELLIGENCE: PERILS 

AND POSSIBILITIES, The Great Democracy Initiative, December 2020 

https://greatdemocracyinitiative.org/document/federal-procurement-of-artificial-intelligence-perils-andpossibilities/ Abstract: The proliferation of Artificial Intelligence (AI) use by federal agencies raises 

urgent questions about how these new technologies should be regulated. Today, AI procurement helps 

streamline processes in agencies like the Social Security Administration, the Food and Drug 

Administration, Homeland Security, and more. Under the right conditions, algorithmic governance can 

be an incredibly useful tool, however, under the wrong conditions, it can lead to widespread 

discrimination, invasion of privacy, and the degradation of democratic principles. Yet, much, if not 

most, of the AI used by federal agencies will be procured from a virtually unregulated private market. 

In this report, David. S Rubenstein shows that when the government acquires AI, it is often procuring 

the policy choices of the nongovernmental actors who designed the technology. 

Rubenstein outlines a plan for ethical AI procurement, going forward: mandating the creation of a 

government-wide inventory that includes clear information on AI systems used by federal agencies, 

requiring agencies to prepare “AI risk assessment” reports prior to acquiring AI services, and 

integrating ethical AI consideration into existing regulations for source selection. Federal procurement 

of AI services must be reimagined as more than just a marketplace but rather a policymaking space that 

promotes trustworthy and ethical AI.

Engstrom and Ho, ALGORITHMIC ACCOUNTABILITY IN THE ADMINISTRATIVE STATE, 37 

YALE J. ON REG. (2020) 

https://digitalcommons.law.yale.edu/cgi/viewcontent.cgi?article=1563&context=yjreg

Storia Law, 8 Points about AI Development Agreements that can be learned from the “Contract 

Guidance on Utilization of AI and Data” at https://storialaw.jp/en/service/bigdata/ai-13 

Rebecca S. Eisner and Brad L. Peterson, Smart Licensing of Artificial Intelligence, (May 16, 2019) 

https://www.mayerbrown.com/en/perspectives-events/publications/2019/05/smart-licensing-ofartificial-intelligence 

Rebecca S. Eisner, (Sept. 2020) https://www.mayerbrown.com/-/media/files/perspectivesevents/publications/2020/09/tbfall20ofnoteipt.pdf 

Practical Law, Expert Q&A with Rebecca Eisner of Mayer Brown LLP on artificial intelligence (AI) 

licensing, https://www.mayerbrown.com/-/media/files/news/2019/01/expert-qanda-on-artificialintelligence-ai-licensing-w0219801.pdf
5

Leslie, David, Understanding Artificial Intelligence Ethics and Safety: A Guide for the Responsible 

Design and Implementation of AI Systems in the Public Sector, (June 10, 2019). Available at SSRN: 

https://ssrn.com/abstract=3403301 or http://dx.doi.org/10.2139/ssrn.3403301

This guide, written for department and delivery leads in the UK public sector and adopted by the 

British Government in its publication, 'Using AI in the Public Sector,' identifies the potential harms 

caused by AI systems and proposes concrete, operationalisable measures to counteract them. It stresses 

that public sector organisations can anticipate and prevent these potential harms by stewarding a culture 

of responsible innovation and by putting in place governance processes that support the design and 

implementation of ethical, fair, and safe AI systems. It also highlights the need for algorithmically 

supported outcomes to be interpretable by their users and made understandable to decision subjects in 

clear, non-technical, and accessible ways. Finally, it builds out a vision of human-centered and contextsensitive implementation that gives a central role to communication, evidence-based reasoning, 

situational awareness, and moral justifiability.

Mulligan, Deirdre K. and Bamberger, Kenneth A., Procurement As Policy: Administrative Process for 

Machine Learning, (October 4, 2019). Berkeley Technology Law Journal, Vol. 34, 2019, Available at 

SSRN: https://ssrn.com/abstract=3464203 or http://dx.doi.org/10.2139/ssrn.3464203

Abstract: At every level of government, officials contract for technical systems that employ machine 

learning—systems that perform tasks without using explicit instructions, relying on patterns and 

inference instead. These systems frequently displace discretion previously exercised by policymakers 

or individual front-end government employees with an opaque logic that bears no resemblance to the 

reasoning processes of agency personnel. However, because agencies acquire these systems through 

government procurement processes, they and the public have little input into—or even knowledge 

about—their design or how well that design aligns with public goals and values. This Article explains 

the ways that the decisions about goals, values, risk, and certainty, along with the elimination of caseby-case discretion, inherent in machine-learning system design create policies—not just once when 

they are designed, but over time as they adapt and change. When the adoption of these systems is 

governed by procurement, the policies they embed receive little or no agency or outside expertise 

beyond that provided by the vendor. Design decisions are left to private third-party developers. There is 

no public participation, no reasoned deliberation, and no factual record, which abdicates Government 

responsibility for policymaking. This Article then argues for a move from a procurement mindset to 

policymaking mindset. When policy decisions are made through system design, processes suitable for 

substantive administrative determinations should be used: processes that foster deliberation reflecting 

both technocratic demands for reason and rationality informed by expertise, and democratic demands 

for public participation and political accountability. Specifically, the Article proposes administrative 

law as the framework to guide the adoption of machine learning governance, describing specific ways 

that the policy choices embedded in machine-learning system design fail the prohibition against 

arbitrary and capricious agency actions absent a reasoned decision-making process that both enlists the 

expertise necessary for reasoned deliberation about, and justification for, such choices, and makes 

visible the political choices being made. Finally, this Article sketches models for machine-learning 

adoption processes that satisfy the prohibition against arbitrary and capricious agency actions. It 

explores processes by which agencies might garner technical expertise and overcome problems of 

system opacity, satisfying administrative law’s technocratic demand for reasoned expert deliberation. It 

further proposes both institutional and engineering design solutions to the challenge of policymaking 

opacity, offering process paradigms to ensure the “political visibility” required for public input and 

political oversight. In doing so, it also proposes the importance of using “contestable design”—design 

that exposes value-laden features and parameters and provides for iterative human involvement in 
6

system evolution and deployment. Together, these institutional and design approaches further both 

administrative law’s technocratic and democratic mandates.

Naudé, Wim and Dimitri, Nicola, Public Procurement and Innovation for Human-Centered Artificial 

Intelligence. IZA Discussion Paper No. 14021, Available at SSRN: https://ssrn.com/abstract=3762891

Abstract-The possible negative consequences of Artificial Intelligence (AI) have given rise to calls for 

public policy to ensure that it is safe, and to prevent improper use and misuse. Human-centered AI 

(HCAI) draws on ethical principles and puts forth actionable guidelines in this regard. So far however, 

these have lacked strong incentives for adherence. In this paper we contribute to the debate on HCAI 

by arguing that public procurement and innovation (PPaI) can be used to incentivize HCAI. We dissect 

the literature on PPaI and HCAI and provide a simple theoretical model to show that procurement of 

innovative AI solutions underpinned by ethical considerations can provide the incentives that scholars 

have called for. Our argument in favor of PPaI for HCAI is also an argument for the more innovative 

use of public procurement, and is consistent with calls for mission-oriented and challenge-led 

innovation policies. Our paper also contributes to the emerging literature on public entrepreneurship, 

given that PPaI for HCAI can advance the transformation of society, but only under uncertainty.

The World Economic Forum, AI Procurement in a Box: Challenges and opportunities during 

implementation, World Economic Forum, June 2020

Workshop participants explored various themes related to the governments use of AI and how 

procurement plays a role in government adoption of the technology. 

http://www3.weforum.org/docs/WEF_AI_Procurement_in_a_Box_Challenges_and_Opportunities_duri

ng_implementation_2020.pdf

The World Economic Forum, AI Government Procurement Guidelines, (Sept. 2019) 

http://www3.weforum.org/docs/WEF_Guidelines_for_AI_Procurement.pdf

Exploring Blockchain Technology for Government Transparency: Blockchain-Based Public 

Procurement to Reduce Corruption | World Economic Forum (weforum.org), June 2020, 

http://www3.weforum.org/docs/WEF_Blockchain_Government_Transparency_Report.pdf

WEF_AI_Procurement_in_a_Box_Workbook_2020.pdf (weforum.org)

http://www3.weforum.org/docs/WEF_AI_Procurement_in_a_Box_Workbook_2020.pdf

European Commission, Emerging technologies in public procurement, (June 2020) 

https://ec.europa.eu/growth/single-market/public-procurement/digital/emerging-technologies_en

Emerging technologies can transform public procurement. From automating repetitive administrative 

tasks to providing unprecedented information and analysis regarding spending patterns and project 

results, new technology can enable better decisions, lower costs, and increase transparency. In 2019, 

the Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs commissioned under 

the ISA2 a study on the uptake of emerging technologies in public procurement. This study examined 

how public authorities around the globe are using new technologies when procuring goods and 

services. Covering technologies including artificial intelligence and machine learning, big data and data 

analytics, blockchain, robotic process automation, augmented and virtual reality, internet of things, and 

drones, the study features:

• a longlist of 96 projects (as of January 2020) where these new technologies have been explored 

or used by public authorities for procurement
7

• 20 detailed case studies presenting the issue addressed, impact, cost, requirements and risks of 

particularly interesting emerging technology projects

• a final report presenting the overall approach and findings of the project, including 10 

recommendations for the application of emerging technologies to public procurement

• Update (22 June 2020): 10 new projects have been added. 3 are from Portugal, 3 are from 

Estonia, and the remaining 4 are from Slovenia, the Netherlands, Italy and Spain. The most 

frequently used emerging technologies in these new projects are business intelligence, 

blockchain and artificial intelligence.

European Commission, Study on up-take of emerging technologies in public procurement, (Feb. 27, 

2020) https://ec.europa.eu/docsroom/documents/40102

Confronting Bias: BSA’s Framework to Build Trust in AI | BSA | The Software Alliance. 

https://ai.bsa.org/confronting-bias-bsas-framework-to-build-trust-in-ai

Raji, et al, Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal 

Algorithmic Auditing, https://dl.acm.org/doi/pdf/10.1145/3351095.3372873

Mark Treveil, Nicolas Omont, Clément Stenac, Kenji Lefevre, Du Phan, Joachim Zentici, Adrien 

Lavoillotte, Makoto Miyazaki, Lynn Heidmann, Introducing MLOps, November 2020, 

https://www.oreilly.com/library/view/introducing-mlops/9781492083283/

More than half of the analytics and machine learning (ML) models created by organizations today 

never make it into production. Some of the challenges and barriers to operationalization are technical, 

but others are organizational. Either way, the bottom line is that models not in production can't provide 

business impact. This book introduces the key concepts of MLOps to help data scientists and 

application engineers not only operationalize ML models to drive real business change but also 

maintain and improve those models over time. Through lessons based on numerous MLOps 

applications around the world, nine experts in machine learning provide insights into the five steps of 

the model life cycle--Build, Preproduction, Deployment, Monitoring, and Governance--uncovering 

how robust MLOps processes can be infused throughout.

This book helps you:

• Fulfill data science value by reducing friction throughout ML pipelines and workflows

• Refine ML models through retraining, periodic tuning, and complete remodeling to ensure longterm accuracy

• Design the MLOps life cycle to minimize organizational risks with models that are unbiased, 

fair, and explainable

• Operationalize ML models for pipeline deployment and for external business systems that are 

more complex and less standardized

Guidelines for AI procurement, A guide to using artificial intelligence in the public sector, (June 8, 

2020) https://www.gov.uk/government/publications/guidelines-for-ai-procurement/guidelines-for-aiprocurement

DHS AI Strategic Plan

https://www.dhs.gov/sites/default/files/publications/21_0730_st_ai_ml_strategic_plan_2021.pdf
8

U.S. Government Accountability Office, Artificial Intelligence: An Accountability Framework for 

Federal Agencies and Other Entities, Highlights of GAO-21-519SP, available at

https://www.gao.gov/assets/gao-21-519sp-highlights.pdf.