Subject: DM feedback on the NIST AI RMF second draft

Dear NIST t eam d eveloping the AI Risk Management Framework,

Thank you for the op portunity to p rovid e comments on the second d raft of the AI Risk 

Management Framework and Playb ook. DeepMind shared a resp onse to the RFIin Septemb er 

20 21 and feed back to the concep t p ap er as well as the initial d raft earlier this year. We 

ap p reciate that ad d itional d etail will b e p rovid ed in the next iterations of the d raft RMF and 

are sharing b elow some high- level feed back.

Int rod uc t ion

As with p revious d rafts, we sup port the d evelop ment of flexib le and voluntary AI standard s. 

Consid ering the p ace of AI d evelop ment and the fact that, in many areas, the field is not yet 

mature enough to d esign app rop riate regulatory req uirements, the d evelop ment of guid ance 

to imp rove companies’ and ind ividuals’ und erstand ing and management of risk is a crucial 

first step . These norms could then, if need ed , mature into regulatory req uirements at the 

ap p rop riate time. The op portunity for many stakehold ers to input on the various d rafts in a 

transp arent and clearly timed manner is also essential and something we have valued in the 

way NIST ap p roached the d evelop ment of the AI RMF.

AI Ris ks & Trus t wort hy AI

We support the changes mad e to the trustworthy AI section and the new structure around 

the seven elements of trustworthy AI - this new classification is better aligned with the 

OECD’s ap p roach of cap turing the main characteristics of AI systems. This section however 

seems to suggest that AI systems are either trustworthy or not, d epend ing on the seven 

characteristics. It might b e more accurate to p resent trustworthiness as a sp ectrum that 

consid ers these characteristics within a p articular context in which an AI ap p lication is b eing 

used . Ultimately, trustworthiness will also b e d etermined b y the users of the system.

As p er our first NIST sub mission, we also want to reiterate the need to incentivise d elib eration 

over comp lex longer- term risks from ad vancing AI capab ilities. We sup port the inclusion of 

ongoing monitoring and information-sharing, given that new AI- related risks and mitigation 

strategies are d iscovered and stud ied over time as these systems evolve. Eq ually we think 

that guidance on tools, b enchmarks and aud it mechanisms may help d ifferent actors b etter 

und erstand the range of options availab le to add ress d ifferent harms.

The new emphasis on test, evaluation, validation and verification (TEVV) comp ared to 

p revious d rafts of the RMF is a p ositive d evelopment and we agree that socio- technical 

consid erations throughout the AI lifecycle are imp ortant. However, we b elieve the p reced ing 

d esign stage deserves special attention as well, as choices made at the initial stage of problem 

formulation and outcome definition will matter greatly. We’d recommend reviewing language to be more 

balanced in the way TEVV is mentioned.
More concretely, on some of the changes and new ad ditions in this section and the seven 

characteristics:

•

•

• Ove rall, within the c harac te ristic s, it see ms that whe the r a syste m is 

p e rformant - b e it acc urate or high- p e rforming

• - is not suffic ie ntly emp hasise d , although we note d it was und e r the ‘valid and 

re liab le ’ c harac te ristic . Data gove rnanc e also d e se rve s treatment as a more 

fund ame ntal c onc e p t that runs ac ross all the d iffe re nt c harac te ristic s. Thinking 

about the governance

• me c hanisms as a key c omp one nt of the RMF should also he lp give it a more 

robust structure.

•

•

•

• ‘Va lid a nd Re lia b le ’ - 

• The me ntion of human- AI te aming asp e c ts in this se c tion is c ruc ial. Ac c urac y 

should b e d e te rmine d within

• the more holistic environme nt of ac tual use , for instanc e , if a me d ic al 

p re d ic tion d e vic e is de ve lop e d to b e used b y nurse s, the n the ac c urac y isn't 

just of the mod e l itse lf, b ut of the mod e l c oup le d with the nurse s’ inte rp re tation 

in the re alm of re al use .

•

•

•

• ‘Sa fe ’ -

• The me ntion of a stateme nt from ISO/IEC TS 5723:20 22 that AI syste ms ‘should 

not, und e r d e fine d c ond itions,

• cause p hysic al or p syc hologic al harm or le ad to a state in whic h human life , 

he alth, p rop e rty, or the e nvironme nt is e nd ange re d ’ d oe sn't se e m to c onsid e r 

risk trad e - offs rathe r than the ab se nc e of risk. The d e finition of a safe system 

will d e p e nd on how we

• und e rstand this risk in re lation to the b e nefit of the syste m ove rall - c larifying 

this p oint in this se c tion would b e he lp ful

•

•

•

• ‘Fa ir - a nd b ia s is m a na ge d ’ 

• -

• NIST rightly note s that fairne ss, b ias and d isc rimination are c omp le x c onc e p ts 

that are d e fine d and use d d iffe re ntly,
• ac ross d iffe re nt c onte xts, for e xamp le ac ross c onsume r, financ ial, and d ata 

p rivac y law, as we ll as statistic s, c omp ute r sc ie nc e , and c ognitive sc ie nc e . For 

this re ason, we sup port NIST’s attemp t to d e ve lop a stand ard for ‘id e ntifying 

and managing b ias in

• AI’ that d e marc ate s this risk into c halle nge s re lating to syste mic bias, 

c omp utational b ias and human b iase s. It would b e he lp ful to furthe r illustrate 

some of the sub - c ategorie s of risk within the se thre e c ate gorie s, sup p orte d by 

e xamp le s, as some are not

• imme d iate ly c le ar: for instanc e , what d oe s NIST me an b y ‘c omp utational b ias’ 

within d atase ts? Similarly, NIST c ould highlight that the ne e d for ongoing 

monitoring of AI syste ms is p artic ularly true for fairne ss risks, as some may only 

b e visib le ove r long

• time sc ale s - for instanc e a loan- ap p roval system that d isp rop ortionate ly 

favors c e rtain group s, b ut in ways that only b e c ome e vid e nt over longe r time 

horizons. In othe r instanc e s, e fforts to mitigate c e rtain fairne ss risks c an le ad 

to ne gative longe r- term

• sid e e ffe c ts - Dee p Mind researc h

• has highlighte d how e fforts to me asure and mitigate language mod e l toxic ity, 

at the language mode training stage , c ould p ote ntially re sult in d ownstre am 

harms, suc h as a re d uc tion in te xt ab out, and d iale c ts of, marginalize d group s.

•

•

•

• ‘Se c ure a nd Re s ilie nt ’ 

• -

• This se c tion se e ms ove rly high le ve l and the re is a ne e d to e xp and on some of 

the te rminology: for instanc e , what d oe s c onfid e ntiality, inte grity and 

availab ility me an in te rms of an AI syste m? More NIST p ublic ations or 

frame works that c ap ture se c urity re q uire ments

• c ould also b e re fe re nc e d in this se c tion suc h as

• NIST

• CSF

• and how it fits into the AI risk manage me nt frame work

• for ad d re ssing se c urity risks.

•

•

•

• ‘Tra ns p a re nt a nd Ac c ount a b le ’ -

• Doc ume ntation b e st p rac tic e s with re sp e ct to AI syste ms c ontinue to e volve , 

p artic ularly in the AI re se arc h world . Efforts to d oc ume nt the c harac te ristic s of

• AI

• mod e ls have e xp ande d to
• datasets

• and re inforc e me nt

• le arning age nts, as we ll as to d oc ume nting the b road e r te c hnic al

• and soc iote c hnic al syste ms that AI mod e ls are inc orp orate d into. NIST c ould 

inc orp orate guid anc e on how and whe n to use suc h re sourc e s, as we ll as 

b road e r d ata manageme nt p rac tic e s, p e rhap s re fe re nc ing NIST’s

• Re se arc h

• Data Frame work (RDaF), to he lp organizations b e tte r gauge e xpe c tations

• around transp are nc y and to p re p are ac c ord ingly. As

• d oc ume ntation p rac tic e de ve lop s and b ec ome s inc re asingly common in AI 

re se arc h, NIST c ould e xp lore and de fine b est p rac tic e s ove r time, and p rovide 

guid e line s as to what, whe n and how to d oc ume nt. Build ing on the notion of 

re sp onsib ility b e ing share d among

• ‘all AI ac tors’, c onsid e ring e arly onward s how d oc ume ntation can c ontrib ute 

to und e rstand ing how that re sp onsib ility is and should b e d istrib ute d would 

also b e he lp ful.

•

• 

• 

• ‘Exp la ina b le a nd int e rp re t a b le ’ - The

• te rms ‘e xp lainab ility’ and ‘inte rp re tab ility’ are use d inc onsiste ntly ac ross the

AI c ommunity, inc lud ing in ways not c ap ture d b y the c urre nt NIST d e finitions. 

For e xamp le , some c onc e p tualisations of e xp lainab ility c apture not just 

‘me c hanistic ’ attemp ts

• to id e ntify c ausal p athways or fe ature s und e rlying a mod e l’s p re d ic tions,

• b ut also the c ritic al work, informe d

• b y soc ial sc ie nc e , to e nsure that the sub seq ue nt e xp lanation

• p rovid e d to a use r is ac c e ssib le , and use ful, for the ir sp e c ific goals. Other 

p rac titione rs stre ss the ne e d for p ost- hoc ‘analysis’ and ‘p rob ing’ of AI 

systems and app lic ations’, and the ir outp uts. Give n the se c halle nge s, it would 

b e he lp ful for NIST to highlight

• how the ir d e finitions d iffe r to othe rs in the fie ld . NIST c ould also p rovid e 

p rac titione rs with p rac tic al e xamp le s and illustrations of the d iffe re nt typ e s of 

e xp lainab ility and inte rp re tab ility c halle nge s the y may fac e , across d iffe re nt 

c onte xts and aud ie nce s,

• as we ll as guid anc e on the trad e - offs and re lationship s that may e xist - for 

e xamp le , b e twe e n inte rp re tab ility, ac c urac y, and fairne ss. 

• 

•

•

• ‘Priva c y- enhance d ’
• - The me ntion that ‘p roc e ssing of d ata c ould c re ate p rivac y- re late d p rob le ms’ 

might

• b e too narrow a view for how p rivac y risks might e me rge : sinc e e xp lic it 

c omb inations of d ata sourc e s c an c re ate p rivac y p rob le ms that the two 

sourc e s in isolation d id n't have , inte rac tions b e twe e n AI syste ms re p re se nt 

imp lic it c omb inations of d ata sourc e s

• that c ould ge ne rate p rivac y risks that e ach of the systems in isolation d on't 

have . It might the re fore not b e e nough for asse ssors to look at d ata p roc e ssing 

ac tivitie s; the y might also have to look at the use and inte rac tions b e twe e n AI 

syste ms. We sugge st

• favoring outc ome - b ase d ap p roac he s, as it is unlike ly that a ‘one size fits all’ 

ap p roac h to p rivac y would ad e q uate ly add re ss the range of p ote ntial p rivac y 

risks.

•

•

•

• ‘Hum a n fa c t ors ’ 

• - The c onc e p t of ‘human in the loop ’ syste m, me ntione d in the human fac tors 

se c tion,

• should b e d e fine d and d isc usse d in more d e tail as this is c urre ntly the main 

me c hanism for managing risk in d e p loying safe ty c ritic al syste ms, a 

me c hanism that is also limite d by the e xte nt the human c an und e rstand what 

the system d oe s. The d e finition should

• c le arly make a d istinc tion b e twe e n systems that c an autonomously make a 

d e fe rral d e c ision to a human e xp e rt and syste ms that are me rely use d b y a 

human d e c ision make r as an ad ditional op inion. In ad d ition, a numb e r of AI 

syste ms are unlike ly to re q uire muc h

• human ove rsight, such as mod e ls use d to imp rove vid e o c omp re ssion.

•

AI RMF p rofile s

Consid ering the current focus on general purpose systems, we’d encourage the inclusion of 

a p rofile on those systems, which could facilitate cross- b ord er interoperab ility as the EU is 

currently looking at regulatory requirements. A profile on general p urpose systems would also 

b ring to light some of the challenges we’d encounter when governing these systems - in this 

case, and consid ering the large numb er of actors involved in the GPAIS value chain, from 

which p ersp ective should a p rofile b e d rafted? It’s unclear whether a use case p rofile would 

effectively p rovid e ecosystem- wid e allocation of resp onsib ilities. In ad d ition, the 

exp erimental and iterative nature of AI d evelop ment resemb les more R&D than trad itional 
p roduct d evelop ment; the risk management system need s to strike the right b alance 

b etween structure/certainty and flexib ility/mod ularity.

More b road ly, on this p oint around AI actors, and d espite the incorp oration of the OECD 

d efinition of AI actors in the second d raft, it is unclear what the various roles of organizations 

will b e with resp ect to the d esign, d evelop ment and d istrib ution and use of an AI system 

within the AI RMF. These d istinctions will b e however critical in informing a risk management 

p rogram. For instance, the mention of ‘joint resp onsib ility of all AI actors’ could b e 

misrep resented . The fact that the d raft RMF does not d istinguish b etween AI research and 

commercialisation is another area where a p rofile/use case would b e ap p rop riate.

RMF p la yb ook

We’ve noted the efforts p ut in d evelop ing the p laybook and in making it an interactive and 

accessib le p latform - what the p laybook shows are some of the trade - offs that might need 

to b e mad e b etween sharing sub stantial information as guidance and the p ossib ility that too 

much information could b ecome overwhelming to the RMF audience, or shift the p ercep tion 

of the framework to that of a b ox- checking comp liance exercise, rather than a resource to 

enab le d eep er d elib eration on the more c omp lex risks facing an organization One 

recommend ation could b e to earmark more clearly which section is likely to b e more relevant 

to which stakehold ers, so that ind ividuals can q uickly grasp what part of the p laybook they 

should b e focusing on.

While we welcome the enhanced level of d etail in the Playbook, we b elieve it’s important to 

p rovid e sufficient examp les to illustrate what ad equate imp lementation looks like. In 

p articular, further guid ance translating socio- regulatory req uirements into technical 

imp lementation would b e b eneficial.

To make the p laybook more actionab le, NIST c ould look to collate and share state of the art 

AI risk evaluation and mitigation tools, across the risks covered b y the RMF. This could build 

upon existing repositories, such the Partnership on AI’s work to aggregate and compare

Exp lainab le AI tools, and help users add ress the ‘choice overload’ posed b y the recent 

p roliferation of such tools.

NIST could work with p artners - in p articular und er- represented groups - to d evelop a 

rep resentative set of case stud ies, showing how organizations p lan to use the RMF, or how 

they currently do AI risk management. This would enab le RMF users to build up a clear p icture 

of what the RMF categories and sub categories might look like in p ractice. The case stud ies 

would also exp lain how risks and resp onsib ilities will d iffer d ep end ing on where an 

organization sits in the AI sup p ly chain - for examp le, an organization d eveloping a new 

lab eled d ataset, or running AI exp eriments with human feed back, will want to closely consid er 

the risks p osed b y d ata enrichment activitie s. Case stud ies could also demonstrate how RMF 

use will d iffer across sectors, b ut also for d ifferent actors within a single AI sup p ly chain, from 

und erlying dataset, mod el and compute d evelop ers, through to app lication d ep loyers, and 

affected users. In d oing so, they could highlight which actor is b est- p laced to act at d ifferent 
stages, but also which risks are d ep end ent on multip le actors and their interactions, and 

illustrate d ifferent app roaches to shared challenges. In add ition, the case stud ies could 

highlight how d ecisions and trad e - offs affect the resulting b enefits and harms of an AI 

ap p lication, and how they are d istrib uted . For examp le, our recent case study documented

how Deep Mind worked with third p arties to id entify and mitigate p otential unintend ed risks 

p osed b y releasing our Alp haFold p rotein structure p red iction model, as well as certain 

d ecisions we took to try and ensure that the b enefits would b e shared more eq uitab ly. 

Conc lus ion

Consid ering the extensive work und ertaken b y NIST to d evelop the AI RMF, and its potential 

to act as a long- term guid ing resource, we suggest that NIST also consid ers ways to share 

the work with relevant international p artners. For examp le, the UK is currently outlining their 

ap p roach to AI governance and regulation and could p otentially b enefit from similar 

resources.

We look forward to continuing to inp ut on the NIST AI RMF, and we would welcome the 

opp ortunity to have a call with you to d iscuss our comments further.

Kind regards, 

Alexandra

-- 

Alexandra Belias

International Public Policy Manager