Model: danicafisher/dfisher-sentence-transformer-fine-tuned
Author: danicafisher
Pipeline: sentence-similarity
Library: sentence-transformers
Tags: sentence-transformers, safetensors, bert, sentence-similarity, feature-extraction, generated_from_trainer, dataset_size:274, loss:MultipleNegativesRankingLoss, arxiv:1908.10084, arxiv:1705.00652, base_model:sentence-transformers/all-MiniLM-L6-v2, base_model:finetune:sentence-transformers/all-MiniLM-L6-v2, text-embeddings-inference, endpoints_compatible, region:us
Base models (finetune): id: sentence-transformers/all-MiniLM-L6-v2
Card metadata: base_model: sentence-transformers/all-MiniLM-L6-v2
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags: sentence-transformers, sentence-similarity, feature-extraction, generated_from_trainer, dataset_size:274, loss:MultipleNegativesRankingLoss
widget: source_sentence: What are the key considerations for conducting diligence on training data use in relation to intellectual property and privacy risks?
sentences: APPENDIX
•
Julia Simon-Mishel, Supervising Attorney, Philadelphia Legal Assistance
•
Dr. Zachary Mahafza, Research & Data Analyst, Southern Poverty Law Center
•
J. Khadijah Abdurahman, Tech Impact Network Research Fellow, AI Now Institute, UCLA C2I1, and
UWA Law School
Panelists separately described the increasing scope of technology use in providing for social welfare, including 
in fraud detection, digital ID systems, and other methods focused on improving efficiency and reducing cost. 
However, various panelists individually cautioned that these systems may reduce burden for government 
agencies by increasing the burden and agency of people using and interacting with these technologies. 
Additionally, these systems can produce feedback loops and compounded harm, collecting data from 
communities and using it to reinforce inequality. Various panelists suggested that these harms could be 
mitigated by ensuring community input at the beginning of the design process, providing ways to opt out of 
these systems and use associated human-driven mechanisms instead, ensuring timeliness of benefit payments, 
and providing clear notice about the use of these systems and clear explanations of how and what the 
technologies are doing. Some panelists suggested that technology should be used to help people receive 
benefits, e.g., by pushing benefits to those in need and ensuring automated decision-making systems are only 
used to provide a positive outcome; technology shouldn't be used to take supports away from people who need 
them. 
Panel 6: The Healthcare System. This event explored current and emerging uses of technology in the 
healthcare system and consumer products related to health. 
Welcome:
•
Alondra Nelson, Deputy Director for Science and Society, White House Office of Science and Technology
Policy
•
Patrick Gaspard, President and CEO, Center for American Progress
Moderator: Micky Tripathi, National Coordinator for Health Information Technology, U.S Department of 
Health and Human Services. 
Panelists: 
•
Mark Schneider, Health Innovation Advisor, ChristianaCare
•
Ziad Obermeyer, Blue Cross of California Distinguished Associate Professor of Policy and Management,
University of California, Berkeley School of Public Health
•
Dorothy Roberts, George A. Weiss University Professor of Law and Sociology and the Raymond Pace and
Sadie Tanner Mossell Alexander Professor of Civil Rights, University of Pennsylvania
•
David Jones, A. Bernard Ackerman Professor of the Culture of Medicine, Harvard University
•
Jamila Michener, Associate Professor of Government, Cornell University; Co-Director, Cornell Center for
Health Equity­
Panelists discussed the impact of new technologies on health disparities; healthcare access, delivery, and 
outcomes; and areas ripe for research and policymaking. Panelists discussed the increasing importance of tech-
nology as both a vehicle to deliver healthcare and a tool to enhance the quality of care. On the issue of 
delivery, various panelists pointed to a number of concerns including access to and expense of broadband 
service, the privacy concerns associated with telehealth systems, the expense associated with health 
monitoring devices, and how this can exacerbate equity issues.  On the issue of technology enhanced care, 
some panelists spoke extensively about the way in which racial biases and the use of race in medicine 
perpetuate harms and embed prior discrimination, and the importance of ensuring that the technologies used 
in medical care were accountable to the relevant stakeholders. Various panelists emphasized the importance 
of having the voices of those subjected to these technologies be heard.
59, 27 
MP-4.1-010 
Conduct appropriate diligence on training data use to assess intellectual property, 
and privacy, risks, including to examine whether use of proprietary or sensitive 
training data is consistent with applicable laws.  
Intellectual Property; Data Privacy 
AI Actor Tasks: Governance and Oversight, Operation and Monitoring, Procurement, Third-party entities 
 
MAP 5.1: Likelihood and magnitude of each identiﬁed impact (both potentially beneﬁcial and harmful) based on expected use, past 
uses of AI systems in similar contexts, public incident reports, feedback from those external to the team that developed or deployed 
the AI system, or other data are identiﬁed and documented. 
Action ID 
Suggested Action 
GAI Risks 
MP-5.1-001 Apply TEVV practices for content provenance (e.g., probing a system's synthetic 
data generation capabilities for potential misuse or vulnerabilities. 
Information Integrity; Information 
Security 
MP-5.1-002 
Identify potential content provenance harms of GAI, such as misinformation or 
disinformation, deepfakes, including NCII, or tampered content. Enumerate and 
rank risks based on their likelihood and potential impact, and determine how well 
provenance solutions address speciﬁc risks and/or harms. 
Information Integrity; Dangerous, 
Violent, or Hateful Content; 
Obscene, Degrading, and/or 
Abusive Content 
MP-5.1-003 
Consider disclosing use of GAI to end users in relevant contexts, while considering 
the objective of disclosure, the context of use, the likelihood and magnitude of the 
risk posed, the audience of the disclosure, as well as the frequency of the 
disclosures. 
Human-AI Conﬁguration 
MP-5.1-004 Prioritize GAI structured public feedback processes based on risk assessment 
estimates. 
Information Integrity; CBRN 
Information or Capabilities; 
Dangerous, Violent, or Hateful 
Content; Harmful Bias and 
Homogenization 
MP-5.1-005 Conduct adversarial role-playing exercises, GAI red-teaming, or chaos testing to 
identify anomalous or unforeseen failure modes. 
Information Security 
MP-5.1-006 
Proﬁle threats and negative impacts arising from GAI systems interacting with, 
manipulating, or generating content, and outlining known and potential 
vulnerabilities and the likelihood of their occurrence. 
Information Security 
AI Actor Tasks: AI Deployment, AI Design, AI Development, AI Impact Assessment, Aﬀected Individuals and Communities, End-
Users, Operation and Monitoring, 18 
GOVERN 3.2: Policies and procedures are in place to deﬁne and diﬀerentiate roles and responsibilities for human-AI conﬁgurations 
and oversight of AI systems. 
Action ID 
Suggested Action 
GAI Risks 
GV-3.2-001 
Policies are in place to bolster oversight of GAI systems with independent 
evaluations or assessments of GAI models or systems where the type and 
robustness of evaluations are proportional to the identiﬁed risks. 
CBRN Information or Capabilities; 
Harmful Bias and Homogenization 
GV-3.2-002 
Consider adjustment of organizational roles and components across lifecycle 
stages of large or complex GAI systems, including: Test and evaluation, validation, 
and red-teaming of GAI systems; GAI content moderation; GAI system 
development and engineering; Increased accessibility of GAI tools, interfaces, and 
systems, Incident response and containment. 
Human-AI Conﬁguration; 
Information Security; Harmful Bias 
and Homogenization 
GV-3.2-003 
Deﬁne acceptable use policies for GAI interfaces, modalities, and human-AI 
conﬁgurations (i.e., for chatbots and decision-making tasks), including criteria for 
the kinds of queries GAI applications should refuse to respond to.  
Human-AI Conﬁguration 
GV-3.2-004 
Establish policies for user feedback mechanisms for GAI systems which include 
thorough instructions and any mechanisms for recourse. 
Human-AI Conﬁguration  
GV-3.2-005 
Engage in threat modeling to anticipate potential risks from GAI systems. 
CBRN Information or Capabilities; 
Information Security 
AI Actors: AI Design 
 
GOVERN 4.1: Organizational policies and practices are in place to foster a critical thinking and safety-ﬁrst mindset in the design, 
development, deployment, and uses of AI systems to minimize potential negative impacts. 
Action ID 
Suggested Action 
GAI Risks 
GV-4.1-001 
Establish policies and procedures that address continual improvement processes 
for GAI risk measurement. Address general risks associated with a lack of 
explainability and transparency in GAI systems by using ample documentation and 
techniques such as: application of gradient-based attributions, occlusion/term 
reduction, counterfactual prompts and prompt engineering, and analysis of 
embeddings; Assess and update risk measurement approaches at regular 
cadences. 
Confabulation 
GV-4.1-002 
Establish policies, procedures, and processes detailing risk measurement in 
context of use with standardized measurement protocols and structured public 
feedback exercises such as AI red-teaming or independent external evaluations. 
CBRN Information and Capability; 
Value Chain and Component 
Integration, source_sentence: What should individuals be able to do when encountering problems with automated systems, according to the context provided?
sentences: 6 
2.2. Confabulation 
“Confabulation” refers to a phenomenon in which GAI systems generate and conﬁdently present 
erroneous or false content in response to prompts. Confabulations also include generated outputs that 
diverge from the prompts or other input or that contradict previously generated statements in the same 
context. These phenomena are colloquially also referred to as “hallucinations” or “fabrications.” 
Confabulations can occur across GAI outputs and contexts.9,10 Confabulations are a natural result of the 
way generative models are designed: they generate outputs that approximate the statistical distribution 
of their training data; for example, LLMs predict the next token or word in a sentence or phrase. While 
such statistical prediction can produce factually accurate and consistent outputs, it can also produce 
outputs that are factually inaccurate or internally inconsistent. This dynamic is particularly relevant when 
it comes to open-ended prompts for long-form responses and in domains which require highly 
contextual and/or domain expertise.  
Risks from confabulations may arise when users believe false content – often due to the conﬁdent nature 
of the response – leading users to act upon or promote the false information. This poses a challenge for 
many real-world applications, such as in healthcare, where a confabulated summary of patient 
information reports could cause doctors to make incorrect diagnoses and/or recommend the wrong 
treatments. Risks of confabulated content may be especially important to monitor when integrating GAI 
into applications involving consequential decision making. 
GAI outputs may also include confabulated logic or citations that purport to justify or explain the 
system’s answer, which may further mislead humans into inappropriately trusting the system’s output. 
For instance, LLMs sometimes provide logical steps for how they arrived at an answer even when the 
answer itself is incorrect. Similarly, an LLM could falsely assert that it is human or has human traits, 
potentially deceiving humans into believing they are speaking with another human. 
The extent to which humans can be deceived by LLMs, the mechanisms by which this may occur, and the 
potential risks from adversarial prompting of such behavior are emerging areas of study. Given the wide 
range of downstream impacts of GAI, it is diﬃcult to estimate the downstream scale and impact of 
confabulations. 
Trustworthy AI Characteristics: Fair with Harmful Bias Managed, Safe, Valid and Reliable, Explainable 
and Interpretable 
2.3. Dangerous, Violent, or Hateful Content 
GAI systems can produce content that is inciting, radicalizing, or threatening, or that gloriﬁes violence, 
with greater ease and scale than other technologies. LLMs have been reported to generate dangerous or 
violent recommendations, and some models have generated actionable instructions for dangerous or 
 
 
9 Confabulations of falsehoods are most commonly a problem for text-based outputs; for audio, image, or video 
content, creative generation of non-factual content can be a desired behavior.  
10 For example, legal confabulations have been shown to be pervasive in current state-of-the-art LLMs. See also, 
e.g.,, SECTION TITLE
HUMAN ALTERNATIVES, CONSIDERATION, AND FALLBACK
You should be able to opt out, where appropriate, and have access to a person who can quickly 
consider and remedy problems you encounter. You should be able to opt out from automated systems in 
favor of a human alternative, where appropriate. Appropriateness should be determined based on reasonable 
expectations in a given context and with a focus on ensuring broad accessibility and protecting the public from 
especially harmful impacts. In some cases, a human or other alternative may be required by law. You should have 
access to timely human consideration and remedy by a fallback and escalation process if an automated system 
fails, it produces an error, or you would like to appeal or contest its impacts on you. Human consideration and 
fallback should be accessible, equitable, effective, maintained, accompanied by appropriate operator training, and 
should not impose an unreasonable burden on the public. Automated systems with an intended use within sensi­
tive domains, including, but not limited to, criminal justice, employment, education, and health, should additional­
ly be tailored to the purpose, provide meaningful access for oversight, include training for any people interacting 
with the system, and incorporate human consideration for adverse or high-risk decisions. Reporting that includes 
a description of these human governance processes and assessment of their timeliness, accessibility, outcomes, 
and effectiveness should be made public whenever possible. 
Definitions for key terms in The Blueprint for an AI Bill of Rights can be found in Applying the Blueprint for an AI Bill of Rights. 
Accompanying analysis and tools for actualizing each principle can be found in the Technical Companion. 
7, FROM 
PRINCIPLES 
TO PRACTICE 
A TECHINCAL COMPANION TO
THE Blueprint for an 
AI BILL OF RIGHTS
12, source_sentence: How did the White House Office of Science and Technology Policy gather input from the American public regarding algorithmic and data-driven harms?
sentences: 23 
MP-1.1-002 
Determine and document the expected and acceptable GAI system context of 
use in collaboration with socio-cultural and other domain experts, by assessing: 
Assumptions and limitations; Direct value to the organization; Intended 
operational environment and observed usage patterns; Potential positive and 
negative impacts to individuals, public safety, groups, communities, 
organizations, democratic institutions, and the physical environment; Social 
norms and expectations. 
Harmful Bias and Homogenization 
MP-1.1-003 
Document risk measurement plans to address identiﬁed risks. Plans may 
include, as applicable: Individual and group cognitive biases (e.g., conﬁrmation 
bias, funding bias, groupthink) for AI Actors involved in the design, 
implementation, and use of GAI systems; Known past GAI system incidents and 
failure modes; In-context use and foreseeable misuse, abuse, and oﬀ-label use; 
Over reliance on quantitative metrics and methodologies without suﬃcient 
awareness of their limitations in the context(s) of use; Standard measurement 
and structured human feedback approaches; Anticipated human-AI 
conﬁgurations. 
Human-AI Conﬁguration; Harmful 
Bias and Homogenization; 
Dangerous, Violent, or Hateful 
Content 
MP-1.1-004 
Identify and document foreseeable illegal uses or applications of the GAI system 
that surpass organizational risk tolerances. 
CBRN Information or Capabilities; 
Dangerous, Violent, or Hateful 
Content; Obscene, Degrading, 
and/or Abusive Content 
AI Actor Tasks: AI Deployment 
 
MAP 1.2: Interdisciplinary AI Actors, competencies, skills, and capacities for establishing context reﬂect demographic diversity and 
broad domain and user experience expertise, and their participation is documented. Opportunities for interdisciplinary 
collaboration are prioritized. 
Action ID 
Suggested Action 
GAI Risks 
MP-1.2-001 
Establish and empower interdisciplinary teams that reﬂect a wide range of 
capabilities, competencies, demographic groups, domain expertise, educational 
backgrounds, lived experiences, professions, and skills across the enterprise to 
inform and conduct risk measurement and management functions. 
Human-AI Conﬁguration; Harmful 
Bias and Homogenization 
MP-1.2-002 
Verify that data or benchmarks used in risk measurement, and users, 
participants, or subjects involved in structured GAI public feedback exercises 
are representative of diverse in-context user populations. 
Human-AI Conﬁguration; Harmful 
Bias and Homogenization 
AI Actor Tasks: AI Deployment, 49 
early lifecycle TEVV approaches are developed and matured for GAI, organizations may use 
recommended “pre-deployment testing” practices to measure performance, capabilities, limits, risks, 
and impacts. This section describes risk measurement and estimation as part of pre-deployment TEVV, 
and examines the state of play for pre-deployment testing methodologies.  
Limitations of Current Pre-deployment Test Approaches 
Currently available pre-deployment TEVV processes used for GAI applications may be inadequate, non-
systematically applied, or fail to reﬂect or mismatched to deployment contexts. For example, the 
anecdotal testing of GAI system capabilities through video games or standardized tests designed for 
humans (e.g., intelligence tests, professional licensing exams) does not guarantee GAI system validity or 
reliability in those domains. Similarly, jailbreaking or prompt engineering tests may not systematically 
assess validity or reliability risks.  
Measurement gaps can arise from mismatches between laboratory and real-world settings. Current 
testing approaches often remain focused on laboratory conditions or restricted to benchmark test 
datasets and in silico techniques that may not extrapolate well to—or directly assess GAI impacts in real-
world conditions. For example, current measurement gaps for GAI make it diﬃcult to precisely estimate 
its potential ecosystem-level or longitudinal risks and related political, social, and economic impacts. 
Gaps between benchmarks and real-world use of GAI systems may likely be exacerbated due to prompt 
sensitivity and broad heterogeneity of contexts of use. 
A.1.5. Structured Public Feedback 
Structured public feedback can be used to evaluate whether GAI systems are performing as intended 
and to calibrate and verify traditional measurement methods. Examples of structured feedback include, 
but are not limited to: 
• 
Participatory Engagement Methods: Methods used to solicit feedback from civil society groups, 
aﬀected communities, and users, including focus groups, small user studies, and surveys. 
• 
Field Testing: Methods used to determine how people interact with, consume, use, and make 
sense of AI-generated information, and subsequent actions and eﬀects, including UX, usability, 
and other structured, randomized experiments.  
• 
AI Red-teaming: A structured testing exercise used to probe an AI system to ﬁnd ﬂaws and 
vulnerabilities such as inaccurate, harmful, or discriminatory outputs, often in a controlled 
environment and in collaboration with system developers. 
Information gathered from structured public feedback can inform design, implementation, deployment 
approval, maintenance, or decommissioning decisions. Results and insights gleaned from these exercises 
can serve multiple purposes, including improving data quality and preprocessing, bolstering governance 
decision making, and enhancing system documentation and debugging practices. When implementing 
feedback activities, organizations should follow human subjects research requirements and best 
practices such as informed consent and subject compensation., ABOUT THIS FRAMEWORK­­­­­
The Blueprint for an AI Bill of Rights is a set of five principles and associated practices to help guide the 
design, use, and deployment of automated systems to protect the rights of the American public in the age of 
artificial intel-ligence. Developed through extensive consultation with the American public, these principles are 
a blueprint for building and deploying automated systems that are aligned with democratic values and protect 
civil rights, civil liberties, and privacy. The Blueprint for an AI Bill of Rights includes this Foreword, the five 
principles, notes on Applying the The Blueprint for an AI Bill of Rights, and a Technical Companion that gives 
concrete steps that can be taken by many kinds of organizations—from governments at all levels to companies of 
all sizes—to uphold these values. Experts from across the private sector, governments, and international 
consortia have published principles and frameworks to guide the responsible use of automated systems; this 
framework provides a national values statement and toolkit that is sector-agnostic to inform building these 
protections into policy, practice, or the technological design process.  Where existing law or policy—such as 
sector-specific privacy laws and oversight requirements—do not already provide guidance, the Blueprint for an 
AI Bill of Rights should be used to inform policy decisions.
LISTENING TO THE AMERICAN PUBLIC
The White House Office of Science and Technology Policy has led a year-long process to seek and distill input 
from people across the country—from impacted communities and industry stakeholders to technology develop-
ers and other experts across fields and sectors, as well as policymakers throughout the Federal government—on 
the issue of algorithmic and data-driven harms and potential remedies. Through panel discussions, public listen-
ing sessions, meetings, a formal request for information, and input to a publicly accessible and widely-publicized 
email address, people throughout the United States, public servants across Federal agencies, and members of the 
international community spoke up about both the promises and potential harms of these technologies, and 
played a central role in shaping the Blueprint for an AI Bill of Rights. The core messages gleaned from these 
discussions include that AI has transformative potential to improve Americans’ lives, and that preventing the 
harms of these technologies is both necessary and achievable. The Appendix includes a full list of public engage-
ments. 
4, source_sentence: What are the suggested actions for establishing transparency policies related to GAI applications?
sentences: 42 
MG-2.4-002 
Establish and maintain procedures for escalating GAI system incidents to the 
organizational risk management authority when speciﬁc criteria for deactivation 
or disengagement is met for a particular context of use or for the GAI system as a 
whole. 
Information Security 
MG-2.4-003 
Establish and maintain procedures for the remediation of issues which trigger 
incident response processes for the use of a GAI system, and provide stakeholders 
timelines associated with the remediation plan. 
Information Security 
 
MG-2.4-004 Establish and regularly review speciﬁc criteria that warrants the deactivation of 
GAI systems in accordance with set risk tolerances and appetites. 
Information Security 
 
AI Actor Tasks: AI Deployment, Governance and Oversight, Operation and Monitoring 
 
MANAGE 3.1: AI risks and beneﬁts from third-party resources are regularly monitored, and risk controls are applied and 
documented. 
Action ID 
Suggested Action 
GAI Risks 
MG-3.1-001 
Apply organizational risk tolerances and controls (e.g., acquisition and 
procurement processes; assessing personnel credentials and qualiﬁcations, 
performing background checks; ﬁltering GAI input and outputs, grounding, ﬁne 
tuning, retrieval-augmented generation) to third-party GAI resources: Apply 
organizational risk tolerance to the utilization of third-party datasets and other 
GAI resources; Apply organizational risk tolerances to ﬁne-tuned third-party 
models; Apply organizational risk tolerance to existing third-party models 
adapted to a new domain; Reassess risk measurements after ﬁne-tuning third-
party GAI models. 
Value Chain and Component 
Integration; Intellectual Property 
MG-3.1-002 
Test GAI system value chain risks (e.g., data poisoning, malware, other software 
and hardware vulnerabilities; labor practices; data privacy and localization 
compliance; geopolitical alignment). 
Data Privacy; Information Security; 
Value Chain and Component 
Integration; Harmful Bias and 
Homogenization 
MG-3.1-003 
Re-assess model risks after ﬁne-tuning or retrieval-augmented generation 
implementation and for any third-party GAI models deployed for applications 
and/or use cases that were not evaluated in initial testing. 
Value Chain and Component 
Integration 
MG-3.1-004 
Take reasonable measures to review training data for CBRN information, and 
intellectual property, and where appropriate, remove it. Implement reasonable 
measures to prevent, ﬂag, or take other action in response to outputs that 
reproduce particular training data (e.g., plagiarized, trademarked, patented, 
licensed content or trade secret material). 
Intellectual Property; CBRN 
Information or Capabilities, DATA PRIVACY 
EXTRA PROTECTIONS FOR DATA RELATED TO SENSITIVE
DOMAINS
•
Continuous positive airway pressure machines gather data for medical purposes, such as diagnosing sleep
apnea, and send usage data to a patient’s insurance company, which may subsequently deny coverage for the
device based on usage data. Patients were not aware that the data would be used in this way or monitored
by anyone other than their doctor.70 
•
A department store company used predictive analytics applied to collected consumer data to determine that a
teenage girl was pregnant, and sent maternity clothing ads and other baby-related advertisements to her
house, revealing to her father that she was pregnant.71
•
School audio surveillance systems monitor student conversations to detect potential "stress indicators" as
a warning of potential violence.72 Online proctoring systems claim to detect if a student is cheating on an
exam using biometric markers.73 These systems have the potential to limit student freedom to express a range
of emotions at school and may inappropriately flag students with disabilities who need accommodations or
use screen readers or dictation software as cheating.74
•
Location data, acquired from a data broker, can be used to identify people who visit abortion clinics.75
•
Companies collect student data such as demographic information, free or reduced lunch status, whether
they've used drugs, or whether they've expressed interest in LGBTQI+ groups, and then use that data to 
forecast student success.76 Parents and education experts have expressed concern about collection of such
sensitive data without express parental consent, the lack of transparency in how such data is being used, and
the potential for resulting discriminatory impacts.
• Many employers transfer employee data to third party job verification services. This information is then used
by potential future employers, banks, or landlords. In one case, a former employee alleged that a
company supplied false data about her job title which resulted in a job offer being revoked.77
37, 14 
GOVERN 1.2: The characteristics of trustworthy AI are integrated into organizational policies, processes, procedures, and practices. 
Action ID 
Suggested Action 
GAI Risks 
GV-1.2-001 
Establish transparency policies and processes for documenting the origin and 
history of training data and generated data for GAI applications to advance digital 
content transparency, while balancing the proprietary nature of training 
approaches. 
Data Privacy; Information 
Integrity; Intellectual Property 
GV-1.2-002 
Establish policies to evaluate risk-relevant capabilities of GAI and robustness of 
safety measures, both prior to deployment and on an ongoing basis, through 
internal and external evaluations. 
CBRN Information or Capabilities; 
Information Security 
AI Actor Tasks: Governance and Oversight 
 
GOVERN 1.3: Processes, procedures, and practices are in place to determine the needed level of risk management activities based 
on the organization’s risk tolerance. 
Action ID 
Suggested Action 
GAI Risks 
GV-1.3-001 
Consider the following factors when updating or deﬁning risk tiers for GAI: Abuses 
and impacts to information integrity; Dependencies between GAI and other IT or 
data systems; Harm to fundamental rights or public safety; Presentation of 
obscene, objectionable, oﬀensive, discriminatory, invalid or untruthful output; 
Psychological impacts to humans (e.g., anthropomorphization, algorithmic 
aversion, emotional entanglement); Possibility for malicious use; Whether the 
system introduces signiﬁcant new security vulnerabilities; Anticipated system 
impact on some groups compared to others; Unreliable decision making 
capabilities, validity, adaptability, and variability of GAI system performance over 
time. 
Information Integrity; Obscene, 
Degrading, and/or Abusive 
Content; Value Chain and 
Component Integration; Harmful 
Bias and Homogenization; 
Dangerous, Violent, or Hateful 
Content; CBRN Information or 
Capabilities 
GV-1.3-002 
Establish minimum thresholds for performance or assurance criteria and review as 
part of deployment approval (“go/”no-go”) policies, procedures, and processes, 
with reviewed processes and approval thresholds reﬂecting measurement of GAI 
capabilities and risks. 
CBRN Information or Capabilities; 
Confabulation; Dangerous, 
Violent, or Hateful Content 
GV-1.3-003 
Establish a test plan and response policy, before developing highly capable models, 
to periodically evaluate whether the model may misuse CBRN information or 
capabilities and/or oﬀensive cyber capabilities. 
CBRN Information or Capabilities; 
Information Security, source_sentence: What methods are suggested for recording and integrating structured feedback about content provenance from various stakeholders in the context of GAI systems?
sentences: 39 
MS-3.3-004 
Provide input for training materials about the capabilities and limitations of GAI 
systems related to digital content transparency for AI Actors, other 
professionals, and the public about the societal impacts of AI and the role of 
diverse and inclusive content generation. 
Human-AI Conﬁguration; 
Information Integrity; Harmful Bias 
and Homogenization 
MS-3.3-005 
Record and integrate structured feedback about content provenance from 
operators, users, and potentially impacted communities through the use of 
methods such as user research studies, focus groups, or community forums. 
Actively seek feedback on generated content quality and potential biases. 
Assess the general awareness among end users and impacted communities 
about the availability of these feedback channels. 
Human-AI Conﬁguration; 
Information Integrity; Harmful Bias 
and Homogenization 
AI Actor Tasks: AI Deployment, Aﬀected Individuals and Communities, End-Users, Operation and Monitoring, TEVV 
 
MEASURE 4.2: Measurement results regarding AI system trustworthiness in deployment context(s) and across the AI lifecycle are 
informed by input from domain experts and relevant AI Actors to validate whether the system is performing consistently as 
intended. Results are documented. 
Action ID 
Suggested Action 
GAI Risks 
MS-4.2-001 
Conduct adversarial testing at a regular cadence to map and measure GAI risks, 
including tests to address attempts to deceive or manipulate the application of 
provenance techniques or other misuses. Identify vulnerabilities and 
understand potential misuse scenarios and unintended outputs. 
Information Integrity; Information 
Security 
MS-4.2-002 
Evaluate GAI system performance in real-world scenarios to observe its 
behavior in practical environments and reveal issues that might not surface in 
controlled and optimized testing environments. 
Human-AI Conﬁguration; 
Confabulation; Information 
Security 
MS-4.2-003 
Implement interpretability and explainability methods to evaluate GAI system 
decisions and verify alignment with intended purpose. 
Information Integrity; Harmful Bias 
and Homogenization 
MS-4.2-004 
Monitor and document instances where human operators or other systems 
override the GAI's decisions. Evaluate these cases to understand if the overrides 
are linked to issues related to content provenance. 
Information Integrity 
MS-4.2-005 
Verify and document the incorporation of results of structured public feedback 
exercises into design, implementation, deployment approval (“go”/“no-go” 
decisions), monitoring, and decommission decisions. 
Human-AI Conﬁguration; 
Information Security 
AI Actor Tasks: AI Deployment, Domain Experts, End-Users, Operation and Monitoring, TEVV, 46 
MG-4.3-003 
Report GAI incidents in compliance with legal and regulatory requirements (e.g., 
HIPAA breach reporting, e.g., OCR (2023) or NHTSA (2022) autonomous vehicle 
crash reporting requirements. 
Information Security; Data Privacy 
AI Actor Tasks: AI Deployment, Aﬀected Individuals and Communities, Domain Experts, End-Users, Human Factors, Operation and 
Monitoring, ENDNOTES
107. Centers for Medicare & Medicaid Services. Biden-Harris Administration Quadruples the Number
of Health Care Navigators Ahead of HealthCare.gov Open Enrollment Period. Aug. 27, 2021.
https://www.cms.gov/newsroom/press-releases/biden-harris-administration-quadruples-number­
health-care-navigators-ahead-healthcaregov-open
108. See, e.g., McKinsey & Company. The State of Customer Care in 2022. July 8, 2022. https://
www.mckinsey.com/business-functions/operations/our-insights/the-state-of-customer-care-in-2022;
Sara Angeles. Customer Service Solutions for Small Businesses. Business News Daily.
Jun. 29, 2022. https://www.businessnewsdaily.com/7575-customer-service-solutions.html
109. Mike Hughes. Are We Getting The Best Out Of Our Bots? Co-Intelligence Between Robots &
Humans. Forbes. Jul. 14, 2022.
https://www.forbes.com/sites/mikehughes1/2022/07/14/are-we-getting-the-best-out-of-our-bots-co­
intelligence-between-robots--humans/?sh=16a2bd207395
110. Rachel Orey and Owen Bacskai. The Low Down on Ballot Curing. Nov. 04, 2020. https://
bipartisanpolicy.org/blog/the-low-down-on-ballot-curing/; Zahavah Levine and Thea Raymond-
Seidel. Mail Voting Litigation in 2020, Part IV: Verifying Mail Ballots. Oct. 29, 2020.
https://www.lawfareblog.com/mail-voting-litigation-2020-part-iv-verifying-mail-ballots
111. National Conference of State Legislatures. Table 15: States With Signature Cure Processes. Jan. 18,
2022.
https://www.ncsl.org/research/elections-and-campaigns/vopp-table-15-states-that-permit-voters-to­
correct-signature-discrepancies.aspx
112. White House Office of Science and Technology Policy. Join the Effort to Create A Bill of Rights for
an Automated Society. Nov. 10, 2021.
https://www.whitehouse.gov/ostp/news-updates/2021/11/10/join-the-effort-to-create-a-bill-of­
rights-for-an-automated-society/
113. White House Office of Science and Technology Policy. Notice of Request for Information (RFI) on
Public and Private Sector Uses of Biometric Technologies. Issued Oct. 8, 2021.
https://www.federalregister.gov/documents/2021/10/08/2021-21975/notice-of-request-for­
information-rfi-on-public-and-private-sector-uses-of-biometric-technologies
114. National Artificial Intelligence Initiative Office. Public Input on Public and Private Sector Uses of
Biometric Technologies. Accessed Apr. 19, 2022.
https://www.ai.gov/86-fr-56300-responses/
115. Thomas D. Olszewski, Lisa M. Van Pay, Javier F. Ortiz, Sarah E. Swiersz, and Laurie A. Dacus.
Synopsis of Responses to OSTP’s Request for Information on the Use and Governance of Biometric
Technologies in the Public and Private Sectors. Science and Technology Policy Institute. Mar. 2022.
https://www.ida.org/-/media/feature/publications/s/sy/synopsis-of-responses-to-request-for­
information-on-the-use-and-governance-of-biometric-technologies/ida-document-d-33070.ashx
73
Likes: 0 | All-time downloads: 10
Created: 2024-09-20 17:34:21