ezesecops/rmf-ato-core huggingface dataset:
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
AI RMF Core > Subcategories > MEASURE 3.2

MEASURE 3.2: Risk tracking approaches are considered for settings where AI risks are difficult to assess using currently available measurement techniques or where metrics are not yet available. Categories Subcategories Continued on next page Table 3: Categories and subcategories for the MEASURE function. (Continued)
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
AI RMF Core > Subcategories > MEASURE 3.3

MEASURE 3.3: Feedback processes for end users and impacted communities to report problems and appeal system outcomes are established and integrated into AI system evaluation metrics. MEASURE 4: Feedback about efficacy of measurement is gathered and assessed.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
AI RMF Core > Subcategories > MEASURE 4.1

MEASURE 4.1: Measurement approaches for identifying AI risks are connected to deployment context(s) and informed through consultation with domain experts and other end users. Approaches are documented.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
AI RMF Core > Subcategories > MEASURE 4.2

MEASURE 4.2: Measurement results regarding AI system trustworthiness in deployment context(s) and across the AI lifecycle are informed by input from domain experts and relevant AI actors to validate whether the system is performing consistently as intended. Results are documented.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
AI RMF Core > Subcategories > MEASURE 4.3

MEASURE 4.3: Measurable performance improvements or declines based on consultations with relevant AI actors, including affected communities, and field data about contextrelevant risks and trustworthiness characteristics are identified and documented. Categories Subcategories 5.4 Manage The MANAGE function entails allocating risk resources to mapped and measured risks on a regular basis and as defined by the GOVERN function. Risk treatment comprises plans to respond to, recover from, and communicate about incidents or events. Contextual information gleaned from expert consultation and input from relevant AI actors - established in GOVERN and carried out in MAP - is utilized in this function to decrease the likelihood of system failures and negative impacts. Systematic documentation practices established in GOVERN and utilized in MAP and MEASURE bolster AI risk management efforts and increase transparency and accountability. Processes for assessing emergent risks are in place, along with mechanisms for continual improvement. After completing the MANAGE function, plans for prioritizing risk and regular monitoring and improvement will be in place. Framework users will have enhanced capacity to manage the risks of deployed AI systems and to allocate risk management resources based on assessed and prioritized risks. It is incumbent on Framework users to continue to apply the MANAGE function to deployed AI systems as methods, contexts, risks, and needs or expectations from relevant AI actors evolve over time. Practices related to managing AI risks are described in the NIST AI RMF Playbook. Table 4 lists the MANAGE function's categories and subcategories. Table 4: Categories and subcategories for the MANAGE function. MANAGE 1: AI risks based on assessments and other analytical output from the MAP and MEASURE functions are prioritized, responded to, and managed.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
AI Risks and Trustworthiness > Accountable and Transparent

Accountable and Transparent

Trustworthy AI depends upon accountability. Accountability presupposes transparency.
Transparency reflects the extent to which information about an AI system and its outputs is
available to individuals interacting with such a system - regardless of whether they are even
aware that they are doing so. Meaningful transparency provides access to appropriate levels
of information based on the stage of the AI lifecycle and tailored to the role or knowledge
of AI actors or individuals interacting with or using the AI system. By promoting higher
levels of understanding, transparency increases confidence in the AI system.
This characteristic's scope spans from design decisions and training data to model train-
ing, the structure of the model, its intended use cases, and how and when deployment,
post-deployment, or end user decisions were made and by whom. Transparency is often
necessary for actionable redress related to AI system outputs that are incorrect or otherwise
lead to negative impacts. Transparency should consider human-AI interaction: for exam-
ple, how a human operator or user is notified when a potential or actual adverse outcome
caused by an AI system is detected. A transparent system is not necessarily an accurate,
privacy-enhanced, secure, or fair system. However, it is difficult to determine whether an
opaque system possesses such characteristics, and to do so over time as complex systems
evolve.
The role of AI actors should be considered when seeking accountability for the outcomes of
AI systems. The relationship between risk and accountability associated with AI and tech-
nological systems more broadly differs across cultural, legal, sectoral, and societal contexts.
When consequences are severe, such as when life and liberty are at stake, AI developers
and deployers should consider proportionally and proactively adjusting their transparency
and accountability practices. Maintaining organizational practices and governing structures
for harm reduction, like risk management, can help lead to more accountable systems.
Measures to enhance transparency and accountability should also consider the impact of
these efforts on the implementing entity, including the level of necessary resources and the
need to safeguard proprietary information.
Maintaining the provenance of training data and supporting attribution of the AI system's
decisions to subsets of training data can assist with both transparency and accountability.
Training data may also be subject to copyright and should follow applicable intellectual
property rights laws.
As transparency tools for AI systems and related documentation continue to evolve, devel-
opers of AI systems are encouraged to test different types of transparency tools in cooper-
ation with AI deployers to ensure that AI systems are used as intended.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Appendix A: > Descriptions of AI Actor Tasks from Figures 2 and 3 > Additional AI Actors

Additional AI Actors

Third-party entities include providers, developers, vendors, and evaluators of data, al-
gorithms, models, and/or systems and related services for another organization or the or-
ganization's customers or clients. Third-party entities are responsible for AI design and
development tasks, in whole or in part. By definition, they are external to the design, devel-
opment, or deployment team of the organization that acquires its technologies or services.
The technologies acquired from third-party entities may be complex or opaque, and risk
tolerances may not align with the deploying or operating organization.
End users of an AI system are the individuals or groups that use the system for specific
purposes. These individuals or groups interact with an AI system in a specific context. End
users can range in competency from AI experts to first-time technology end users.
Affected individuals/communities encompass all individuals, groups, communities, or
organizations directly or indirectly affected by AI systems or decisions based on the output
of AI systems. These individuals do not necessarily interact with the deployed system or
application.
Other AI actors may provide formal or quasi-formal norms or guidance for specifying
and managing AI risks. They can include trade associations, standards developing or-
ganizations, advocacy groups, researchers, environmental groups, and civil society
organizations.
The general public is most likely to directly experience positive and negative impacts of
AI technologies. They may provide the motivation for actions taken by the AI actors. This
group can include individuals, communities, and consumers associated with the context in
which an AI system is developed or deployed.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Appendix C: > AI Risk Management and Human-AI Interaction

AI Risk Management and Human-AI Interaction

Organizations that design, develop, or deploy AI systems for use in operational settings
may enhance their AI risk management by understanding current limitations of human-
AI interaction. The AI RMF provides opportunities to clearly define and differentiate the
various human roles and responsibilities when using, interacting with, or managing AI
systems.
Many of the data-driven approaches that AI systems rely on attempt to convert or represent
individual and social observational and decision-making practices into measurable quanti-
ties. Representing complex human phenomena with mathematical models can come at the
cost of removing necessary context. This loss of context may in turn make it difficult to
understand individual and societal impacts that are key to AI risk management efforts.
Issues that merit further consideration and research include:
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
AI RMF Core

AI RMF Core

The AI RMF Core provides outcomes and actions that enable dialogue, understanding, and
activities to manage AI risks and responsibly develop trustworthy AI systems. As illus-
trated in Figure 5, the Core is composed of four functions: GOVERN, MAP, MEASURE,
and MANAGE. Each of these high-level functions is broken down into categories and sub-
categories. Categories and subcategories are subdivided into specific actions and outcomes.
Actions do not constitute a checklist, nor are they necessarily an ordered set of steps.
Fig. 5. Functions organize AI risk management activities at their highest level to govern, map,
measure, and manage AI risks. Governance is designed to be a cross-cutting function to inform
and be infused throughout the other three functions.
Risk management should be continuous, timely, and performed throughout the AI system
lifecycle dimensions. AI RMF Core functions should be carried out in a way that reflects
diverse and multidisciplinary perspectives, potentially including the views of AI actors out-
side the organization. Having a diverse team contributes to more open sharing of ideas and
assumptions about purposes and functions of the technology being designed, developed,
deployed, or evaluated - which can create opportunities to surface problems and identify
existing and emergent risks.
An online companion resource to the AI RMF, the NIST AI RMF Playbook, is available
to help organizations navigate the AI RMF and achieve its outcomes through suggested
tactical actions they can apply within their own contexts. Like the AI RMF, the Playbook
is voluntary and organizations can utilize the suggestions according to their needs and
interests. Playbook users can create tailored guidance selected from suggested material
for their own use and contribute their suggestions for sharing with the broader community.
Along with the AI RMF, the Playbook is part of the NIST Trustworthy and Responsible AI
Resource Center.
Framework users may apply these functions as best suits their needs for managing
AI risks based on their resources and capabilities. Some organizations may choose
to select from among the categories and subcategories; others may choose and have
the capacity to apply all categories and subcategories. Assuming a governance struc-
ture is in place, functions may be performed in any order across the AI lifecycle as
deemed to add value by a user of the framework. After instituting the outcomes in
GOVERN, most users of the AI RMF would start with the MAP function and con-
tinue to MEASURE or MANAGE. However users integrate the functions, the process
should be iterative, with cross-referencing between functions as necessary. Simi-
larly, there are categories and subcategories with elements that apply to multiple
functions, or that logically should take place before certain subcategory decisions.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Appendix D: > Attributes of the AI RMF

Attributes of the AI RMF

NIST described several key attributes of the AI RMF when work on the Framework first
began. These attributes have remained intact and were used to guide the AI RMF's devel-
opment. They are provided here as a reference.
The AI RMF strives to:
1. Be risk-based, resource-efficient, pro-innovation, and voluntary.
2. Be consensus-driven and developed and regularly updated through an open, trans-
parent process. All stakeholders should have the opportunity to contribute to the AI
RMF's development.
3. Use clear and plain language that is understandable by a broad audience, including
senior executives, government officials, non-governmental organization leadership,
and those who are not AI professionals - while still of sufficient technical depth to
be useful to practitioners. The AI RMF should allow for communication of AI risks
across an organization, between organizations, with customers, and to the public at
large.
4. Provide common language and understanding to manage AI risks. The AI RMF
should offer taxonomy, terminology, definitions, metrics, and characterizations for
AI risk.
5. Be easily usable and fit well with other aspects of risk management. Use of the
Framework should be intuitive and readily adaptable as part of an organization's
broader risk management strategy and processes. It should be consistent or aligned
with other approaches to managing AI risks.
6. Be useful to a wide range of perspectives, sectors, and technology domains. The AI
RMF should be universally applicable to any AI technology and to context-specific
use cases.
7. Be outcome-focused and non-prescriptive. The Framework should provide a catalog
of outcomes and approaches rather than prescribe one-size-fits-all requirements.
8. Take advantage of and foster greater awareness of existing standards, guidelines, best
practices, methodologies, and tools for managing AI risks - as well as illustrate the
need for additional, improved resources.
9. Be law- and regulation-agnostic.
The Framework should support organizations'
abilities to operate under applicable domestic and international legal or regulatory
regimes.
10. Be a living document. The AI RMF should be readily updated as technology, under-
standing, and approaches to AI trustworthiness and uses of AI change and as stake-
holders learn from implementing AI risk management generally and this framework
in particular.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Effectiveness of the AI RMF

Effectiveness of the AI RMF

Evaluations of AI RMF effectiveness - including ways to measure bottom-line improve-
ments in the trustworthiness of AI systems - will be part of future NIST activities, in
conjunction with the AI community.
Organizations and other users of the Framework are encouraged to periodically evaluate
whether the AI RMF has improved their ability to manage AI risks, including but not lim-
ited to their policies, processes, practices, implementation plans, indicators, measurements,
and expected outcomes. NIST intends to work collaboratively with others to develop met-
rics, methodologies, and goals for evaluating the AI RMF's effectiveness, and to broadly
share results and supporting information. Framework users are expected to benefit from:
• enhanced processes for governing, mapping, measuring, and managing AI risk, and
clearly documenting outcomes;
• improved awareness of the relationships and tradeoffs among trustworthiness char-
acteristics, socio-technical approaches, and AI risks;
• explicit processes for making go/no-go system commissioning and deployment deci-
sions;
• established policies, processes, practices, and procedures for improving organiza-
tional accountability efforts related to AI system risks;
• enhanced organizational culture which prioritizes the identification and management
of AI system risks and potential impacts to individuals, communities, organizations,
and society;
• better information sharing within and across organizations about risks, decision-
making processes, responsibilities, common pitfalls, TEVV practices, and approaches
for continuous improvement;
• greater contextual knowledge for increased awareness of downstream risks;
• strengthened engagement with interested parties and relevant AI actors; and
• augmented capacity for TEVV of AI systems and associated risks.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
AI Risks and Trustworthiness > Explainable and Interpretable

Explainable and Interpretable

Explainability refers to a representation of the mechanisms underlying AI systems' oper-
ation, whereas interpretability refers to the meaning of AI systems' output in the context
of their designed functional purposes. Together, explainability and interpretability assist
those operating or overseeing an AI system, as well as users of an AI system, to gain
deeper insights into the functionality and trustworthiness of the system, including its out-
puts. The underlying assumption is that perceptions of negative risk stem from a lack of
ability to make sense of, or contextualize, system output appropriately. Explainable and
interpretable AI systems offer information that will help end users understand the purposes
and potential impact of an AI system.
Risk from lack of explainability may be managed by describing how AI systems function,
with descriptions tailored to individual differences such as the user's role, knowledge, and
skill level. Explainable systems can be debugged and monitored more easily, and they lend
themselves to more thorough documentation, audit, and governance.
Risks to interpretability often can be addressed by communicating a description of why
an AI system made a particular prediction or recommendation. (See "Four Principles of
Explainable Artificial Intelligence" and "Psychological Foundations of Explainability and
Interpretability in Artificial Intelligence" found here.)
Transparency, explainability, and interpretability are distinct characteristics that support
each other. Transparency can answer the question of "what happened" in the system. Ex-
plainability can answer the question of "how" a decision was made in the system. Inter-
pretability can answer the question of "why" a decision was made by the system and its
meaning or context to the user.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
AI Risks and Trustworthiness > Fair – with Harmful Bias Managed

Fair - with Harmful Bias Managed

Fairness in AI includes concerns for equality and equity by addressing issues such as harm-
ful bias and discrimination. Standards of fairness can be complex and difficult to define be-
cause perceptions of fairness differ among cultures and may shift depending on application.
Organizations' risk management efforts will be enhanced by recognizing and considering
these differences. Systems in which harmful biases are mitigated are not necessarily fair.
For example, systems in which predictions are somewhat balanced across demographic
groups may still be inaccessible to individuals with disabilities or affected by the digital
divide or may exacerbate existing disparities or systemic biases.
Bias is broader than demographic balance and data representativeness. NIST has identified
three major categories of AI bias to be considered and managed: systemic, computational
and statistical, and human-cognitive. Each of these can occur in the absence of prejudice,
partiality, or discriminatory intent. Systemic bias can be present in AI datasets, the orga-
nizational norms, practices, and processes across the AI lifecycle, and the broader society
that uses AI systems. Computational and statistical biases can be present in AI datasets
and algorithmic processes, and often stem from systematic errors due to non-representative
samples. Human-cognitive biases relate to how an individual or group perceives AI sys-
tem information to make a decision or fill in missing information, or how humans think
about purposes and functions of an AI system. Human-cognitive biases are omnipresent
in decision-making processes across the AI lifecycle and system use, including the design,
implementation, operation, and maintenance of AI.
Bias exists in many forms and can become ingrained in the automated systems that help
make decisions about our lives. While bias is not always a negative phenomenon, AI sys-
tems can potentially increase the speed and scale of biases and perpetuate and amplify
harms to individuals, groups, communities, organizations, and society. Bias is tightly asso-
ciated with the concepts of transparency as well as fairness in society. (For more informa-
tion about bias, including the three categories, see NIST Special Publication 1270, Towards
a Standard for Identifying and Managing Bias in Artificial Intelligence.)
4.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Framework (AI RMF 1.0)

Framework (AI RMF 1.0)

January 2023
Gina M. Raimondo, Secretary
Laurie E. Locascio, NIST Director and Under Secretary of Commerce for Standards and Technology
Certain commercial entities, equipment, or materials may be identified in this document in order to describe
an experimental procedure or concept adequately. Such identification is not intended to imply recommenda-
the entities, materials, or equipment are necessarily the best available for the purpose.
Update Schedule and Versions
The Artificial Intelligence Risk Management Framework (AI RMF) is intended to be a living document.
NIST will review the content and usefulness of the Framework regularly to determine if an update is appro-
priate; a review with formal input from the AI community is expected to take place no later than 2028. The
Framework will employ a two-number versioning system to track and identify major and minor changes. The
first number will represent the generation of the AI RMF and its companion documents (e.g., 1.0) and will
change only with major revisions. Minor revisions will be tracked using ".n" after the generation number
(e.g., 1.1). All changes will be tracked using a Version Control Table which identifies the history, including
version number, date of change, and description of change. NIST plans to update the AI RMF Playbook
frequently. Comments on the AI RMF Playbook may be sent via email to AIframework@nist.gov at any time
and will be reviewed and integrated on a semi-annual basis.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Framing Risk

Framing Risk

AI risk management offers a path to minimize potential negative impacts of AI systems,
such as threats to civil liberties and rights, while also providing opportunities to maximize
positive impacts. Addressing, documenting, and managing AI risks and potential negative
impacts effectively can lead to more trustworthy AI systems.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Appendix C: > AI Risk Management and Human-AI Interaction > need to be clearly defined and differentiated. Human-AI configurations can span

1. Human roles and responsibilities in decision making and overseeing AI systems

need to be clearly defined and differentiated. Human-AI configurations can span

from fully autonomous to fully manual. AI systems can autonomously make deci-
sions, defer decision making to a human expert, or be used by a human decision
maker as an additional opinion. Some AI systems may not require human oversight,
such as models used to improve video compression. Other systems may specifically
require human oversight.
2. Decisions that go into the design, development, deployment, evaluation, and use