ezesecops/rmf-ato-core huggingface dataset:
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Framing Risk > Organizational Integration and Management of Risk

Organizational Integration and Management of Risk

AI risks should not be considered in isolation. Different AI actors have different responsi-
bilities and awareness depending on their roles in the lifecycle. For example, organizations
developing an AI system often will not have information about how the system may be
used. AI risk management should be integrated and incorporated into broader enterprise
risk management strategies and processes. Treating AI risks along with other critical risks,
such as cybersecurity and privacy, will yield a more integrated outcome and organizational
efficiencies.
The AI RMF may be utilized along with related guidance and frameworks for managing
AI system risks or broader enterprise risks. Some risks related to AI systems are common
across other types of software development and deployment. Examples of overlapping risks
include: privacy concerns related to the use of underlying data to train AI systems; the en-
ergy and environmental implications associated with resource-heavy computing demands;
security concerns related to the confidentiality, integrity, and availability of the system and
its training and output data; and general security of the underlying software and hardware
for AI systems.
Organizations need to establish and maintain the appropriate accountability mechanisms,
roles and responsibilities, culture, and incentive structures for risk management to be ef-
fective. Use of the AI RMF alone will not lead to these changes or provide the appropriate
incentives. Effective risk management is realized through organizational commitment at
senior levels and may require cultural change within an organization or industry. In addi-
tion, small to medium-sized organizations managing AI risks or implementing the AI RMF
may face different challenges than large organizations, depending on their capabilities and
resources.
2.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
AI Risks and Trustworthiness > Privacy-Enhanced

Privacy-Enhanced

Privacy refers generally to the norms and practices that help to safeguard human autonomy,
identity, and dignity. These norms and practices typically address freedom from intrusion,
limiting observation, or individuals' agency to consent to disclosure or control of facets of
their identities (e.g., body, data, reputation). (See The NIST Privacy Framework: A Tool
for Improving Privacy through Enterprise Risk Management.)
Privacy values such as anonymity, confidentiality, and control generally should guide choices
for AI system design, development, and deployment. Privacy-related risks may influence
security, bias, and transparency and come with tradeoffs with these other characteristics.
Like safety and security, specific technical features of an AI system may promote or reduce
privacy. AI systems can also present new risks to privacy by allowing inference to identify
individuals or previously private information about individuals.
Privacy-enhancing technologies ("PETs") for AI, as well as data minimizing methods such
as de-identification and aggregation for certain model outputs, can support design for
privacy-enhanced AI systems. Under certain conditions such as data sparsity, privacy-
enhancing techniques can result in a loss in accuracy, affecting decisions about fairness
and other values in certain domains.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Framing Risk > Risk Prioritization

Risk Prioritization

Attempting to eliminate negative risk entirely can be counterproductive in practice because
not all incidents and failures can be eliminated. Unrealistic expectations about risk may
lead organizations to allocate resources in a manner that makes risk triage inefficient or
impractical or wastes scarce resources. A risk management culture can help organizations
recognize that not all AI risks are the same, and resources can be allocated purposefully.
Actionable risk management efforts lay out clear guidelines for assessing trustworthiness
of each AI system an organization develops or deploys. Policies and resources should be
prioritized based on the assessed risk level and potential impact of an AI system. The extent
to which an AI system may be customized or tailored to the specific context of use by the
AI deployer can be a contributing factor.
When applying the AI RMF, risks which the organization determines to be highest for the
AI systems within a given context of use call for the most urgent prioritization and most
thorough risk management process. In cases where an AI system presents unacceptable
negative risk levels - such as where significant negative impacts are imminent, severe harms
are actually occurring, or catastrophic risks are present - development and deployment
should cease in a safe manner until risks can be sufficiently managed. If an AI system's
development, deployment, and use cases are found to be low-risk in a specific context, that
may suggest potentially lower prioritization.
Risk prioritization may differ between AI systems that are designed or deployed to directly
interact with humans as compared to AI systems that are not. Higher initial prioritization
may be called for in settings where the AI system is trained on large datasets comprised of
sensitive or protected data such as personally identifiable information, or where the outputs
of the AI systems have direct or indirect impact on humans. AI systems designed to interact
only with computational systems and trained on non-sensitive datasets (for example, data
collected from the physical environment) may call for lower initial prioritization. Nonethe-
less, regularly assessing and prioritizing risk based on context remains important because
non-human-facing AI systems can have downstream safety or social implications.
Residual risk - defined as risk remaining after risk treatment (Source: ISO GUIDE 73) -
directly impacts end users or affected individuals and communities. Documenting residual
risks will call for the system provider to fully consider the risks of deploying the AI product
and will inform end users about potential negative impacts of interacting with the system.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Framing Risk > Risk Tolerance

Risk Tolerance

While the AI RMF can be used to prioritize risk, it does not prescribe risk tolerance. Risk
tolerance refers to the organization's or AI actor's (see Appendix A) readiness to bear the
risk in order to achieve its objectives. Risk tolerance can be influenced by legal or regula-
tory requirements (Adapted from: ISO GUIDE 73). Risk tolerance and the level of risk that
is acceptable to organizations or society are highly contextual and application and use-case
specific. Risk tolerances can be influenced by policies and norms established by AI sys-
tem owners, organizations, industries, communities, or policy makers. Risk tolerances are
likely to change over time as AI systems, policies, and norms evolve. Different organiza-
tions may have varied risk tolerances due to their particular organizational priorities and
resource considerations.
Emerging knowledge and methods to better inform harm/cost-benefit tradeoffs will con-
tinue to be developed and debated by businesses, governments, academia, and civil society.
To the extent that challenges for specifying AI risk tolerances remain unresolved, there may
be contexts where a risk management framework is not yet readily applicable for mitigating
negative AI risks.
The Framework is intended to be flexible and to augment existing risk practices
which should align with applicable laws, regulations, and norms. Organizations
should follow existing regulations and guidelines for risk criteria, tolerance, and
response established by organizational, domain, discipline, sector, or professional
requirements. Some sectors or industries may have established definitions of harm or
established documentation, reporting, and disclosure requirements. Within sectors,
risk management may depend on existing guidelines for specific applications and
use case settings. Where established guidelines do not exist, organizations should
define reasonable risk tolerance. Once tolerance is defined, this AI RMF can be used
to manage risks and to document risk management processes.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
AI Risks and Trustworthiness > Safe

Safe

AI systems should "not under defined conditions, lead to a state in which human life,
health, property, or the environment is endangered" (Source: ISO/IEC TS 5723:2022). Safe
operation of AI systems is improved through:
• responsible design, development, and deployment practices;
• clear information to deployers on responsible use of the system;
• responsible decision-making by deployers and end users; and
• explanations and documentation of risks based on empirical evidence of incidents.
Different types of safety risks may require tailored AI risk management approaches based
on context and the severity of potential risks presented. Safety risks that pose a potential
risk of serious injury or death call for the most urgent prioritization and most thorough risk
management process.
Employing safety considerations during the lifecycle and starting as early as possible with
planning and design can prevent failures or conditions that can render a system dangerous.
Other practical approaches for AI safety often relate to rigorous simulation and in-domain
testing, real-time monitoring, and the ability to shut down, modify, or have human inter-
vention into systems that deviate from intended or expected functionality.
AI safety risk management approaches should take cues from efforts and guidelines for
safety in fields such as transportation and healthcare, and align with existing sector- or
application-specific guidelines or standards.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
AI Risks and Trustworthiness > Secure and Resilient

Secure and Resilient

AI systems, as well as the ecosystems in which they are deployed, may be said to be re-
silient if they can withstand unexpected adverse events or unexpected changes in their envi-
ronment or use - or if they can maintain their functions and structure in the face of internal
and external change and degrade safely and gracefully when this is necessary (Adapted
from: ISO/IEC TS 5723:2022). Common security concerns relate to adversarial examples,
data poisoning, and the exfiltration of models, training data, or other intellectual property
through AI system endpoints. AI systems that can maintain confidentiality, integrity, and
availability through protection mechanisms that prevent unauthorized access and use may
be said to be secure. Guidelines in the NIST Cybersecurity Framework and Risk Manage-
ment Framework are among those which are applicable here.
Security and resilience are related but distinct characteristics. While resilience is the abil-
ity to return to normal function after an unexpected adverse event, security includes re-
silience but also encompasses protocols to avoid, protect against, respond to, or recover
from attacks. Resilience relates to robustness and goes beyond the provenance of the data
to encompass unexpected or adversarial use (or abuse or misuse) of the model or data.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Framing Risk > Understanding and Addressing Risks, Impacts, and Harms

Understanding and Addressing Risks, Impacts, and Harms

In the context of the AI RMF, risk refers to the composite measure of an event's probability
of occurring and the magnitude or degree of the consequences of the corresponding event.
The impacts, or consequences, of AI systems can be positive, negative, or both and can
result in opportunities or threats (Adapted from: ISO 31000:2018). When considering the
negative impact of a potential event, risk is a function of 1) the negative impact, or magni-
tude of harm, that would arise if the circumstance or event occurs and 2) the likelihood of
occurrence (Adapted from: OMB Circular A-130:2016). Negative impact or harm can be
experienced by individuals, groups, communities, organizations, society, the environment,
and the planet.
"Risk management refers to coordinated activities to direct and control an organiza-
tion with regard to risk" (Source: ISO 31000:2018).
While risk management processes generally address negative impacts, this Framework of-
fers approaches to minimize anticipated negative impacts of AI systems and identify op-
portunities to maximize positive impacts. Effectively managing the risk of potential harms
could lead to more trustworthy AI systems and unleash potential benefits to people (individ-
uals, communities, and society), organizations, and systems/ecosystems. Risk management
can enable AI developers and users to understand impacts and account for the inherent lim-
itations and uncertainties in their models and systems, which in turn can improve overall
system performance and trustworthiness and the likelihood that AI technologies will be
used in ways that are beneficial.
The AI RMF is designed to address new risks as they emerge. This flexibility is particularly
important where impacts are not easily foreseeable and applications are evolving. While
some AI risks and benefits are well-known, it can be challenging to assess negative impacts
and the degree of harms. Figure 1 provides examples of potential harms that can be related
to AI systems.
AI risk management efforts should consider that humans may assume that AI systems work
- and work well - in all settings. For example, whether correct or not, AI systems are
often perceived as being more objective than humans or as offering greater capabilities
than general software.
Fig. 1. Examples of potential harms related to AI systems. Trustworthy AI systems and their
responsible use can mitigate negative risks and contribute to benefits for people, organizations, and
ecosystems.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
AI Risks and Trustworthiness > Valid and Reliable

Valid and Reliable

Validation is the "confirmation, through the provision of objective evidence, that the re-
quirements for a specific intended use or application have been fulfilled" (Source: ISO
9000:2015). Deployment of AI systems which are inaccurate, unreliable, or poorly gener-
alized to data and settings beyond their training creates and increases negative AI risks and
reduces trustworthiness.
Reliability is defined in the same standard as the "ability of an item to perform as required,
without failure, for a given time interval, under given conditions" (Source: ISO/IEC TS
5723:2022). Reliability is a goal for overall correctness of AI system operation under the
conditions of expected use and over a given period of time, including the entire lifetime of
the system.
Accuracy and robustness contribute to the validity and trustworthiness of AI systems, and
can be in tension with one another in AI systems.
Accuracy is defined by ISO/IEC TS 5723:2022 as "closeness of results of observations,
computations, or estimates to the true values or the values accepted as being true." Mea-
sures of accuracy should consider computational-centric measures (e.g., false positive and
false negative rates), human-AI teaming, and demonstrate external validity (generalizable
beyond the training conditions). Accuracy measurements should always be paired with
clearly defined and realistic test sets - that are representative of conditions of expected use
- and details about test methodology; these should be included in associated documen-
tation. Accuracy measurements may include disaggregation of results for different data
segments.
Robustness or generalizability is defined as the "ability of a system to maintain its level
of performance under a variety of circumstances" (Source: ISO/IEC TS 5723:2022). Ro-
bustness is a goal for appropriate system functionality in a broad set of conditions and
circumstances, including uses of AI systems not initially anticipated. Robustness requires
not only that the system perform exactly as it does under expected uses, but also that it
should perform in ways that minimize potential harms to people if it is operating in an
unexpected setting.
Validity and reliability for deployed AI systems are often assessed by ongoing testing or
monitoring that confirms a system is performing as intended. Measurement of validity,
accuracy, robustness, and reliability contribute to trustworthiness and should take into con-
sideration that certain types of failures can cause greater harm. AI risk management efforts
should prioritize the minimization of potential negative impacts, and may need to include
human intervention in cases where the AI system cannot detect or correct errors.
NIST AI RMF Playbook
AI RMF Playbook > Govern > GOVERN 2.3

GOVERN 2.3: Executive leadership of the organization takes responsibility for decisions about risks associated with AI system development and deployment.

About:
Senior leadership and members of the C-Suite in organizations that maintain an AI portfolio, should maintain awareness of AI risks, affirm the organizational appetite for such risks, and be responsible for managing those risks..

Accountability ensures that a specific team and individual is responsible for AI risk management efforts. Some organizations grant authority and resources (human and budgetary) to a designated officer who ensures adequate performance of the institution's AI portfolio (e.g. predictive modeling, machine learning).

Suggested Actions:
- Organizational management can:
- Declare risk tolerances for developing or using AI systems.
- Support AI risk management efforts, and play an active role in such efforts.
- Integrate a risk and harm prevention mindset throughout the AI lifecycle as part of organizational culture
- Support competent risk management executives.
- Delegate the power, resources, and authorization to perform risk management to each appropriate level throughout the management chain.
- Organizations can establish board committees for AI risk management and oversight functions and integrate those functions within the organization's broader enterprise risk management approaches.

Transparency and Documentation:
### Organizations can document the following
- Did your organization's board and/or senior management sponsor, support and participate in your organization's AI governance?
- What are the roles, responsibilities, and delegation of authorities of personnel involved in the design, development, deployment, assessment and monitoring of the AI system?
- Do AI solutions provide sufficient information to assist the personnel to make an informed decision and take actions accordingly?
- To what extent has the entity clarified the roles, responsibilities, and delegated authorities to relevant stakeholders?

### AI Transparency Resources
- WEF Companion to the Model AI Governance Framework- 2020. [URL](https://www.pdpc.gov.sg/-/media/Files/PDPC/PDF-Files/Resource-for-Organisation/AI/SGIsago.pdf)
- GAO-21-519SP: AI Accountability Framework for Federal Agencies & Other Entities. [URL](https://www.gao.gov/products/gao-21-519sp)

References:
Bd. Governors Fed. Rsrv. Sys., Supervisory Guidance on Model Risk Management, SR Letter 11-7 (Apr. 4, 2011)

Off. Superintendent Fin. Inst. Canada, Enterprise-Wide Model Risk Management for Deposit-Taking Institutions, E-23 (Sept. 2017).
NIST AI RMF Playbook
AI RMF Playbook > Govern > GOVERN 6.2

GOVERN 6.2: Contingency processes are in place to handle failures or incidents in third-party data or AI systems deemed to be high-risk.

About:
To mitigate the potential harms of third-party system failures, organizations may implement policies and procedures that include redundancies for covering third-party functions.

Suggested Actions:
- Establish policies for handling third-party system failures to include consideration of redundancy mechanisms for vital third-party AI systems.
- Verify that incident response plans address third-party AI systems.

Transparency and Documentation:
### Organizations can document the following
- To what extent does the plan specifically address risks associated with acquisition, procurement of packaged software from vendors, cybersecurity controls, computational infrastructure, data, data science, deployment mechanics, and system failure?
- Did you establish a process for third parties (e.g. suppliers, end users, subjects, distributors/vendors or workers) to report potential vulnerabilities, risks or biases in the AI system?
- If your organization obtained datasets from a third party, did your organization assess and manage the risks of using such datasets?

### AI Transparency Resources
- GAO-21-519SP: AI Accountability Framework for Federal Agencies & Other Entities. [URL](https://www.gao.gov/products/gao-21-519sp)
- WEF Model AI Governance Framework Assessment 2020. [URL](https://www.pdpc.gov.sg/-/media/Files/PDPC/PDF-Files/Resource-for-Organisation/AI/SGModelAIGovFramework2.pdf)
- WEF Companion to the Model AI Governance Framework- 2020. [URL](https://www.pdpc.gov.sg/-/media/Files/PDPC/PDF-Files/Resource-for-Organisation/AI/SGIsago.pdf)
- AI policies and initiatives, in Artificial Intelligence in Society, OECD, 2019. [URL](https://www.oecd.org/publications/artificial-intelligence-in-society-eedfee77-en.htm)

References:
Bd. Governors Fed. Rsrv. Sys., Supervisory Guidance on Model Risk Management, SR Letter 11-7 (Apr. 4, 2011)

"Proposed Interagency Guidance on Third-Party Relationships: Risk Management," 2021. [URL](https://www.occ.gov/news-issuances/news-releases/2021/nr-occ-2021-74a.pdf)

Off. Comptroller Currency, Comptroller's Handbook: Model Risk Management (Aug. 2021). [URL](https://www.occ.gov/publications-and-resources/publications/comptrollers-handbook/files/model-risk-management/index-model-risk-management.html)