ezesecops/rmf-ato-core huggingface dataset:
NIST AI RMF Playbook
AI RMF Playbook > Measure > MEASURE 2.10 (part 2)

Zhanglong Ji, Zachary C. Lipton, and Charles Elkan. "Differential Privacy and Machine Learning: a Survey and Review." arXiv preprint, submitted December 24,2014. [URL](https://arxiv.org/abs/1412.7584)

Michael B. Hawes. "Implementing Differential Privacy: Seven Lessons From the 2020 United States Census." Harvard Data Science Review 2, no. 2 (2020). [URL](https://doi.org/10.1162/99608f92.353c6f99)

Harvard University Privacy Tools Project. "Differential Privacy." Harvard University, n.d. [URL](https://privacytools.seas.harvard.edu/differential-privacy)

John M. Abowd, Robert Ashmead, Ryan Cumings-Menon, Simson Garfinkel, Micah Heineck, Christine Heiss, Robert Johns, Daniel Kifer, Philip Leclerc, Ashwin Machanavajjhala, Brett Moran, William Matthew Spence Sexton and Pavel Zhuravlev. "The 2020 Census Disclosure Avoidance System TopDown Algorithm." United States Census Bureau, April 7, 2022. [URL](https://www2.census.gov/adrm/CED/Papers/CY22/2022-002-AbowdAshmeadCumingMenonGarfinkelEtal.pdf)

Nicolas Papernot and Abhradeep Guha Thakurta. "How to deploy machine learning with differential privacy." National Institute of Standards and Technology (NIST), December 21, 2021. [URL](https://www.nist.gov/blogs/cybersecurity-insights/how-deploy-machine-learning-differential-privacy)

Claire McKay Bowen. "Utility Metrics for Differential Privacy: No One-Size-Fits-All." National Institute of Standards and Technology (NIST), November 29, 2021. [URL](https://www.nist.gov/blogs/cybersecurity-insights/utility-metrics-differential-privacy-no-one-size-fits-all)

Helen Nissenbaum. "Contextual Integrity Up and Down the Data Food Chain." Theoretical Inquiries in Law 20, L. 221 (2019): 221-256. [URL](https://nissenbaum.tech.cornell.edu/papers/Contextual%20Integrity%20Up%20and%20Down.pdf)

Sebastian Benthall, Seda Gürses, and Helen Nissenbaum. "Contextual Integrity through the Lens of Computer Science." Foundations and Trends in Privacy and Security 2, no. 1 (December 22, 2017): 1-69. [URL](https://doi.org/10.1561/3300000016)

Jenifer Sunrise Winter and Elizabeth Davidson. "Big Data Governance of Personal Health Information and Challenges to Contextual Integrity." The Information Society: An International Journal 35, no. 1 (2019): 36-51. [URL](https://doi.org/10.1080/01972243.2018.1542648.
NIST AI RMF Playbook
AI RMF Playbook > Measure > MEASURE 2.11 (part 1)

MEASURE 2.11: Fairness and bias - as identified in the MAP function - are evaluated and results are documented.

About:
Fairness in AI includes concerns for equality and equity by addressing issues such as harmful bias and discrimination. Standards of fairness can be complex and difficult to define because perceptions of fairness differ among cultures and may shift depending on application. Organizations' risk management efforts will be enhanced by recognizing and considering these differences. Systems in which harmful biases are mitigated are not necessarily fair. For example, systems in which predictions are somewhat balanced across demographic groups may still be inaccessible to individuals with disabilities or affected by the digital divide or may exacerbate existing disparities or systemic biases.

Bias is broader than demographic balance and data representativeness. NIST has identified three major categories of AI bias to be considered and managed: systemic, computational and statistical, and human-cognitive. Each of these can occur in the absence of prejudice, partiality, or discriminatory intent.

- Systemic bias can be present in AI datasets, the organizational norms, practices, and processes across the AI lifecycle, and the broader society that uses AI systems.
- Computational and statistical biases can be present in AI datasets and algorithmic processes, and often stem from systematic errors due to non-representative samples.
- Human-cognitive biases relate to how an individual or group perceives AI system information to make a decision or fill in missing information, or how humans think about purposes and functions of an AI system. Human-cognitive biases are omnipresent in decision-making processes across the AI lifecycle and system use, including the design, implementation, operation, and maintenance of AI.

Bias exists in many forms and can become ingrained in the automated systems that help make decisions about our lives. While bias is not always a negative phenomenon, AI systems can potentially increase the speed and scale of biases and perpetuate and amplify harms to individuals, groups, communities, organizations, and society.
NIST AI RMF Playbook
AI RMF Playbook > Measure > MEASURE 4.1 (part 2)

George Margetis, Stavroula Ntoa, Margherita Antona, and Constantine Stephanidis. "Human-Centered Design of Artificial Intelligence." In Handbook of Human Factors and Ergonomics, edited by Gavriel Salvendy and Waldemar Karwowski, 5th ed., 1085-1106. John Wiley & Sons, 2021. [URL](https://onlinelibrary.wiley.com/doi/10.1002/9781119636113.ch42)

Caitlin Thompson. "Who's Homeless Enough for Housing? In San Francisco, an Algorithm Decides." Coda, September 21, 2021. [URL](https://www.codastory.com/authoritarian-tech/san-francisco-homeless-algorithm/)

John Zerilli, Alistair Knott, James Maclaurin, and Colin Gavaghan. "Algorithmic Decision-Making and the Control Problem." Minds and Machines 29, no. 4 (December 11, 2019): 555-78. [URL](https://doi.org/10.1007/s11023-019-09513-7)

Fry, Hannah. Hello World: Being Human in the Age of Algorithms. New York: W.W. Norton & Company, 2018. [URL](https://wwnorton.com/books/Hello-World)

Sasha Costanza-Chock. Design Justice: Community-Led Practices to Build the Worlds We Need. Cambridge: The MIT Press, 2020. [URL](https://direct.mit.edu/books/book/4605/Design-JusticeCommunity-Led-Practices-to-Build-the)

David G. Robinson. Voices in the Code: A Story About People, Their Values, and the Algorithm They Made. New York: Russell Sage Foundation, 2022. [URL](https://www.russellsage.org/publications/voices-code)

Diane Hart, Gabi Diercks-O'Brien, and Adrian Powell. "Exploring Stakeholder Engagement in Impact Evaluation Planning in Educational Development Work." Evaluation 15, no. 3 (2009): 285-306. [URL](https://doi.org/10.1177/1356389009105882)

Asit Bhattacharyya and Lorne Cummings. "Measuring Corporate Environmental Performance - Stakeholder Engagement Evaluation." Business Strategy and the Environment 24, no. 5 (2013): 309-25. [URL](https://doi.org/10.1002/bse.1819)

Hendricks, Sharief, Nailah Conrad, Tania S. Douglas, and Tinashe Mutsvangwa. "A Modified Stakeholder Participation Assessment Framework for Design Thinking in Health Innovation." Healthcare 6, no. 3 (September 2018): 191-96. [URL](https://doi.org/10.1016/j.hjdsi.2018.06.003)

Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. "Stakeholder Participation in AI: Beyond 'Add Diverse Stakeholders and Stir.'" arXiv preprint, submitted November 1, 2021. [URL](https://arxiv.org/abs/2111.01122)

Emanuel Moss, Elizabeth Watkins, Ranjit Singh, Madeleine Clare Elish, and Jacob Metcalf. "Assembling Accountability: Algorithmic Impact Assessment for the Public Interest." SSRN, July 8, 2021. [URL](https://doi.org/10.2139/ssrn.3877437)

Alexandra Reeve Givens, and Meredith Ringel Morris. "Centering Disability Perspectives in Algorithmic Fairness, Accountability, & Transparency." FAT* '20: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, January 27, 2020, 684-84. [URL](https://doi.org/10.1145/3351095.3375686)
Standards for Security Categorization of Federal Information and Information Systems
APPENDIX A TERMS AND DEFINITIONS > AVAILABILITY

AVAILABILITY: Ensuring timely and reliable access to and use of information. [44 U.S.C., SEC. 3542]
Standards for Security Categorization of Federal Information and Information Systems
APPENDIX A TERMS AND DEFINITIONS > CONFIDENTIALITY

CONFIDENTIALITY: Preserving authorized restrictions on information access and disclosure, including means for protecting personal privacy and proprietary information. [44 U.S.C., SEC. 3542]
Standards for Security Categorization of Federal Information and Information Systems
APPENDIX A TERMS AND DEFINITIONS > EXECUTIVE AGENCY

EXECUTIVE AGENCY: An executive department specified in 5 U.S.C., SEC. 101; a military department specified in 5 U.S.C., SEC. 102; an independent establishment as defined in 5 U.S.C., SEC. 104(1); and a wholly owned Government corporation fully subject to the provisions of 31 U.S.C., CHAPTER 91. [41 U.S.C., SEC. 403]
Standards for Security Categorization of Federal Information and Information Systems
APPENDIX A TERMS AND DEFINITIONS > FEDERAL INFORMATION SYSTEM

FEDERAL INFORMATION SYSTEM: An information system used or operated by an executive agency, by a contractor of an executive agency, or by another organization on behalf of an executive agency. [40 U.S.C., SEC. 11331]
Standards for Security Categorization of Federal Information and Information Systems
APPENDIX A TERMS AND DEFINITIONS > INFORMATION RESOURCES

INFORMATION RESOURCES: Information and related resources, such as personnel, equipment, funds, and information technology. [44 U.S.C., SEC. 3502]
Standards for Security Categorization of Federal Information and Information Systems
APPENDIX A TERMS AND DEFINITIONS > INFORMATION SECURITY

INFORMATION SECURITY: The protection of information and information systems from unauthorized access, use, disclosure, disruption, modification, or destruction in order to provide confidentiality, integrity, and availability. [44 U.S.C., SEC. 3542]
Standards for Security Categorization of Federal Information and Information Systems
APPENDIX A TERMS AND DEFINITIONS > INFORMATION SYSTEM

INFORMATION SYSTEM: A discrete set of information resources organized for the collection, processing, maintenance, use, sharing, dissemination, or disposition of information. [44 U.S.C., SEC. 3502]
Standards for Security Categorization of Federal Information and Information Systems
APPENDIX A TERMS AND DEFINITIONS > INFORMATION TECHNOLOGY

INFORMATION TECHNOLOGY: Any equipment or interconnected system or subsystem of equipment that is used in the automatic acquisition, storage, manipulation, management, movement, control, display, switching, interchange, transmission, or reception of data or information by the executive agency. For purposes of the preceding sentence, equipment is used by an executive agency if the equipment is used by the executive agency directly or is used by a contractor under a contract with the executive agency which: (i) requires the use of such equipment; or (ii) requires the use, to a significant extent, of such equipment in the performance of a service or the furnishing of a product. The term information technology includes computers, ancillary equipment, software, firmware and similar procedures, services (including support services), and related resources. [40 U.S.C., SEC. 1401]
Standards for Security Categorization of Federal Information and Information Systems
APPENDIX A TERMS AND DEFINITIONS > INFORMATION TYPE

INFORMATION TYPE: A specific category of information (e.g., privacy, medical, proprietary, financial, investigative, contractor sensitive, security management), defined by an organization, or in some instances, by a specific law, Executive Order, directive, policy, or regulation.
Standards for Security Categorization of Federal Information and Information Systems
APPENDIX A TERMS AND DEFINITIONS > INTEGRITY

INTEGRITY: Guarding against improper information modification or destruction, and includes ensuring information non-repudiation and authenticity. [44 U.S.C., SEC. 3542]
Standards for Security Categorization of Federal Information and Information Systems
APPENDIX A TERMS AND DEFINITIONS > NATIONAL SECURITY SYSTEM

NATIONAL SECURITY SYSTEM: Any information system (including any telecommunications system) used or operated by an agency or by a contractor of an agency, or other organization on behalf of an agency- (i) the function, operation, or use of which involves intelligence activities; involves cryptologic activities related to national security; involves command and control of military forces; involves equipment that is an integral part of a weapon or weapons system; or is critical to the direct fulfillment of military or intelligence missions (excluding a system that is to be used for routine administrative and business applications, for example, payroll, finance, logistics, and personnel management applications); or, (ii) is protected at all times by procedures established for information that have been specifically authorized under criteria established by an Executive Order or an Act of Congress to be kept classified in the interest of national defense or foreign policy. [44 U.S.C., SEC. 3542]
Standards for Security Categorization of Federal Information and Information Systems
APPENDIX A TERMS AND DEFINITIONS > SECURITY CATEGORY

SECURITY CATEGORY: The characterization of information or an information system based on an assessment of the potential impact that a loss of confidentiality, integrity, or availability of such information or information system would have on organizational operations, organizational assets, or individuals.
Standards for Security Categorization of Federal Information and Information Systems
APPENDIX A TERMS AND DEFINITIONS > SECURITY CONTROLS

SECURITY CONTROLS: The management, operational, and technical controls (i.e., safeguards or countermeasures) prescribed for an information system to protect the confidentiality, integrity, and availability of the system and its information.
Standards for Security Categorization of Federal Information and Information Systems
1 PURPOSE

1 PURPOSE

The E-Government Act of 2002 (Public Law 107-347), passed by the one hundred and seventh
Congress and signed into law by the President in December 2002, recognized the importance of
information security to the economic and national security interests of the United States. Title III of
the E-Government Act, entitled the Federal Information Security Management Act of 2002 (FISMA),
tasked NIST with responsibilities for standards and guidelines, including the development of:
•
Standards to be used by all federal agencies to categorize all information and information systems
collected or maintained by or on behalf of each agency based on the objectives of providing
appropriate levels of information security according to a range of risk levels;
•
Guidelines recommending the types of information and information systems to be included in
each category; and
•
Minimum information security requirements (i.e., management, operational, and technical
controls), for information and information systems in each such category.
FIPS Publication 199 addresses the first task cited-to develop standards for categorizing information
and information systems. Security categorization standards for information and information systems
provide a common framework and understanding for expressing security that, for the federal
government, promotes: (i) effective management and oversight of information security programs,
including the coordination of information security efforts throughout the civilian, national security,
emergency preparedness, homeland security, and law enforcement communities; and (ii) consistent
reporting to the Office of Management and Budget (OMB) and Congress on the adequacy and
effectiveness of information security policies, procedures, and practices. Subsequent NIST standards
and guidelines will address the second and third tasks cited.
Standards for Security Categorization of Federal Information and Information Systems
2 APPLICABILITY

2 APPLICABILITY

These standards shall apply to: (i) all information within the federal government other than that
information that has been determined pursuant to Executive Order 12958, as amended by Executive
Order 13292, or any predecessor order, or by the Atomic Energy Act of 1954, as amended, to require
protection against unauthorized disclosure and is marked to indicate its classified status; and (ii) all
federal information systems other than those information systems designated as national security
systems as defined in 44 United States Code Section 3542(b)(2). Agency officials shall use the
security categorizations described in FIPS Publication 199 whenever there is a federal requirement to
provide such a categorization of information or information systems. Additional security designators
may be developed and used at agency discretion. State, local, and tribal governments as well as
private sector organizations comprising the critical infrastructure of the United States may consider
the use of these standards as appropriate. These standards are effective upon approval by the
Secretary of Commerce.
Standards for Security Categorization of Federal Information and Information Systems
3 CATEGORIZATION OF INFORMATION AND INFORMATION SYSTEMS

3 CATEGORIZATION OF INFORMATION AND INFORMATION SYSTEMS

This publication establishes security categories for both information1 and information systems. The
security categories are based on the potential impact on an organization should certain events occur
which jeopardize the information and information systems needed by the organization to accomplish
its assigned mission, protect its assets, fulfill its legal responsibilities, maintain its day-to-day
functions, and protect individuals. Security categories are to be used in conjunction with vulnerability
and threat information in assessing the risk to an organization.
1 Information is categorized according to its information type. An information type is a specific category of information
(e.g., privacy, medical, proprietary, financial, investigative, contractor sensitive, security management) defined by an
organization or, in some instances, by a specific law, Executive Order, directive, policy, or regulation.
Standards for Security Categorization of Federal Information and Information Systems
REFERENCES

REFERENCES

[1]
Privacy Act of 1974 (Public Law 93-579), September 1975.
[2]
Paperwork Reduction Act of 1995 (Public Law 104-13), May 1995.
[3]
OMB Circular A-130, Transmittal Memorandum #4, Management of Federal Information
Resources, November 2000.
[4]
Information Technology Management Reform Act of 1996 (Public Law 104-106), August
1996.
[5]
Federal Information Security Management Act of 2002 (Public Law 107-347), December
2002.
Standards for Security Categorization of Federal Information and Information Systems
3 CATEGORIZATION OF INFORMATION AND INFORMATION SYSTEMS > Potential Impact on Organizations and Individuals

Potential Impact on Organizations and Individuals

FIPS Publication 199 defines three levels of potential impact on organizations or individuals should
there be a breach of security (i.e., a loss of confidentiality, integrity, or availability). The application
of these definitions must take place within the context of each organization and the overall national
interest.
The potential impact is LOW if-
- The loss of confidentiality, integrity, or availability could be expected to have a limited adverse
effect on organizational operations, organizational assets, or individuals.2
AMPLIFICATION: A limited adverse effect means that, for example, the loss of confidentiality,
integrity, or availability might: (i) cause a degradation in mission capability to an extent and duration
that the organization is able to perform its primary functions, but the effectiveness of the functions is
noticeably reduced; (ii) result in minor damage to organizational assets; (iii) result in minor financial
loss; or (iv) result in minor harm to individuals.
The potential impact is MODERATE if-
- The loss of confidentiality, integrity, or availability could be expected to have a serious adverse
effect on organizational operations, organizational assets, or individuals.
AMPLIFICATION: A serious adverse effect means that, for example, the loss of confidentiality,
integrity, or availability might: (i) cause a significant degradation in mission capability to an extent
and duration that the organization is able to perform its primary functions, but the effectiveness of the
functions is significantly reduced; (ii) result in significant damage to organizational assets; (iii) result
in significant financial loss; or (iv) result in significant harm to individuals that does not involve loss
of life or serious life threatening injuries.
2 Adverse effects on individuals may include, but are not limited to, loss of the privacy to which individuals are entitled
under law.
The potential impact is HIGH if-
- The loss of confidentiality, integrity, or availability could be expected to have a severe or
catastrophic adverse effect on organizational operations, organizational assets, or individuals.
AMPLIFICATION: A severe or catastrophic adverse effect means that, for example, the loss of
confidentiality, integrity, or availability might: (i) cause a severe degradation in or loss of mission
capability to an extent and duration that the organization is not able to perform one or more of its
primary functions; (ii) result in major damage to organizational assets; (iii) result in major financial
loss; or (iv) result in severe or catastrophic harm to individuals involving loss of life or serious life
threatening injuries.
Standards for Security Categorization of Federal Information and Information Systems
3 CATEGORIZATION OF INFORMATION AND INFORMATION SYSTEMS > Security Categorization Applied to Information Types

Security Categorization Applied to Information Types

The security category of an information type can be associated with both user information and system
information3 and can be applicable to information in either electronic or non-electronic form. It can
also be used as input in considering the appropriate security category of an information system (see
description of security categories for information systems below). Establishing an appropriate
security category of an information type essentially requires determining the potential impact for each
security objective associated with the particular information type.
The generalized format for expressing the security category, SC, of an information type is:
SC information type = {(confidentiality, impact), (integrity, impact), (availability, impact)},
where the acceptable values for potential impact are LOW, MODERATE, HIGH, or NOT APPLICABLE.4
EXAMPLE 1: An organization managing public information on its web server determines that there is
no potential impact from a loss of confidentiality (i.e., confidentiality requirements are not applicable),
a moderate potential impact from a loss of integrity, and a moderate potential impact from a loss of
availability. The resulting security category, SC, of this information type is expressed as:
SC public information = {(confidentiality, NA), (integrity, MODERATE), (availability, MODERATE)}.
EXAMPLE 2: A law enforcement organization managing extremely sensitive investigative information
determines that the potential impact from a loss of confidentiality is high, the potential impact from a
loss of integrity is moderate, and the potential impact from a loss of availability is moderate. The
resulting security category, SC, of this information type is expressed as:
SC investigative information = {(confidentiality, HIGH), (integrity, MODERATE), (availability, MODERATE)}.
EXAMPLE 3: A financial organization managing routine administrative information (not privacy-related
information) determines that the potential impact from a loss of confidentiality is low, the potential
impact from a loss of integrity is low, and the potential impact from a loss of availability is low. The
resulting security category, SC, of this information type is expressed as:
SC administrative information = {(confidentiality, LOW), (integrity, LOW), (availability, LOW)}.
3 System information (e.g., network routing tables, password files, and cryptographic key management information)
must be protected at a level commensurate with the most critical or sensitive user information being processed, stored,
or transmitted by the information system to ensure confidentiality, integrity, and availability.
4 The potential impact value of not applicable only applies to the security objective of confidentiality.