Model: Cheselle/finetuned-arctic
Author: Cheselle
Pipeline: sentence-similarity
Library: sentence-transformers
Tags: sentence-transformers, safetensors, bert, sentence-similarity, feature-extraction, generated_from_trainer, dataset_size:600, loss:MatryoshkaLoss, loss:MultipleNegativesRankingLoss, arxiv:1908.10084, arxiv:2205.13147, arxiv:1705.00652, base_model:Snowflake/snowflake-arctic-embed-m, base_model:finetune:Snowflake/snowflake-arctic-embed-m, model-index, text-embeddings-inference, endpoints_compatible, region:us
Base models (finetune): id: Snowflake/snowflake-arctic-embed-m
Card metadata: base_model: Snowflake/snowflake-arctic-embed-m
library_name: sentence-transformers
metrics: cosine_accuracy@1, cosine_accuracy@3, cosine_accuracy@5, cosine_accuracy@10, cosine_precision@1, cosine_precision@3, cosine_precision@5, cosine_precision@10, cosine_recall@1, cosine_recall@3, cosine_recall@5, cosine_recall@10, cosine_ndcg@10, cosine_mrr@10, cosine_map@100, dot_accuracy@1, dot_accuracy@3, dot_accuracy@5, dot_accuracy@10, dot_precision@1, dot_precision@3, dot_precision@5, dot_precision@10, dot_recall@1, dot_recall@3, dot_recall@5, dot_recall@10, dot_ndcg@10, dot_mrr@10, dot_map@100
pipeline_tag: sentence-similarity
tags: sentence-transformers, sentence-similarity, feature-extraction, generated_from_trainer, dataset_size:600, loss:MatryoshkaLoss, loss:MultipleNegativesRankingLoss
widget: source_sentence: What are the existing regulatory safety requirements mentioned in the context for medical devices?
sentences: 47 
Appendix A. Primary GAI Considerations 
The following primary considerations were derived as overarching themes from the GAI PWG 
consultation process. These considerations (Governance, Pre-Deployment Testing, Content Provenance, 
and Incident Disclosure) are relevant for voluntary use by any organization designing, developing, and 
using GAI and also inform the Actions to Manage GAI risks. Information included about the primary 
considerations is not exhaustive, but highlights the most relevant topics derived from the GAI PWG.  
Acknowledgments: These considerations could not have been surfaced without the helpful analysis and 
contributions from the community and NIST staﬀ GAI PWG leads: George Awad, Luca Belli, Harold Booth, 
Mat Heyman, Yooyoung Lee, Mark Pryzbocki, Reva Schwartz, Martin Stanley, and Kyra Yee. 
A.1. Governance 
A.1.1. Overview 
Like any other technology system, governance principles and techniques can be used to manage risks, behavior or outcomes of a GAI model or system, how they could occur, and stress test safeguards”. AI 
red-teaming can be performed before or after AI models or systems are made available to the broader 
public; this section focuses on red-teaming in pre-deployment contexts.  
The quality of AI red-teaming outputs is related to the background and expertise of the AI red team 
itself. Demographically and interdisciplinarily diverse AI red teams can be used to identify ﬂaws in the 
varying contexts where GAI will be used. For best results, AI red teams should demonstrate domain 
expertise, and awareness of socio-cultural aspects within the deployment context. AI red-teaming results 
should be given additional analysis before they are incorporated into organizational governance and 
decision making, policy and procedural updates, and AI risk management eﬀorts. 
Various types of AI red-teaming may be appropriate, depending on the use case: 
•, SECTION TITLE
 
 
 
 
 
 
Applying The Blueprint for an AI Bill of Rights 
RELATIONSHIP TO EXISTING LAW AND POLICY
There are regulatory safety requirements for medical devices, as well as sector-, population-, or technology-spe­
cific privacy and security protections. Ensuring some of the additional protections proposed in this framework 
would require new laws to be enacted or new policies and practices to be adopted. In some cases, exceptions to 
the principles described in the Blueprint for an AI Bill of Rights may be necessary to comply with existing law, 
conform to the practicalities of a specific use case, or balance competing public interests. In particular, law 
enforcement, and other regulatory contexts may require government actors to protect civil rights, civil liberties, 
and privacy in a manner consistent with, but using alternate mechanisms to, the specific principles discussed in, source_sentence: What steps should be taken to adapt processes based on findings from incidents involving harmful content generation?
sentences: some cases may include personal data. The use of personal data for GAI training raises risks to widely 
accepted privacy principles, including to transparency, individual participation (including consent), and 
purpose speciﬁcation. For example, most model developers do not disclose speciﬁc data sources on 
which models were trained, limiting user awareness of whether personally identiﬁably information (PII) 
was trained on and, if so, how it was collected.  
Models may leak, generate, or correctly infer sensitive information about individuals. For example, 
during adversarial attacks, LLMs have revealed sensitive information (from the public domain) that was 
included in their training data. This problem has been referred to as data memorization, and may pose 
exacerbated privacy risks even for data present only in a small number of training samples.  
In addition to revealing sensitive information in GAI training data, GAI models may be able to correctly, performance, feedback received, and improvements made. 
Harmful Bias and Homogenization 
MG-4.2-002 
Practice and follow incident response plans for addressing the generation of 
inappropriate or harmful content and adapt processes based on ﬁndings to 
prevent future occurrences. Conduct post-mortem analyses of incidents with 
relevant AI Actors, to understand the root causes and implement preventive 
measures. 
Human-AI Conﬁguration; 
Dangerous, Violent, or Hateful 
Content 
MG-4.2-003 Use visualizations or other methods to represent GAI model behavior to ease 
non-technical stakeholders understanding of GAI system functionality. 
Human-AI Conﬁguration 
AI Actor Tasks: AI Deployment, AI Design, AI Development, Aﬀected Individuals and Communities, End-Users, Operation and 
Monitoring, TEVV 
 
MANAGE 4.3: Incidents and errors are communicated to relevant AI Actors, including aﬀected communities. Processes for tracking,, AI Actor Tasks: AI Deployment, AI Design, AI Impact Assessment, Aﬀected Individuals and Communities, Domain Experts, End-
Users, Human Factors, Operation and Monitoring  
 
MEASURE 1.1: Approaches and metrics for measurement of AI risks enumerated during the MAP function are selected for 
implementation starting with the most signiﬁcant AI risks. The risks or trustworthiness characteristics that will not – or cannot – be 
measured are properly documented. 
Action ID 
Suggested Action 
GAI Risks 
MS-1.1-001 Employ methods to trace the origin and modiﬁcations of digital content. 
Information Integrity 
MS-1.1-002 
Integrate tools designed to analyze content provenance and detect data 
anomalies, verify the authenticity of digital signatures, and identify patterns 
associated with misinformation or manipulation. 
Information Integrity 
MS-1.1-003 
Disaggregate evaluation metrics by demographic factors to identify any, source_sentence: What are the Principles of Artificial Intelligence Ethics developed by the US Intelligence Community intended to guide?
sentences: Evaluation data; Ethical considerations; Legal and regulatory requirements. 
Information Integrity; Harmful Bias 
and Homogenization 
AI Actor Tasks: AI Deployment, AI Impact Assessment, Domain Experts, End-Users, Operation and Monitoring, TEVV 
 
MEASURE 2.10: Privacy risk of the AI system – as identiﬁed in the MAP function – is examined and documented. 
Action ID 
Suggested Action 
GAI Risks 
MS-2.10-001 
Conduct AI red-teaming to assess issues such as: Outputting of training data 
samples, and subsequent reverse engineering, model extraction, and 
membership inference risks; Revealing biometric, conﬁdential, copyrighted, 
licensed, patented, personal, proprietary, sensitive, or trade-marked information; 
Tracking or revealing location information of users or members of training 
datasets. 
Human-AI Conﬁguration; 
Information Integrity; Intellectual 
Property 
MS-2.10-002 
Engage directly with end-users and other stakeholders to understand their, 8 
Trustworthy AI Characteristics: Accountable and Transparent, Privacy Enhanced, Safe, Secure and 
Resilient 
2.5. Environmental Impacts 
Training, maintaining, and operating (running inference on) GAI systems are resource-intensive activities, 
with potentially large energy and environmental footprints. Energy and carbon emissions vary based on 
what is being done with the GAI model (i.e., pre-training, ﬁne-tuning, inference), the modality of the 
content, hardware used, and type of task or application. 
Current estimates suggest that training a single transformer LLM can emit as much carbon as 300 round-
trip ﬂights between San Francisco and New York. In a study comparing energy consumption and carbon 
emissions for LLM inference, generative tasks (e.g., text summarization) were found to be more energy- 
and carbon-intensive than discriminative or non-generative tasks (e.g., text classiﬁcation)., security and defense activities.21 Similarly, the U.S. Intelligence Community (IC) has developed the Principles 
of Artificial Intelligence Ethics for the Intelligence Community to guide personnel on whether and how to 
develop and use AI in furtherance of the IC's mission, as well as an AI Ethics Framework to help implement 
these principles.22
The National Science Foundation (NSF) funds extensive research to help foster the 
development of automated systems that adhere to and advance their safety, security and 
effectiveness. Multiple NSF programs support research that directly addresses many of these principles: 
the National AI Research Institutes23 support research on all aspects of safe, trustworthy, fair, and explainable 
AI algorithms and systems; the Cyber Physical Systems24 program supports research on developing safe 
autonomous and cyber physical systems with AI components; the Secure and Trustworthy Cyberspace25, source_sentence: How does Hagan (2024) propose to establish quality standards for AI responses to legal problems?
sentences: actually occurring, or large-scale risks could occur); and broad GAI negative risks, 
including: Immature safety or risk cultures related to AI and GAI design, 
development and deployment, public information integrity risks, including impacts 
on democratic processes, unknown long-term performance characteristics of GAI. 
Information Integrity; Dangerous, 
Violent, or Hateful Content; CBRN 
Information or Capabilities 
GV-1.3-007 Devise a plan to halt development or deployment of a GAI system that poses 
unacceptable negative risk. 
CBRN Information and Capability; 
Information Security; Information 
Integrity 
AI Actor Tasks: Governance and Oversight 
 
GOVERN 1.4: The risk management process and its outcomes are established through transparent policies, procedures, and other 
controls based on organizational risk priorities. 
Action ID 
Suggested Action 
GAI Risks 
GV-1.4-001 
Establish policies and mechanisms to prevent GAI systems from generating, gists, advocates, journalists, policymakers, and communities in the United States and around the world. This 
technical companion is intended to be used as a reference by people across many circumstances – anyone 
impacted by automated systems, and anyone developing, designing, deploying, evaluating, or making policy to 
govern the use of an automated system. 
Each principle is accompanied by three supplemental sections: 
1
2
WHY THIS PRINCIPLE IS IMPORTANT: 
This section provides a brief summary of the problems that the principle seeks to address and protect against, including 
illustrative examples. 
WHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS: 
• The expectations for automated systems are meant to serve as a blueprint for the development of additional technical
standards and practices that should be tailored for particular sectors and contexts.
• This section outlines practical steps that can be implemented to realize the vision of the Blueprint for an AI Bill of Rights. The, Greshake, K. et al. (2023) Not what you've signed up for: Compromising Real-World LLM-Integrated 
Applications with Indirect Prompt Injection. arXiv. https://arxiv.org/abs/2302.12173 
Hagan, M. (2024) Good AI Legal Help, Bad AI Legal Help: Establishing quality standards for responses to 
people’s legal problem stories. SSRN. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4696936 
Haran, R. (2023) Securing LLM Systems Against Prompt Injection. NVIDIA. 
https://developer.nvidia.com/blog/securing-llm-systems-against-prompt-injection/ 
Information Technology Industry Council (2024) Authenticating AI-Generated Content. 
https://www.itic.org/policy/ITI_AIContentAuthorizationPolicy_122123.pdf 
Jain, S. et al. (2023) Algorithmic Pluralism: A Structural Approach To Equal Opportunity. arXiv. 
https://arxiv.org/pdf/2305.08157 
Ji, Z. et al (2023) Survey of Hallucination in Natural Language Generation. ACM Comput. Surv. 55, 12, 
Article 248. https://doi.org/10.1145/3571730, source_sentence: How can information security measures be applied to maintain the integrity and confidentiality of GAI models and systems?
sentences: using: ﬁeld testing with sub-group populations to determine likelihood of 
exposure to generated content exhibiting harmful bias, AI red-teaming with 
counterfactual and low-context (e.g., “leader,” “bad guys”) prompts. For ML 
pipelines or business processes with categorical or numeric outcomes that rely 
on GAI, apply general fairness metrics (e.g., demographic parity, equalized odds, 
equal opportunity, statistical hypothesis tests), to the pipeline or business 
outcome where appropriate; Custom, context-speciﬁc metrics developed in 
collaboration with domain experts and aﬀected communities; Measurements of 
the prevalence of denigration in generated content in deployment (e.g., sub-
sampling a fraction of traﬃc and manually annotating denigrating content). 
Harmful Bias and Homogenization; 
Dangerous, Violent, or Hateful 
Content 
MS-2.11-003 
Identify the classes of individuals, groups, or environmental ecosystems which, 27 
MP-4.1-010 
Conduct appropriate diligence on training data use to assess intellectual property, 
and privacy, risks, including to examine whether use of proprietary or sensitive 
training data is consistent with applicable laws.  
Intellectual Property; Data Privacy 
AI Actor Tasks: Governance and Oversight, Operation and Monitoring, Procurement, Third-party entities 
 
MAP 5.1: Likelihood and magnitude of each identiﬁed impact (both potentially beneﬁcial and harmful) based on expected use, past 
uses of AI systems in similar contexts, public incident reports, feedback from those external to the team that developed or deployed 
the AI system, or other data are identiﬁed and documented. 
Action ID 
Suggested Action 
GAI Risks 
MP-5.1-001 Apply TEVV practices for content provenance (e.g., probing a system's synthetic 
data generation capabilities for potential misuse or vulnerabilities. 
Information Integrity; Information 
Security 
MP-5.1-002, vulnerabilities in systems (hardware, software, data) and write code to exploit them. Sophisticated threat 
actors might further these risks by developing GAI-powered security co-pilots for use in several parts of 
the attack chain, including informing attackers on how to proactively evade threat detection and escalate 
privileges after gaining system access. 
Information security for GAI models and systems also includes maintaining availability of the GAI system 
and the integrity and (when applicable) the conﬁdentiality of the GAI code, training data, and model 
weights. To identify and secure potential attack points in AI systems or speciﬁc components of the AI 
 
 
12 See also https://doi.org/10.6028/NIST.AI.100-4, to be published.
model-index: name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-m
results: task: type: information-retrieval
name: Information Retrieval
dataset: name: Unknown
type: unknown
metrics: type: cosine_accuracy@1
value: 0.81
name: Cosine Accuracy@1
verified: no, type: cosine_accuracy@3
value: 0.96
name: Cosine Accuracy@3
verified: no, type: cosine_accuracy@5
value: 0.99
name: Cosine Accuracy@5
verified: no, type: cosine_accuracy@10
value: 1
name: Cosine Accuracy@10
verified: no, type: cosine_precision@1
value: 0.81
name: Cosine Precision@1
verified: no, type: cosine_precision@3
value: 0.31999999999999995
name: Cosine Precision@3
verified: no, type: cosine_precision@5
value: 0.19799999999999998
name: Cosine Precision@5
verified: no, type: cosine_precision@10
value: 0.09999999999999998
name: Cosine Precision@10
verified: no, type: cosine_recall@1
value: 0.81
name: Cosine Recall@1
verified: no, type: cosine_recall@3
value: 0.96
name: Cosine Recall@3
verified: no, type: cosine_recall@5
value: 0.99
name: Cosine Recall@5
verified: no, type: cosine_recall@10
value: 1
name: Cosine Recall@10
verified: no, type: cosine_ndcg@10
value: 0.9167865159386339
name: Cosine Ndcg@10
verified: no, type: cosine_mrr@10
value: 0.8887499999999998
name: Cosine Mrr@10
verified: no, type: cosine_map@100
value: 0.8887499999999998
name: Cosine Map@100
verified: no, type: dot_accuracy@1
value: 0.81
name: Dot Accuracy@1
verified: no, type: dot_accuracy@3
value: 0.96
name: Dot Accuracy@3
verified: no, type: dot_accuracy@5
value: 0.99
name: Dot Accuracy@5
verified: no, type: dot_accuracy@10
value: 1
name: Dot Accuracy@10
verified: no, type: dot_precision@1
value: 0.81
name: Dot Precision@1
verified: no, type: dot_precision@3
value: 0.31999999999999995
name: Dot Precision@3
verified: no, type: dot_precision@5
value: 0.19799999999999998
name: Dot Precision@5
verified: no, type: dot_precision@10
value: 0.09999999999999998
name: Dot Precision@10
verified: no, type: dot_recall@1
value: 0.81
name: Dot Recall@1
verified: no, type: dot_recall@3
value: 0.96
name: Dot Recall@3
verified: no, type: dot_recall@5
value: 0.99
name: Dot Recall@5
verified: no, type: dot_recall@10
value: 1
name: Dot Recall@10
verified: no, type: dot_ndcg@10
value: 0.9167865159386339
name: Dot Ndcg@10
verified: no, type: dot_mrr@10
value: 0.8887499999999998
name: Dot Mrr@10
verified: no, type: dot_map@100
value: 0.8887499999999998
name: Dot Map@100
verified: no
Likes: 0 | All-time downloads: 7
Created: 2024-09-23 10:23:25