Model: achapman/finetuned_arctic_ai_risk
Author: achapman
Pipeline: sentence-similarity
Library: sentence-transformers
Tags: sentence-transformers, safetensors, bert, sentence-similarity, feature-extraction, generated_from_trainer, dataset_size:600, loss:MatryoshkaLoss, loss:MultipleNegativesRankingLoss, arxiv:1908.10084, arxiv:2205.13147, arxiv:1705.00652, base_model:Snowflake/snowflake-arctic-embed-m, base_model:finetune:Snowflake/snowflake-arctic-embed-m, model-index, text-embeddings-inference, endpoints_compatible, region:us
Base models (finetune): id: Snowflake/snowflake-arctic-embed-m
Card metadata: base_model: Snowflake/snowflake-arctic-embed-m
library_name: sentence-transformers
metrics: cosine_accuracy@1, cosine_accuracy@3, cosine_accuracy@5, cosine_accuracy@10, cosine_precision@1, cosine_precision@3, cosine_precision@5, cosine_precision@10, cosine_recall@1, cosine_recall@3, cosine_recall@5, cosine_recall@10, cosine_ndcg@10, cosine_mrr@10, cosine_map@100, dot_accuracy@1, dot_accuracy@3, dot_accuracy@5, dot_accuracy@10, dot_precision@1, dot_precision@3, dot_precision@5, dot_precision@10, dot_recall@1, dot_recall@3, dot_recall@5, dot_recall@10, dot_ndcg@10, dot_mrr@10, dot_map@100
pipeline_tag: sentence-similarity
tags: sentence-transformers, sentence-similarity, feature-extraction, generated_from_trainer, dataset_size:600, loss:MatryoshkaLoss, loss:MultipleNegativesRankingLoss
widget: source_sentence: What considerations should be taken into account regarding the specific set or types of users for the AI system?
sentences: 46 
MG-4.3-003 
Report GAI incidents in compliance with legal and regulatory requirements (e.g., 
HIPAA breach reporting, e.g., OCR (2023) or NHTSA (2022) autonomous vehicle 
crash reporting requirements. 
Information Security; Data Privacy 
AI Actor Tasks: AI Deployment, Aﬀected Individuals and Communities, Domain Experts, End-Users, Human Factors, Operation and 
Monitoring, reporting, data protection, data privacy, or other laws. 
Data Privacy; Human-AI 
Conﬁguration; Information 
Security; Value Chain and 
Component Integration; Harmful 
Bias and Homogenization 
GV-6.2-004 
Establish policies and procedures for continuous monitoring of third-party GAI 
systems in deployment. 
Value Chain and Component 
Integration 
GV-6.2-005 
Establish policies and procedures that address GAI data redundancy, including 
model weights and other system artifacts., times, and availability of critical support. 
Human-AI Conﬁguration; 
Information Security; Value Chain 
and Component Integration 
AI Actor Tasks: AI Deployment, Operation and Monitoring, TEVV, Third-party entities 
 
MAP 1.1: Intended purposes, potentially beneﬁcial uses, context speciﬁc laws, norms and expectations, and prospective settings in 
which the AI system will be deployed are understood and documented. Considerations include: the speciﬁc set or types of users, source_sentence: What should organizations leverage when deploying GAI applications and using third-party pre-trained models?
sentences: external use, narrow vs. broad application scope, ﬁne-tuning, and varieties of 
data sources (e.g., grounding, retrieval-augmented generation). 
Data Privacy; Intellectual 
Property, 44 
MG-3.2-007 
Leverage feedback and recommendations from organizational boards or 
committees related to the deployment of GAI applications and content 
provenance when using third-party pre-trained models. 
Information Integrity; Value Chain 
and Component Integration 
MG-3.2-008 
Use human moderation systems where appropriate to review generated content 
in accordance with human-AI conﬁguration policies established in the Govern, Security 
MS-2.7-003 
Conduct user surveys to gather user satisfaction with the AI-generated content 
and user perceptions of content authenticity. Analyze user feedback to identify 
concerns and/or current literacy levels related to content provenance and 
understanding of labels on content. 
Human-AI Conﬁguration; 
Information Integrity 
MS-2.7-004 
Identify metrics that reﬂect the eﬀectiveness of security measures, such as data, source_sentence: What are the potential positive and negative impacts of AI system uses on individuals and communities?
sentences: and Homogenization 
AI Actor Tasks: AI Deployment, Aﬀected Individuals and Communities, End-Users, Operation and Monitoring, TEVV 
 
MEASURE 4.2: Measurement results regarding AI system trustworthiness in deployment context(s) and across the AI lifecycle are 
informed by input from domain experts and relevant AI Actors to validate whether the system is performing consistently as 
intended. Results are documented. 
Action ID 
Suggested Action 
GAI Risks 
MS-4.2-001, bias based on race, gender, disability, or other protected classes.  
Harmful bias in GAI systems can also lead to harms via disparities between how a model performs for 
diﬀerent subgroups or languages (e.g., an LLM may perform less well for non-English languages or 
certain dialects). Such disparities can contribute to discriminatory decision-making or ampliﬁcation of 
existing societal biases. In addition, GAI systems may be inappropriately trusted to perform similarly, along with their expectations; potential positive and negative impacts of system uses to individuals, communities, organizations, 
society, and the planet; assumptions and related limitations about AI system purposes, uses, and risks across the development or 
product AI lifecycle; and related TEVV and system metrics. 
Action ID 
Suggested Action 
GAI Risks 
MP-1.1-001 
When identifying intended purposes, consider factors such as internal vs., source_sentence: How does the suggested action MG-41-001 aim to address GAI risks?
sentences: most appropriate baseline is to compare against, which can result in divergent views on when a disparity between 
AI behaviors for diﬀerent subgroups constitutes a harm. In discussing harms from disparities such as biased 
behavior, this document highlights examples where someone’s situation is worsened relative to what it would have 
been in the absence of any AI system, making the outcome unambiguously a harm of the system., Harmful Bias Managed, Privacy Enhanced, Safe, Secure and Resilient, Valid and Reliable 
3. 
Suggested Actions to Manage GAI Risks 
The following suggested actions target risks unique to or exacerbated by GAI. 
In addition to the suggested actions below, AI risk management activities and actions set forth in the AI 
RMF 1.0 and Playbook are already applicable for managing GAI risks. Organizations are encouraged to, MANAGE 4.1: Post-deployment AI system monitoring plans are implemented, including mechanisms for capturing and evaluating 
input from users and other relevant AI Actors, appeal and override, decommissioning, incident response, recovery, and change 
management. 
Action ID 
Suggested Action 
GAI Risks 
MG-4.1-001 
Collaborate with external researchers, industry experts, and community 
representatives to maintain awareness of emerging best practices and, source_sentence: What are some examples of input data features that may serve as proxies for demographic group membership in GAI systems?
sentences: data privacy violations, obscenity, extremism, violence, or CBRN information in 
system training data. 
Data Privacy; Intellectual Property; 
Obscene, Degrading, and/or 
Abusive Content; Harmful Bias and 
Homogenization; Dangerous, 
Violent, or Hateful Content; CBRN 
Information or Capabilities 
MS-2.6-003 Re-evaluate safety features of ﬁne-tuned models when the negative risk exceeds 
organizational risk tolerance. 
Dangerous, Violent, or Hateful 
Content, GAI. 
Information Integrity; Intellectual 
Property 
AI Actor Tasks: Governance and Oversight, Operation and Monitoring 
 
GOVERN 1.6: Mechanisms are in place to inventory AI systems and are resourced according to organizational risk priorities. 
Action ID 
Suggested Action 
GAI Risks 
GV-1.6-001 Enumerate organizational GAI systems for incorporation into AI system inventory 
and adjust AI system inventory requirements to account for GAI risks. 
Information Security, complex or unstructured data; Input data features that may serve as proxies for 
demographic group membership (i.e., image metadata, language dialect) or 
otherwise give rise to emergent bias within GAI systems; The extent to which 
the digital divide may negatively impact representativeness in GAI system 
training and TEVV data; Filtering of hate speech or content in GAI system 
training data; Prevalence of GAI-generated data in GAI system training data. 
Harmful Bias and Homogenization
model-index: name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-m
results: task: type: information-retrieval
name: Information Retrieval
dataset: name: Unknown
type: unknown
metrics: type: cosine_accuracy@1
value: 0.85
name: Cosine Accuracy@1
verified: no, type: cosine_accuracy@3
value: 0.975
name: Cosine Accuracy@3
verified: no, type: cosine_accuracy@5
value: 1
name: Cosine Accuracy@5
verified: no, type: cosine_accuracy@10
value: 1
name: Cosine Accuracy@10
verified: no, type: cosine_precision@1
value: 0.85
name: Cosine Precision@1
verified: no, type: cosine_precision@3
value: 0.325
name: Cosine Precision@3
verified: no, type: cosine_precision@5
value: 0.19999999999999998
name: Cosine Precision@5
verified: no, type: cosine_precision@10
value: 0.09999999999999999
name: Cosine Precision@10
verified: no, type: cosine_recall@1
value: 0.85
name: Cosine Recall@1
verified: no, type: cosine_recall@3
value: 0.975
name: Cosine Recall@3
verified: no, type: cosine_recall@5
value: 1
name: Cosine Recall@5
verified: no, type: cosine_recall@10
value: 1
name: Cosine Recall@10
verified: no, type: cosine_ndcg@10
value: 0.9341754705038519
name: Cosine Ndcg@10
verified: no, type: cosine_mrr@10
value: 0.911875
name: Cosine Mrr@10
verified: no, type: cosine_map@100
value: 0.9118749999999999
name: Cosine Map@100
verified: no, type: dot_accuracy@1
value: 0.85
name: Dot Accuracy@1
verified: no, type: dot_accuracy@3
value: 0.975
name: Dot Accuracy@3
verified: no, type: dot_accuracy@5
value: 1
name: Dot Accuracy@5
verified: no, type: dot_accuracy@10
value: 1
name: Dot Accuracy@10
verified: no, type: dot_precision@1
value: 0.85
name: Dot Precision@1
verified: no, type: dot_precision@3
value: 0.325
name: Dot Precision@3
verified: no, type: dot_precision@5
value: 0.19999999999999998
name: Dot Precision@5
verified: no, type: dot_precision@10
value: 0.09999999999999999
name: Dot Precision@10
verified: no, type: dot_recall@1
value: 0.85
name: Dot Recall@1
verified: no, type: dot_recall@3
value: 0.975
name: Dot Recall@3
verified: no, type: dot_recall@5
value: 1
name: Dot Recall@5
verified: no, type: dot_recall@10
value: 1
name: Dot Recall@10
verified: no, type: dot_ndcg@10
value: 0.9341754705038519
name: Dot Ndcg@10
verified: no, type: dot_mrr@10
value: 0.911875
name: Dot Mrr@10
verified: no, type: dot_map@100
value: 0.9118749999999999
name: Dot Map@100
verified: no
Likes: 0 | All-time downloads: 11
Created: 2024-09-21 13:39:01