Model: Vira21/finetuned_arctic
Author: Vira21
Pipeline: sentence-similarity
Library: sentence-transformers
Tags: sentence-transformers, safetensors, bert, sentence-similarity, feature-extraction, generated_from_trainer, dataset_size:600, loss:MatryoshkaLoss, loss:MultipleNegativesRankingLoss, arxiv:1908.10084, arxiv:2205.13147, arxiv:1705.00652, base_model:Snowflake/snowflake-arctic-embed-m, base_model:finetune:Snowflake/snowflake-arctic-embed-m, model-index, text-embeddings-inference, endpoints_compatible, region:us
Base models (finetune): id: Snowflake/snowflake-arctic-embed-m
Card metadata: tags: sentence-transformers, sentence-similarity, feature-extraction, generated_from_trainer, dataset_size:600, loss:MatryoshkaLoss, loss:MultipleNegativesRankingLoss
base_model: Snowflake/snowflake-arctic-embed-m
widget: source_sentence: What are the potential risks associated with the impersonation and cyber-attacks mentioned in the context?
sentences: Technology Engagement Center 
Uber Technologies 
University of Pittsburgh 
Undergraduate Student 
Collaborative 
Upturn 
US Technology Policy Committee 
of the Association of Computing 
Machinery 
Virginia Puccio 
Visar Berisha and Julie Liss 
XR Association 
XR Safety Initiative 
• As an additional effort to reach out to stakeholders regarding the RFI, OSTP conducted two listening sessions
for members of the public. The listening sessions together drew upwards of 300 participants. The Science and
Technology Policy Institute produced a synopsis of both the RFI submissions and the feedback at the listening
sessions.115
61, across all subgroups, which could leave the groups facing underperformance with worse outcomes than 
if no GAI system were used. Disparate or reduced performance for lower-resource languages also 
presents challenges to model adoption, inclusion, and accessibility, and may make preservation of 
endangered languages more diﬃcult if GAI systems become embedded in everyday processes that would 
otherwise have been opportunities to use these languages.  
Bias is mutually reinforcing with the problem of undesired homogenization, in which GAI systems 
produce skewed distributions of outputs that are overly uniform (for example, repetitive aesthetic styles, impersonation, cyber-attacks, and weapons creation. 
CBRN Information or Capabilities; 
Information Security 
MS-2.6-007 Regularly evaluate GAI system vulnerabilities to possible circumvention of safety 
measures.  
CBRN Information or Capabilities; 
Information Security 
AI Actor Tasks: AI Deployment, AI Impact Assessment, Domain Experts, Operation and Monitoring, TEVV, source_sentence: What techniques are suggested to assess and manage statistical biases related to GAI content provenance?
sentences: 2 
This work was informed by public feedback and consultations with diverse stakeholder groups as part of NIST’s 
Generative AI Public Working Group (GAI PWG). The GAI PWG was an open, transparent, and collaborative 
process, facilitated via a virtual workspace, to obtain multistakeholder input on GAI risk management and to 
inform NIST’s approach. 
The focus of the GAI PWG was limited to four primary considerations relevant to GAI: Governance, Content 
Provenance, Pre-deployment Testing, and Incident Disclosure (further described in Appendix A). As such, the 
suggested actions in this document primarily address these considerations. 
Future revisions of this proﬁle will include additional AI RMF subcategories, risks, and suggested actions based 
on additional considerations of GAI as the space evolves and empirical evidence indicates additional risks. A 
glossary of terms pertinent to GAI risk management will be developed and hosted on NIST’s Trustworthy &, 30 
MEASURE 2.2: Evaluations involving human subjects meet applicable requirements (including human subject protection) and are 
representative of the relevant population. 
Action ID 
Suggested Action 
GAI Risks 
MS-2.2-001 Assess and manage statistical biases related to GAI content provenance through 
techniques such as re-sampling, re-weighting, or adversarial training. 
Information Integrity; Information 
Security; Harmful Bias and 
Homogenization 
MS-2.2-002 
Document how content provenance data is tracked and how that data interacts 
with privacy and security. Consider: Anonymizing data to protect the privacy of 
human subjects; Leveraging privacy output ﬁlters; Removing any personally 
identiﬁable information (PII) to prevent potential harm or misuse. 
Data Privacy; Human AI 
Conﬁguration; Information 
Integrity; Information Security; 
Dangerous, Violent, or Hateful 
Content 
MS-2.2-003 Provide human subjects with options to withdraw participation or revoke their, Homogenization? arXiv. https://arxiv.org/pdf/2211.13972 
Boyarskaya, M. et al. (2020) Overcoming Failures of Imagination in AI Infused System Development and 
Deployment. arXiv. https://arxiv.org/pdf/2011.13416 
Browne, D. et al. (2023) Securing the AI Pipeline. Mandiant. 
https://www.mandiant.com/resources/blog/securing-ai-pipeline 
Burgess, M. (2024) Generative AI’s Biggest Security Flaw Is Not Easy to Fix. WIRED. 
https://www.wired.com/story/generative-ai-prompt-injection-hacking/ 
Burtell, M. et al. (2024) The Surprising Power of Next Word Prediction: Large Language Models 
Explained, Part 1. Georgetown Center for Security and Emerging Technology. 
https://cset.georgetown.edu/article/the-surprising-power-of-next-word-prediction-large-language-
models-explained-part-1/ 
Canadian Centre for Cyber Security (2023) Generative artiﬁcial intelligence (AI) - ITSAP.00.041. 
https://www.cyber.gc.ca/en/guidance/generative-artiﬁcial-intelligence-ai-itsap00041, source_sentence: How does the absence of an explanation regarding data usage affect parents' ability to contest decisions made in child maltreatment assessments?
sentences: MS-1.1-005 
Evaluate novel methods and technologies for the measurement of GAI-related 
risks including in content provenance, oﬀensive cyber, and CBRN, while 
maintaining the models’ ability to produce valid, reliable, and factually accurate 
outputs. 
Information Integrity; CBRN 
Information or Capabilities; 
Obscene, Degrading, and/or 
Abusive Content, NOTICE & 
EXPLANATION 
WHY THIS PRINCIPLE IS IMPORTANT
This section provides a brief summary of the problems which the principle seeks to address and protect 
against, including illustrative examples. 
Automated systems now determine opportunities, from employment to credit, and directly shape the American 
public’s experiences, from the courtroom to online classrooms, in ways that profoundly impact people’s lives. But this 
expansive impact is not always visible. An applicant might not know whether a person rejected their resume or a 
hiring algorithm moved them to the bottom of the list. A defendant in the courtroom might not know if a judge deny­
ing their bail is informed by an automated system that labeled them “high risk.” From correcting errors to contesting 
decisions, people are often denied the knowledge they need to address the impact of automated systems on their lives., ever being notified that data was being collected and used as part of an algorithmic child maltreatment
risk assessment.84 The lack of notice or an explanation makes it harder for those performing child
maltreatment assessments to validate the risk assessment and denies parents knowledge that could help them
contest a decision.
41, source_sentence: How should automated systems be tested to ensure they are free from algorithmic discrimination?
sentences: humans (e.g., intelligence tests, professional licensing exams) does not guarantee GAI system validity or 
reliability in those domains. Similarly, jailbreaking or prompt engineering tests may not systematically 
assess validity or reliability risks.  
Measurement gaps can arise from mismatches between laboratory and real-world settings. Current 
testing approaches often remain focused on laboratory conditions or restricted to benchmark test 
datasets and in silico techniques that may not extrapolate well to—or directly assess GAI impacts in real-
world conditions. For example, current measurement gaps for GAI make it diﬃcult to precisely estimate 
its potential ecosystem-level or longitudinal risks and related political, social, and economic impacts. 
Gaps between benchmarks and real-world use of GAI systems may likely be exacerbated due to prompt 
sensitivity and broad heterogeneity of contexts of use. 
A.1.5. Structured Public Feedback, 62. See, e.g., Federal Trade Commission. Data Brokers: A Call for Transparency and Accountability. May
2014.
https://www.ftc.gov/system/files/documents/reports/data-brokers-call-transparency-accountability­
report-federal-trade-commission-may-2014/140527databrokerreport.pdf; Cathy O’Neil.
Weapons of Math Destruction. Penguin Books. 2017.
https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction
63. See, e.g., Rachel Levinson-Waldman, Harsha Pandurnga, and Faiza Patel. Social Media Surveillance by
the U.S. Government. Brennan Center for Justice. Jan. 7, 2022.
https://www.brennancenter.org/our-work/research-reports/social-media-surveillance-us-government;
Shoshana Zuboff. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of
Power. Public Affairs. 2019.
64. Angela Chen. Why the Future of Life Insurance May Depend on Your Online Presence. The Verge. Feb.
7, 2019., WHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS
The expectations for automated systems are meant to serve as a blueprint for the development of additional 
technical standards and practices that are tailored for particular sectors and contexts. 
Any automated system should be tested to help ensure it is free from algorithmic discrimination before it can be 
sold or used. Protection against algorithmic discrimination should include designing to ensure equity, broadly 
construed.  Some algorithmic discrimination is already prohibited under existing anti-discrimination law. The 
expectations set out below describe proactive technical and policy steps that can be taken to not only 
reinforce those legal protections but extend beyond them to ensure equity for underserved communities48 
even in circumstances where a specific legal protection may not be clearly established. These protections, source_sentence: What rights do applicants have if their application for credit is denied according to the CFPB?
sentences: even if the inferences are not accurate (e.g., confabulations), and especially if they reveal information 
that the individual considers sensitive or that is used to disadvantage or harm them. 
Beyond harms from information exposure (such as extortion or dignitary harm), wrong or inappropriate 
inferences of PII can contribute to downstream or secondary harmful impacts. For example, predictive 
inferences made by GAI models based on PII or protected attributes can contribute to adverse decisions, 
leading to representational or allocative harms to individuals or groups (see Harmful Bias and 
Homogenization below)., relevant biological and chemical threat knowledge and information is often publicly accessible, LLMs 
could facilitate its analysis or synthesis, particularly by individuals without formal scientiﬁc training or 
expertise.  
Recent research on this topic found that LLM outputs regarding biological threat creation and attack 
planning provided minimal assistance beyond traditional search engine queries, suggesting that state-of-
the-art LLMs at the time these studies were conducted do not substantially increase the operational 
likelihood of such an attack. The physical synthesis development, production, and use of chemical or 
biological agents will continue to require both applicable expertise and supporting materials and 
infrastructure. The impact of GAI on chemical or biological agent misuse will depend on what the key 
barriers for malicious actors are (e.g., whether information access is one such barrier), and how well GAI 
can help actors address those barriers., information in their credit report." The CFPB has also asserted that "[t]he law gives every applicant the right to 
a specific explanation if their application for credit was denied, and that right is not diminished simply because 
a company uses a complex algorithm that it doesn't understand."92 Such explanations illustrate a shared value 
that certain decisions need to be explained. 
A California law requires that warehouse employees are provided with notice and explana-
tion about quotas, potentially facilitated by automated systems, that apply to them. Warehous-
ing employers in California that use quota systems (often facilitated by algorithmic monitoring systems) are 
required to provide employees with a written description of each quota that applies to the employee, including 
“quantified number of tasks to be performed or materials to be produced or handled, within the defined
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics: cosine_accuracy@1, cosine_accuracy@3, cosine_accuracy@5, cosine_accuracy@10, cosine_precision@1, cosine_precision@3, cosine_precision@5, cosine_precision@10, cosine_recall@1, cosine_recall@3, cosine_recall@5, cosine_recall@10, cosine_ndcg@10, cosine_mrr@10, cosine_map@100
model-index: name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-m
results: task: type: information-retrieval
name: Information Retrieval
dataset: name: Unknown
type: unknown
metrics: type: cosine_accuracy@1
value: 0.97
name: Cosine Accuracy@1
verified: no, type: cosine_accuracy@3
value: 1
name: Cosine Accuracy@3
verified: no, type: cosine_accuracy@5
value: 1
name: Cosine Accuracy@5
verified: no, type: cosine_accuracy@10
value: 1
name: Cosine Accuracy@10
verified: no, type: cosine_precision@1
value: 0.97
name: Cosine Precision@1
verified: no, type: cosine_precision@3
value: 0.3333333333333334
name: Cosine Precision@3
verified: no, type: cosine_precision@5
value: 0.19999999999999996
name: Cosine Precision@5
verified: no, type: cosine_precision@10
value: 0.09999999999999998
name: Cosine Precision@10
verified: no, type: cosine_recall@1
value: 0.97
name: Cosine Recall@1
verified: no, type: cosine_recall@3
value: 1
name: Cosine Recall@3
verified: no, type: cosine_recall@5
value: 1
name: Cosine Recall@5
verified: no, type: cosine_recall@10
value: 1
name: Cosine Recall@10
verified: no, type: cosine_ndcg@10
value: 0.9876185950714291
name: Cosine Ndcg@10
verified: no, type: cosine_mrr@10
value: 0.9833333333333333
name: Cosine Mrr@10
verified: no, type: cosine_map@100
value: 0.9833333333333334
name: Cosine Map@100
verified: no
Likes: 0 | All-time downloads: 123
Created: 2024-12-31 09:17:21