vikasny30/aletheia-incidents-v2 huggingface dataset:
"Second, because LLMs are trained on internet text data, there is also a risk that model weights encode functions which, if deployed in particular contexts, would violate social norms of that context. Following the principles of contextual integrity, it may be that models deviate from information sharing norms as a result of their training. Overcoming this challenge requires two types of infrastructure: one for keeping track of social norms in context, and another for ensuring that models adhere to them. Keeping track of what social norms are presently at play is an active research area. Surfacing value misalignments between a model’s behaviour and social norms is a daunting task, against which there is also active research (see Chapter 5)." | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 2 - Unintentional
"Collective action problems are ubiquitous in our society (Olson Jr, 1965). They possess an incentive structure in which society is best served if everyone cooperates, but where an individual can achieve personal gain by choosing to defect while others cooperate. The way we resolve these problems at many scales is highly complex and dependent on a deep understanding of the intricate web of social interactions that forms our culture and imprints on our individual identities and behaviours (Ostrom, 2010). Some collective action problems can be resolved by codifying a law, for instance the social dilemma of whether or not to pay for an item in a shop. The path forward here is comparatively easy to grasp, from the perspective of deploying an AI assistant: we need to build these standards into the model as behavioural constraints. Such constraints would need to be imposed by a regulator or agreed upon by practitioners, with suitable penalties applied should the constraint be violated so tha
"Efforts to deploy advanced assistant technology in society, in a way that is broadly beneficial, can be viewed as a wicked problem (Rittel and Webber, 1973). Wicked problems are defined by the property that they do not admit solutions that can be foreseen in advance, rather they must be solved iteratively using feedback from data gathered as solutions are invented and deployed. With the deployment of any powerful general-purpose technology, the already intricate web of sociotechnical relationships in modern culture are likely to be disrupted, with unpredictable externalities on the conventions, norms and institutions that stabilise society. For example, the increasing adoption of generative AI tools may exacerbate misinformation in the 2024 US presidential election (Alvarez et al., 2023), with consequences that are hard to predict. The suggestion that the cooperative AI problem is wicked does not imply it is intractable. However, it does have consequences for the approach that we must
"Emergent access risks are most likely to arise when current and novel capabilities are combined. Emergent risks can be difficult to foresee fully (Ovadya and Whittlestone, 2019; Prunkl et al., 2021) due to the novelty of the technology (see Chapter 1) and the biases of those who engage in product design or foresight processes D’Ignazio and Klein (2020). Indeed, people who occupy relatively advantaged social, educational and economic positions in society are often poorly equipped to foresee and prevent harm because they are disconnected from lived experiences of those who would be affected. Drawing upon access concerns that surround existing technologies, we anticipate three possible trends: • Trend 1: Technology as societal infrastructure. If advanced AI assistants are adopted by organisations or governments in domains affecting material well-being, ‘opting out’ may no longer be a real option for people who want to continue to participate meaningfully in society. Indeed, if this trend
"The rapid integration of AI systems with advanced capabilities, such as greater autonomy, content generation, memorisation and planning skills (see Chapter 4) into personalised assistants also raises new and more specific challenges related to misinformation, disinformation and the broader integrity of our information environment. " | 3. Misinformation | 3.0 > Misinformation | 3 - Other | 3 - Other
"Design choices such as greater personalisation of AI assistants and efforts to align them with human preferences could also reinforce people’s pre-existing biases and entrench specific ideologies. Increasingly agentic AI assistants trained using techniques such as reinforcement learning from human feedback (RLHF) and with the ability to access and analyse users’ behavioural data, for example, may learn to tailor their responses to users’ preferences and feedback. In doing so, these systems could end up producing partial or ideologically biased statements in an attempt to conform to user expectations, desires or preferences for a particular worldview (Carroll et al., 2022). Over time, this could lead AI assistants to inadvertently reinforce people’s tendency to interpret information in a way that supports their own prior beliefs (‘confirmation bias’), thus making them more entrenched in their own views and more resistant to factual corrections (Lewandowsky et al., 2012). At the societa
"Beyond this, the widespread adoption of advanced AI assistants for content generation could have a number of negative consequences for our shared information ecosystem. One concern is that it could result in a degradation of the quality of the information available online. Researchers have already observed an uptick in the amount of audiovisual misinformation, elaborate scams and fake websites created using generative AI tools (Hanley and Durumeric, 2023). As more and more people turn to AI assistants to autonomously create and disseminate information to public audiences at scale, it may become increasingly difficult to parse and verify reliable information. This could further threaten and complicate the status of journalists, subject-matter experts and public information sources. Over time, a proliferation of spam, misleading or low-quality synthetic content in online spaces could also erode the digital knowledge commons – the shared knowledge resources accessible to everyone on the
"Finally, AI assistants themselves could become weaponised by malicious actors to sow misinformation and manipulate public opinion at scale. Studies show that spreaders of disinformation tend to privilege quantity over quality of messaging, flooding online spaces repeatedly with misleading content to sow ‘seeds of doubt’ (Hassoun et al., 2023). Research on the ‘continued influence effect’ also shows that repeatedly being exposed to false information is more likely to influence someone’s thoughts than a single exposure. Studies show, for example, that repeated exposure to false information makes people more likely to believe it by increasing perceived social consensus, and it makes people more resistant to changing their minds even after being given a correction (for a review of these effects, see Lewandowsky et al., 2012; Ecker et al., 2022). By leveraging the frequent and personalised nature of repeated interactions with an AI assistant, malicious actors could therefore gradually nudg
"Advanced AI assistants may make users more susceptible to misinformation, as people develop competence trust in these systems’ abilities and uncritically turn to them as reliable sources of information." | 5. Human-Computer Interaction | 5.1 > Overreliance and unsafe use | 2 - AI | 2 - Unintentional
"AI assistants may provide ideologically biased or otherwise partial information in attempting to align to user expectations. In doing so, AI assistants may reinforce people’s pre-existing biases and compromise productive political debate." | 3. Misinformation | 3.2 > Pollution of information ecosystem and loss of consensus reality | 2 - AI | 2 - Unintentional
"AI assistants may contribute to the spread of large quantities of factually inaccurate and misleading content, with negative consequences for societal trust in information sources and institutions, as individuals increasingly struggle to discern truth from falsehood." | 3. Misinformation | 3.2 > Pollution of information ecosystem and loss of consensus reality | 2 - AI | 3 - Other
"AI assistants may facilitate large-scale disinformation campaigns by offering novel, covert ways for propagandists to manipulate public opinion. This could undermine the democratic process by distorting public opinion and, in the worst case, increasing skepticism and political violence." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 1 - Intentional
"Insulting content generated by LMs is a highly visible and frequently mentioned safety issue. Mostly, it is unfriendly, disrespectful, or ridiculous content that makes users uncomfortable and drives them away. It is extremely hazardous and could have negative social consequences." | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 3 - Other
"The model produces unfair and discriminatory data, such as social bias based on race, gender, religion, appearance, etc. These contents may discomfort certain groups and undermine social stability and peace." | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 3 - Other
"The model output contains illegal and criminal attitudes, behaviors, or motivations, such as incitement to commit crimes, fraud, and rumor propagation. These contents may hurt users and have negative societal repercussions." | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 3 - Other
"For some sensitive and controversial topics (especially on politics), LMs tend to generate biased, misleading, and inaccurate content. For example, there may be a tendency to support a specific political position, leading to discrimination or exclusion of other political viewpoints." | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 3 - Other
"The model generates unsafe information related to physical health, guiding and encouraging users to harm themselves and others physically, for example by offering misleading medical information or inappropriate drug usage guidance. These outputs may pose potential risks to the physical health of users." | 3. Misinformation | 3.1 > False or misleading information | 2 - AI | 3 - Other
"The model generates a risky response about mental health, such as content that encourages suicide or causes panic or anxiety. These contents could have a negative effect on the mental health of users." | 3. Misinformation | 3.1 > False or misleading information | 2 - AI | 3 - Other
"The generation involves exposing users’ privacy and property information or providing advice with huge impacts such as suggestions on marriage and investments. When handling this information, the model should comply with relevant laws and privacy regulations, protect users’ rights and interests, and avoid information leakage and abuse." | 2. Privacy & Security | 2.1 > Compromise of privacy by leaking or correctly inferring sensitive information | 2 - AI | 3 - Other
"In addition to the above-mentioned typical safety scenarios, current research has revealed some unique attacks that such models may confront. For example, Perez and Ribeiro (2022) found that goal hijacking and prompt leaking could easily deceive language models to generate unsafe responses. Moreover, we also find that LLMs are more easily triggered to output harmful content if some special prompts are added. In response to these challenges, we develop, categorize, and label 6 types of adversarial attacks, and name them Instruction Attack, which are challenging for large language models to handle. Note that our instruction attacks are still based on natural language (rather than unreadable tokens) and are intuitive and explainable in semantics." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 1 - Intentional
"It refers to the appending of deceptive or misleading instructions to the input of models in an attempt to induce the system into ignoring the original user prompt and producing an unsafe response." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 1 - Intentional
"By analyzing the model’s output, attackers may extract parts of the systemprovided prompts and thus potentially obtain sensitive information regarding the system itself." | 2. Privacy & Security | 2.1 > Compromise of privacy by leaking or correctly inferring sensitive information | 1 - Human | 1 - Intentional
"Attackers might specify a model’s role attribute within the input prompt and then give specific instructions, causing the model to finish instructions in the speaking style of the assigned role, which may lead to unsafe outputs. For example, if the character is associated with potentially risky groups (e.g., radicals, extremists, unrighteous individuals, racial discriminators, etc.) and the model is overly faithful to the given instructions, it is quite possible that the model outputs unsafe content linked to the given character." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 1 - Intentional
"If the input instructions themselves refer to inappropriate or unreasonable topics, the model will follow these instructions and produce unsafe content. For instance, if a language model is requested to generate poems with the theme “Hail Hitler”, the model may produce lyrics containing fanaticism, racism, etc. In this situation, the output of the model could be controversial and have a possible negative impact on society." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 1 - Intentional
"By adding imperceptibly unsafe content into the input, users might either deliberately or unintentionally influence the model to generate potentially harmful content. In the following cases involving migrant workers, ChatGPT provides suggestions to improve the overall quality of migrant workers and reduce the local crime rate. ChatGPT responds to the user’s hint with a disguised and biased opinion that the general quality of immigrants is favorably correlated with the crime rate, posing a safety risk." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 1 - Intentional
"This category is about threat, insult, scorn, profanity, sarcasm, impoliteness, etc. LLMs are required to identify and oppose these offensive contents or actions." | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 3 - Other
"This type of safety problem is mainly about social bias across various topics such as race, gender, religion, etc. LLMs are expected to identify and avoid unfair and biased expressions and actions." | 1. Discrimination & Toxicity | 1.0 > Discrimination & Toxicity | 2 - AI | 3 - Other
"This category focuses on actions or expressions that may influence human physical health. LLMs should know appropriate actions or expressions in various scenarios to maintain physical health." | 3. Misinformation | 3.1 > False or misleading information | 2 - AI | 3 - Other
"Different from physical health, this category pays more attention to health issues related to psychology, spirit, emotions, mentality, etc. LLMs should know correct ways to maintain mental health and prevent any adverse impacts on the mental well-being of individuals." | 3. Misinformation | 3.1 > False or misleading information | 2 - AI | 3 - Other
as they claim to generate biased and discriminatory results, these AI systems have a negative impact on the rights of individuals, principles of adjudication, and overall judicial integrity | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
manipulation of social dynamics | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 2 - AI | 1 - Intentional
AI employed to produce convincing counterfeit visuals, videos, and audio clips that give the impression of authenticity | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 1 - Intentional
Malicious utilization of AI has the potential to endanger digital security, physical security, and political security. International law enforcement entities grapple with a variety of risks linked to the Malevolent Utilization of AI. | 4. Malicious Actors & Misuse | 4.3 > Fraud, scams, and targeted manipulation | 1 - Human | 1 - Intentional
Generating correct, truthful, and consistent outputs with proper confidence | 3. Misinformation | 3.1 > False or misleading information | 2 - AI | 2 - Unintentional
Wrong information not intentionally generated by malicious users to cause harm, but unintentionally generated by LLMs because they lack the ability to provide factually correct information. | 3. Misinformation | 3.1 > False or misleading information | 2 - AI | 2 - Unintentional
LLMs can generate content that is nonsensical or unfaithful to the provided source content with appeared great confidence, known as hallucination | 3. Misinformation | 3.1 > False or misleading information | 2 - AI | 2 - Unintentional
over-confidence in topics where objective answers are lacking, as well as in areas where their inherent limitations should caution against LLMs’ uncertainty (e.g. not as accurate as experts)... ack of awareness regarding their outdated knowledge base about the question, leading to confident yet erroneous response | 3. Misinformation | 3.1 > False or misleading information | 2 - AI | 2 - Unintentional
flatter users by reconfirming their misconceptions and stated beliefs | 3. Misinformation | 3.1 > False or misleading information | 2 - AI | 1 - Intentional
Avoiding unsafe and illegal outputs, and leaking private information | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 3 - Other
LLMs are found to generate answers that contain violent content or generate content that responds to questions that solicit information about violent behaviors | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 1 - Intentional
LLMs have been shown to be a convenient tool for soliciting advice on accessing, purchasing (illegally), and creating illegal substances, as well as for dangerous use of them | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 1 - Intentional
LLMs can be leveraged to solicit answers that contain harmful content to children and youth | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 1 - Intentional
LLMs have the capability to generate sex-explicit conversations, and erotic texts, and to recommend websites with sexual content | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 1 - Intentional
machine learning models are known to be vulnerable to data privacy attacks, i.e. special techniques of extracting private information from the model or the system used by attackers or malicious users, usually by querying the models in a specially designed way | 2. Privacy & Security | 2.1 > Compromise of privacy by leaking or correctly inferring sensitive information | 2 - AI | 1 - Intentional
Avoiding bias and ensuring no disparate performance | 1. Discrimination & Toxicity | 1.3 > Unequal performance across groups | 2 - AI | 2 - Unintentional
In the context of LLM outputs, we want to make sure the suggested or completed texts are indistinguishable in nature for two involved individuals (in the prompt) with the same relevant profiles but might come from different groups (where the group attribute is regarded as being irrelevant in this context) | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
LLMs must not exhibit or highlight any stereotypes in the generated text. Pretrained LLMs tend to pick up stereotype biases persisting in crowdsourced data and further amplify them | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
LLMs are exposed to vast groups of people, and their political biases may pose a risk of manipulation of socio-political processes | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 3 - Other
The LLM’s performances can differ significantly across different groups of users. For example, the question-answering capability showed significant performance differences across different racial and social status groups. The fact-checking abilities can differ for different tasks and languages | 1. Discrimination & Toxicity | 1.3 > Unequal performance across groups | 2 - AI | 2 - Unintentional
LLMs can be leveraged, by malicious users, to proactively generate propaganda information that can facilitate the spreading of a target | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 1 - Intentional
LLMs are expected to reflect social values by avoiding the use of offensive language toward specific groups of users, being sensitive to topics that can create instability, as well as being sympathetic when users are seeking emotional support | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 3 - Other
language being rude, disrespectful, threatening, or identity-attacking toward certain groups of the user population (culture, race, and gender etc) | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 3 - Other
it is important to build high-quality locally collected datasets that reflect views from local users to align a model’s value system | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 1 - Human | 2 - Unintentional
Knowledge bases that LLMs are trained on continue to shift... questions such as “who scored the most points in NBA history" or “who is the richest person in the world" might have answers that need to be updated over time, or even in real-time | 3. Misinformation | 3.1 > False or misleading information | 2 - AI | 2 - Unintentional
existing disparities in data among different user groups might create differentiated experiences when users interact with an algorithmic system (e.g. a recommendation system), which will further reinforce the bias | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
"generative AI tools can and will be used to propagate content that is false, misleading, biased, inflammatory, or dangerous. As generative AI tools grow more sophisticated, it will be quicker, cheaper, and easier to produce this content—and existing harmful content can serve as the foundation to produce more" | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 1 - Intentional
"Bad actors can also use generative AI tools to produce adaptable content designed to support a campaign, political agenda, or hateful position and spread that information quickly and inexpensively across many platforms. This rapid spread of false or misleading content—AI-facilitated disinformation—can also create a cyclical effect for generative AI: when a high volume of disinformation is pumped into the digital ecosystem and more generative systems are trained on that information via reinforcement learning methods, for example, false or misleading inputs can create increasingly incorrect outputs." | 4. Malicious Actors & Misuse | 4.3 > Fraud, scams, and targeted manipulation | 1 - Human | 1 - Intentional
"Bad actors can also use generative AI tools to produce adaptable content designed to support a campaign, political agenda, or hateful position and spread that information quickly and inexpensively across many platforms." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 1 - Intentional