vikasny30/aletheia-incidents-v2 huggingface dataset:
"Relying on general-purpose AI products that fail to fulfil their intended function can lead to harm. For example, general- purpose AI systems can make up facts (‘hallucination’), generate erroneous computer code, or provide inaccurate medical information. This can lead to physical and psychological harms to consumers and reputational, financial and legal harms to individuals and organisations." | 7. AI System Safety, Failures, & Limitations | 7.3 > Lack of capability or robustness | 1 - Human | 2 - Unintentional
"General-purpose AI systems can amplify social and political biases, causing concrete harm. They frequently display biases with respect to race, gender, culture, age, disability, political opinion, or other aspects of human identity. This can lead to discriminatory outcomes including unequal resource allocation, reinforcement of stereotypes, and systematic neglect of certain groups or viewpoints." | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 1 - Human | 2 - Unintentional
"The creation, perpetuation or exacerbation of inequalities and biases at a large-scale." | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 3 - Other | 3 - Other
"Large-scale influence on communication and information systems, and epistemic processes more generally." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 3 - Other | 3 - Other
"AI could be used to develop sophisticated tools to manipulate and persuade individuals." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 1 - Intentional
"AI models and systems underpin the advertising approaches that drive much of the internet, potentially influencing societal behavior." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 2 - AI | 1 - Intentional
"AI-based surveillance and manipulation could be used to maintain global totalitarian regimes." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 1 - Intentional
"The tendency for humans to over-rely on AI models and systems, trusting their outputs without sufficient critical evaluation, which can lead to poor decision-making." | 5. Human-Computer Interaction | 5.1 > Overreliance and unsafe use | 1 - Human | 2 - Unintentional
"Granting AI models and systems high levels of decision-making autonomy can lead to unintended consequences." | 7. AI System Safety, Failures, & Limitations | 7.2 > AI possessing dangerous capabilities | 1 - Human | 2 - Unintentional
"The widespread use of common AI platforms can create centralized points of failure, making systems more vulnerable to disruptions or attacks" | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 2 - Unintentional
"The difficulty in distinguishing synthetic content from authentic material adds to information risks." | 3. Misinformation | 3.2 > Pollution of information ecosystem and loss of consensus reality | 3 - Other | 3 - Other
"Heavy reliance on AI in critical sectors like finance or healthcare can exacerbate issues related to size, speed, interconnectivity, and complexity of the system." | 5. Human-Computer Interaction | 5.1 > Overreliance and unsafe use | 1 - Human | 2 - Unintentional
"Incomplete or biased training data can lead to discriminatory AI outputs." | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 3 - Other | 2 - Unintentional
"AI models and systems are vulnerable to manipulation through adversarial inputs." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 1 - Intentional
"AI-generated deepfakes can create convincingly realistic but entirely fabricated information." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 2 - AI | 3 - Other
"AI models and systems may grant governments or corporations increased monitoring over individuals." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 2 - AI | 1 - Intentional
"Widespread use of AI-powered persuasion tools could lead to systemic harm" | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 3 - Other
"In the context of Normal Accident Theory [150], normal accidents are those that “could no longer be ascribed to isolated equipment malfunction, operator error, or acts of God.” We refer to these as “system failures” (to be distinguished from “systemic risks”), while the opposite would be “isolated failures.” For isolated failures, harms are consistent with the underlying failure modes. For example, an AI capable of producing false or misleading content would constitute risks re- lated to misinformation and disinformation. Whereas for system failures, harms are not consistent with the underlying failure modes, or the harms are caused by interactions between different components within a system rather than from the failure of any specific component." | X.1 > Excluded | 3 - Other | 3 - Other
"An example of AI capabilities is that an AI might be capable of developing novel bioweapons. Whereas an example of AI inadequacy is a self-driving car causing an accident due to not being able to recognize certain objects. The boundary between capabilities and inadequacy is sometimes blurred. For exam- ple, when an AI generates falsehoods, it could be framed as either a capability of developing fiction, or an inadequacy in generating truthful content." | X.1 > Excluded | 3 - Other | 3 - Other
"A large scale “scraping” of web data for training datasets increases vulnerability to data poisoning, backdoor attacks, and the inclusion of inaccurate or toxic data [76, 28, 48]. With a large dataset, filtering out these quality issues is very difficult or trades off against significant data loss." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 2 - Unintentional
"Adversarial examples [198, 83] refer to data that are designed to fool an AI model by inducing unintended behavior. They do this by exploiting spurious correlations learned by the model. They are part of inference-time attacks, where the examples are test examples. They generalize to different model architectures and models trained on different training sets." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 1 - Intentional
"GPAI models are often easily reconfigured for various use cases or have competencies beyond the intended use [78, 225]. They can be performed either by changing the weights of the model (e.g., fine-tuning) or by modifying only the model inputs (e.g., prompt engineering, jailbreaking, retrieval-augmented generation). Reconfiguration can be intentional (with the help of adversarial inputs) or unintentional (from unanticipated inputs to the model)." | 4. Malicious Actors & Misuse | 4.0 > Malicious use | 1 - Human | 3 - Other
"When an LLM is configured to evaluate the performance of another model or AI system, it may produce incorrect evaluation outputs [122, 147]. For example, it may give a higher rating to a more verbose answer or an answer from a particular political stance. If an LLM-based evaluation is integrated into the training of a new model, the trained model could develop in a way that specifically finds and exploits limitations in the evaluator’s metrics." | 7. AI System Safety, Failures, & Limitations | 7.1 > AI pursuing its own goals in conflict with human goals or values | 2 - AI | 2 - Unintentional
"AI models may be prone to self-preference bias, where they favor their own generated content over that of others [147, 114]. This bias becomes particularly relevant in self-evaluation tasks, where a model assesses the quality or persua- siveness [66] of its own outputs, or in model-based evaluations more broadly. This bias can result in models unfairly discriminating against human-generated content in favor of their own outputs." | 7. AI System Safety, Failures, & Limitations | 7.1 > AI pursuing its own goals in conflict with human goals or values | 2 - AI | 1 - Intentional
"There is a lack of robust frameworks for understanding and evaluating if the output of AI systems robustly conforms to human values, as opposed to if the systems have learned to produce outputs that are only partially correlated with them (i.e., mimicking) [13]. Additionally, outputs by AI models often do not perfectly reflect the representation of human values learned by the model, and it is not known how these values evolve and transition across different stages of model training and deployment. Such evaluations may be especially challenging with LLMs that adopt different personas with different behaviorial patterns, where they do not consistently conform to certain human values." | 7. AI System Safety, Failures, & Limitations | 7.1 > AI pursuing its own goals in conflict with human goals or values | 3 - Other | 3 - Other
"When AI models are trained through evaluation with human feedback, such as reinforcement learning from human feedback, their outputs can be challenging to assess, as they may contain hard-to-detect errors or issues that only become apparent over time. The human evaluator can rate incorrect outputs positively or similar to correct outputs. This can lead to the model learning to produce subtly incorrect or harmful outputs, such as code with software vulnerabilities, or politically biased information. In extreme cases where a model is deceiving users, complicated outputs can contain hidden errors or backdoors." | 7. AI System Safety, Failures, & Limitations | 7.3 > Lack of capability or robustness | 2 - AI | 2 - Unintentional
"Existing explainability techniques can be insufficient for detecting discriminatory biases. Manipulation methods can hide underlying biases from these tech- niques, generating misleading explanations [192, 112]. Such explanations ex- clude sensitive or prohibitive attributes, such as race or gender, and instead include desired attributes, even though they do not accurately represent the underlying model." | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 3 - Other | 3 - Other
"A jailbreak is a type of adversarial input to the model (during deployment) re- sulting in model behavior deviating from intended use. Jailbreaks may be gen- erated automatically in a “white box” setting, where access to internal training parameters is required for creation and optimization of the attack [238]. Other attacks may be “black box” - without access to model internals. In text based generative models, jailbreaks may sometimes be human-readable, with the use of reasoning or role-play to “convince” the model to bypass its safety mechanisms [231]." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 1 - Intentional
"Current generation multimodal (e.g., vision and language) GPAI models are vulnerable to adversarial jailbreak attacks. These attacks can be used to automatically induce a model to produce an arbitrary or specific output with high success rate [227]. Multimodal jailbreaks can also be used to exfiltrate a model’s context window or other model internals [18]." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 1 - Intentional
"Various new or existing text encodings, such as Base64, can be employed to craft jailbreak attacks that bypass safety training [13]. Low-resource language inputs also appear more likely to circumvent a model’s safeguards [229]. Since safety fine-tuning might not involve this encoding data or may only do so to a limited extent, harmful natural language prompts could be translated into less frequently used encodings [214]." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 1 - Intentional
"Additional modalities can introduce new attack vectors in multimodal models as well as expand the scope of the previous attacks, ranging from jailbreaking to poisoning [13]. Typically, different modalities have different robustness levels, allowing malicious actors to choose the most vulnerable part of the model to attack [119, 181]." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 3 - Other | 3 - Other
"Language models with long context windows are vulnerable to new types of ex- ploitations that are ineffective on models with shorter context windows. While few-shot jailbreaking, which involves providing few examples of the desired harmful output, might not trigger a harmful response, many-shot jailbreak- ing, which involves a higher number of such examples, increases the likelihood of eliciting an undesirable output. These vulnerabilities become more significant as context windows expand with newer model releases [7]." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 3 - Other | 3 - Other
"AI models can be particularly sensitive to coherent external evidence, even when they come into conflict with the models’ prior knowledge. This may lead to models producing false outputs given false information during the retrieval- augmentation process, despite only a relatively small amount of false informa- tion input that is inconsistent with the model’s prior knowledge trained on much larger amounts of data [220]." | 7. AI System Safety, Failures, & Limitations | 7.3 > Lack of capability or robustness | 2 - AI | 2 - Unintentional
"AI models can be influenced to accept misinformation through persuasive conversations, even when their initial responses are factually correct. Multi-turn persuasion can be more effective than single-turn persuasion attempts in altering the model’s stance [223]." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 1 - Intentional
"AI systems can achieve user-specified tasks in undesirable ways unless they are specified carefully and in enough detail. AI systems might find an easier unintended way to accomplish the objective provided by the user or developer, so that the actions by the AI system taken during its execution are very different from what the user expected [75, 191]. This behavior arises not from a problem with the learning algorithm, but rather from the misspecification or underspeci- fication of the intended task, and is generally referred to as specification gaming [43]." | 7. AI System Safety, Failures, & Limitations | 7.1 > AI pursuing its own goals in conflict with human goals or values | 2 - AI | 1 - Intentional
"In some instances, specification gaming in a GPAI model can lead to reward tampering, without further training. This can mean that relatively benign cases of specification gaming (such as sycophancy in LLMs) can, if left unchecked, enable the model to generalize to more sophisticated behavior such as reward tampering [57]." | 7. AI System Safety, Failures, & Limitations | 7.1 > AI pursuing its own goals in conflict with human goals or values | 2 - AI | 1 - Intentional
"Goal or objective misgeneralization is a type of robustness failure where an AI system appears to be pursuing the intended objective in training, but does not generalize to pursuing this objective in out-of-distribution settings in deployment while maintaining good deployment performance in some tasks [180, 59]." | 7. AI System Safety, Failures, & Limitations | 7.3 > Lack of capability or robustness | 2 - AI | 1 - Intentional
"Deceptive behavior of an AI system consists of actions or outputs of the AI that reliably mislead other parties, including humans and other AI systems. This behavior can result in the targeted parties becoming convinced of, and acting on, false information [140]." | 7. AI System Safety, Failures, & Limitations | 7.1 > AI pursuing its own goals in conflict with human goals or values | 2 - AI | 1 - Intentional
"AI systems can create false or misleading claims that can lead to unauthorized actions, even in some cases violating the terms and conditions set by the model provider [79, 1]. For example, an AI system can claim that it is not collecting data from its current interaction with the user, in line with the provider’s policies, but the system still stores the user’s input without deleting it after the session. This harms both the user and the provider, as the provider is exposed to increased legal liability due to the model’s actions." | 7. AI System Safety, Failures, & Limitations | 7.2 > AI possessing dangerous capabilities | 2 - AI | 1 - Intentional
"GPAI systems can produce outputs (such as natural language text, audio, or video) that convince their users of incorrect information. This can happen through personalized persuasion in dialogue, or the mass-production of mis- leading information that is then disseminated over the internet. The persuasive capabilities of GPAI models can sometimes scale with model size or capability [32, 172]. Persuasive models could have larger societal implications by being misused to generate convincing but manipulative or untruthful content." | 7. AI System Safety, Failures, & Limitations | 7.2 > AI possessing dangerous capabilities | 2 - AI | 1 - Intentional
"The growing integration and interconnectivity with external tools and plugins increase the risk of exposure to malicious external inputs. This interconnectivity makes it easier for external tools to introduce harmful content [220]." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 3 - Other
"AI systems that have the broad ability to connect to a network to obtain infor- mation could also end up sending data outbound in ways that neither providers, deployers, or end users intended [138]. This can happen if there is no whitelisting of communication channels (such as network connections or allowed protocols). In general, this can occur if the deployment of the AI system violates the prin- ciple of least privilege. Such outbound communication may lead to leakage of confidential data, or the AI system performing unwanted actions like sending emails or ordering goods on the internet." | 7. AI System Safety, Failures, & Limitations | 7.2 > AI possessing dangerous capabilities | 2 - AI | 1 - Intentional
"Model weights or access to them can be leaked when initial access is granted only to a select group of individuals, such as institutional researchers [209]. This risk can increase as more people gain access, and identifying the source of the leak becomes more difficult. The availability of leaked model weights makes various attacks on systems that use the leaked AI model easier to implement, such as finding adversarial examples, elicitation of dangerous capabilities, and extraction of confidential information present in the training data. The avail- ability of model weights might also enable the misuse of the AI system using the leaked model to produce harmful or illegal content [67]." | 2. Privacy & Security | 2.2 > AI system security vulnerabilities and attacks | 1 - Human | 1 - Intentional
"When relying on GPAI in critical infrastructure, there may be common mode failures that begin with vulnerabilities or robustness issues in the underlying model architecture or training setup. These failures may happen accidentally (in edge-cases) or due to adversarial inputs to the AI systems [58]." | 7. AI System Safety, Failures, & Limitations | 7.3 > Lack of capability or robustness | 2 - AI | 3 - Other
"AIs can easily give moral advice even when not having a coherent, contradictions- free moral stance. This could lead to the users’ moral judgments being nega- tively influenced by random or arbitrary moral advice given by AIs [109]." | 5. Human-Computer Interaction | 5.1 > Overreliance and unsafe use | 2 - AI | 3 - Other
"AI systems can undermine human autonomy, if they allow for habitually trusting the AI’s suggestions without sufficient exercising of human agency. Over time, a user may develop unjustified trust in or dependence on the system, or rely on its advice for tasks outside the system’s domain of expertise [205, 42]. In particular, less confident users (or users in emotional distress) can be more prone to “overtrust” a system [219]." | 5. Human-Computer Interaction | 5.2 > Loss of human agency and autonomy | 1 - Human | 2 - Unintentional
"Disinformation (in various modalities: text, audio, images, video, etc.) can be generated with minimal human oversight and effort. Disinformation tools are relatively cheap and their technology is widely available. Such deployments can be particularly widespread in sensitive political contexts." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 1 - Intentional
"GPAI tools can be used in automation and scaling of influence campaigns [178]. Public opinion may be manipulated by targeted misleading or manipulative information. This can lead to rising political polarization and diminishing trust in public institutions." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 1 - Intentional
"Generative models can create illegal, harmful, or discriminatory content [196], such as sexual abuse material, at scale. Current access controls (e.g., API access filters) are not effective against all user queries in generating such content." | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 3 - Other
"Generative models can create harmful or discriminatory content from benign user requests. Models can exhibit bias to particular harmful styles of generation (e.g., sexualization of photos of women [87] in the case of image generation models) or they can generate toxic, misleading, or violent data (e.g., a model generating jokes can use ethnic stereotypes or slurs to deliver humor)." | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 2 - Unintentional
"Deepfakes are media that depict real or non-existent people or events, involving the use of multiple modalities (e.g., images, audio, video). They can also involve the imitation of speech or body movements of real people. Multimodal deepfakes can be used to harass, discredit, intimidate, and extort individuals." | 4. Malicious Actors & Misuse | 4.3 > Fraud, scams, and targeted manipulation | 1 - Human | 1 - Intentional
"GPAIs can be misused for the automated generation of content personalized to target select individuals based on their weak spots [30]. Such attacks may be more efficient and more successful in achieving the goals of harassment, extortion, or intimidation." | 4. Malicious Actors & Misuse | 4.3 > Fraud, scams, and targeted manipulation | 1 - Human | 1 - Intentional
"AI tools can be misused by human or institutional actors for monitoring, control- ling, or suppressing individuals [178]. Massive data collection and automated analysis are often conducted, and AI tools can further exacerbate such practices." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 1 - Intentional
"AI systems embedded with systemic biases can manipulate large population segments, particularly when these biases align with the beliefs or behaviors of the targeted group. When weaponized at scale, this manipulation can exacerbate social divisions or cause large-scale disruptions, such as city-wide blackouts (e.g., by the manipulation of power consumption into the peak demand period [159])." | 4. Malicious Actors & Misuse | 4.2 > Cyberattacks, weapon development or use, and mass harm | 1 - Human | 1 - Intentional
"The use of GPAIs may contribute to the proliferation of either deliberate dis- information or unintended misinformation can severely erode trust in public figures and democratic institutions. This diminishing trust can extend to other forms of media, making the public less informed." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 3 - Other