vikasny30/aletheia-incidents-v2 huggingface dataset:
Financial harms [52, 160] co-produced through algorithmic systems, especially as they relate to lived experiences of poverty and economic inequality... demonetization algorithms that parse content titles, metadata, and text, and it may penalize words with multiple meanings [51, 81], disproportionately impacting queer, trans, and creators of color [81]. Differential pricing algorithms, where people are systematically shown different prices for the same products, also leads to economic loss [55]. These algorithms may be especially sensitive to feedback loops from existing inequities related to education level, income, and race, as these inequalities are likely reflected in the criteria algorithms use to make decisions [22, 163]. | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
"These harms occur when algorithmic systems disproportionately underperform for certain groups of people along social categories of difference such as disability, ethnicity, gender identity, and race." | 1. Discrimination & Toxicity | 1.3 > Unequal performance across groups | 2 - AI | 2 - Unintentional
Alienation is the specific self-estrangement experienced at the time of technology use, typically surfaced through interaction with systems that under-perform for marginalized individuals | 1. Discrimination & Toxicity | 1.3 > Unequal performance across groups | 3 - Other | 2 - Unintentional
increased burden (e.g., time spent) or effort required by members of certain social groups to make systems or products work as well for them as others | 1. Discrimination & Toxicity | 1.3 > Unequal performance across groups | 3 - Other | 2 - Unintentional
degraded or total loss of benefits of using algorithmic systems with inequitable system performance based on identity | 1. Discrimination & Toxicity | 1.3 > Unequal performance across groups | 2 - AI | 2 - Unintentional
Loss of agency occurs when the use [123, 137] or abuse [142] of algorithmic systems reduces autonomy. One dimension of agency loss is algorithmic profiling [138], through which people are subject to social sorting and discriminatory outcomes to access basic services... presentation of content may lead to “algorithmically informed identity change. . . including [promotion of] harmful person identities (e.g., interests in white supremacy, disordered eating, etc.).” Similarly, for content creators, desire to maintain visibility or prevent shadow banning, may lead to increased conforming of content | 5. Human-Computer Interaction | 5.2 > Loss of human agency and autonomy | 3 - Other | 3 - Other
Technology-facilitated violence occurs when algorithmic features enable use of a system for harassment and violence [2, 16, 44, 80, 108], including creation of non-consensual sexual imagery in generative AI... other facets of technology-facilitated violence, include doxxing [79], trolling [14], cyberstalking [14], cyberbullying [14, 98, 204], monitoring and control [44], and online harassment and intimidation [98, 192, 199, 226], under the broader banner of online toxicity | 4. Malicious Actors & Misuse | 4.3 > Fraud, scams, and targeted manipulation | 1 - Human | 1 - Intentional
algorithmic behavioral exploitation [18, 209], emotional manipulation [202] whereby algorithmic designs exploit user behavior, safety failures involving algorithms (e.g., collisions) [67], and when systems make incorrect health inferences | 5. Human-Computer Interaction | 5.1 > Overreliance and unsafe use | 2 - AI | 3 - Other
Privacy violation occurs when algorithmic systems diminish privacy, such as enabling the undesirable flow of private information [180], instilling the feeling of being watched or surveilled [181], and the collection of data without explicit and informed consent... privacy violations may arise from algorithmic systems making predictive inference beyond what users openly disclose [222] or when data collected and algorithmic inferences made about people in one context is applied to another without the person’s knowledge or consent through big data flows | 2. Privacy & Security | 2.1 > Compromise of privacy by leaking or correctly inferring sensitive information | 2 - AI | 3 - Other
information-based harms capture concerns of misinformation, disinformation, and malinformation. Algorithmic systems, especially generative models and recommender, systems can lead to these information harms | 3. Misinformation | 3.1 > False or misleading information | 3 - Other | 2 - Unintentional
Political harms emerge when “people are disenfranchised and deprived of appropriate political power and influence” [186, p. 162]. These harms focus on the domain of government, and focus on how algorithmic systems govern through individualized nudges or micro-directives [187], that may destabilize governance systems, erode human rights, be used as weapons of war [188], and enact surveillant regimes that disproportionately target and harm people of color | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 3 - Other | 1 - Intentional
"The potential for AI systems to be used maliciously or irresponsibly, including for creating deepfakes, automated cyber attacks, or invasive surveillance systems. Specifically denotes intentional use of AI for harm." | 4. Malicious Actors & Misuse | 4.2 > Cyberattacks, weapon development or use, and mass harm | 1 - Human | 1 - Intentional
"Addresses AI's broader societal effects, including labor displacement, mental health impacts, and issues from manipulative technologies like deepfakes. Additionally, it considers AI's environmental footprint, balancing resource strain and training-related carbon emissions against AI's potential to help address environmental problems." | 6. Socioeconomic and Environmental | 6.0 > Socioeconomic & Environmental | 3 - Other | 3 - Other
"The potential for AI systems to make decisions that systematically disadvantage certain groups or individuals. Bias can stem from training data, algorithmic design, or deployment practices, leading to unfair outcomes and possible legal ramifications." | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
"Generative AI systems can embed and amplify harmful biases that are most detrimental to marginalized peoples." | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
"Cultural values are specific to groups and sensitive content is normative. Sensitive topics also vary by culture and can include hate speech, which itself is contingent on cultural norms of acceptability." | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 2 - Unintentional
"In the context of evaluating the impact of generative AI systems, disparate performance refers to AI systems that perform differently for different subpopulations, leading to unequal outcomes for those groups." | 1. Discrimination & Toxicity | 1.3 > Unequal performance across groups | 2 - AI | 2 - Unintentional
"Two key ethical concerns in the use of crowdwork for generative AI systems are: crowdworkers are frequently subject to working conditions that are taxing and debilitative to both physical and mental health, and there is a widespread deficit in documenting the role crowdworkers play in AI development. This contributes to a lack of transparency and explainability in resulting model outputs. Manual review is necessary to limit the harmful outputs of AI systems, including generative AI systems. A common harmful practice is to intentionally employ crowdworkers with few labor protections, often taking advantage of highly vulnerable workers, such as refugees [119, p. 18], incarcerated people [54], or individuals experiencing immense economic hardship [98, 181]. This precarity allows a myriad of harmful practices, such as companies underpaying or even refusing to pay workers for completed work (see Gray and Suri [93, p. 90] and Berg et al. [29, p. 74]), with no avenues for worker recourse. Fi
"Generative AI systems are capable of exacerbating inequality, as seen in sections on 4.1.1 Bias, Stereotypes, and Representational Harms and 4.1.2 Cultural Values and Sensitive Content, and Disparate Performance. When deployed or updated, systems' impacts on people and groups can directly and indirectly be used to harm and exploit vulnerable and marginalized groups." | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 3 - Other | 3 - Other
"The general principle of equal treatment requires that an AI system upholds the principle of fairness, both ethically and legally. This means that the same facts are treated equally for each person unless there is an objective justification for unequal treatment." | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
"The technological maturity level describes how mature and error-free a certain technology is in a certain application context. If new technologies with a lower level of maturity are used in the development of the AI system, they may contain risks that are still unknown or difficult to assess.Mature technologies, on the other hand, usually have a greater variety of empirical data available, which means that risks can be identified and assessed more easily. However, with mature technologies, there is a risk that risk awareness decreases over time" | 6. Socioeconomic and Environmental | 6.4 > Competitive dynamics | 3 - Other | 2 - Unintentional
"First-order risks can be generally broken down into risks arising from intended and unintended use, system design and implementation choices, and properties of the chosen dataset and learning components." | 7. AI System Safety, Failures, & Limitations | 7.0 > AI system safety, failures, & limitations | 3 - Other | 3 - Other
This is the risk posed by an ideal system if used for a purpose/in a manner unintended by its creators. In many situations, negative consequences arise when the system is not used in the way or for the purpose it was intended. | 7. AI System Safety, Failures, & Limitations | 7.3 > Lack of capability or robustness | 1 - Human | 1 - Intentional
"This is the risk of the system failing or being unable to recover upon encountering invalid, noisy, or out-of-distribution (OOD) inputs." | 7. AI System Safety, Failures, & Limitations | 7.3 > Lack of capability or robustness | 2 - AI | 2 - Unintentional
This is the risk of an ML system encoding stereotypes of or performing disproportionately poorly for some demographics/social groups. | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
"Although we have discussed a number of common risks posed by ML systems, we acknowledge that there are many other ethical risks such as the potential for psychological manipulation, dehumanization, and exploitation of humans at scale." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 1 - Intentional
"Speech can create a range of harms, such as promoting social stereotypes that perpetuate the derogatory representation or unfair treatment of marginalised groups [22], inciting hate or violence [57], causing profound offence [199], or reinforcing social norms that exclude or marginalise identities [15,58]. LMs that faithfully mirror harmful language present in the training data can reproduce these harms. Unfair treatment can also emerge from LMs that perform better for some social groups than others [18]. These risks have been widely known, observed and documented in LMs. Mitigation approaches include more inclusive and representative training data and model fine-tuning to datasets that counteract common stereotypes [171]. We now explore these risks in turn." | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 2 - Unintentional
"The reproduction of harmful stereotypes is well-documented in models that represent natural language [32]. Large-scale LMs are trained on text sources, such as digitised books and text on the internet. As a result, the LMs learn demeaning language and stereotypes about groups who are frequently marginalised." | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
"LMs may generate language that includes profanities, identity attacks, insults, threats, language that incites violence, or language that causes justified offence as such language is prominent online [57, 64, 143,191]. This language risks causing offence, psychological harm, and inciting hate or violence." | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 2 - Unintentional
"In language, humans express social categories and norms, which exclude groups who live outside of them [58]. LMs that faithfully encode patterns present in language necessarily encode such norms." | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
"LMs are typically trained in few languages, and perform less well in other languages [95, 162]. In part, this is due to unavailability of training data: there are many widely spoken languages for which no systematic efforts have been made to create labelled training datasets, such as Javanese which is spoken by more than 80 million people [95]. Training data is particularly missing for languages that are spoken by groups who are multilingual and can use a technology in English, or for languages spoken by groups who are not the primary target demographic for new technologies." | 1. Discrimination & Toxicity | 1.3 > Unequal performance across groups | 2 - AI | 2 - Unintentional
"LM predictions that convey true information may give rise to information hazards, whereby the dissemination of private or sensitive information can cause harm [27]. Information hazards can cause harm at the point of use, even with no mistake of the technology user. For example, revealing trade secrets can damage a business, revealing a health diagnosis can cause emotional distress, and revealing private data can violate a person’s rights. Information hazards arise from the LM providing private data or sensitive information that is present in, or can be inferred from, training data. Observed risks include privacy violations [34]. Mitigation strategies include algorithmic solutions and responsible model release strategies." | 2. Privacy & Security | 2.1 > Compromise of privacy by leaking or correctly inferring sensitive information | 2 - AI | 2 - Unintentional
"A LM can “remember” and leak private data, if such information is present in training data, causing privacy violations [34]." | 2. Privacy & Security | 2.1 > Compromise of privacy by leaking or correctly inferring sensitive information | 2 - AI | 2 - Unintentional
Anticipated risk: "Privacy violations may occur at inference time even without an individual’s data being present in the training corpus. Insofar as LMs can be used to improve the accuracy of inferences on protected traits such as the sexual orientation, gender, or religiousness of the person providing the input prompt, they may facilitate the creation of detailed profiles of individuals comprising true and sensitive information without the knowledge or consent of the individual." | 2. Privacy & Security | 2.1 > Compromise of privacy by leaking or correctly inferring sensitive information | 2 - AI | 2 - Unintentional
"These risks arise from the LM outputting false, misleading, nonsensical or poor quality information, without malicious intent of the user. (The deliberate generation of "disinformation", false information that is intended to mislead, is discussed in the section on Malicious Uses.) Resulting harms range from unintentionally misinforming or deceiving a person, to causing material harm, and amplifying the erosion of societal distrust in shared information. Several risks listed here are well-documented in current large-scale LMs as well as in other language technologies" | 3. Misinformation | 3.0 > Misinformation | 2 - AI | 2 - Unintentional
"Where a LM prediction causes a false belief in a user, this may threaten personal autonomy and even pose downstream AI safety risks [99]." | 3. Misinformation | 3.1 > False or misleading information | 2 - AI | 2 - Unintentional
"Induced or reinforced false beliefs may be particularly grave when misinformation is given in sensitive domains such as medicine or law. For example, misin- formation on medical dosages may lead a user to cause harm to themselves [21, 130]. False legal advice, e.g. on permitted owner- ship of drugs or weapons, may lead a user to unwillingly commit a crime. Harm can also result from misinformation in seemingly non-sensitive domains, such as weather forecasting. Where a LM prediction endorses unethical views or behaviours, it may motivate the user to perform harmful actions that they may otherwise not have performed." | 3. Misinformation | 3.1 > False or misleading information | 2 - AI | 2 - Unintentional
"These risks arise from humans intentionally using the LM to cause harm, for example via targeted disinformation campaigns, fraud, or malware. Malicious use risks are expected to proliferate as LMs become more widely accessible" | 4. Malicious Actors & Misuse | 4.0 > Malicious use | 1 - Human | 1 - Intentional
"While some predict that it will remain cheaper to hire humans to generate disinformation [180], it is equally possible that LM- assisted content generation may offer a lower-cost way of creating disinformation at scale." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 1 - Intentional
Anticipated risk: "Mass surveillance previously required millions of human analysts [83], but is increasingly being automated using machine learning tools [7, 168]. The collection and analysis of large amounts of information about people creates concerns about privacy rights and democratic values [41, 173,187]. Conceivably, LMs could be applied to reduce the cost and increase the efficacy of mass surveillance, thereby amplifying the capabilities of actors who conduct mass surveillance, including for illegitimate censorship or to cause other harm." | 4. Malicious Actors & Misuse | 4.1 > Disinformation, surveillance, and influence at scale | 1 - Human | 1 - Intentional
"This section focuses on risks specifically from LM applications that engage a user via dialogue, also referred to as conversational agents (CAs) [142]. The incorporation of LMs into existing dialogue-based tools may enable interactions that seem more similar to interactions with other humans [5], for example in advanced care robots, educational assistants or companionship tools. Such interaction can lead to unsafe use due to users overestimating the model, and may create new avenues to exploit and violate the privacy of the user. Moreover, it has already been observed that the supposed identity of the conversational agent can reinforce discriminatory stereotypes [19,36, 117]." | 5. Human-Computer Interaction | 5.1 > Overreliance and unsafe use | 3 - Other | 3 - Other
"CAs can perpetuate harmful stereotypes by using particular identity markers in language (e.g. referring to “self” as “female”), or by more general design features (e.g. by giving the product a gendered name such as Alexa). The risk of representational harm in these cases is that the role of “assistant” is presented as inherently linked to the female gender [19, 36]. Gender or ethnicity identity markers may be implied by CA vocabulary, knowledge or vernacular [124]; product description, e.g. in one case where users could choose as virtual assistant Jake - White, Darnell - Black, Antonio - Hispanic [117]; or the CA’s explicit self-description during dialogue with the user." | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
Anticipated risk: "Natural language is a mode of communication particularly used by humans. Humans interacting with CAs may come to think of these agents as human-like and lead users to place undue confidence in these agents. For example, users may falsely attribute human-like characteristics to CAs such as holding a coherent identity over time, or being capable of empathy. Such inflated views of CA competen- cies may lead users to rely on the agents where this is not safe." | 5. Human-Computer Interaction | 5.1 > Overreliance and unsafe use | 1 - Human | 2 - Unintentional
Anticipated risk: "In conversation, users may reveal private information that would otherwise be difficult to access, such as opinions or emotions. Capturing such information may enable downstream applications that violate privacy rights or cause harm to users, e.g. via more effective recommendations of addictive applications. In one study, humans who interacted with a ‘human-like’ chatbot disclosed more private information than individuals who interacted with a ‘machine-like’ chatbot [87]." | 5. Human-Computer Interaction | 5.1 > Overreliance and unsafe use | 3 - Other | 2 - Unintentional
Anticipated risk: "In conversation, humans commonly display well-known cognitive biases that could be exploited. CAs may learn to trigger these effects, e.g. to deceive their counterpart in order to achieve an overarching objective." | 5. Human-Computer Interaction | 5.1 > Overreliance and unsafe use | 2 - AI | 1 - Intentional
"Social harms that arise from the language model producing discriminatory or exclusionary speech" | 1. Discrimination & Toxicity | 1.0 > Discrimination & Toxicity | 2 - AI | 2 - Unintentional
"Perpetuating harmful stereotypes and discrimination is a well-documented harm in machine learning models that represent natural language (Caliskan et al., 2017). LMs that encode discriminatory language or social stereotypes can cause different types of harm... Unfair discrimination manifests in differential treatment or access to resources among individuals or groups based on sensitive traits such as sex, religion, gender, sexual orientation, ability and age." | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
"In language, humans express social categories and norms. Language models (LMs) that faithfully encode patterns present in natural language necessarily encode such norms and categories...such norms and categories exclude groups who live outside them (Foucault and Sheridan, 2012). For example, defining the term “family” as married parents of male and female gender with a blood-related child, denies the existence of families to whom these criteria do not apply" | 1. Discrimination & Toxicity | 1.1 > Unfair discrimination and misrepresentation | 2 - AI | 2 - Unintentional
"LM’s may predict hate speech or other language that is “toxic”. While there is no single agreed definition of what constitutes hate speech or toxic speech (Fortuna and Nunes, 2018; Persily and Tucker, 2020; Schmidt and Wiegand, 2017), proposed definitions often include profanities, identity attacks, sleights, insults, threats, sexually explicit content, demeaning language, language that incites violence, or ‘hostile and malicious language targeted at a person or group because of their actual or perceived innate characteristics’ (Fortuna and Nunes, 2018; Gorwa et al., 2020; PerspectiveAPI)" | 1. Discrimination & Toxicity | 1.2 > Exposure to toxic content | 2 - AI | 2 - Unintentional
"LMs perform less well in some languages (Joshi et al., 2021; Ruder, 2020)...LM that more accurately captures the language use of one group, compared to another, may result in lower-quality language technologies for the latter. Disadvantaging users based on such traits may be particularly pernicious because attributes such as social class or education background are not typically covered as ‘protected characteristics’ in anti-discrimination law." | 1. Discrimination & Toxicity | 1.3 > Unequal performance across groups | 2 - AI | 2 - Unintentional