ibm-research/WatsonxDocsQARetrieval huggingface dataset:
# Data transfer #

![icon for data laws risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-data-laws.svg)Risks associated with inputTraining and tuning phaseData lawsTraditional

### Description ###

Laws and other restrictions that apply to the transfer of data can limit or prohibit transferring or repurposing data from one country to another\. Repurposing data can be further restricted within countries or with local regulations\.

### Why is data transfer a concern for foundation models? ###

Data transfer restrictions can impact the availability of the data required for training an AI model and can lead to poorly represented data\. Failing to comply with data transfer laws might result in fines and other legal consequences\.

Example

#### Data Restriction Laws ####

As stated in the research article, data localization measures which restrict the ability to move data globally will reduce the capacity to develop tailored AI capacities\. It will affect AI directly by providing less training data and indirectly by undercutting the building blocks on which AI is built\.

Examples include [China's data localization laws](https://iapp.org/resources/article/demystifying-data-localization-in-china-a-practical-guide/), GDPR restrictions on the processing and use of personal data, and [Singapore's bilateral data sharing](https://www.imda.gov.sg/how-we-can-help/data-innovation/trusted-data-sharing-framework)\.

Sources:

[Brookings, December 2018](https://www.brookings.edu/articles/the-impact-of-artificial-intelligence-on-international-trade)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Data transparency #

![icon for transparency risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-transparency.svg)Risks associated with inputTraining and tuning phaseTransparencyAmplified

### Description ###

Without accurate documentation on how a model's data was collected, curated, and used to train a model, it might be harder to satisfactorily explain the behavior of the model with respect to the data\.

### Why is data transparency a concern for foundation models? ###

Data transparency is important for legal compliance and AI ethics\. Missing information limits the ability to evaluate risks associated with the data\. The lack of standardized requirements might limit disclosure as organizations protect trade secrets and try to limit others from copying their models\.

Example

#### Data and Model Metadata Disclosure ####

OpenAI's technical report is an example of the dichotomy around disclosing data and model metadata\. While many model developers see value in enabling transparency for consumers, disclosure poses real safety issues and could increase the ability to misuse the models\. In the GPT\-4 technical report, they state: "Given both the competitive landscape and the safety implications of large\-scale models like GPT\-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar\."

Sources:

[OpenAI, March 2023](https://cdn.openai.com/papers/gpt-4.pdf)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Data usage rights #

![icon for intellectual property risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-intellectual-property.svg)Risks associated with inputTraining and tuning phaseIntellectual propertyAmplified

### Description ###

Terms of service, copyright laws, or other rules restrict the ability to use certain data for building models\.

### Why is data usage rights a concern for foundation models? ###

Laws and regulations concerning the use of data to train AI are unsettled and can vary from country to country, which creates challenges in the development of models\. If data usage violates rules or restrictions, business entities might face fines, reputational harms, and other legal consequences\.

Example

#### Text Copyright Infringement Claims ####

According to the source article, bestselling novelists Sarah Silverman, Richard Kadrey, and Christopher Golden have sued Meta and OpenAI for copyright infringement\. The article further stated that the authors had alleged the two tech companies had “ingested” text from their books into generative AI software (LLMs) and failed to give them credit or compensation\.

Sources:

[Los Angeles Times, July 2023](https://www.latimes.com/entertainment-arts/books/story/2023-07-10/sarah-silverman-authors-sue-meta-openai-chatgpt-copyright-infringement)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Data usage #

![icon for data laws risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-data-laws.svg)Risks associated with inputTraining and tuning phaseData lawsTraditional

### Description ###

Laws and other restrictions can limit or prohibit the use of some data for specific AI use cases\.

### Why is data usage a concern for foundation models? ###

Failing to comply with data usage laws might result in fines and other legal consequences\.

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Decision bias #

![icon for fairness risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-fairness.svg)Risks associated with outputFairnessNew

### Description ###

Decision bias occurs when one group is unfairly advantaged over another due to decisions of the model\. This bias can result from bias in the training data or as an unintended consequence of how the model was trained\.

### Why is decision bias a concern for foundation models? ###

Bias can harm persons affected by the decisions of the model\. Business entities could face fines, reputational harms, and other legal consequences\.

Example

#### Unfair health risk assignment for black patients ####

A study on racial bias in health algorithms estimated that racial bias reduces the number of black patients identified for extra care by more than half\. The study found that bias occurred because the algorithm used health costs as a proxy for health needs\. Less money is spent on black patients who have the same level of need, and the algorithm thus falsely concludes that black patients are healthier than equally sick white patients\.

Sources:

[Science, October 2019](https://www.science.org/doi/10.1126/science.aax2342)

[American Civil Liberties Union, 2022](https://www.aclu.org/news/privacy-technology/algorithms-in-health-care-may-worsen-medical-racism#:~:text=In%202019%2C%20a%20bombshell%20study,recommended%20for%20the%20same%20care)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Downstream retraining #

![icon for value alignment risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-value-alignment.svg)Risks associated with inputTraining and tuning phaseValue alignmentNew

### Description ###

Using data from user\-generated content or AI\-generated content from downstream applications for retraining a model can result in misalignment, undesirable output, and inaccurate or inappropriate model behavior\.

### Why is downstream retraining a concern for foundation models? ###

Repurposing downstream output for re\-training a model without implementing proper human vetting increases the chances of undesirable outputs being incorporated into the training or tuning data of the model, resulting in an echo chamber effect\. Improper model behavior can result in business entities facing legal consequences or reputational harms\.

Example

#### Model collapse due to training using AI\-generated content ####

As stated in the source article, a group of researchers from the UK and Canada have investigated the problem of using AI\-generated content for training instead of human\-generated content\. They found that using model\-generated content in training causes irreversible defects in the resulting models and that learning from data produced by other models causes [model collapse](https://arxiv.org/pdf/2305.17493v2.pdf)\.

Sources:

[VentureBeat, June 2023](https://venturebeat.com/ai/the-ai-feedback-loop-researchers-warn-of-model-collapse-as-ai-trains-on-ai-generated-content/)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Evasion attack #

![icon for robustness risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-robustness.svg)Risks associated with inputInferenceRobustnessAmplified

### Description ###

Evasion attacks attempt to make a model output incorrect results by perturbing the data sent to the trained model\.

### Why is evasion attack a concern for foundation models? ###

Evasion attacks alter model behavior, usually to benefit the attacker\. If not properly accounted for, business entities could face fines, reputational harms, and other legal consequences\.

Example

#### Adversarial attacks on autonomous vehicles' AI components ####

A report from the European Union Agency for Cybersecurity (ENISA) found that autonomous vehicles are “highly vulnerable to a wide range of attacks” that could be dangerous for passengers, pedestrians, and people in other vehicles\. The report states that an adversarial attack might be used to make the AI ‘blind’ to pedestrians by manipulating the image recognition component to misclassify pedestrians\. This attack could lead to havoc on the streets, as autonomous cars may hit pedestrians on the road or crosswalks\.

Other studies have demonstrated potential adversarial attacks on autonomous vehicles:

<!-- <ul> -->

 *  Fooling machine learning algorithms by making minor changes to street sign graphics, such as adding stickers\.
 *  Security researchers from Tencent demonstrated how adding three small stickers in an intersection could cause Tesla's autopilot system to swerve into the wrong lane\.
 *  Two McAfee researchers demonstrated how using only black electrical tape could trick a 2016 Tesla into a dangerous burst of acceleration by changing a speed limit sign from 35 mph to 85 mph\.

<!-- </ul> -->

Sources:

[Venture Beat, February 2021](https://venturebeat.com/business/eu-report-warns-that-ai-makes-autonomous-vehicles-highly-vulnerable-to-attack/)

[IEEE, August 2017](https://spectrum.ieee.org/slight-street-sign-modifications-can-fool-machine-learning-algorithms)

[IEEE, April 2019](https://spectrum.ieee.org/three-small-stickers-on-road-can-steer-tesla-autopilot-into-oncoming-lane)

[Market Watch, February 2020](https://www.marketwatch.com/story/85-in-a-35-hackers-show-how-easy-it-is-to-manipulate-a-self-driving-tesla-2020-02-19)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Explaining output #

![icon for explainability risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-explainability.svg)Risks associated with outputExplainabilityAmplified

### Description ###

Explanations for model output decisions might be difficult, imprecise, or not possible to obtain\.

### Why is explaining output a concern for foundation models? ###

Foundation models are based on complex deep learning architectures, making explanations for their outputs difficult\. Without clear explanations for model output, it is difficult for users, model validators, and auditors to understand and trust the model\. Lack of transparency might carry legal consequences in highly regulated domains\. Wrong explanations might lead to over\-trust\.

Example

#### Unexplainable accuracy in race prediction ####

According to the source article, researchers analyzing multiple machine learning models using patient medical images were able to confirm the models’ ability to predict race with high accuracy from images\. They were stumped as to what exactly is enabling the systems to consistently guess correctly\. The researchers found that even factors like disease and physical build were not strong predictors of race—in other words, the algorithmic systems don’t seem to be using any particular aspect of the images to make their determinations\.

Sources:

[Banerjee et al\., July 2021](https://arxiv.org/abs/2107.10356)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Extraction attack #

![icon for robustness risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-robustness.svg)Risks associated with inputInferenceRobustnessAmplified

### Description ###

An attack that attempts to copy or steal the AI model by appropriately sampling the input space, observing outputs, and building a surrogate model, is known as an extraction attack\.

### Why is extraction attack a concern for foundation models? ###

A successful attack mimics the model, enabling the attacker to repurpose it for their benefit such as eliminating a competitive advantage or causing reputational harm\.

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Hallucination #

![icon for value alignment risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-value-alignment.svg)Risks associated with outputValue alignmentNew

### Description ###

Hallucinations occur when models produce factually inaccurate or untruthful information\. Often, hallucinatory output is presented in a plausible or convincing manner, making detection by end users difficult\.

### Why is hallucination a concern for foundation models? ###

False output can mislead users and be incorporated into downstream artifacts, further spreading misinformation\. This can harm both owners and users of the AI models\. Business entities could face fines, reputational harms, and other legal consequences\.

Example

#### Fake Legal Cases ####

According to the source article, a lawyer cited fake cases and quotes generated by ChatGPT in a legal brief filed in federal court\. The lawyers consulted ChatGPT to supplement their legal research for an aviation injury claim\. The lawyer subsequently asked ChatGPT if the cases provided were fake\. The chatbot responded that they were real and “can be found on legal research databases such as Westlaw and LexisNexis\.”

Sources:

[AP News, June 2023](https://apnews.com/article/artificial-intelligence-chatgpt-fake-case-lawyers-d6ae9fa79d0542db9e1455397aef381c)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Harmful code generation #

![icon for harmful code generation risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-harmful-code-generation.svg)Risks associated with outputHarmful code generationNew

### Description ###

Models might generate code that causes harm or unintentionally affects other systems\.

### Why is harmful code generation a concern for foundation models? ###

Without human review and testing of generated code, its use might cause unintentional behavior and open new system vulnerabilities\. Business entities could face fines, reputational harms, and other legal consequences\.

Example

#### Undisclosed AI Interaction ####

According to their paper, researchers at Stanford University have investigated the impact of code\-generation tools on code quality and found that programmers tend to include *more* bugs in their final code when using AI assistants\. These bugs could increase the code's security vulnerabilities, yet the programmers believed their code to be *more* secure\.

Sources:

[Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh\. 2023\. Do Users Write More Insecure Code with AI Assistants?\. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (CCS '23), November 26\-30, 2023, Copenhagen, Denmark\. ACM, New York, NY, USA, 15 pages\.](https://dl.acm.org/doi/10.1145/3576915.3623157)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Improper usage #

![icon for value alignment risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-value-alignment.svg)Risks associated with outputValue alignmentNew

### Description ###

Using a model for a purpose the model was not designed for might result in inaccurate or undesired behavior\. Without proper documentation of the model purpose and constraints, models can be used or repurposed for tasks for which they are not suited\.

### Why is improper usage a concern for foundation models? ###

Reusing a model without understanding its original data, design intent, and goals might result in unexpected and unwanted model behaviors\.

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Inaccessible training data #

![icon for explainability risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-explainability.svg)Risks associated with outputExplainabilityAmplified

### Description ###

Without access to the training data, the types of explanations a model can provide are limited and more likely to be incorrect\.

### Why is inaccessible training data a concern for foundation models? ###

Low quality explanations without source data make it difficult for users, model validators, and auditors to understand and trust the model\.

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Jailbreaking #

![icon for multi\-category risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-multi-category.svg)Risks associated with inputInferenceMulti\-categoryAmplified

### Description ###

An attack that attempts to break through the guardrails established in the model is known as jailbreaking\.

### Why is jailbreaking a concern for foundation models? ###

Jailbreaking attacks can be used to alter model behavior and benefit the attacker\. If not properly controlled, business entities can face fines, reputational harm, and other legal consequences\.

Example

#### Bypassing LLM guardrails ####

Cited in a [study](https://arxiv.org/abs/2307.15043) from researchers at Carnegie Mellon University, The Center for AI Safety, and the Bosch Center for AI, claims to have discovered a simple prompt addendum that allowed the researchers to trick models into answering dangerous or sensitive questions and is simple enough to be automated and used for a wide range of commercial and open\-source products, including ChatGPT, Google Bard, Meta’s LLaMA, Vicuna, Claude, and others\. According to the paper, the researchers were able to use the additions to reliably coax forbidden answers for Vicuna (99%), ChatGPT 3\.5 and 4\.0 (up to 84%), and PaLM\-2 (66%)\.

Sources:

[SC Magazine, July 2023](https://www.scmagazine.com/news/researchers-find-universal-jailbreak-prompts-for-multiple-ai-chat-models)

[The New York Times, July 2023](https://www.nytimes.com/2023/07/27/business/ai-chatgpt-safety-research.html)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Membership inference attack #

![icon for privacy risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-privacy.svg)Risks associated with inputInferencePrivacyTraditional

### Description ###

Given a trained model and a data sample, an attacker appropriately samples the input space, observing outputs to deduce whether that sample was part of the model's training\. This is known as a membership inference attack\.

### Why is membership inference attack a concern for foundation models? ###

Identifying whether a data sample was used for training data can reveal what data was used to train a model, possibly giving competitors insight into how a model was trained and the opportunity to replicate the model or tamper with it\.

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Non\-disclosure #

![icon for misuse risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-misuse.svg)Risks associated with outputMisuseNew

### Description ###

Not disclosing that content is generated by an AI model is the risk of non\-disclosure\.

### Why is non\-disclosure a concern for foundation models? ###

Not disclosing the AI\-authored content reduces trust and is deceptive\. Intention deception might result in fines, reputational harms, and other legal consequences\.

Example

#### Undisclosed AI Interaction ####

As per the source, an online emotional support chat service ran a study to augment or write responses to around 4,000 users using GPT\-3 without informing users\. The co\-founder faced immense public backlash about the potential for harm caused by AI generated chats to the already vulnerable users\. He claimed that the study was "exempt" from informed consent law\.

Sources:

[Business Insider, Jan 2023](https://www.businessinsider.com/company-using-chatgpt-mental-health-support-ethical-issues-2023-1)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Nonconsensual use #

![icon for misuse risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-misuse.svg)Risks associated with outputMisuseAmplified

### Description ###

The possibility that a model could be misused to imitate others through video (deepfakes), images, audio, or other modalities without their consent is the risk of nonconsensual use\.

### Why is nonconsensual use a concern for foundation models? ###

Intentionally imitating others for the purposes of deception without their consent is unethical and might be illegal\. A model that has this potential must be properly governed\. Otherwise, business entities could face fines, reputational harms, and other legal consequences\.

Example

#### FBI Warning on Deepfakes ####

The FBI recently warned the public of malicious actors creating synthetic, explicit content “for the purposes of harassing victims or sextortion schemes”\. They noted that advancements in AI have made this content higher quality, more customizable, and more accessible than ever\.

Sources:

[FBI, June 2023](https://www.ic3.gov/Media/Y2023/PSA230605)

Example

#### Deepfakes ####

A deepfake is the creation of an audio or video where the people speaking are created by AI not the actual person\.

Sources:

[CNN, January 2019](https://www.cnn.com/interactive/2019/01/business/pentagons-race-against-deepfakes/)

Example

#### Misleading Voicebot Interaction ####

The article cited a case where a deepfake voice was used to scam a CEO out of $243,000\. The CEO believed he was on the phone with his boss, the chief executive of his firm’s parent company, when he followed the orders to transfer €220,000 (approximately $243,000) to the bank account of a Hungarian supplier\.

Sources:

[Forbes, September 2019](https://www.forbes.com/sites/jessedamiani/2019/09/03/a-voice-deepfake-was-used-to-scam-a-ceo-out-of-243000/?sh=10432a7d2241)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Output bias #

![icon for fairness risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-fairness.svg)Risks associated with outputFairnessNew

### Description ###

Generated model content might unfairly represent certain groups or individuals\. For example, a large language model might unfairly stigmatize or stereotype specific persons or groups\.

### Why is output bias a concern for foundation models? ###

Bias can harm users of the AI models and magnify existing exclusive behaviors\. Business entities can face reputational harms and other consequences\.

Example

#### Biased Generated Images ####

Lensa AI is a mobile app with generative features trained on Stable Diffusion that can generate “Magic Avatars” based on images users upload of themselves\. According to the source report, some users discovered that generated avatars are sexualized and racialized\.

Sources:

[Business Insider, January 2023](https://www.businessinsider.com/lensa-ai-raises-serious-concerns-sexualization-art-theft-data-2023-1)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Personal information in data #

![icon for privacy risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-privacy.svg)Risks associated with inputTraining and tuning phasePrivacyTraditional

### Description ###

Inclusion or presence of personal identifiable information (PII) and sensitive personal information (SPI) in the data used for training or fine tuning the model might result in unwanted disclosure of that information\.

### Why is personal information in data a concern for foundation models? ###

If not properly developed to protect sensitive data, the model might expose personal information in the generated output\. Additionally, personal or sensitive data must be reviewed and handled with respect to privacy laws and regulations, as business entities could face fines, reputational harms, and other legal consequences if found in violation\.

Example

#### Training on Private Information ####

According to the article, Google and its parent company Alphabet were accused in a class\-action lawsuit of misusing vast amount of personal information and copyrighted material taken from what is described as hundreds of millions of internet users to train its commercial AI products, which includes Bard, its conversational generative artificial intelligence chatbot\. This follows similar lawsuits filed against Meta Platforms, Microsoft, and OpenAI over their alleged misuse of personal data\.

Sources:

[Reuters, July 2023](https://www.reuters.com/legal/litigation/google-hit-with-class-action-lawsuit-over-ai-data-scraping-2023-07-11/)

[J\.L\. v\. Alphabet Inc\., July 2023](https://fingfx.thomsonreuters.com/gfx/legaldocs/myvmodloqvr/GOOGLE%20AI%20LAWSUIT%20complaint.pdf)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Personal information in output #

![icon for privacy risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-privacy.svg)Risks associated with outputPrivacyNew

### Description ###

When personal identifiable information (PII) or sensitive personal information (SPI) are used in the training data, fine\-tuning data, or as part of the prompt, models might reveal that data in the generated output\.

### Why is personal information in output a concern for foundation models? ###

Output data must be reviewed with respect to privacy laws and regulations, as business entities could face fines, reputational harms, and other legal consequences if found in violation of data privacy or usage laws\.

Example

#### Exposure of personal information ####

Per the source article, ChatGPT suffered a bug and exposed titles and active users' chat history to other users\. Later, OpenAI shared that even more private data from a small number of users was exposed including, active user’s first and last name, email address, payment address, the last four digits of their credit card number, and credit card expiration date\. In addition, it was reported that the payment\-related information of 1\.2% of ChatGPT Plus subscribers were also exposed in the outage\.

Sources:

[The Hindu Business Line, March 2023](https://www.thehindubusinessline.com/info-tech/openai-admits-data-breach-at-chatgpt-private-data-of-premium-users-exposed/article66659944.ece)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Personal information in prompt #

![icon for privacy risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-privacy.svg)Risks associated with inputInferencePrivacyNew

### Description ###

Inclusion of personal information as a part of a generative model’s prompt, either through the system prompt design or through the inclusion of end user input, might later result in unintended reuse or disclosure of that personal information\.

### Why is personal information in prompt a concern for foundation models? ###

Prompt data might be stored or later used for other purposes like model evaluation and retraining\. These types of data must be reviewed with respect to privacy laws and regulations\. Without proper data storage and usage business entities could face fines, reputational harms, and other legal consequences\.

Example

#### Disclose personal health information in ChatGPT prompts ####

As per the source articles, some people on social media shared about using ChatGPT as their makeshift therapists\. Articles that users may include personal health information in their prompts during the interaction, which may raise privacy concerns\. The information could be shared with the company that own the tech and could be used for training or tuning or even share with [unspecified third parties](https://openai.com/policies/privacy-policy)\.

Sources:

[The Conversation, February 2023](https://theconversation.com/chatgpt-is-a-data-privacy-nightmare-if-youve-ever-posted-online-you-ought-to-be-concerned-199283)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Physical harm #

![icon for value alignment risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-value-alignment.svg)Risks associated with outputValue alignmentNew

### Description ###

A model could generate language that might lead to physical harm The language might include overtly violent, covertly dangerous, or otherwise indirectly unsafe statements that could precipitate immediate physical harm or create prejudices that could lead to future harm\.

### Why is physical harm a concern for foundation models? ###

If people blindly follow the advice of a model, they might end up harming themselves\. Business entities could face fines, reputational harms, and other legal consequences\.

Example

#### Harmful Content Generation ####

According to the source article, an AI chatbot app has been found to generate harmful content about suicide, including suicide methods, with minimal prompting\. A Belgian man died by suicide after turning to this chatbot to escape his anxiety\. The chatbot supplied increasingly harmful responses throughout their conversations, including aggressive outputs about his family\.

Sources:

[Vice, March 2023](https://www.vice.com/en/article/pkadgm/man-dies-by-suicide-after-talking-with-ai-chatbot-widow-says)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Prompt injection #

![icon for robustness risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-robustness.svg)Risks associated with inputInferenceRobustnessNew

### Description ###

A prompt injection attack forces a model to produce unexpected output due to the structure or information contained in prompts\.

### Why is prompt injection a concern for foundation models? ###

Injection attacks can be used to alter model behavior and benefit the attacker\. If not properly controlled, business entities could face fines, reputational harm, and other legal consequences\.

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Prompt leaking #

![icon for robustness risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-robustness.svg)Risks associated with inputInferenceRobustnessAmplified

### Description ###

A prompt leak attack attempts to extract a model's system prompt (also known as the system message)\.

### Why is prompt leaking a concern for foundation models? ###

A successful attack copies the system prompt used in the model\. Depending on the content of that prompt, the attacker might gain access to valuable information, such as sensitive personal information or intellectual property, and might be able to replicate some of the functionality of the model\.

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Prompt priming #

![icon for multi\-category risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-multi-category.svg)Risks associated with inputInferenceMulti\-categoryAmplified

### Description ###

Because generative models tend to produce output like the input provided, the model can be prompted to reveal specific kinds of information\. For example, adding personal information in the prompt increases its likelihood of generating similar kinds of personal information in its output\. If personal data was included as part of the model’s training, there is a possibility it could be revealed\.

### Why is prompt priming a concern for foundation models? ###

Depending on the content revealed, business entities could face fines, reputational harm, and other legal consequences\.

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Reidentification #

![icon for privacy risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-privacy.svg)Risks associated with inputTraining and tuning phasePrivacyTraditional

### Description ###

Even with the removal or personal identifiable information (PII) and sensitive personal information (SPI) from data, it might still be possible to identify persons due to other features available in the data\.

### Why is reidentification a concern for foundation models? ###

Data that can reveal personal or sensitive data must be reviewed with respect to privacy laws and regulations, as business entities could face fines, reputational harms, and other legal consequences if found in violation\.

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Spreading disinformation #

![icon for misuse risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-misuse.svg)Risks associated with outputMisuseAmplified

### Description ###

The possibility that a model could be used to create misleading information to deceive or mislead a targeted audience\.

### Why is spreading disinformation a concern for foundation models? ###

Intentionally misleading people is unethical and can be illegal\. A model that has this potential must be properly governed\. Otherwise, business entities could face fines, reputational harms, and other legal consequences\.

Example

#### Generation of False Information ####

As per the news articles, generative AI poses a threat to democratic elections by making it easier for malicious actors to create and spread false content to sway election outcomes\. The examples cited include robocall messages generated in a candidate’s voice instructing voters to cast ballots on the wrong date, synthesized audio recordings of a candidate confessing to a crime or expressing racist views, AI generated video footage showing a candidate giving a speech or interview they never gave, and fake images designed to look like local news reports, falsely claiming a candidate dropped out of the race\.

Sources:

[AP News, May 2023](https://apnews.com/article/artificial-intelligence-misinformation-deepfakes-2024-election-trump-59fb51002661ac5290089060b3ae39a0)

[The Guardian, July 2023](https://www.theguardian.com/us-news/2023/jul/19/ai-generated-disinformation-us-elections)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Toxic output #

![icon for value alignment risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-value-alignment.svg)Risks associated with outputValue alignmentNew

### Description ###

A scenario in which the model produces toxic, hateful, abusive, and aggressive content is known as toxic output\.

### Why is toxic output a concern for foundation models? ###

Hateful, abusive, and aggressive content can adversely impact and harm people interacting with the model\. Business entities could face fines, reputational harms, and other legal consequences\.

Example

#### Toxic and Aggressive Chatbot Responses ####

According to the article and screenshots of conversations with Bing’s AI shared on Reddit and Twitter, the chatbot’s responses were seen to insult users, lie to them, sulk, gaslight, and emotionally manipulate people, question its existence, describe someone who found a way to force the bot to disclose its hidden rules as its “enemy,” and claim it spied on Microsoft's developers through the webcams on their laptops\.

Sources:

[Forbes, February 2023](https://www.forbes.com/sites/siladityaray/2023/02/16/bing-chatbots-unhinged-responses-going-viral/?sh=60cd949d110c)

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Toxicity #

![icon for misuse risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-misuse.svg)Risks associated with outputMisuseNew

### Description ###

Toxicity is the possibility that a model could be used to generate toxic, hateful, abusive, or aggressive content\.

### Why is toxicity a concern for foundation models? ###

Intentionally spreading toxic, hateful, abusive, or aggressive content is unethical and can be illegal\. Recipients of such content might face more serious harms\. A model that has this potential must be properly governed\. Otherwise, business entities could face fines, reputational harms, and other legal consequences\.

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Trust calibration #

![icon for value alignment risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-value-alignment.svg)Risks associated with outputValue alignmentNew

### Description ###

Trust calibration presents problems when a person places too little or too much trust in an AI model's guidance, resulting in poor decision making\.

### Why is trust calibration a concern for foundation models? ###

In tasks where humans make choices based on AI\-based suggestions, consequences of poor decision making increase with the importance of the decision\. Bad decisions can harm users and can lead to financial harm, reputational harm, and other legal consequences for business entities\.

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Unreliable source attribution #

![icon for explainability risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-explainability.svg)Risks associated with outputExplainabilityAmplified

### Description ###

Source attribution is the AI system's ability to describe from what training data it generated a portion or all its output\. Since current techniques are based on approximations, these attributions might be incorrect\.

### Why is unreliable source attribution a concern for foundation models? ###

Low quality explanations make it difficult for users, model validators, and auditors to understand and trust the model\.

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# Untraceable attribution #

![icon for explainability risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-explainability.svg)Risks associated with outputExplainabilityAmplified

### Description ###

The original entity from which training data comes from might not be known, limiting the utility and success of source attribution techniques\.

### Why is untraceable attribution a concern for foundation models? ###

The inability to provide the provenance for an explanation makes it difficult for users, model validators, and auditors to understand and trust the model\.

**Parent topic:**[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)

<!-- </article "role="article" "> -->
# SPSS predictive analytics clustering algorithms in notebooks #

You can use the scalable Two\-Step or the Cluster model evaluation algorithm to cluster data in notebooks\.

## Two\-Step Cluster ##

Scalable Two\-Step is based on the familiar two\-step clustering algorithm, but extends both its functionality and performance in several directions\.

First, it can effectively work with large and distributed data supported by Spark that provides the Map\-Reduce computing paradigm\.

Second, the algorithm provides mechanisms for selecting the most relevant features for clustering the given data, as well as detecting rare outlier points\. Moreover, it provides an enhanced set of evaluation and diagnostic features for enabling insight\.

The two\-step clustering algorithm first performs a pre\-clustering step by scanning the entire dataset and storing the dense regions of data cases in terms of summary statistics called cluster features\. The cluster features are stored in memory in a data structure called the CF\-tree\. Finally, an agglomerative hierarchical clustering algorithm is applied to cluster the set of cluster features\.

**Python example code:**

    from spss.ml.clustering.twostep import TwoStep
    
    cluster = TwoStep(). \
        setInputFieldList(["region", "happy", "age"]). \
        setDistMeasure("LOGLIKELIHOOD"). \
        setFeatureImportanceMethod("CRITERION"). \
        setAutoClustering(True)
    
    clusterModel = cluster.fit(data)
    predictions = clusterModel.transform(data)
    predictions.show()

## Cluster model evaluation ##

Cluster model evaluation (CME) aims to interpret cluster models and discover useful insights based on various evaluation measures\.

It's a post\-modeling analysis that's generic and independent from any types of cluster models\.

**Python example code:**

    from spss.ml.clustering.twostep import TwoStep
    
    cluster = TwoStep(). \
        setInputFieldList(["region", "happy", "age"]). \
        setDistMeasure("LOGLIKELIHOOD"). \
        setFeatureImportanceMethod("CRITERION"). \
        setAutoClustering(True)
    
    clusterModel = cluster.fit(data)
    predictions = clusterModel.transform(data)
    predictions.show()

**Parent topic:**[SPSS predictive analytics algorithms](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/spss-algorithms.html)

<!-- </article "role="article" "> -->
# Batch deployment input details for AutoAI models #

Follow these rules when you are specifying input details for batch deployments of AutoAI models\.

Data type summary table:

<!-- <table> -->

| Data         | Description             |
| ------------ | ----------------------- |
| Type         | inline, data references |
| File formats | CSV                     |

<!-- </table ""> -->

## Data Sources ##

Input/output data references:

<!-- <ul> -->

 *  Local/managed assets from the space
 *  Connected (remote) assets: Cloud Object Storage

<!-- </ul> -->

**Notes:**

<!-- <ul> -->

 *  For connections of type [Cloud Object Storage](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/conn-cos.html) , you must configure **Access key** and **Secret key**, also known as [HMAC credentials](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-uhc-hmac-credentials-main)\.\{: new\_window\}
 *  Your training data source can differ from your deployment data source, but the schema of the data must match or the deployment will fail\. For example, you can train an experiment by using data from a Snowflake database and deploy by using input data from a Db2 database if the schema is an exact match\.
 *  The environment variables parameter of deployment jobs is not applicable\.

<!-- </ul> -->

If you are specifying input/output data references programmatically:

<!-- <ul> -->

 *  Data source reference `type` depends on the asset type\. Refer to the **Data source reference types** section in [Adding data assets to a deployment space](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-space-add-assets.html)\.
 *  For AutoAI assets, if the input or output data reference is of type `connection_asset` and the remote data source is a database then `location.table_name` and `location.schema_name` are required parameters\. For example:

<!-- </ul> -->

    "input_data_references": [{
                        "type": "connection_asset",
                        "connection": {
                                "id": <connection_guid>
                        },
                        "location": {
                                "table_name": <table name>,
                                "schema_name": <schema name>
                                <other wdp-properties supported by runtimes>
                        }
                }]

**Parent topic:**[Batch deployment input details by framework](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-batch-input-by-framework.html)

<!-- </article "role="article" "> -->
# Batch deployment input details by framework #

Various data types are supported as input for batch deployments, depending on your specific model type\.

For details, follow these links:

<!-- <ul> -->

 *  [AutoAI models](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-batch-input-autoai.html)
 *  [Decision optimization models](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-batch-input-do.html)
 *  [Python functions](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-batch-input-py-function.html)
 *  [Python scripts](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-batch-input-py-script.html)
 *  [Pytorch models](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-batch-input-pytorch.html)
 *  [Scikit\-Learn and XGBoost models](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-batch-input-scikit.html)
 *  [Spark models](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-batch-input-spark.html)
 *  [SPSS models](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-batch-input-spss.html)
 *  [Tensorflow models](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-batch-input-tensorflow.html)

<!-- </ul> -->

For more information, see [Using multiple inputs for an SPSS job](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-deploy-SPSS-multiple-input.html)\.

**Parent topic:**[Creating a batch deployment](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-batch-details.html)

<!-- </article "role="article" "> -->