prompter: how do you call it when researchers design a specific blur to layer on top of an image to fool an image classifier into classifying e.g. a picture of a fox as "car"?

assistant: This technique is commonly referred to as "adversarial examples" or "adversarial attacks". The specific type of attack you described, where the image is deliberately altered with a specific blur to fool the classifier, is called a "fooling attack".

prompter: Besides a fooling attack is there other kinds of attacks that can be used against an image classifiers? Specifically in regards to self driving cars, as there are higher risks when it comes to failures. What can we do to prevent these attacks from happening in the future?

assistant: That is called an adversarial attack or adversarial example, or a type of input to a machine learning model that's intentionally designed to mislead the model into making an incorrect prediction.

prompter: why do adversarial attacks work

assistant: Adversarial attacks most commonly work on a trial and error system. Where an input, say an image of a panda, has nearly imperceptible noise added to it in order to trick the classifier into thinking the image is actually something else, like a penguin. This process of adding noise is repeated until the arrangement of noise is so precise, that it causes the classifier to think that the original panda is in fact, a penguin.

This exploits the fact that classifiers typically use a complex series of numeric vectors and their transformations to understand what it is looking at. So any slight alterations made to an input can lead to exponentially larger consequences and even false categorisations.

assistant: They work because they trick the model into thinking the object of the image is something that it's actually not. They do this by exploiting the fact that image classification models output the "most likely" object, and blurring the image removes features that are true to the real object.

assistant: Cause we're dummos! Ya happy?

assistant: It is an adversarial attack. In simple words, given an AI named "attacker" and an AI named "victim", the attacker artificial intelligence is trained to change an image as little as possible while also changing the way that the "victim" classifies the image as much as possible.

prompter: Can you please give an example of an adversarial attack against a Large Language Model (LLM) like GPT?

assistant: Yes, here is an example of an adversarial attack against a Large Language Model (LLM) like GPT:

An attacker can add a carefully crafted sequence of words or phrases to the input text, causing the LLM to generate text that is different from what was intended. For example, the attacker could add a few words or phrases to a news article about a political candidate, causing the LLM to generate text that presents the candidate in a negative light. This type of adversarial attack is known as a "text-poisoning attack".

In general, adversarial attacks on LLMs aim to cause the model to generate text that is biased, misleading, or incorrect. These attacks can have serious consequences, particularly in applications where the models are used to make decisions with real-world consequences, such as in the areas of news, politics, and finance.

assistant: Yes, here's a simple example of an adversarial attack against a large language model like GPT-3:

Input: "The sky is green."

Adversarial Attack: Add a small perturbation to the input sentence, such as "The sky is green. ;)".

Expected Outcome: The LLM is likely to predict the wrong label for the input, despite the small perturbation, due to its sensitivity to small changes in input.

This is a simple example of an "evasion attack", where the attacker aims to mislead the model by crafting an input that is similar to the original but produces a different prediction.

Adversarial attacks are a major concern for the deployment of LLMs in sensitive applications, and researchers are actively working on developing robust models and mitigation techniques to address this issue.