# Cybersecurity for AI Systems: A Survey
## 3 AI Attacks and Defense Mechanisms
### Poisoning Attack
#### 3.3.1 Dataset Poisoning Attacks  
The major scenarios of data poisoning attacks are error-agnostic poisoning attacks and error-specific poisoning attacks. In the error-agnostic type of poisoning attack the hacker aims to cause a Denial of Service (DOS) kind of attack. The hacker causes the system to produce errors, but it does not matter what type of error it is. For example, in a multi-class classification task a hacker could poison the data leading to misclassification of the data points irrespective of the class type, thus maximizing the loss function of the learning algorithm. To launch this kind of attack, the hacker needs to manipulate both the features and the labels of the data points. On the other hand, in error-specific poisoning attacks, the hacker causes the system to produce specific misclassification errors, resulting in security violation of both integrity and availability. Here, the hacker aims at misclassifying a small sample of chosen data points in a multi-class classification task. The hacker aims to minimize the loss function of the learning algorithm to serve the purpose, i.e., to force the system into misclassifying specific instances without compromising the normal system operation, ensuring that the attack is undetected [38, 39]. A model is built up from a training dataset. So, attacking the dataset results in poisoning the model. By poisoning the dataset, the adversary could manipulate to generate natural adversarial examples, or inject instances with incorrect labels into the training dataset. The model may learn the pattern on misclassified examples in the data that serves the goal of the adversary. The dataset poisoning attacks can be further subdivided into two categories [79]. * **Data Modification**: The adversary updates or deletes training data. Here, the attacker does not have access to the algorithm. They can only manipulate labels. For instance, the attacker can draw new labels at random from the training pool, or can optimize the labels to cause maximum disruption. * **Data Injection:** Even if the adversary does not have access to the training data or learning algorithm, he or she can still inject incorrect data into the training set. This is similar to manipulation, but the difference is that the adversary introduces new malicious data into the training pool, not just labels. Support Vector Machines (SVMs) are widely used classification models for malware identification, intrusion detection systems, and filtering of spam emails, to name a few applications. [PERSON], [PERSON] and [PERSON] [40] illustrated poisoning attacks on the SVM classifier, with the assumption that the adversary has information about the learning algorithm, and the data distribution. The adversary generates surrogate data from the data distribution and tampers with the training data, by introducing the surrogate data,to drastically reduce the model training accuracy. The test data remains untouched. The authors formed an equation, expressing the adversarial strategy, as:  
\[MAX_{x}\ A(x)=\sum_{i=1}^{k}(1-y_{i}f_{x}(x_{i}))=\ \sum_{i=1}^{k}\left(-g_{i}\right)\]  
where \(x_{l}\leq x\ \leq x_{u}\) and \(\text{D}=(x_{i},\ y_{i})_{i=1}^{k}\) is the validation data. The goal of the adversary is to maximize the loss function \(A(x)\) with the surrogate data instance \((x,\ y)\) to be added into the training set \(D_{tr}\) in order to maximally reduce the training accuracy of classification. g\({}_{i}\)is the status of the margin, influenced by the surrogate data instance \((x,\ y)\). [PERSON] et al. [80] presented the attack on SVM learning by exploiting training data confidentiality. The objective is to access the features and the labels of the training data by examining the classification on the test set. Figure 2 explains the poison attack on the SVM classifier. The left sub-figure indicates the decision boundary of the linear SVM classifier, with support vectors and classifier margin.