# Cybersecurity for AI Systems: A Survey
## 2 Literature Review
Their taxonomy includes four dimensions to categorize attacks on AI systems, including attack classes, attack targets, vulnerabilities and exploits used by the attacks and whether the attack has a payload or effect beyond itself. Their taxonomical structure is very comprehensive and can be used to analyze a system for its dependability, reliability and security. Despite the benefits of machine learning technologies, the learning algorithms can be abused by cybercriminals to conduct illicit and undesirable activities. It was shown in [33, 34] that attackers might gain a significant benefit by exploiting vulnerabilities in the learning algorithms, which can sometimes become a weakest link in the security chain. Several studies related to attacks on machine learning algorithms have been reported in the literature using different threat models. [PERSON] et al. [35, 36], [PERSON] et al. [36], [PERSON] et al. [37] and [PERSON] et al. [38] discussed different attack scenarios against machine learning models with different attack models. The frameworks they proposed characterize the attacks according to the attacker's goal, their capabilities to manipulate the data and influence the learning system, familiarity with the algorithms, the data used by the defender and the attacker's strategy. For example, data poisoning attacks, also known as causative attacks, are a major emerging security threat to data-driven technologies. In these types of attacks, it can be assumed that the hacker has control over the training dataset that is being used by the learning algorithm. The hacker can actively influence the training dataset in order to subvert the entire learning process, thus decreasing the overall performance of the system, or to produce particular types of errors in the system output. For example, in a classification task, the hacker may poison the data to modify the decision boundaries learned by the learning algorithm, thus resulting in misclassification of instances, or a higher error rate for a specific type of class. This is a kind of threat that is related to the reliability of the large amount of data collected by the systems [38, 39]. This survey is distinct from [31, 39] in studying attacks on an AI system from the perspective of a software engineering team, that organizes its work around different stages of an AI system's development life cycle. For these different stages of an AI system, and their corresponding attacks, potential defense mechanisms are also provided. Organizing the literature using this perspective can be valuable to systematically study the design of AI systems for security purposes, to explore the trade offs that result from using different defense mechanisms, and to develop a catalog of patterns and tactics for designing AI systems for security purposes. Table 1 lists various attacks carried out at different stages of the AI system development processes and the countermeasures that are taken against these attacks. \begin{table}
\begin{tabular}{p{56.9 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Attacks** & **AI System Development** & **Vulnerabilities** & **Defense Mechanisms** \\ \hline Poisoning attacks [1, 37] model & During training of the algorithms, resulting in stealing of the data and algorithm from individual user devices. & Weakness in the federated learning algorithms, resulting in stealing of the data and algorithm from individual user devices. & See list of defense mechanisms for both the data and model poisoning attacks. \\ \hline Data poisoning attacks [38, 39, 40] & During the training stage & Tampering of the features and class information in the training dataset & Adversarial training, Feature squeezing, Transferability blocking, MagNet, Defense-GAN, Local intrinsic dimensionality, Reject On Negative Impact (RONI), L-2 Defense, Slab Defense, Loss Defense and K-NN Defense. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Attacks on AI systems at different stages of its development.