## 4 Anomalies and Sources
### Adversarial Attack  
Whether in reality or in theory, AGVs have sensors to sense different information. For example, camera sensors can sense pictures, and some sensors can learn location or voice information. These sensors receive real-world information, such as obstacles. The device should stop moving or bypass obstacles when it encounters an object. When the device encounters a stop sign, it should stop immediately and adjust its route. At this time, a safety problem arises. If the image received by the sensor has been intentionally modified, this modification would be invisible to humans, but it would lead the AGV to make an incorrect decision--for example, speeding up when it is time to stop. As another example, if someone deliberately adds some insignificant noise to the obstacle, this could cause the AGV to directly hit the obstacle. These are the safety concerns that should be considered in the industrial environment. This kind of noise is called adversarial noise and cannot be detected by humans but will change the noise predicted by the AGV. Data with this kind of noise are called adversarial examples. This is expressed more formally below.  
**Definition: Adversarial Examples** Let \(f\) be a trained neural network used for classification tasks. Let \(H\) be a human oracle with the same classification capabilities. Assume that for a given legitimate input \(x\), the following equation holds: \(f(x)=H(x)\) Let \(x^{\prime}\) be a modified version of \(x\) that is close to \(x\), i.e., \(||x^{\prime}-x||\leq e\), for all small \(e\), \(e\in\mathbb{R}\). Then, \(x\) is an adversarial example, if the following holds:  
\[H(x)=H(x^{\prime})\bigwedge f(x)\
eq f(x^{\prime}). \tag{1}\]  
In the real world, the way to implement adversarial attacks is to use adversarial noise, such as an adversarial sticker, attached to one or more AGVs. When the AGV's sensor transmits these artificial noises to the decision-making system, it will make incorrect decisions. These prediction errors can produce runaway AGVs or disabled AGVs, which in turn lead to many safety and security problems. Hence, this adversarial attack is considered a kind of anomaly. In this context, the following two possible scenarios are introduced: the first one is that there is no obstacle ahead, but the AGV thinks one or several obstacles are present because of an adversarial attack. The AGV stops and does not move (forward) for a long time. In this situation, the AGV should communicate with other AGVs to find or update a new feasible path to complete the task.  
The second case is the opposite: an obstacle is in front of the AGV, but the attacker makes the AGV think that there is no obstacle (see Figure 1). This will cause the AGV to go forward and hit the object directly (with high speed). At this moment, the AGV may be damaged due to this unexpected collision. The damage means that one or more components of the AGV stop working. If the motor is damaged or the wheels are blocked as consequence of the collision, the adversarial attack can lead to the anomaly described in Section 4.1.  
Figure 1: An example of an adversarial attack. On the left, the normal situation is shown. On the right figure, one object has adversarial patch(es), which results in the anomaly. More specifically, the AGV cannot detect the object correctly.