# Resilience and Resilient Systems of Artificial Intelligence: Taxonomy, Models and Methods
## 5 Models and Methods to Ensure and Assess AIS Resilience
### Methods to Assess AIS Resilience
In this case, there are two types of attacks that give the most diverse results--\"strong\" attacks on one/few pixels and \"weak\" attacks on all pixels. The formation of both attacks is realized on the basis of the Covariance matrix adaptation evolution strategy (CMA-ES) [132; 133]. For the first type of attacks, the constraint on the perturbation amplitude (th) is given by the L0-norm, and for the second type of attacks, by the Lo-norm. Testing the model's resilience to drift usually involves the most complex cases of drift, such as the emergence of a new class or real concept drift. The ability to adapt to concept drift can be tested by passing a sample of classes with swapped labels to the model for continual training. Successful adaptation means that performance post-recovery reaches at least 95% of the pre-disturbance performance. The condition for stopping the adaptation process is lack of improving performance within a given number of iterations or reaching the maximum number of steps (mini-batches). In [30; 31; 133], the resilience of the image classification system to faults, adversarial attacks, and real concept drift was tested. Figure 11 shows a diagram of the resilience testing method. In the testing diagram presented in Figure 11 shows, the AIS is considered as a black box. Only information about the AIS inputs and outputs is used, information about gradients AIS is ignored. Testing is performed by generating a disturbance and observing the AIS performance changing during disturbance absorption and adaptation. Empirical testing is more suitable for comparative analysis, as an analytical tool in assessing AIS resilience--however, empirical testing does not provide exact guarantees. To increase the effectiveness of empirical testing, it is necessary to improve test coverage. Another disadvantage of empirical testing is the lack of clear quantification of confidence in the truth of the desired property after testing [133; 134]. Formal verification methods are used to provide rigorous guarantees of test results. The works related to the formal verification of deep neural networks consider various options for encoding the model in a form convenient for the solver and implemented in accordance with the chosen theory [135; 136; 137]. The most well-known approaches to formal verification are as follows: constraint solver-based appro  
Figure 11: Functional diagram of AIS resilience testing. network is encoded as a set of constraints [138]; approaches based on the calculation of approximate bounds, where approximation operations are applied to the space of inputs, outputs, or functions of neural layers to simplify the search for guaranteed bounds [139; 140]; approaches based on the calculation of converging bounds, where the search and iterative refinement of guaranteed bounds are performed [141; 142]. These approaches are designed to provide qualitative verification and rely on deterministic results, but characterized by high computational complexity. However, factors such as the stochastic nature of learning, the appearance of data from an unknown distribution in the inference mode, the development of probabilistic neural networks and randomized model architectures narrow the possibilities and effectiveness of qualitative verification. In response to that, probabilistic (statistical) methods of verification (certification) are being developed; such methods are the most generalized and computationally efficient. An example of such method is the Lipschitz stability estimation method based on the theory of extreme values -but such methods have a reliability problem [143]. In addition, the vast majority of publications only consider verification of AIS robustness to disturbances [144; 145]. Not nearly enough attention is paid to the behavior of the AIS in the performance recovery mode. Recovery speed after a disturbance is still not verified. The paper [144; 145] proposes an approach to black-box probabilistic verification of robustness.