# Improved Surprise Adequacy Tools for Corner Case Data Description and Detection
## 1 Introduction  
As the fast development of machine learning technologies in the last decade, especially Deep Learning (DL) based techniques, it makes Artificial Intelligent (AI) a widely-known term in both industries and our daily life. However, its fast development also makes us concerning AI applications' quality, e.g., the safety of DL-based systems [1]. As more complex architectures of deep neural networks are applied to obtain high accuracy in applications, the safety becomes more critical than ever before, especially in life- and property-related fields, like medical diagnosis [2], malware detection [3], autonomous driving [4] and so on. For instance, autonomous driving is being developed in many big companies, like Tesla, Ford, Waymo/Google, but several serious accidents have happened in the real world due to DLs' safety issues [5, 6]. These accidents imminently grab scholars' attention around the world on studies about finding factors affecting the safety in DL systems testing and taking measurements to eliminate them. In both traditional software and DL-based systems, incorrect or unexpected corner case behaviors are always regarded to play an important part in affecting systems' safety [7]. For example, in autonomous driving, many reported real-world collision cases are related to those rare or previously unseen corner cases. In [8], a collision between a Tesla car and a trailer was reported since the Tesla's DL system failed to deal with the corner cases of \"white color against a brightly lit sky\" and the \"high ride height\". Another collision was reported on a Google self-driving car crashed to a bus since its DL system made wrong decision on rare conditions (corner cases) [9]. Besides the given two examples, many otherwrong decisions made by AI systems are more or less related to different corner cases. For instances, in cerebral small vessel disease, due to the heterogeneous parenchymal damage in morphology and size [10], corner cases happen to cause DL models to make wrong decisions. The other corner case study in wind engineering is ramp study, which brings harm to the electricity industry's safety [11]. Therefore, to detect and fix those potential flaws or undesired corner case behaviors, namely corner case detection, is surely crucial for system designers to construct safety- and security-critical DL systems, and it should also be a necessary part of systematical AI testing. Currently, many scholars are studying corner cases and related AI testing techniques [12; 13]. Firstly, from the perspective of concepts, corner cases are namely those erroneous behaviors in the DL-based software, which are analogous to bugs in traditional software [14]. Different to traditional software, corner cases of DL systems pay more attention to data distribution instead of system architectures. Hence, traditional software usually aims at patch fixing after bug detection, DL-based systems need to consider these detected erroneous corner case data in the AI model retraining process for improving its robustness on structure and parameters. Secondly, research on corner cases mainly includes the following two parts. One kind of study is to test a given DL model's robustness and stability when facing corner case data. For example, some adversarial attack strategies, such as in [15; 16], are applied to the original testing data to generate adversarial samples which can reflect the feature of corner cases to some extent, then the DL model's accuracy and robustness are tested. The DeepFool proposed in [17] can also realize the generation of adversarial corner cases and robustness testing. The DeepMutation proposed in [18] aimed at mutating DL models' structure and parameters directly, and testing the modified model's performance on the original testing data. Moreover, through mutation and testing, some undetected corner cases can be further detected.