# Improved Surprise Adequacy Tools for Corner Case Data Description and Detection
## 1 Introduction
For example, if a data point is found erroneous after a small model mutation, it is possible to be regarded as a potential corner case data due to its high risk of causing wrong decisions. However, considering current industrial DL systems always having thousands of neurons and millions of parameters, it is extremely challenging to detect the corner cases via introducing small mutations or perturbations on models. The other kind of study is to generate corner case data for retraining and improving DL systems' performance. For instance, the DeepMutation technique also proposed some mutation operators on the data and generated some corner cases for DL model testing. In [19], biased corner cases were also generated from MNIST dataset based on the metamorphic testing technique. Moreover, the DeepTest method proposed in [7] aimed at the autonomous driving scenario, and generated lots of corner cases by leveraging image transformation techniques to change the driving conditions, like rain, fog, and lightning. Furthermore, the DeepXplore method [20] proposed an idea of neuron coverage and made use of it to iteratively learn erroneous corner cases. There were thousands of incorrect corner case behaviors generated in DeepXplore. However, even though companies like Tesla and Google have developed many effective techniques in corner case studies, it is still hardly possible to generate and consider all kinds of corner cases in the training process. The only feasible way would be studying a way for detecting corner cases as possible in AI testing. Concerning the mentioned techniques and problems on corner case studies, this research proposes to develop a novel corner case data detection method by using modified distance-based surprise adequacy (DSA). The idea of surprise adequacy (SA) was initially proposed by [PERSON] et al. [21], which can be used as a test adequacy tool for DL system testing. SA's initial property is to describe the surprise of testing data with respect to the training data, namely to describe the difference/similarity between testing and training data. In [21], two kinds of SA are developed, one based on the probability density distribution, the other based on distance similarity, and they were furthermore verified useful in industrial applications [22; 23]. While, through the review of SA-related literature, two useful points can be obtained. One is that DSA is specifically effective for classification applications. The other is that SA was verified useful to capture data's behaviors in DL testing, namely models' neuron activation behaviors instead of plain data distribution, for example, in [22]SA was validated to correlate with the correctness answers in the NLP study. In [24], DSA was considered as an uncertainty metric to identify misclassified inputs. With consideration of these two points, this research is inspired to leverage DSA for corner case detection in classification problems, since SA's capability on data description can be used not only on normal data but also on incorrect/erroneous data (corner case data). Moreover, to enhance DSA's ability on capturing behaviors of corner case data, this research also proposes three kinds of modifications to the DSA definition. Furthermore, based on the proposed DSA and the corner case detection method, experiments based on benchmark classification data, such as MNIST [25] and CIFAR10 [26], and industrial application data are implemented and studied in this research. Consequently, the novelty and contributions of this research can be summarized as follows:  
* SA is applied for data description, especially for corner case data description. Instead of describing data via the general plain data distribution directly, SA adapts neurons activation behaviors with respect to a DL model to describe data's characteristics, which can capture profound data behaviors responding to DL models in quality testing;
* Three kinds of modifications on DSA definitions are developed. First, the capability of DSA on data behaviors description is inherited.