# ExtendAIST: Exploring the Space of AI-in-the-Loop System Testing
## 3 Extendaist
### Design
That is, \(\phi(x,n)\) may locate \((-\infty,low_{n})\cup(high_{n},+\infty)\), which is referred to as the corner-case region of neuron \(n\). Therefore, the _neuron boundary coverage_\(Cov_{NBC}\), the rate of the number of neurons falling in corner-case region and the total number of corner cases as in Equation (4), is used to measure how many corner-case regions are covered by test dataset \(T\). \(N_{\textit{LUPPE}}=\{n\in N|\exists x\in T:\phi(x,n)\in(high_{n},+\infty)\}\) is the set of neurons located in the upper corner-case region, and \(N_{\textit{LOWER}}=\{n\in N|\exists x\in T:\phi(x,n)\in(-\infty,low_{n})\}\) is the set of neurons located in the lower corner-case region. The total number of corner cases of neuron boundary coverage is equal to\(2\times|N|\), because \((-\infty,low_{n})\) and \((high_{n},+\infty)\) are mutually exclusive and neuron cannot fall in two regions at the same time. \[Cov_{NBC}=\frac{|N_{UIPPER}|+|N_{LOWER}|}{2\times|N|}\] (4)
* Strong neuron activation coverage. Because hyperactive corner-case neurons affect the training of DNN significantly, it is essential to measure the coverage of hyperactive corner-case neurons, namely _strong neuron activation coverage_. Similar to the neuron boundary coverage, strong neuron activation coverage \(Cov_{SNA}\) is the rate of the number of neurons falling in the upper corner-case region and the total number of corner cases as in Equation (5). \[Cov_{SNA}=\frac{|N_{UIPPER}|}{|N|}\] (5)

#### 4.1.3 Layer Level (EP_LL)

In the layer level sub-extension point EP_LL, the neuron coverage from the perspective of top hyperactive neurons in each layer has been further investigated [15]. An effective input test dataset should cover more and more hyperactive neurons in one layer. In this sub-extension point, the top-\(k\) neuron coverage has been implemented to cover the top-\(k\) neurons in the same layer. * Top-\(k\) neuron coverage. Given test input \(x\) and neurons \(n_{1}\) and \(n_{2}\) in the same layer, if \(\phi(x,n_{1})>\phi(x,n_{2})\), then \(n_{1}\) is more active than \(n_{2}\). Here, the _top-\(k\) neuron coverage_ is designed to measure how many neurons are activated as the most \(k\) neurons in each layer by the input test dataset \(T\). As shown in Equation (6), \(top_{k}(x,i)\) is the set of first \(k\) neurons, which are ranked in descending order of their outputs. \[Cov_{TKN}=\frac{|\cup_{x\in T}\left(\cup_{1\leq i\leq l}top_{k}(x,i)\right)|}{ |N|}\] (6)

#### 4.1.4 Neuron Pair Level (EP_NPL)

The neuron pair level coverage focuses on the propagation of changes between neurons from adjacent layers. Inspired by the MC/DC criterion, [PERSON] et al. [16] proposed the neuron pair level coverage by taking \(\alpha=(n_{l}^{i},n_{l+1}^{j})\) as the neuron pair, in which \(n_{l}^{i}\) is a neuron regarded as a condition in the \(l\)th layer and \(n_{l+1}^{j}\) is a neuron regarded as a decision in the \((l+1)\)th layer. Hence, neuron pair level coverage is presented to inspect the influence of neurons in the \(l\)th layer on neurons in the \((l+1)\)th layer. The change of neuron \(n_{l}^{k}\) when given two test inputs \(x_{1}\) and \(x_{2}\) could be a _sign change_ (denoted as \(sc(n_{l}^{k},x_{1},x_{2})\)), a _value change_ (denoted as \(vc(g,n_{l}^{k},x_{1},x_{2})\), where \(g\) is a value change function), and a _distance change_ (denoted as \(dc(h,l,x_{1},x_{2})\), where \(h\) is a distance change function) for neurons in the \(l\)th layer. Therefore, given the neuron pair \(\alpha=(n_{l}^{i},n_{l+1}^{j})\) and two test inputs \(x_{1}\) and \(x_{2}\), there exit four implemented techniques in this sub-extension point of neuron pair level coverage. * Sign-Sign cover. The sign change of condition neuron \(n_{l}^{i}\) and signs of other neurons in the \(l\)th layer not changing affect the sign of decision neuron \(n_{l+1}^{j}\) in the next layer.