# ExtendAIST: Exploring the Space of AI-in-the-Loop System Testing
## 3 Extendaist
### Design  
According to the testing workflow defined in Section 2.2, we present the main steps executed in ExtendAIST, as shown in Algorithm 1, which outputs a recommendation of testing strategy for the input testing requirements of AI-in-the-loop system under test. ExtendAIST first investigates each testing requirement from the input testing requirements (Lines 2-20), such as the testing coverage requirement, testing property requirement, and application area requirement. According to the requirement, it inspects and selects the corresponding extension point available for the testing requirements (Lines 6-18). Then, the algorithm inspects and selects the appropriate sub-extension points for the testing requirements (Lines 11-17). All the existing techniques in the selected sub-extension points can be used (Line 16). It generates the testing strategy for the selected testing requirement (Line 19). Finally, it returns the recommended testing strategy for all of the input testing requirements (Line 21). The time complexity of Algorithm 1 is \(O(n^{2})\). ```
1:Testing requirement set \(R\) of AI-in-the-loop system under test
2:Testing strategy
3:\(Strategy\)\(\leftarrow\)\(\varnothing\) // Initialize the testing strategy
4:for\(i=1\to n\)do // There are \(n\) requirements in the set \(R\)
5:\(ExtenP\)\(\leftarrow\)\(\varnothing\) // Initialize the set of selected extension points for the \(i\)th requirement
6:\(SubExtenP\)\(\leftarrow\)\(\varnothing\) // Initialize the set of selected sub-extension points for the \(i\)th requirement
7: Investigate(\(R_{i}\)) // Investigate the \(i\)th testing requirement
8:for\(j=1\to 5\)do // Five extension points
9:\(f_{ep}\leftarrow\) Inspect(\(EP_{j}\)) // Inspect the \(j\)th extension point
10:if\(f_{ep}==\) true then
11:\(ExtenP\)\(\leftarrow\)\(ExtenP\cup EP_{j}\) // The \(j\)th extension point is appropriate for the \(i\)th testing requirement
12:endif
13:for\(k=1\to m_{j}\)do // \(m_{i}\) sub-extension points in the \(j\)th extension point
14:\(f_{sep}\leftarrow\) Inspect(\(SEP_{j}^{k}\)) // Inspect the \(k\)th sub-extension point
15:if\(f_{sep}==\) true then
16:\(SubExtenP\)\(\leftarrow\)\(SubExtenP\cup SEP_{j}^{k}\) // The \(k\)th sub-extension point in the \(j\)th extension point is appropriate for the \(i\)th testing requirement
17:endif
18:\(Tech\)\(\leftarrow\) Select(\(Tech_{j}^{k}\)) // Select the existing techniques in the \(k\)th sub-extension point of the \(j\)th extension point to test the AIS
19:endfor
20:\(Strategy[i]\)\(\leftarrow\)\((R_{i},ExtenP,SubExtenP,Tech)\) // Generate the strategy for the \(i\)th testing requirement
21:endfor
22:returnStrategy // Output the recommendation of testing strategy for the input requirement

### Illustration

In this section, we take the security and stability intelligent control unit in power system terminal as an illustration example to show the detailed implementation steps of testing AIS following the design space of ExtendAIST and the integration of other available techniques in ExtendAIST. The security and stability intelligent control unit in power system terminal is an AI-in-the-loop system which aims at predicting exceptions according to the learnt control strategies and taking relevant actions to avoid the potential failure for power transmission. As shown in Figure 4, the security and stability intelligent control unit in power system terminal obtains the transmission sections between the power supply center and electric load center, including the data of current, voltage, power, and frequency. This information is then transferred to the control strategies table to predict the decisions according to the learned strategies. For example, if the current value \(s_{1}\) is greater than the threshold \(P_{1}\), then decision \(D_{1}\) is decided according to the learned control strategies and transferred to switch off the actuator. Error decisions lead to illegal actions on actuators which can cause incalculable loss.