# Smart Grid Testing Management Platform (SGTMP) +
## 3 Implementing SGTMP
### Test Execution and Result Processing
#### 3.4.2 The Test Run Scheduler  
Once a test run is created, its status is set as _CREATED_. Given that a single test might create several test runs, depending on the amount of configuration value permutations, it is not sufficient to start each test run as soon as it is created. Besides, each test/test run has several requirements. From the scheduling perspective, the most important are the simulator requirements, which describe the simulators that must be made available for the duration of the test run. Given that it is desirable to have as many running tests as possible at the same time, test runs must be selected so that they will not attempt to use each other's resources, which could result in a deadlock or unpredictable simulation results. Figure 4 shows the life cycle of an individual test run. The scheduler is responsible for transitioning test runs into the _STARTED_ stage, ideally with the best efficiency as specified by the business or SG laboratory metrics [48].  
Possible selection strategies include, but are not limited to, the following:  
* _Maximize test runs_: This strategy tries to maximize the number of test runs that can be executed in parallel by trying to find the best combination of simulator utilization and the number of test runs. If a solution is found, the result will be the largest possible amount of test runs executed at once, at the cost of increased selection time; however, test case generation plays a role in the extensiveness of the test results [49].
* _Maximize simulator usage_: This strategy is a complement of the _\"Maximize test runs\"_ strategy. Here the amount of simulators (resources) is maximized per each test. The reason might be that tests with more simulators are more important (integration tests as opposed to single simulator tests).
* _OS scheduling-based_: This strategy uses well-known OS scheduling algorithms for selecting test runs to execute. The simulation requirements and currently used simulators can be used to ensure fairness and resource usage optimization. All of the queued test runs for execution are considered.
* _Heuristics-based_: This strategy uses different metrics from the ones described above for prioritizing test runs, such as risk or business importance. This aspect can be the _Feature_ priority, where the highest priority features would be selected first, using a secondary selection strategy for selecting a test that currently has all of the available resources. Other metrics, such as the maximization of diagnostic information available per test [50], can be considered.  
Figure 4: State diagram of a test run execution.  
For all scheduling algorithms, one important consideration is that the tests involve simulations, and as such tests cannot be paused once started, only aborted and transitioned to the finished state. Given the time complexity of the _\"Maximize test runs\"_ and _\"Maximize simulator usage\"_ strategies, dynamically selected strategy combinations could be used. For more complex scenarios, another process could attempt to pre-compute an optimal solution, taking into account the expected run-time of currently running tests and the timeliness of another test run to be executed. The process of test launching can be repeated until there are no more available test runs to perform.