# Test Suite Prioritization Based on Optimization Approach Using Reinforcement Learning
## 3 Materials and Methods  
This section presents the proposed methodology for TSP based on a four-fold methodology. Figure 2 shows the block diagram of the proposed methodology. The proposed four-fold methodology of test suite prioritization is based on reinforcement learning. First, the testers' and users' log datasets are prepared using the proposed interaction recording systems for the android application. Second, the proposed reinforcement learning model is used to predict the highest future reward sequence list from the data collected in the first step. Third, the proposed prioritization algorithm signifies the prioritized test suite. Figure 2: Proposed methodology for test suite prioritization based on RL based optimization. Lastly, the fault seeding approach is used to validate the results from software engineering experts. In the proposed interaction recording systems, the testers' and users' interactions are recorded when the user or tester interacts with the system. Then, the sequence of activities or states is extracted from those users' and testers' logs. Next, the number of states from the testers' log is computed for the environment construction of the proposed RL model. Next, the agent is trained on the matrix representing the testing environment using the Q-learning algorithm. During learning, the agent updates and maintains its Q-table. The final updated Q-table is obtained when the agent learns the complete environment. In the next step, the algorithm identifies the sequence of states with the highest reward values from starting state to the final state. The prioritization algorithm is then applied based on the sequence with the highest reward values. For prioritization of the user and tester interactions, the system records the interaction when users and testers use the application. User interactions are recorded when the user interacts with the GUI of the application. An interaction list is generated when the user logs in to the application or performs any other application activity, such as 'Sign up' and 'Forget password'. The list maintains and adds all actions performed by the user in the GUI of the application. At each instance or button click, the activity name and the button ID are stored in the list. The addition to the list ends when the user logs out from the system, or the application is terminated forcefully. The final list is added to the database for persistence. Tester Log is also recorded in the same way when the tester interacts with the system to implement the test cases for testing the system. A tester list is generated by clicking buttons on the application interface while testing the application. By executing all test cases, the logs of every test case according to its type are collected. The user and tester logs are then processed to get the sequence of states visited by users or testers. Next, all possible states are extracted from the tester logs for the construction of the matrix representing the testing environment and computing frequency count and complexity from the list of user logs. Finally, the mean, median, and complexity of the states are calculated from the list, excluding the initial and final states for the reward matrix calculation. Faults are seeded or injected through software. In the first step, frequent and rare paths are extracted by sequential pattern mining [13; 27] method. Sequence mining is a handy data mining technique for understanding the user and tester behavior with the system [13]. After applying sequential pattern mining, the system gets frequent paths and rare paths from the test suite. In the second step, the program is given to the software in which we want to inject the faults for evaluation [30]. The software scans the source code and generates the control flow graph according to the program [28]. The software gets the program's seed point injection locations and workflow in the third step.