# Attention Transfer Reinforcement Learning for Test Case Prioritization in Continuous Integration
## 2 Background
### Problem Formulation  
The objective of TCP is to determine an optimal ordering of test cases that maximizes the early detection of failures during the regression testing process. In other words, within a limited execution time, the goal is to execute as many defect-detecting test cases as possible. The problem of sorting test cases is described below.  
_TCP_ aims at finding a test suite \(TS_{i}\), such that \(Q(TS_{i})\) is maximized and the total duration of execution of \(TS_{i}\) is less than \(M\). In other words, _TCP_ aims at finding \(TS_{i}\), such that \(\forall TS_{i}\in PT:Q(TS_{i})\geq Q(TS_{i}^{\prime})\cap\sum_{t_{k}\in TS_{i} ^{\prime}}t_{k}\), duration \(\leq M\cap\sum_{t_{k}\in TS_{i}}t_{k}\), duration \(\leq M\). We select the sequence that achieves the best testing performance among all sequences that satisfy the test execution time constraints.  
From this, it can be seen that the TCP problem is an optimization problem. TCP integrates the concepts of constrained time, selection, and performance evaluation, relying solely on historical test execution data from CI cycles. It does not necessitate additional information beyond what is provided by previous test results.  
Currently, numerous studies have applied reinforcement learning methods to the TCP task. As illustrated in Figure 1, the process begins with the random initialization of the reinforcement learning strategy. Subsequently, a set of test cases is input into the model, which calculates the priority of each test case based on its features. The test cases are then ordered according to their computed priorities. Following this, test scheduling and execution are carried out based on the prioritized order. The test results provide feedback to the reinforcement learning model, which dynamically adjusts its strategy based on this feedback to maximize the predefined reward function. When an action yields a positive reward, its selection tendency is reinforced. Agents can learn the best strategy through repeated experimentation.  
In TCP tasks, reinforcement learning directly optimizes the final goal, avoiding reliance on intermediate metrics like coverage or diversity. The reinforcement learning algorithm adjusts its prioritization strategy based on feedback from the ranking results of each cycle. The current cycle's prioritization strategy is formed through training on experimental data from all previous cycles. This training data include features and execution results of test cases from the recent past, where code changes are minimal between closely related cycles. Consequently, these data can be used to predict the execution outcomes of the current test cases. When the number of test case executions is too low, the historical execution data for each test case are insufficient. In such cases, the reinforcement learning model lacks effective reward signals, making it difficult to determine which test cases should be prioritized in the sequence to detect more defects. Methods like RETECS perform poorly in test case prioritization tasks with limited data. Additionally, if test cases have been executed over multiple cycles and accumulated a substantial amount of test results, their feature information can be leveraged for prioritization. However, methods such as PRM, which focus on the relative relationships between test cases, fail to fully utilize individual test case characteristics, leading to suboptimal performance. Consequently, existing approaches do not adequately evaluate the feature information of test cases and rely on fixed strategies for prioritization. This limits their applicability across different prioritization scenarios. Enhancing the generalization ability of the model is the key focus of this study.