# Attention Transfer Reinforcement Learning for Test Case Prioritization in Continuous Integration
## 4 ATRL-TCP: Attention Transfer Reinforcement Learning for TCP
 \tag{1}\]  
This reward function returns the test case's verdict as each test case's individual reward. After obtaining a \(TS_{i}\) through prioritization, if a test case \(t\) within it fails (\(t.verdict_{i}=0\)), a positive reward is accumulated. Conversely, if the test case executes correctly, no reward is given to the model. Executing test cases that can detect errors helps identify software issues. This behavior should be reinforced. No reward is given if the test case passes. Because the core goal of TCP is to prioritize the execution of test cases that can detect defects. Still, the order of test cases is not explicitly included in the reward. It is implicitly included by encouraging the agent to focus on failing test cases and prioritizing them higher. For test case subsets, the reward function should be designed based on global features, directly using the final objective, NAPFD, as the reward function:  
\[\text{NAPFD}=P-\frac{\sum_{i=1}^{m}TF_{i}}{m\times n}+\frac{P}{2n} \tag{2}\]  
In Equation (2), there are \(n\) test cases and \(m\) faults. \(P\) represents the number of faults detected by the prioritized test suite divided by the total number of faults detected in the full test suite. This ratio measures the fault detection coverage of the prioritized test suite. A higher \(P\) value indicates that the prioritized test suite detects more faults. If a fault \(i\) is never detected, we set \(TF_{i}=0\). For a given test case ordering, if fault-revealing test cases are placed earlier while passing test cases are placed later, we consider this sequence to be more effective in detecting faults with fewer executed test cases. NAPFD serves as a metric to evaluate the fault detection performance of such sequences. In this study, NAPFD is used as the reward function. The more faults the prioritized and scheduled test case set can reveal, and the higher the fault-revealing test cases are ranked, the larger the NAPFD value. A higher NAPFD indicates a more successful test case prioritization, providing more positive feedback to the model. _Design of the Attention Transfer Mechanism_  
To achieve attention transfer, we design the second part of the test case features. A threshold \(\alpha\) is set for History length. When a test case's History length is less than \(0.5\alpha\), its historical information is deemed insufficient. Similarly, a threshold \(\beta\) is set for Error rate. When a test case's Error rate is less than \(0.5\beta\), its fault-detection capability is considered inadequate. We designed a vector called Guide, which is of the same length as the State, State = \(\left[0.5,0.5,0.5,\frac{\text{Length}}{a},\frac{\text{Error rate}}{\beta}\right]\). Through this design pattern, when the historical information of a single test case is relatively rich and its fault-detection capability is strong, the agent focuses more on the individual characteristics of that test case. For example, when the History Length of a test case exceeds half of the threshold, its ratio to the threshold is greater than 0.5, causing the overall \"focus\" to shift backward. Conversely, when the historical information is limited and the ratio of History Length to the threshold is less than 0.5, the overall \"focus\" shifts forward. The agent will pay more attention to the overall characteristics of the test case set. The Guide vector is used to direct the generation of weights in the Attention mechanism, ensuring that the attention weights of the resulting State are adjusted in a directional and purposeful manner. We refer to this approach as Guide Attention, with the formula for computing the attention weights given as follows:  
\[\text{attn}=\text{softmax}(\text{noise}+\text{guide}) \tag{3}\]  
The output of Equation (3) is the attention weight, and noise is computed using Equation (4).