# An Intelligent Method for C++ Test Case Synthesis Based on a Q-Learning Agent
## 2 Materials and Methods
### Test Case Synthesis Algorithm
Subfigure (**b**) shows the growth pattern across 54 training sessions for Hjson, which exhibits prolonged and more gradual Q-table expansion due to higher API complexity and state diversity. the median for RAND is only 23.2%, with significantly higher variance. Figure 5b presents results for the Boltzmann (Softmax) selection policy, which also maintains stable median coverage values. The QLB3 configuration reaches a median of 28.5%. Figure 5c shows branch coverage distributions for Hjson under the \(\varepsilon\)-Greedy policy. The QLEG configurations yield consistent performance in the range from 30.92% to 35.6%, surpassing the RAND baseline. The highest median, achieved by QLEG2, is 33%. In Figure 5d, the Boltzmann (Softmax) policy demonstrates high stability, with branch coverage consistently ranging between 32.42% and 36.6%. The highest median achieved is 33.7%. Figure 5: Branch coverage distributions obtained using different action selection policies compared to the RAND baseline. For each configuration, 150 synthesized test cases were evaluated, and branch coverage was computed. The boxplots display the median, interquartile range, and outliers (\({}^{\circ}\)). Horizontal brackets indicate pairwise differences assessed by the Mann-Whitney U-test. An asterisk (*) indicates that the difference is statistically significant (\(p<0.05\)). (**a**) BitmapPlusPlus \(\varepsilon\)-Greedy. (**b**) BitmapPlusPlus Boltzmann. (**c**) Hjson \(\varepsilon\)-Greedy. (**d**) Hjson Boltzmann. #### 3.5.2 Comparison of Action Selection Policies by Branch Coverage Growth Dynamics

Figure 6 illustrates the progression of branch coverage during the execution of TCs synthesized using QLEG2 and QLB2 policies in Algorithm 3, in comparison with the RAND (baseline), for the BitmapPlusPlus and Hjson libraries, respectively. In Figure 6, which corresponds to the BitmapPlusPlus library, the learned policies QLEG2 and QLB2 exhibit a rapid increase in coverage during the first 40 instructions. Both policies significantly outperform RAND across all instruction ranges. QLB2 achieves the highest coverage, reaching the original TC's maximum around 50 instructions, whereas RAND saturates around 24% and fails to reach the reference line. The coverage curves for QLEG2 and QLB2 are closely aligned, yet QLB2 consistently shows a slight advantage. Although QLB2 ultimately outperforms, RAND does not fall far behind and demonstrates moderate success. This behavior can be attributed to the lower API complexity and more uniform coverage opportunities in the BitmapPlusPlus library, which increase the chances for even random strategies to hit effective action sequences without exhaustive search. In Figure 6 for Hjson library, the differences become even more pronounced. QLEG2 and QLB2 reach over 30% branch coverage within the first 200-250 instructions, while the RAND policy requires more than 400 instructions to approach 29%. The original TC's maximum is surpassed by QLB2 and nearly matched by QLEG2. The learned policies demonstrate higher efficiency and stability, especially in the plateau region after 300 instructions, where RAND continues to grow slowly and remains below the optimal level. The weaker performance of RAND in this case is a consequence of higher API complexity, which leads to frequent invalid action sequences and early termination due to the maxRollback constraint. Since invalid actions cause full resets and exhaust the retry budget, RAND is unable to form long, coherent sequences that effectively explore deep control-flow paths. Figure 6: The dynamics of branch coverage growth as a function of the number of executed instructions for test cases synthesized using the QLEG2 and QLB2 policies, in comparison to the RAND baseline. The instructions count axis is aligned. The curves show the mean branch coverage across 150 synthesized test cases per configuration. The dashed purple line represents the maximum coverage achieved by the original test case. Subfigure (**a**) corresponds to the BitmapPlusPlus library and (**b**) to Hjson. #### 3.5.3.