# An Intelligent Method for C++ Test Case Synthesis Based on a Q-Learning Agent
## 2 Materials and Methods
### Test Case Synthesis Algorithm
loss and stabilizing rewards indicate that the agent successfully learns an effective test generation policy under constrained exploration budgets. ### Scalability Evaluation Based on Q-Table Resource Consumption

Table 6 presents a comparative analysis of training time and the resulting Q-table size for the tested libraries. Figure 4 illustrates the growth dynamics of the Q-table size during training sessions for the evaluated libraries. Each data point on the plot corresponds to the completion of \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Library** & **Number of Training** & **Number of Unique** & **Training Time** & **Final Q-Table Size** \\  & **Sessions (TC)** & **States** & **(Hours)** & **(Mb)** \\ \hline BitmapPlusPlus & 7 & \(\sim\)7000 & \(\sim\)1 & \(\sim\)3.1 \\ Hjson & 46 & \(\sim\)17,000 & \(\sim\)20 & \(\sim\)8.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Resource consumption metrics for the Q-learning agent across the two target libraries. The table compares the number of training sessions, total number of unique states stored in the Q-table, cumulative training time, and the final memory footprint of the learned Q-table. Figure 3: Evolution of the agent’s performance over 1000 training episodes for two representative test cases: BitmapPlusPlus and Hjson. Subfigures (**a,c**) show the total accumulated reward over episodes for BitmapPlusPlus and Hjson, respectively, reflecting execution quality, coverage gains, and redundancy penalties. Subfigures (**b,d**) depict the corresponding average loss per episode, indicating Q-value convergence and policy stability over time. a training session for a specific TC. In case of BitmapPlusPlus library training sessions (Figure 4a), the Q-table size stabilizes quickly after the fourth session, reaching approximately 7000 entries. In contrast, for the Hjson case (Figure 4b), a more irregular and prolonged growth is observed, eventually reaching around 17,000 entries. After a sharp increase in the Q-table size for a particular TC, the growth dynamics begin to stabilize. ### Q-Table Exploitation Phase

#### 3.5.1 Stability Analysis of Branch Coverage Under Different Action Selection Policies

To assess the impact of different action selection strategies on test case quality, we performed test case synthesis using the trained Q-table obtained after the reinforcement learning phase is complete. For each action selection policy configuration, 150 test cases were independently generated using Algorithm 3 and _maxRollback_ parameter equal 30. The synthesized test cases were executed and evaluated based on branch coverage, allowing us to analyze the stability and effectiveness of different policy settings. The parameters of the evaluated action selection policies are summarized in Table 7.

Figure 5a shows the distribution of branch coverage achieved by the \(\epsilon\)-Greedy policy series for TC synthesis in the BitmapPlusPlus library. As evident from the plot, all three policy configurations exhibit consistently higher median coverage values compared to the RAND baseline policy. QLEG1 achieves the highest median coverage of 28.2%, whereas \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Configuration** & **Policy Type** & **Parameters** & **Value** \\ \hline QLEG1 & & & 0.1 \\ QLEG2 & \(\epsilon Greedy\) & \(\epsilon\) & 0.15 \\ QLEG3 & & & 0.2 \\ QLB1 & & & 0.8 \\ QLB2 & (Softmax) & \(\tau\) & 1.5 \\ QLB3 & & & 3.0 \\ RAND (Baseline) &
\begin{tabular}{c} Uniform Random \\ Distribution \\ \end{tabular} & - & - \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Parameters of the action selection policies used in Algorithm 3 for test case synthesis. Figure 4: Growth dynamics of the number of unique states stored in the Q-table across training sessions. Subfigure (**a**) shows state accumulation over 7 test cases for BitmapPlusPlus, where saturation is reached quickly.