# QRLIT: Quantum Reinforcement Learning for Database Index Tuning
## 5 Performance Evaluation
### Overall Performance  
In this section, experiments were conducted to study the overall performance of the two algorithms when the database is fixed at 10 MB. This study is based on the following metrics: number of queries processed per hour, episode execution time, temporal difference error, and number of Grover iterations. The first metric defines the quality of the algorithms in terms of their ability to identify a policy that maximizes the cumulative reward (queries per hour) over time. The episode execution time metric measures the velocity of the algorithms in executing an episode. The temporal difference error (TD Error) (Equation (1)) metric demonstrates the algorithm's convergence to an optimal policy; in other words, the closer the values are to 0, the better the policy is. Finally, the Grover iterations metric measures the relation between exploration and exploitation; the lower the value, the higher the rate of exploration relative to exploitation. This metric allows for the analysis of the agent's exploration capacity, which is directly correlated with the number of iterations. The results obtained for each metric in each of 50 episodes for the two algorithms are shown in Figures 3-6. The average results of each metric over 50 episodes of the two algorithms are summarized in Table 2. The analysis of Figure 3 and Table 2 reveals that on average, the hybrid algorithm exhibits a higher number of queries processed per hour by 0.61% compared to its classical counterpart. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Test Name** & **Database Size** & \(\boldsymbol{\alpha}\) & \(\boldsymbol{\gamma}\) & \(\mathbf{k}\) & **Episodes** & **Steps** & **Total Time Steps** \\ \hline Overall Performance & 10 MB & 0.001 & 0.8 & 0.00017 & 50 & 100 & 5000 \\ Impact of Database Size & 10 MB, 20 MB, 30 MB, 40 MB, 70 MB, 100 MB & 0.001 & 0.8 & 0.00017 & 25 & 100 & 2500 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Configuration parameters for the tests. Figure 4: Average temporal difference error in each episode. Figure 5: Grover iterations in each episode with hybrid algorithm. Figure 3: Number of queries processed per hour in each episode. Moreover, from the analysis of Figure 4, the hybrid algorithm has a much faster convergence to a low temporal difference error showing a more stable learning, displaying a temporal difference error trajectory closer to 0 (Table 2). To find a balance between exploring and exploiting, the classical algorithm implements a strategy known as Epsilon-greedy. This strategy uses an exploration rate epsilon \(\varepsilon=0.9\), which decreases with an exploration discount factor of 0.1 at the end of each episode. Therefore, with this algorithm, the agent explores with a probability of \(\varepsilon\) or follows the learned policy (exploits) with a probability of \(1-\varepsilon\). In the case of the quantum-classical algorithm, as the agent learns and adjusts its policy, the number of Grover iterations also increases, consequently reducing the exploration probability (Equation (20)) (Figure 5). The quantum-classical algorithm provides a better index recommendation, resulting in a higher number of queries processed per hour than the classical algorithm because as the agent of the quantum-classical algorithm refines its policy through learning, the exploration rate decreases. This leads to a decrease in unnecessary explorations and allows for more effective learning. However, the quantum-classical algorithm takes on average 21.67% more time to complete an episode than its classical counterpart (Figure 6). This discrepancy is related to the additional computational overhead to create and execute the quantum circuit at each time step. Figure 7 shows the time required to build and execute [PERSON]'s algorithm.