## 4 Performance Evaluation: Results
### Synthetic Data-Hard Configuration  
Figures 1a-3a present results when a Bernoulli distribution is adopted for the reward PDF in the _Hard_ configuration. In this case, the POKER algorithm is the algorithm that presents best overall behavior and leads to the lowest regret at the time horizon for \(T_{U}/T_{M}=5\) and \(T_{U}/T_{M}=10\), while UCB1 provides the best performance at the time horizon when \(T_{U}=T_{M}\). Moving to the new algorithms, their performance compared to the other ones improves as the \(T_{U}/T_{M}\) ratio increases; this is true in particular for muUCB1, that moves from an extremely bad performance when \(T_{U}/T_{M}=1\) to a regret comparable to the POKER algorithm when \(T_{U}/T_{M}=10\), furthermore showing a trend as a function of time suggesting that muUCB1 might be the best option on a longer time horizon. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Acu-Edu** & **Acadiau-Ca** & **Adrian-Edu** & **Agnesscott-Edu** & **Aims-Edu** \\ \hline
396 & 381 & 488 & 506 & 333 \\
271 & 261 & 488 & 504 & 276 \\
271 & 141 & 325 & 545 & 266 \\
268 & 136 & 324 & 1946 & 331 \\
273 & 136 & 321 & 549 & 290 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Excerpt from the data used in [17], downloaded from [24]; data express latencies measured in milliseconds. The trend is confirmed as well when the reward PDF has a truncated Gaussian distribution (Figures 4a-6a). With the exception of the case \(T_{U}=T_{M}\), where UCB1 is clearly the best algorithm at the time horizon, the MLI and muUCB1 perform very well. MLI is the best option when \(T_{U}/T_{M}=5\) and \(T_{U}/T_{M}=10\), but muUCB1 improves its performance as the \(T_{U}/T_{M}\) ratio increases, and achieves similar performance to MLI for \(T_{U}/T_{M}=10\). The results obtained with an exponential distribution for the reward PDF, presented in Figures 7a-9a, highlight again the performance improvement for muUCB1 as the \(T_{U}/T_{M}\) ratio increases, leading the new algorithm to be clearly the best option when \(T_{U}/T_{M}=10\). Notably, the adoption of the exponential distribution has a strong impact on the performance of the POKER algorithm, which presents a linear growth of regret with time; a similar behavior is observed for the MLI algorithm. Figure 1: Performance in terms of regret of the six considered algorithms, with a Bernoulli distribution for the reward Probability Density Function (PDF) and \(T_{U}/T_{M}=1\). (**a**) _Hard_ configuration; (**b**) _Easy_ configuration. Figure 2: Performance in terms of regret of the six considered algorithms, with a Bernoulli distribution for the reward PDF and \(T_{U}/T_{M}=5\). (**a**) _Hard_ configuration; (**b**) _Easy_ configuration. Figure 4: Performance in terms of regret of the six considered algorithms, with a truncated Gaussian distribution for the reward PDF and \(T_{U}/T_{M}=1\). (**a**) _Hard_ configuration; (**b**) _Easy_ configuration. Figure 5: Performance in terms of regret of the six considered algorithms, with a truncated Gaussian distribution for the reward PDF and \(T_{U}/T_{M}=5\). (**a**) _Hard_ configuration; (**b**) _Easy_ configuration. Figure 3: Performance in terms of regret of the six considered algorithms, with a Bernoulli distribution for the reward PDF and \(T_{U}/T_{M}=10\). (**a**) _Hard_ configuration; (**b**) _Easy_ configuration. As a general comment on the \(\varepsilon\)-greedy and \(\varepsilon\)-decreasing algorithms, not discussed so far, they present a linear growth of regret with time for all considered distributions, and lead in general to the worst performance. The algorithms, that despite their simplicity have shown good results with a classical MAB model [16; 17], are thus not suitable for adoption in the new model with the considered PDFs and mean value configuration, in particular as the \(T_{U}/T_{M}\) ratio grows. Figure 6: Performance in terms of regret of the six considered algorithms, with a truncated Gaussian distribution for the reward PDF and \(T_{U}/T_{M}=10\). (**a**) _Hard_ configuration; (**b**) _Easy_ configuration.