## 4 Performance Evaluation: Results
### Synthetic Data-Easy Configuration  
Results obtained in the case of the _Easy_ configuration, when one arm is significantly more rewarding than the other ones, are rather different from those observed in Section 4.1 for the _Hard_ configuration. Figures 1b-3b, presenting results for the Bernoulli distribution, show that UCB1, muUCB1 and MLI algorithms perform better than in the _Hard_ configuration, while the opposite is true for a second group of algorithms including POKER, \(\varepsilon\)-decreasing and \(\varepsilon\)-greedy, that in most cases lead to a worse performance, in particular as the \(T_{U}/T_{M}\) ratio increases. These results can be explained by observing that the latter group includes the traditional algorithms that are more aggressive in using the estimated best arm: as a result, they are able to achieve almost the maximum gain when they correctly select the best arm, but incur in a high penalty when they select a sub-optimal arm. Among algorithms in the first group, results show that the gap between muUCB1 and UCB1 is small for \(T_{U}/T_{M}=1\), and the muUCB1 algorithm performs significantly better than UCB1 when the ratio grows. The performance of the two proposed algorithms is particularly good for \(T_{U}/T_{M}=10\), with MLI providing the best overall performance. Figure 8: Performance in terms of regret of the six considered algorithms, with a exponential distribution for the reward PDF and \(T_{U}/T_{M}=5\). **(a)**_Hard_ configuration; **(b)**_Easy_ configuration. Figure 9: Performance in terms of regret of the six considered algorithms, with an exponential distribution for the reward PDF and \(T_{U}/T_{M}=10\). **(a)**_Hard_ configuration; **(b)**_Easy_ configuration. The different performance between the two groups of algorithms can be better understood by observing Figure 10, comparing the regret measured at the time horizon for the six algorithms over the 500 runs for the Bernoulli distribution with \(T_{U}/T_{M}=5\) in the _Hard_ vs. _Easy_ configurations (Figure 10a vs. Figure 10b, respectively). Results highlight that POKER and \(\varepsilon\)-decreasing algorithms are characterized, both in the _Hard_ and _Easy_ configurations, by several spikes in the regret at the time horizon, corresponding to runs where a sub-optimal arm was selected, leading to linear regret over time. This behavior is observed for MLI as well in the _Hard_ configuration, but disappears in the _Easy_ one, where the algorithm is highly consistent in eventually selecting the best arm in each run. The results for the truncated Gaussian distribution, presented in Figures 4b-6b, show a similar behavior: the performance of UCB1 decreases significantly as \(T_{U}/T_{M}\) increases, while muUCB1 and MLI show the opposite trend. The POKER, \(\varepsilon\)-decreasing and \(\varepsilon\)-greedy algorithms confirm a high variability in the experienced regret when \(T_{U}/T_{M}>1\); even in the case \(T_{U}/T_{M}=1\), the only algorithm in this group that achieves good performance is the \(\varepsilon\)-decreasing. Finally, results for the exponential distribution, presented in Figures 7b-9b, also confirm the behavior observed for the other distributions: MLI and muUCB1 algorithms lead to the best performance, and UCB1 is the only traditional algorithm leading to a logarithmic regret when \(T_{U}/T_{M}>1\), while \(\varepsilon\)-greedy, \(\varepsilon\)-decreasing and POKER algorithms show a steep linear regret over time. Overall, results obtained in the _Easy_ configuration show that when an arm provides a significant advantage over the others, traditional algorithms that tend to favor exploitation over exploration, and in particular POKER and \(\varepsilon\)-decreasing, typically perform either extremely well o extremely poorly, depending on the selection of the arm in the early steps.