## 3 Results  
### Agents' Performance  
The results of the different RL agent configurations are summarized in terms of key financial metrics. These metrics were calculated for each configuration of agent type, feature extractor, lookback period, and rebalancing frequency and are shown in Table 5.  
For example, the DQN agent using an MLP feature extractor and a 16-period lookback window achieved an annualized return of 18.25% with a Sharpe ratio of 0.66. A similar configuration with a 28-period lookback showed a higher annualized return of 25.76% and a Sharpe ratio of 0.80. These results suggest that longer lookback periods can yield higher returns but introduce additional risk, particularly during periods of market volatility. On the other hand, the PPO agent with an MLP feature extractor exhibited under-whelming performance, with an annualized return of \(-1.27\%\) and a low Sharpe ratio, indicating inferior results when compared to the baselines, as shown in Table 6.  
Figure 4: Cumulative returns for the market (S&P 500 proxy) and the EWP for the period 2016–April 2024. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline
**Ann.** & **Ann.** & **Ann.** & **Max** & **Max** & **Agent** & **Feat.Ext.** & **Lookback** & **Reb.** & **Indicators** \\
**Return** & **Volatility** & **Calmar** & **Sharpe** & **Sortino** & **Drawdown** & **Agent** & **Feat.Ext.** & **Lookback** & **Reb.** & **Indicators** \\ \hline
0.26 & 0.38 & 0.56 & 0.8 & 1.12 & \(-\)0.47 & DQN & CNN & 16 & 1 & No \\
0.26 & 0.46 & 0.45 & 0.74 & 1.02 & \(-\)0.57 & DQN & CNN & 28 & 1 & Yes \\
0.26 & 0.38 & 0.55 & 0.8 & 1.12 & \(-\)0.47 & DQN & MLP & 28 & 1 & No \\
0.19 & 0.37 & 0.34 & 0.66 & 0.91 & \(-\)0.57 & DDPG & MLP & 16 & 10 & No \\
0.19 & 0.31 & 0.39 & 0.72 & 1 & \(-\)0.49 & DDPG & MLP & 28 & 1 & Yes \\
0.19 & 0.32 & 0.37 & 0.7 & 0.96 & \(-\)0.5 & DDPG & CNN & 28 & 10 & Yes \\
0.19 & 0.25 & 0.55 & 0.8 & 1.18 & \(-\)0.33 & PPO & MLP & 16 & 1 & Yes \\
0.18 & 0.35 & 0.31 & 0.66 & 0.91 & \(-\)0.59 & DQN & MLP & 16 & 1 & No \\
0.18 & 0.34 & 0.5 & 0.66 & 0.95 & \(-\)0.36 & DQN & MLP & 16 & 1 & Yes \\
0.18 & 0.29 & 0.44 & 0.7 & 0.97 & \(-\)0.39 & DDPG & MLP & 16 & 1 & Yes \\
0.14 & 0.19 & 0.57 & 0.8 & 1.12 & \(-\)0.25 & \({}^{\text{A}}\)GSPC & - & - & **1** & - \\
0.09 & 0.14 & 0.35 & 0.69 & 0.94 & \(-\)0.25 & \({}^{\text{EWP}}\) & - & - & **1** & - \\ \hline \hline \end{tabular} Top-10 experimental results by annualized return compared to the benchmark portfolios (in grey).  
\end{table} 
Table 5: Performance metrics for top-10 RL agent configurations. When comparing RL agents to the market benchmark (S&P 500) and the equally weighted portfolio (EWP), we observed the following:  
* The S&P 500 baseline had a Sharpe ratio of 0.80 and an annualized return of 14.18%, which outperformed several agent configurations, particularly those with shorter lookback periods and high rebalancing frequency. * The equally weighted portfolio baseline yielded an annualized return of 8.73% and a Sharpe ratio of 0.68, serving as a middle-ground benchmark that several RL agents surpassed. Additionally, it is worth mentioning that the SAC experiments experienced convergence issues during training. Despite several tuning attempts, the models did not converge consistently, and as a result, the SAC agent configurations were excluded from the final analysis due to their unreliable performance. In summary, the DQN consistently outperforms the other agent families in terms of annualized return and risk-adjusted metrics such as the Sharpe, Sortino, and Calmar ratios, especially when using longer lookback windows. On the other hand, DDPG excels in managing risk, demonstrating the lowest volatility, downside risk, and max drawdown among the agent families, making it more robust in risk-averse scenarios. Regarding feature extractors, the CNN tends to perform better in terms of maximizing returns and achieving higher risk-adjusted ratios (Sharpe, Sortino, Calmar), whereas the MLP shows superior performance in risk mitigation, as evidenced by lower volatility and max drawdown.