## 4 Discussion  
### RL Agents  
The primary goal of the RL agents was to maximize cumulative log-returns, aiming for a consistent increase in the portfolio's value over time. While risk management is fundamental in traditional portfolio optimization, in this study, risk was implicitly addressed only when the agent incurred losses, reflected as negative log-returns. The allocation weights assigned to different instruments, such as \({}^{\Lambda}\)IXIC, \({}^{\Lambda}\)GSPC, and \({}^{\Lambda}\)N225, illustrate how each agent family approached asset allocation. SAC models, however, faced convergence issues and are excluded from further discussion due to unreliable results. Distinct patterns emerged among the different RL agent families. DDPG agents, which operate in continuous action spaces, exhibited greater portfolio diversification, distributing allocations across multiple instruments. For example, the _DDPG-CNN-16_ configuration allocated equal weights (0.25) to \({}^{\Lambda}\)RUT, \({}^{\Lambda}\)IXIC, and \({}^{\Lambda}\)FTSE while maintaining a CASH allocation of 0.25, demonstrating a balanced approach. In contrast, DQN and PPO agents, operating in discrete action spaces, tended to focus on identifying and fully investing in a single, high-performing asset. For instance, _DQN-MLP-28_ allocated 100% of its portfolio to \({}^{\Lambda}\)GDAXI, reflecting a concentrated strategy aimed at maximizing returns, with less consideration for risk diversification. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline \({}^{\Lambda}\)**RUT** & \({}^{\Lambda}\)**IXIC** & \({}^{\Lambda}\)**GDAXI** & \({}^{\Lambda}\)**FTSE** & \({}^{\Lambda}\)**N225** & \({}^{\Lambda}\)**GSPC** & **CASH** & **Agent** & **Feat.Ext.** & **Lookback** &
\begin{tabular}{c} **Reb.** \\ **Freq.** \\ \end{tabular} & **Indicators** \\ \hline
0.10 & 0.27 & 0.05 & 0.26 & 0.07 & 0.04 & 0.21 & PPO & CNN & 28 & 1 & No \\
0.04 & 0.24 & 0.24 & 0.03 & 0.00 & 0.24 & 0.20 & DDPG & CNN & 16 & 10 & Yes \\
0.13 & 0.00 & 0.22 & 0.21 & 0.22 & 0.22 & 0.00 & DDPG & CNN & 16 & 1 & Yes \\
0.00 & 0.20 & 0.20 & 0.00 & 0.20 & 0.20 & 0.20 & DDPG & CNN & 28 & 10 & Yes \\
0.02 & 0.37 & 0.23 & 0.06 & 0.32 & 0.00 & 0.00 & DDPG & CNN & 28 & 1 & Yes \\
0.00 & 0.00 & 0.00 & 0.00 & 0.00 & 1.00 & 0.00 & DQN & CNN & 16 & 10 & Yes \\
0.00 & 0.00 & 0.01 & 0.00 & 0.00 & 0.99 & DQN & CNN & 16 & 1 & Yes \\
0.98 & 0.00 & 0.00 & 0.00 & 0.01 & 0.00 & 0.00 & DQN & CNN & 28 & 10 & Yes \\
0.05 & 0.00 & 0.95 & 0.00 & 0.00 & 0.00 & 0.00 & DQN & CNN & 28 & 1 & Yes \\
0.20 & 0.04 & 0.10 & 0.22 & 0.29 & 0.03 & 0.12 & PPO & CNN & 16 & 10 & Yes \\
0.03 & 0.49 & 0.17 & 0.05 & 0.11 & 0.00 & 0.14 & PPO & CNN & 16 & 1 & Yes \\
0.11 & 0.19 & 0.08 & 0.34 & 0.25 & 0.02 & 0.01 & PPO & CNN & 28 & 10 & Yes \\
0.07 & 0.31 & 0.06 & 0.01 & 0.29 & 0.18 & 0.08 & PPO & CNN & 28 & 1 & Yes \\ \hline \hline \end{tabular} Instrument allocation weights.  
\end{table} 
Table 7: _Cont._Quantitatively, DDPG agents averaged an annualized return of approximately 15.4%, with an annual volatility of 31%, suggesting a well-balanced approach between risk and reward. DQN agents achieved a slightly lower average annualized return of 13.2%, with a higher volatility of 34%, reflecting a more aggressive strategy with greater exposure to risk. PPO agents, on the other hand, delivered more moderate returns of around 9%, paired with volatility levels of approximately 25%, indicating a more conservative approach with lower risk but also reduced return potential. In comparison to the baseline models, RL agents consistently outperformed both the \({}^{\wedge}\)GSPC and equally weighted portfolio (EWP) benchmarks in terms of cumulative log-returns as reflected in Figure 5. The \({}^{\wedge}\)GSPC achieved an annualized return of 14.18%, with a volatility of 18.7%, while the EWP benchmark yielded a lower annualized return of 8.73%, paired with 13.5% volatility. Several RL configurations, particularly DDPG and DQN agents, exceeded these returns--DDPG with CNN reached over 18%--but with the trade-off of higher volatility.