## 5 Conclusions  
This study has demonstrated the potential of deep reinforcement learning (RL) agents in portfolio optimization by evaluating various configurations of agents, feature extractors, and rebalancing frequencies. The findings indicate that DQN and DDPG agents generally outperform traditional baselines, such as the market portfolio (S&P 500) and the equally weighted portfolio, in terms of both annualized returns and risk-adjusted performance, with the best portfolios providing an additional yearly return of 10% compared to the market portfolio and 17% compared to the equally weighted portfolio. Notably, CNN-based feature extractors, particularly with longer lookback periods, were more effective at identifying market patterns and adapting to volatile conditions, yielding superior Sharpe and Sortino ratios compared to MLP-based agents.  
The analysis of rebalancing strategies revealed that while continuous rebalancing can capture short-term opportunities, it often incurs higher transaction costs and slippage. As a result, periodic rebalancing (e.g., every 10 periods) emerged as a more efficient strategy for balancing risk and managing costs. Furthermore, the inclusion of technical indicators provided marginal improvements in certain configurations, suggesting that while they may enhance the agent's understanding of market dynamics, they do not consistently outperform raw OHLC data.  
These findings underscore the dynamic adaptability of RL agents in real-time market conditions, moving beyond the static assumptions of traditional portfolio optimization models like the efficient frontier. RL agents offer robust portfolio strategies that adjust dynamically to shifting market environments. Their strong performance, particularly in high-volatility settings, highlights their potential as valuable tools for portfolio management. Nonetheless, challenges remain, including the optimization of hyperparameters, the reduction in transaction costs, and the exploration of advanced architectures such as Transformers, offering promising directions for future research.  
Conceptualization, F.E.-F., A.G.-S. and J.O.-M.; methodology, F.E.-F., A.G.-S. and J.O.-M.; software, F.E.-F.; validation, A.G.-S. and J.O.-M.; formal analysis, F.E.-F., [PERSON]-S. and J.O.-M.; investigation, F.E.-F.; resources, F.E.-F. and J.O.-M.; data curation, F.E.-F.; writing--original draft preparation, F.E.-F.; writing--review and editing, F.E.-F., [PERSON]-S. and [PERSON]-M.; visualization, F.E.-F.; supervision, [PERSON]-S. and [PERSON]-M.; project administration, [PERSON]-[PERSON] and [PERSON]-M. All authors have read and agreed to the published version of the manuscript.  
The authors want to thank the Spanish Agencia Estatal de Investigacion because of this research has been partially supported by the Ministerio de Ciencia e Innovacion of Spain (Grant Ref. PID2022-137748 OB-C31 funded by MCIN/AEI/10.13039/501100011033) and \"ERDF A way of making Europe\".  
Data Availability StatementData are available using the _vfinance_ library for the OHLC prices and _TALIB_ to compute the technical indicators. All technical indicator parameters have been kept as the library defaults.  
The authors declare no conflicts of interest.  
The following abbreviations are used in this manuscript:  
\begin{tabular}{l l} RL & Reinforcement learning \\ MLP & Multi-layer perceptron \\ CNN & Convolutional Neural Network \\ EMH & Efficient Market Hypothesis \\ DQN & Deep Q-Network \\ DDPG & Deep Deterministic Policy Gradient \\ PPO & Proximal Policy Optimization \\ SAC & Soft Actor-Critic \\ RSI & Relative Strength Index \\ NATR & Normalized Average True Range \\ AROONOSC & Aroon Oscillator \\ OHLC & Open-High-Low-Close \\ CCI & Commodity Channel Index \\ CMO & Chande Momentum Oscillator \\ MFI & Money Flow Index \\ QP & Quadratic Programming \\ GA & Genetic Algorithm \\ MDP & Markov Decision Process \\ \end{tabular}