## 2 Materials and Methods  
### Research Methodology  
To establish a comprehensive RL algorithm framework and explore the available information from the environment, several configurations were tested, as explained in Section 2.7. This approach allowed for a ceteris paribus analysis, where the remaining variables are held constant to isolate the impact of different agent configurations or environmental factors on the reward. By keeping key factors, such as the training and evaluation data, constant across all experiments, we ensured a fair comparison of the agents. Additionally, the performance of each configuration was compared against a market benchmark and a simple equally weighted portfolio, as detailed in Section 2.7.6, to measure the relative advantage of using RL for the portfolio optimization problem. This structure ensures that the observed differences in outcomes are attributable solely to the variations in RL algorithms, feature extractors, or environmental inputs. As for the impact of slippage and rebalancing frequency, two experimental setups were used: one with continuous rebalancing, where the portfolio is adjusted every day, and another with rebalancing every 10 periods (approximately bi-weekly). These setups were designed to evaluate the trade-off between more frequent rebalancing, which can capture market opportunities more effectively, and the higher transaction costs and slippage incurred from increased trading activity. More frequent rebalancing not only raises transaction costs but also exposes the portfolio to slippage, particularly from bid-ask spreads and the potential difference between market close prices and the next day's open. The choice of rebalancing every 10 periods was made as a compromise between daily and more widely used industry practices, such as monthly rebalancing, which tends to take a longer-term view. Additionally, this frequency ensures sufficient overlap between consecutive observations in the environment, which provides historical data with lookback periods of 16 and 28 periods, making it more likely that both rebalancing configurations will be similarly impacted by any potential correlations in market behavior. The goal is to analyze whether the gains in returns from more frequent rebalancing can outweigh the additional trading costs, providing insight into the optimal rebalancing frequency for maximizing performance in real-world scenarios [34]. With respect to the value of technical indicators versus raw prices, two different signal configurations were tested, as detailed in Section 2.4. By maintaining all other parameters constant between experiments, we aimed to isolate and assess the additional value provided by technical indicators. On the one hand, as neural networks are powerful universal function approximators [35] and feature learners [36; 37], it could be argued that the raw price data alone should be sufficient for the agent to learn meaningful patterns, and the technical indicators could be learned as latent-state representations. However, given the limited amount of data available, technical indicators may offer a concise and informative summary of the market environment, potentially enhancing learning efficiency and convergence. This is particularly relevant in light of the EMH [7; 33], which posits that all available information is reflected in the price. Some studies in the literature suggest that additional market indicators may contradict the EMH, providing opportunities for better decision-making by RL agents. Therefore, the experiment sought to explore the balance between raw data and engineered features in portfolio optimization. With respect to capturing patterns and anticipating behavior in the time series of the indicators for each financial instrument, two different feature extractors--MLP and CNN--were employed.