## 2 Materials and Methods
### Experimental Design
#### 2.7.5 Financial Metrics  
In portfolio optimization, relying solely on final reward values or loss metrics, such as actor and critic losses, may not provide a comprehensive evaluation of an RL agent's performance in real-world financial contexts. These internal metrics are primarily focused on the agent's learning process but do not adequately reflect the key financial outcomes that investors prioritize, such as risk-adjusted returns or resilience to large drawdowns. To provide a robust and meaningful comparison of RL agents, it is essential to assess performance using established financial metrics that account for both return and risk. These metrics offer a clearer perspective on how each strategy would perform in real-world market conditions.  
For this study, we employ the following financial metrics: annualized return, annualized volatility, [PERSON] ratio [45], Sortino ratio [46], and Calmar ratio [47]. Additionally, we include maximum drawdown to measure the worst-case scenario in terms of wealth loss. Together, these metrics provide a comprehensive assessment of portfolio performance, considering different dimensions of risk and return.  
1. Annualized Return: This metric measures the yearly compounded return of the portfolio, making it easier to compare results across different time horizons. \[R_{\text{Pf,annualized}}=\left(\prod_{t=1}^{T}(1+r_{t})\right)^{\frac{1}{T}}-1\] (14) where \(r_{t}\) represents the return at time \(t\), and \(T\) is the number of time periods in a year.
2. Annualized Volatility: Volatility is a measure of risk, reflecting the standard deviation of the portfolio's returns over time, annualized to match the return horizon. \[\sigma_{\text{Pf,annualized}}=\sigma\times\sqrt{T}\] (15) where \(\sigma\) is the standard deviation of returns, and \(T\) is the number of time periods in a year.
3. Sharpe Ratio: The Sharpe ratio measures risk-adjusted returns, considering the excess return over the risk-free rate relative to volatility. \[R_{\text{Sharpe}}=\frac{R_{\text{Pf,annualized}}-R_{\text{f}}}{\sigma_{\text{ Pf,annualized}}}\] (16) where \(R_{\text{f}}\) is the risk-free rate.
4. Sortino Ratio: The Sortino ratio is similar to the Sharpe ratio, but it penalizes only downside volatility, ignoring positive deviations from the mean return. \[R_{\text{Sortino}}=\frac{R_{\text{annualized}}-R_{\text{risk-free}}}{\sigma_{ \text{downside}}}\] (17) where \(\sigma_{\text{downside}}\) is the standard deviation of negative returns.
5. Maximum Drawdown: This metric measures the largest peak-to-trough decline in portfolio value, providing insight into the worst-case wealth loss. \[\text{Maximum Drawdown}=\frac{\max(V_{t})-\min(V_{t})}{\max(V_{t})}\] (18) where \(V_{t}\) is the portfolio value at time \(t\).  
Calmar Ratio: The Calmar ratio is a risk-adjusted return metric that evaluates the portfolio's performance relative to its maximum drawdown.  
\[R_{\text{Calmar}}=\frac{R_{\text{Pf, annualized}}}{\text{Maximum Drawdown}} \tag{19}\]  
These metrics complement each other by offering a balanced view of portfolio performance. While the Sharpe ratio uses symmetrical volatility as a risk measure, the Sortino ratio focuses solely on downside risk, making it more suitable for risk-averse investors. The Calmar ratio and maximum drawdown, on the other hand, emphasize worst-case loss scenarios, which are particularly relevant for investors focused on capital preservation. By combining these metrics, we can assess which RL agents and configurations excel in maximizing returns, minimizing risk, or avoiding significant drawdowns, depending on the specific investment objectives.