# Empirical Analysis of Automated Stock Trading Using Deep Reinforcement Learning
## 5 Empirical Analysis  
The experiments are divided into two parts. One is an experiment based on Dow Jones 30 stocks using the DRL algorithm, which is A2C, PPO, DDPG, and the ensemble strategy shown in [10]. We used Remake Ensemble as an additional agent. Another is an experiment based on expanding the stock market area to KOSPI and JPX. KOSPI stands for the Korea Composite Stock Price Index, which refers to the composite stock index of the Korea Exchange's securities market. However, its meaning has expanded, and the securities market is called the KOSPI market. In addition to the Tokyo Stock Exchange, JPX refers to the Japanese Exchange Group, which includes the Osaka Stock Exchange. In this paper, the stock market listed on the Tokyo Stock Exchange under JPX is referred to as the JPX market. The performance of each agent in each stock market is expressed by evaluation metrics: cumulative return, annual return, annual volatility, Sharpe ratio, and max drawdown.  
The cumulative return, which is also called aggregate return, is the total change in the investment price over a set time period, and its formula is as follows:  
\[\centering Cumulative Return=\frac{(Final\ Price\ of\ Stock)-(Initial\ Price\ of\ Stock)}{Initial\ Price\ of\ Stock}\times 100\@add@centering \tag{12}\]  
The annual return is the return on the investment that occurred during the year and is calculated as a percentage of the initial investment. In other words, the annual return is the cumulative return on an annual basis, and its formula is as follows:  
\[\centering Annual Return=\frac{(End\ of\ Year\ Stock\ Price)-(Begining\ of\ Year\ Stock\ Price)}{Begining\ of\ Year\ Stock\ Price}\times 100\@add@centering \tag{13}\]  
Volatility is a statistical metric of the variance of returns on a given security or market index. It is often calculated from the standard deviation between returns from that same portfolio. The annual volatility is volatility measured on a yearly basis, and its formula is as follows:  
* \(P_{i}\): the daily price % change of portfolio on an \(i\)th day
* \(P_{av}\): the mean price % change of portfolio for all trading days\[Annual\ Volatility=\sqrt{\sum\left(P_{av}-P_{i}\right)^{2}} \tag{14}\]  
The Sharpe ratio is the average return earned in excess of the risk-free rate per unit of volatility. It is a mathematical measure of the insight that excess returns over a certain period of time can mean greater volatility and risk in an investment strategy. The Sharpe ratio is calculated by the return of the portfolio, the risk-free rate, and the standard deviation of the portfolio value, and the formula is as follows:  
* \(R_{p}\): the return of portfolio (annual return)
* \(R_{f}\): the risk-free rate
* \(\sigma_{p}\): standard deviation of the portfolio value  
\[Sharpe\ Ratio=\frac{R_{p}-R_{f}}{\sigma_{p}} \tag{15}\]  
The max drawdown, which is an indicator of downside risk over a specified period, means the maximum observed loss from the peak to the low point, which is called the trough point, in the portfolio before reaching a new peak. The formula of the max drawdown is as follows:  
\[Max\ Drawdown=\frac{\left(Tough\ Value\right)-\left(Peak\ Value\right)}{ Peak\ Value}\times 100 \tag{16}\]  
These EMs were obtained using pyfolio ([[https://github.com/quantopian/pyfolio](https://github.com/quantopian/pyfolio)]([https://github.com/quantopian/pyfolio](https://github.com/quantopian/pyfolio))). The risk-free rate is set to zero by default in pyfolio, and we applied it to calculate the Sharpe ratio without changing the setting.