# Empirical Analysis of Automated Stock Trading Using Deep Reinforcement Learning
## 6 Diversification of Trading Strategies  
As shown in the previous sections, several empirical analyses were conducted in different stock markets. Although some agents generated cumulative returns higher than the benchmark indexes, the differences in the cumulative return values for each experiment were huge. Moreover, the ensemble strategy failed to show stunning results in the Dow Jones 30. As shown in Figure 6, only one experiment iteration exceeded the CR of the DJIA. For this reason, we tried to improve it to make it a more potent DRL-based stock trading model. There were two main criteria for robust models that we thought of. The first was whether the performance of the DRL-based trading agent used for automated stock trading could be improved over the original model. Performance here refers to the average cumulative return for each agent. The second was whether at least one of the agents could consistently guarantee a higher cumulative return than the benchmark index in all experiments over the same period. No matter how much the average cumulative return is higher than the original model, stable profit cannot be ensured if the trading agent gets a lower value than the benchmark index in the other iterations. We conducted experiments based on the KOSPI market because all trading agents' average return was lower than the benchmark index. This means the original model performed the worst in the KOSPI market. If the performance improvement in the KOSPI market was successful, we tried to determine whether a general application is possible by expanding the model application to the JPX and DJIA later. We thought about how to improve the original model and tried various methods. As part of these methods, we decided to change the trading strategy of the model according to a hypothesis that we thought could improve the performance. First, we adjusted the values of the validation window and the rebalance window. This attempt was based on the hypothesis that changing the validation term to a shorter period than the existing 63 days would make the trading agent learn better and perform better. However, the performance did not improve, so this method did not pay off. Figure 6: Cumulative return curves of the original ensemble strategy agent and DJIA (\(h_{max}=100\), buying threshold = 0, selling threshold = 0, initial portfolio value $1,000,000, from 4 January 2016 to 6 July 2020, (1134 days)). \begin{table}
\begin{tabular}{c c c c c c c} \hline
**EM** & **JYX30** & **A2C** & **DDPG** & **PPO** & **Ensemble** & **Remake** \\ \hline CR & 71.51\% & 67.328\% & 73.855\% & 65.893\% & 96.081\% & 87.650\% \\ AR & 12.736\% & 11.714\% & 12.823\% & 11.597\% & 15.671\% & 14.791\% \\ AV & 22.027\% & 21.810\% & 22.701\% & 20.357\% & 21.652\% & 21.288\% \\ SR & 0.65 & 0.614 & 0.646 & 0.640 & 0.776 & 0.744 \\ MD & \(-\)28.252\% & \(-\)32.153\% & \(-\)30.315\% & \(-\)30.527\% & \(-\)30.821\% & \(-\)29.591\% \\ \hline \end{tabular}
\end{table} 
Table 3: Average performance of five RL-based agents on top 30 JPX Stocks from 4 January 2016 to 21 July 2020 (1134 days). Secondly, we adjusted the turbulence threshold. The turbulence threshold is a hyperparameter generated by the authors of [10] that reflects risk aversion for market crashes such as wars, the collapse of stock market bubbles, sovereign debt default, and financial crises. To regulate the risk in a worst-case scenario such as the 2008 global financial crisis, they used the financial turbulence index to measure extreme asset price movements [47]. According to them, when the turbulence index is higher than a threshold, which indicates low market conditions, trading agents stop buying and sell all stocks. When the turbulence index is below the threshold, then agents resume trading. We hypothesized that the appropriate turbulence threshold is different for each market, and we expected trading agents to perform better in the KOSPI market by adjusting the turbulence index.