# Empirical Analysis of Automated Stock Trading Using Deep Reinforcement Learning
###### Abstract  
There are several automated stock trading programs using reinforcement learning, one of which is an ensemble strategy. The main idea of the ensemble strategy is to train DRL agents and make an ensemble with three different actor-critic algorithms: Advantage Actor-Critic (A2C), Deep Deterministic Policy Gradient (DDPG), and Proximal Policy Optimization (PPO). This novel idea was the concept mainly used in this paper. However, we did not stop there, but we refined the automated stock trading in two areas. First, we made another DRL-based ensemble and employed it as a new trading agent. We named it Remake Ensemble, and it combines not only A2C, DDPG, and PPO but also Actor-Critic using Kronecker-Factored Trust Region (ACKTR), Soft Actor-Critic (SAC), Twin Delayed DDPG (TD3), and Trust Region Policy Optimization (TRPO). Furthermore, we expanded the application domain of automated stock trading. Although the existing stock trading method treats only 30 Dow Jones stocks, ours handles KOSPI stocks, JPX stocks, and Dow Jones stocks. We conducted experiments with our modified automated stock trading system to validate its robustness in terms of cumulative return. Finally, we suggested some methods to gain relatively stable profits following the experiments. empirical analysis; automated stock trading; deep reinforcement learning; policy gradient method; actor-critic algorithms; ensemble strategy +
Footnote †: journal: _Applied Sciences_  
Even though we believe that the ensemble strategy of [10] is an innovative idea, we intended to verify it through empirical analysis. We judged that it was not easy to grasp the actual performance of DRL agents, including the ensemble strategy, with only the single case experiment shown in [10]. Thus, we conducted five experiments for each trading agent and analyzed the results. As an extension to empirical analysis, we added a new ensemble-based algorithm as an agent with the existing DRL-based trading strategy. Secondly, we extended the model to various markets and applied the model. These attempts stemmed from the question of whether automated stock trading could generally assure high returns. The results and insights obtained through the empirical analysis are as follows: First, the new ensemble-based Remake Ensemble did not show noticeable performance compared to other agents. Therefore, we concluded that adding a new algorithm is not a sure solution to contribute to the general performance of automated stock trading, even if the algorithm is based on a novel idea. Second, the model showed lower performance than the benchmark indexes in the Dow Jones 30, KOSPI, and JPX markets. Along with this, the model performance in each market was also different. For this reason, we reasoned that for the model to obtain stable returns for each market, a trading strategy suitable for each market would be needed. On that count, we refined the trading strategy of the model in a conservative direction to suit the KOSPI market and conducted experiments again. As a result, the performance of each agent in the KOSPI market increased overall, so we extended the model to two other markets and observed the results. In the JPX market, the conservative trading strategy seemed to have a substantial effect, whereas the performance worsened in the Dow Jones 30. Accordingly, we tentatively concluded that while conservative-trading-strategy-based DRL stock trading might be effective in the KOSPI and JPX markets, an aggressive trading strategy in Dow Jones may yield stabler and higher returns. To summarize, the main contributions of this paper are as follows: (a) We conducted empirical analysis of a stock trading agent trained using an ensemble of multiple reinforcement learning algorithms claiming to achieve outstanding performance. We added more reinforcement learning algorithms to train the ensemble model and tested its performance. (b) We applied the trading strategy to various stock markets such as KOSPI and JPX, in contrast to the previous work that only used DJIA as the evaluation data.