# Empirical Analysis of Automated Stock Trading Using Deep Reinforcement Learning
## 2 Related Work  
With the development of artificial intelligence, machine learning naturally has wide applications in finance, such as fundamentals analysis, behavioral finance, technical analysis, financial engineering, and financial technology (FinTech) [18]. RL can suggest an answer to some sequential decision-making finance problems, such as option pricing, multi-period portfolio optimization, and trading [19; 20]. RL also has many applications in business management, such as ads, recommendations, customer management, and marketing [18; 21].  
In the option pricing area, ref. [22] handles Monte Carlo methods. The authors of [23] deal with the least squares Monte Carlo (LSM) method, which is a standard approach in the finance area for pricing American options. For American-type option pricing, it is critical to calculate the conditional expected value of continuation [18; 23; 24; 25].  
In the portfolio optimization area, following the empirical evidence of return predictability [26], dynamic portfolio optimization in multi-periods is increasing its value [27; 28]. The authors of [29] introduce direct reinforcement learning to trade with no forecasting, and the method is extended with deep neural networks [30]. This could be a better solution to deal with some problems in risk management with DRL. Generalizing the continuous action spaces and state is essential to applying DRL to dynamic portfolio optimization [18].  
In the trading area, the RL methodology is extensively used to predict future stock prices and decide stock trading policies [1; 10; 13; 16; 29; 30; 31; 32; 33; 34; 35; 36; 37]. These methods differ in which RL algorithm to use, which data to use as input, and how to define states, actions, and rewards. Some methods use artificial intelligence techniques other than RL, such as genetic algorithms [38]. Some other methods use information other than historical price data and technical indicators, such as applying sentiment analysis based on finance blog posts and using the results for stock price prediction [39].  
Most of these work claim that their methods can outperform baseline strategies such as \"buy and hold\", achieving higher profit. However, most evaluations only cover a single or a few trading environments. Therefore, it is questionable if their methods can be generalized to other environments such as markets other than the New York Stock Exchange. If an RL-based trading algorithm can always gain higher profit in various environments, it may contradict the Efficient Market Hypothesis (EMH) of [PERSON] [40]. According to EMH, our stock market is an \"efficient\" market, where price fully reflects \"all available information\". Thus, it is impossible to predict future prices and gain profit that exceeds market returns. Among the three forms of \"available information\" specified in EMH, most previous work on RL-based automatic stack trading has been related to the weak form, which includes only market transaction data such as past stock prices and trading volume. A major goal of our analysis is to study whether an RL-based stock trading method can find clues that can reliably predict future stock prices and therefore gain profit in a stable manner.