# Machine Learning for Quantitative Finance Applications: A Survey
## 9 Reinforcement Learning (RL)  
In this section, we surveyed some considerable approaches based on reinforcement learning. Reinforcement learning refers to a machine-learning paradigm which involves an \"agent\" in order to perform a task. The goal of agent is to maximize a numerical reward by interacting with an unknown environment. In financial field, this emerging paradigm has gained attention due to their ability to develop portfolio construction including prediction in one integrated step. Basically, the goal of the agent is to include some constraints such as liquidity and transactions cost when a decision-making process is performed. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Article** & **Techniques** & **Main Goal** & **Dataset** \\ \hline [PERSON] et al. (2009) [44] & SVM & Discrete stock price estimation & S\&P 500 \\ \hline [PERSON] et al. (2016) [45] & Sentiment Analysis & Stock market movements prediction &
\begin{tabular}{c} Dow Jones Industrial \\ Average Index (DJIA) \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Summary of studies based on sentiment analysis. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Article** & **Techniques** & **Main Goal** & **Dataset** \\ \hline [PERSON] et al. (2012) [39] & ESM, ARIMA, BPNN & Forecasting & \
\begin{tabular}{c} Shenzhen Integrated Index (SZII), \\ Dow Jones Industrial Average Index (DJIA) \\ \end{tabular} \\ \hline [PERSON] et al. (2013) [40] & Hybrid Genetic Fuzzy Systems–ANN model & Stock price forecasting & IBM, British Airlines, RyanAir, DELL \\ \hline [PERSON] et al. (2014) [41] & Hybrid ARIMA–ANN model & Forecasting &
\begin{tabular}{c} Sunspot data, electricity data, \\ L\&T stock market \\ \end{tabular} \\ \hline [PERSON] et al. (2017) [42] & Hybrid ETS–ANN model & Forecasting & IBM \\ \hline [PERSON] et al. (2018) [43] & Hybrid LSTM–multiple GARCH-type model & Volatility prediction & KOSPI 200 index \\ \hline \hline \end{tabular}
\end{table} 
Table 5: _Cont._In [48] the authors presented a time-driven feature-aware jointly deep reinforcement learning (RL) model (TFJ-DLR) in order to learn features representation considering noisy financial time series. Furthermore, the proposed method is developed to perform a decision-making system by taking advantage of environmental representation. Specifically, the authors selected various features of financial time series in order to summarize the useful values through analyzing historical data. Results demonstrated the robustness and effectiveness of the proposed TFJ-DLR system by using real-world financial data. In particular, the proposed method is able to increase investment returns. A recent work is [49], in which the author implemented an algorithm based on supervised DL and reinforcement learning algorithm. The main goal is to forecast the short-term trend in the currency FOREX (FOReign EXchange) market with the aim to maximize the return of investment in a high-frequency trend (HFT) algorithm. Also, the author proposed a grid trading engine to perform high frequency operations. In order to validate the proposed model, the trading system has been validated over several financial years and on the EUR/USD cross. The value of return of investment (98.23%) and reduced drawdown (15.97%) confirmed the effectiveness of the proposed trading system. Table 7 reports surveyed articles based on RL approach. In [50], the author proposed a model that combines dynamic programming (DP) and RL techniques. The proposed Q-Learner in the Black-Scholes (QLBS) model is developed to estimate future changes for stock market using [PERSON]'s model. After comparing fitted Q iteration to the DP model in order to evaluate the performance, the author formulated an inverse reinforcement learning (IRL) setting for the QLBS model aimed at analyzing prices and actions. Finally, the author discussed the usage of QLBS model for option portfolios.