# Smart Robotic Strategies and Advice for Stock Trading Using Deep Transformer Reinforcement Learning
## 3 Methodology and Proposed Model
### Feature Extraction  
This subsection focuses on describing how features are extracted from the stock data, which will be used later to perform the analysis. We start by outlining the selection of our sets of features. In [45], feature extraction was defined as the process of obtaining derived values (features) from the input data and producing unique and informative properties to enhance the learning and generalization tasks of ML algorithms. Stock data are generally subject to a high amount of noise, and we can reduce the effect of noise by using appropriate technical indicators. These technical indicators can also provide a different perspective on the market, based on their use, and provide a higher degree of predictive ability to predict future price movements. Statistically, these characteristics should generally have a reasonable correlation with market movements, which will complement our forecasted data. Combining these features should lead to a nice mix of practical observations, from which our model can benefit. Inputs to a Transformer network are used to more effectively capture trends in the stock market. These inputs include the daily closing price, trading volume, and six technical indicators for each ticker and its linked index. These technical indicators have been proposed by [53]. It is usual practice to classify technical indicators into four categories: Volume, Volatility, Momentum, and Trend. Summaries of the technical indicators that were utilized are given below:  
1. Exponential moving average (EMA) These are utilized to reduce the noise and point out the short- and long-term trends in time-series data. The exponential moving average EMA \((x,\alpha)\) is generated by exponentially reducing the weight of observations \(X_{t}\) regarding their distance from \(X_{t}\) using a weighted multiplier \(\alpha\). \[ \begin{split}\text{EMA}(x_{t},\alpha)=\alpha x_{t}+(1-\alpha) \,\text{EMA}(x_{t-1},\alpha).\\ \alpha=\frac{2}{N+1}\end{split} \] (7)
2. Money flow index (MFI): Based on price and volume, the money flow index determines the amount of money moving into and out of a specific ticker, or, to put it another way, if a given stock has been over-bought or over-sold. This is what is known as a momentum indicator. When the MFI is over 80, it indicates an over-bought condition; meanwhile, when it is below 20, it suggests an over-sold condition. The MFI may be computed using the following formula: \[ \begin{split}\text{MFI}=100-\frac{100}{1+\text{MFR}},\\ \text{where MFR }=\frac{\text{Positive Money Flow}}{\text{Negative Money Flow}}\\ \text{Money Flow}=\bigg{(}\frac{\text{High }+\text{Low }+\text{ Close}}{3} \bigg{)}Volume.\end{split} \] (8)
3. Relative strength index (RSI): This index is also used as a momentum indicator, which determines whether a ticker is over-bought or over-sold by considering both the velocity and magnitude of price fluctuations. Its value may vary anywhere from 0 to 100, with low values indicating a stock that is being over-sold and high values indicating an over-bought stock. The following formula may be used to easily determine the value of this indicator: \[ \begin{split}\text{RSI}=100-\frac{100}{1+\text{RS}},\\ \text{where RS }=\frac{\text{Average of Up closes}}{\text{Average of Down closes}}. \end{split} \] (9)4. Moving average convergence-divergence (MACD): The MACD is another indicator used to illustrate the relationship between two exponential moving averages (EMAs): Slow (\(\theta_{1}\)) and fast (\(\theta_{2}\)). According to [54], the moving averages are calculated based on the following criteria: \(\theta_{1}\) comprises 26 periods (market standard), \(\theta_{2}\) consists of 12 periods (usual for the financial markets), \(\theta_{1}-\theta_{2}\) for constructing the MACD line. Finally, the moving average is constructed using the MACD line (standardized with 9 periods). This indicator can be used to assess the trend-following momentum within a stock. The formula for calculating this indicator is: \[\text{MACD}=\text{EMA}(\theta_{1})-\text{EMA}(\theta_{2}).\] (10)
5.