# Smart Robotic Strategies and Advice for Stock Trading Using Deep Transformer Reinforcement Learning
## 5 Discussion  
In this section, we discuss the performance of our model on the out-of-sample test data set, the limitations of our work, and future research directions. During the training of the model, our technique automatically selected the best reward function to increase the net worth and profit on the initial investment. This automatic selection of the best reward function allowed the model to learn the optimal hyperparameters. According to our discussion in the previous section, the Sortino ratio, max drawdown, annual volatility, and cumulative returns showed the best performance for both outcomes (i.e., net worth and target return). All of these reward functions ensure the robustness of the model, along with risk management in the return. The performance associated to all the reward functions, along with their descriptions, are summarized in Table 4. The average increase in the net worth of each stock is also provided in the table. The Sortino ratio showed the highest average increase in the case of all stocks, with an average increase of 21.54% for Saudi Telecom Company, 18.54% for Saudi Electricity Company, 17% for Saudi Basic Industries Corporation, and 19.36% for Al-Rajibi Banking and Investment. Note that, in the reinforcement network, failures are represented by agents not achieving the target return. These failures were analysed using the following return functions: Sortino ratio, max drawdown, annual volatility, and cumulative return. All of these reward functions ensure the robustness of the model, in addition to risk management on the return. Figures 22-24 show the results for net worth on the testing data samples. Saudi Telecom Company (Figure 22), Saudi Electricity Company (Figure 23), and Saudi Basic Industries Corporation (Figure 24) showed maximum increases of 13%, more than 14%, and slightly over 13% of the initial net worth, respectively. These results indicate that our model can effectively predict the trading of stocks to maximize profit in the long run, as the net worth follows an increasing trend for almost 10 months in each case. Notably, even with a slight decrease in the value for a few months, the trend of net worth began increasing again. Hence, our model is effective for trading suggestions to provide profit for long-term investments by balancing risk. \begin{table}
\begin{tabular}{l l c c c c} \hline \hline \multirow{2}{*}{**Reward Function**} & \multirow{2}{*}{**Description**} & \multicolumn{4}{c}{**Average Increase in Net worth**} \\ \cline{3-6}  & & **7010** & **5110** & **2010** & **1120** \\ \hline Sortino ratio & Measures the risk on return by & 21.54\% & 18.54\% & 17\% & 9.36\% \\ Cumulative returns & & Indicates the total return at the end of & 8.02\% & 10.61\% & 12.92\% & 14.57\% \\  & the trading phase. & & & & \\ Annual volatility & & shows the annual standard deviation of & 7.84\% & 10.61\% & 12.92\% & 14.57\% \\  & portfolio return. & & & & \\ Calmar ratio & Risk quantification measure. High ratio & & & & \\  & indicates better portfolio performance. & 1.02\% & 5.81\% & 10.90\% & 4.75\% \\ Omega & & Weighted gain to loss probability ratio & 8.12\% & 0.61\% & 2.92\% & 1.57\% \\ Max Drawdown & & Weighted gain to loss probability ratio & 0.024 & 1.61\% & 1.28\% & 0.07\% \\ Normal & & & & & \\  & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Reward functions used in this work, in order of their performance (best to worst from top to bottom). It is evident from the results that our algorithm is capable of generating consistent profit, considering all the transactions. Moreover, among the reward functions, the Sortino  
Figure 23: 5110 stock net worth prediction on test data set (Initial portfolio value $10,000).  
Figure 22: 7010 stock net worth prediction on test data set (Initial portfolio value $10,000). ratio, cumulative returns, and annual volatility were the best, based on their performance and parameter optimization.