# High-Frequency Direction Forecasting of the Futures Market Using a Machine-Learning-Based Method
## 5 Results and Discussion
### Results of Classifiers
#### 5.1.2 Accumulated Return  
As described in Section 4.5, whether the investment portfolio designed in simulation trading could generate considerable profits is even more critical for market investors. In Table 6, the trading return results of all methods in four quarters are reported. Note that the two passive investment strategies, BAH and SAH, are also included in Table 6 to better evaluate the profitability of the proposed method. From the return results shown in Table 6, the largest average return result was generated by XGBoost-SMOTE-NSGA-II, reaching approximately 0.349, which is superior to that of other benchmark methods. Among the four single-algorithm-based models (ANN, SVM, RF, and XGBoost), we find that although none were able to produce positive average returns, XGBoost yielded an average return of \(-0.064\). It outperformed ANN, SVM, and RF, which incurred losses of \(-0.183\), \(-0.267\), and \(-0.354\) on rebar futures simulation transaction, respectively. Specifically, the two traditional machine learning methods (ANN and SVM) continuously suffered from huge losses, thus both generated negative returns in the four testing periods. It demonstrates that it would be less effective to improve return predictions on the rebar futures price movements by implementing a single machine learning method. In addition, it can be observed that the method XGBoost-SMOTE still failed to obtain a satisfactory result in terms of the average trading return, demonstrating that solving the class imbalanced dataset problem could not lead to a significant improvement on the profitability of the trading rule. Hence, to facilitate trading performance, the optimization of the trading rule is employed in our proposed method. By comparing the results of BAH and SAH with our proposed approach, it is found that the SAH strategy produced negative returns in the last three testing periods, resulting in an average loss of \(-0.056\). For the BAH, despite yielding a positive average return, the yield is significantly worse than that of our proposed method. Besides the best average return, XGBoost-SMOTE-NSGA-II achieved return results of 0.278, 0.237, 0.462, and 0.418 in the four testing periods, which are much better than other benchmark methods. According to the return results and analysis mentioned above, it should be noted that the proposed approach achieved optimal profitability among all benchmark methods after considering the NSGA-II method. Therefore, the effectiveness of the proposed method in rebar futures simulation trading has been further verified. To verify whether the profitability of the proposed method XGBoost-SMOTE-NSGA-II was significantly better than that of the benchmark methods, the Friedman test [51] was employed to investigate whether the accumulated return results achieved by the \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Testing Period** & **BAH** & **SAH** & **ANN** & **SVM** & **RF** & **XGBoost** & **XGBoost-SMOTE** & **XGBoost-SMOTE-NSGA-II** \\ \hline Period 1 & \(-0.087\) & \(0.085\) & \(-0.241\) & \(-0.489\) & \(-0.315\) & \(-0.091\) & \(-0.119\) & 0.278 \\ Period 2 & \(0.075\) & \(-0.077\) & \(-0.290\) & \(-0.069\) & \(-0.305\) & \(0.257\) & \(-0.162\) & 0.237 \\ Period 3 & \(0.002\) & \(-0.004\) & \(-0.156\) & \(-0.363\) & \(-0.817\) & \(-0.488\) & \(-0.023\) & 0.462 \\ Period 4 & \(0.226\) & \(-0.228\) & \(-0.046\) & \(-0.148\) & \(0.022\) & \(0.068\) & \(-0.007\) & 0.418 \\ Average & \(0.054\) & \(-0.056\) & \(-0.183\) & \(-0.267\) & \(-0.354\) & \(-0.064\) & \(-0.078\) & 0.349 \\ Sum & \(0.216\) & \(-0.224\) & \(-0.733\) & \(-1.069\) & \(-1.415\) & \(-0.254\) & \(-0.311\) & 1.395 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Accumulated return results for rebar futures simulation trading. proposed method and the benchmark methods were significantly different. Table 7 reports the Friedman test results on the accumulated return results, from which we find that the conclusion is significant at the 0.05 level for one-tailed test.