# High-Frequency Direction Forecasting of the Futures Market Using a Machine-Learning-Based Method
## 4 Experimental Design
### Evaluation Metrics  
To assess the direction prediction accuracy and profitability of the proposed method, four measures are employed in the experiments. The formulas to calculate those four metrics are presented in Table 4.  
First, hit ratio (\(HR\)) calculates the correct direction prediction rate, which is generally used to identify the classification accuracy of the proposed approach and benchmark methods on rebar futures forecasting. Since our proposed model would not perform direction prediction if the predicted trading signal belongs to the \"No Trading\" class, the number of predictions in this range is not included in the formula. As shown in formula 1 of Table 4, \(N\) denotes the number of all prediction times, \(PN\) and \(NN\) respectively refers to the number of correct \"upward\" predictions and correct \"downward\" predictions. Second, accumulated return (\(AR\)) is defined and utilized to measure the profit-making ability of simulation trading in each testing period. There are a total of three conditions for simulating trading: (1) Suppose that if the predictor predicts that the rebar futures price is in a rising trend, and the predicted return is larger than the level threshold for long, a \"long\" position would be executed. (2) If the price movement is downward and the predicted value is smaller than the level threshold for short, the proposed method would perform a \"short-selling\" transaction in the rebar futures market. (3) The proposed method would not execute trading simulation if the predicted return is in the range \([T_{2},T_{3}]\). Additionally, the profit-taking and loss-cutting thresholds are utilized in the proposed method, so that the trading position will be automatically closed in the case that the floating return exceeds one of those values during the position holding period. In formula 2 of Table 4, \(n\) represents the number of periods that lasts for one quarter in each testing period; \(cl\) and \(op\) denote the closing and opening prices of rebar futures, respectively, \(l\) means the trading leverage, and \(t\) refers to the number of periods ahead for simulation trading. As recommended by  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Formula** & **Evaluation Metrics** & **Calculation Formula** \\ \hline
1 & Hit Ratio (\(HR\)) & \(HR=\frac{PN+NN}{NN}\) \\ \hline
2 & Accumulated Return (\(AR\)) & \(AR=\sum\limits_{i=1}^{L}\left(\frac{(\sum\limits_{i=1}^{L}-\sigma_{i=1})}{ \sigma_{i=1}}-cost\right)\times\text{sgn}\pi\), \(\text{sgn}\pi=\left\{\begin{array}{l}-1,\text{ }\[PERSON] [48], five-periods-ahead prediction is designed in this experiment. Additionally, the transaction cost \(cost\) is required when the model executes a long or short transaction in the futures market. The transaction cost includes the liquidity, execution, and slippage cost. It is set to be 0.1% per round trip based upon an arbitrary assumption [49]. Furthermore, sgn \(\varphi\) is a symbolic function, in which \(\varphi\) refers to the movement magnitude level. Third, except for generating excellent prediction ability and profitability, trading risk also exerts a fundamental influence in examining the performance of a portfolio that market investors should pay attention to. As a commonly used risk indicator, maximum drawdown (\(MDD\)) describes the maximum retracement magnitude of the expected return generated by an investment portfolio over a selected period. This indicator could be used to help the market investors understand the stability of the profits and the possibility of potential losses of the model. As shown in formula 3 of Table 4, \(AR_{i}\) represents the accumulated return at the time \(i\), \(AR_{j}\) denotes the value of accumulated return at the time \(j\) before the time \(i\), and \(D_{i}\) represents the maximum retracement magnitude of the accumulated return at the time \(i\). Fourth, the Sharpe ratio (\(SR\)) was proposed by [PERSON] in 1994 [50], which is also employed as one of the famous indicators for transaction risk evaluation.