# Importance of Event Binary Features in Stock Price Prediction
## 5 Performance Evaluation through Fund Simulation  
We conducted fund simulations to measure the cumulative investment profits of each model. To set up a precise fund-simulation environment, we set up the fund simulation using a period that is different from the training and test data periods. We conducted fund simulations using data from September 2015 to November 2019. In addition, we used a previously developed fund simulation system [34] that can find the best trading policy for a set of recommended stocks. When fund simulation is conducted, validation data is applied to the prediction model to obtain recommendations for every date and every stock symbol. In addition, ach prediction result has an additional probability field for the prediction. We call this a \"_prediction value_\", and this value is estimated between 0 and 1. We set a certain threshold \(\theta\), to exclude too low _prediction value_. If the _prediction value_ is too low, the number of stock trading will increase, while the profit per trade decrease in fund simulation. Therefore, fund simulation is performed only when it is above a certain threshold. When the prediction model defined by the parameter \(w\) is \(f\) and the input expression of the stock for \(f\) on a specific date \(t\) is \(s_{t}\), the equation for the _prediction value_ value is as follows. We set \(\theta\) above 0.1 in the fund simulation. \[\textit{prediction value}=f(s_{t};w)\geq\theta \tag{13}\]  
Our fund simulation program combines various policies based on recommended items and dates. Then it calculates the profit and hit rate to get the best trading policy. The Trade Policy generated by the Fund Simulation Program consists of a total of eight fields. The first factor is the \"Buy discount rate\", which is the percentage of purchase price compared to the previous day's close price at buying the recommended stocks. Second is \"Target profit rate\". This factor represents the target rate of profit that we set at buying stocks. Third factor is \"Stop-loss rate\". Stop-loss rate is the loss rate when the purchased stocks should be loss-cut. If the stop-loss rate is \(-12\%\), it will be sold automatically if the current price is \(-12\%\) lower than the buying price. Fourth, \"Profit rate\" is the ratio of the actual profit. Fifth the \"Maximum holding period\" is the maximum number of days to hold the purchased stock. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Forecasting Model**} & \multirow{2}{*}{**MSE**} & \multirow{2}{*}{**MAE**} & **Comparison of Prediction Accuracy** & **Comparison of Prediction Accuracy** \\  & & & **with Model 1\% (MSE)** & **with Model 2\% (MSE)** & **with Model 3\% (MSE)** \\ \hline
**Model 1** & 0.1105 & 0.247 & \(\chi\) & 28.05 & \(-5.97\) \\
**Model 2** & 0.0795 & 0.2162 & \(-38.99\) & \(\chi\) & \(-47.29\) \\
**Model 3** & 0.1171 & 0.2498 & 5.63 & 32.10 & X \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Performance comparison of three models. For example, if the maximum holding date is 5, and the stock price does not reach the target profit rate or stop-loss rate for five days after the purchase date, it will be automatically sold at the closing price of the final date. Sixth, \"Profit rate per trade\" means the ratio of the profit that can be obtained in one trading of the whole trading. \"Profit rate per daily trade\" is a daily calculation of \"Profit rate per trade\". Finally, \"Hit ratio\" refers to the percentage of stocks that were sold at the target profit price. We conducted the fund simulation using only the results with a predictor value of 0.1 or higher for each model, and the result of optimal trading policy is as shown in Table 8  
Model 1 purchases stock as the same price as the previous day's close price because the \"buy discount rate\" is +0. According to fund simulation, we can get the best profit when we sell stock that reach at +24% profit or \(-\)12% loss for up to 22 days. As a result, Model 1 was able to get 52.6% profit of the investment, and the hit ratio was 27.3%.