# Tensor Implementation of Monte-Carlo Tree Search for Model-Based Reinforcement Learning
## 4 Experiments
### Results
We further investigated influence of the number of MCTS simulations \(C_{S}\) and report results for two best performing implementations in Tables 3, 4 and Figure 6. The \(C_{T}\) parameter was set to 100. We can observe that the computation time increases significantly as the number of simulations increases, especially in constant action scenario. Proposed GPU implementation is 12.1 times faster for \(C_{S}=50\) and 8.7 times faster for \(C_{S}=400\) in random action scenario. In constant action scenario, GPU implementation is 4.2 times faster for \(C_{S}=50\) and 2 times faster for \(C_{S}=400\). Figure 5: Comparison of execution time of the two best performing implementations with respect to the number of observations. Results for both scenarios without DNN. Figure 6: Comparison of execution time of the two best performing implementations with respect to the number of simulations. Results for both scenarios without DNN. Last experiments measured performance with DNN model. We omitted CPU method from results due to the high computation requirements associated with the execution of DNN model (e.g., 338 s for \(C_{S}=50\), and \(C_{T}=50\)). Similarly to previous experiments, we fixed \(C_{S}\) to 50 and set \(C_{T}\) to 50, 100, 250, 500 and 750. Results presented in Tables 5, 6 and Figure 7 show similar trend as in experiments without DNN. Proposed GPU is the best performing implementation for all values of parameter \(C_{T}\). Even with the DNN model, which consumes most of the computation, GPU implementation is 4.7 times faster for random action scenario and 7.7 times faster for constant action scenario than the second best performing CPUGPU_P implementations (with 10 and 25 processes respectively), for \(C_{T}=750\). \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{\(C_{S}\)} & \multirow{2}{*}{**GPU**} & \multicolumn{3}{c}{**CPUGPU\_P**} \\ \cline{3-5}  & & 5 & **10** & **25** \\ \hline \multirow{2}{*}{50} & 1.704 & 7.825 & 7.24 & 8.812 \\  & \(\pm\)0.006 & \(\pm\)0.03 & \(\pm\)0.018 & \(\pm\)0.033 \\ \hline \multirow{2}{*}{100} & 6.371 & 22.36 & 19.344 & 19.734 \\  & \(\pm\)0.029 & \(\pm\)0.072 & \(\pm\)0.03 & \(\pm\)0.037 \\ \hline \multirow{2}{*}{200} & 24.281 & 75.867 & 61.968 & 60.144 \\  & \(\pm\)0.135 & \(\pm\)0.095 & \(\pm\)0.066 & \(\pm\)0.099 \\ \hline \multirow{2}{*}{400} & 95.291 & 285.212 & 217.657 & 193.77 \\  & \(\pm\)0.571 & \(\pm\)0.216 & \(\pm\)0.139 & \(\pm\)0.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Effect of number of simulations on computation time. Results of experiments for constant action without DNN (\(C_{T}=100\)). \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{\(C_{S}\)} & \multirow{2}{*}{**GPU**} & \multicolumn{3}{c}{**CPUGPU\_P**} \\ \cline{3-5}  & & **5** & **10** & **25** \\ \hline \multirow{2}{*}{50} & 0.234 & 2.832 & 3.749 & 6.282 \\  & \(\pm\)0.001 & \(\pm\)0.011 & \(\pm\)0.016 & \(\pm\)0.032 \\ \hline \multirow{2}{*}{100} & 0.481 & 4.906 & 5.454 & 9.945 \\  & \(\pm\)0.006 & \(\pm\)0.015 & \(\pm\)0.019 & \(\pm\)0.032 \\ \hline \multirow{2}{*}{200} & 1.068 & 9.908 & 10.225 & 18.109 \\  & \(\pm\)0.001 & \(\pm\)0.028 & \(\pm\)0.023 & \(\pm\)0.034 \\ \hline \multirow{2}{*}{400} & 2.294 & 20.422 & 19.844 & 34.562 \\  & \(\pm\)0.002 & \(\pm\)0.134 & \(\pm\)0.031 & \(\pm\)0.082 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Effect of number of simulations on computation time. Results of experiments for random action without DNN (\(C_{T}=100\)). Figure 7: Comparison of execution time of the two best performing implementations with respect to the number of observations. Results for both scenarios with DNN.