## 2 Methodology
### Monte Carlo Tree Search  
Monte Carlo tree search has earned the researchers' consideration thanks to its fascinating results in a wild variety of challenging domains such as real-time strategy games (RTS) [28], general video game playing (GVGP) [29], and autonomous driving [30]. Compared to other search-based methods such as uninformed approaches, MCTS can mitigate the required computational needs and provide guarantees that purely heuristic-based methods cannot. The component that distinguishes MCTS from other methods is the upper confidence bound for trees (UCT) algorithm proposed by [31] that handles the exploration-exploitation trade-off inside the tree during the iteration process. The MCTS algorithm builds the tree-based form of the control problem with the help of the generative model of the environment; hence it is utilized as a model-based planner. Every iteration during the building of the tree uses four phases, which are the following:  
* Selection: Starting from the current root node, the algorithm recursively selects the child node on every branch with the highest UCT value until a leaf node occurs.
* Expand: This phase encounters whether the leaf node found during selection is not terminal and child nodes can be populated from it. In that case, the algorithm populates the child nodes with the possible actions.
* Simulation: This phase provides the first value assigned to a leaf node. To do so, it conducts a Monte Carlo rollout from the given leaf node until a terminal state of the environment encounters and assigns the value of the terminal state to the leaf node.
* Backpropagation: In this final phase, the assigned state-value is backpropagated along the path traveled during the selection phase from root to leaf. The above-mentioned UCT algorithm looks as follows: \[\overline{X_{i}}+2C_{p}\sqrt{\frac{2\ln N_{i}}{n_{i}}},\] (2) where the exploration-exploitation trade-off is controlled via \(C_{p}\) called the temperature parameter, \(N_{i}\) is the visit count of the parent node, \(n_{i}\) is the visit count of the given node, and \(\overline{X_{i}}\) is the given node's average value. Another important advantage of the MCTS algorithm over its alternatives is that if enough iteration is given, then it can provide the globally optimal solution [31].  
During operating, the algorithm builds a search tree with the help of the four phases. The depth of the tree, hence the horizon, is determined by the provided number of iterations. After reaching the iterations, the information gathered inside the tree is utilized in the root node's state. A policy has to be chosen beforehand to exploit the knowledge inside the children of the root node. The utilized policy, in this case, is the robust-child approach that chooses the action that populates the child node that has the most visit count compared to the other children. It is also important to mention that one decision only determines the next step of one train on the network. Consequently, the number of trains in the network determines how many layers in the tree belong to the same round where every train gets its \(i\)-th action. For a more comprehensive study on MCTS, see [32].