## 9 Conclusions  
This study proposes an MFDRL method, termed MD4-AQF, to address the RTSC problem within the context of DRBPS. The purpose is to narrow the gap between research and traffic engineering practices since DRBPS is quite commonly used in real-world intersections. To manage several challenges posed by DRBPS, we make three major contributions:  
1. We present a \"decision point aligning\"-based action definition for the concurrent phases in two rings. This approach produces a consistent action space, which is used to determine the green times for both phases simultaneously. 2. We present a \"multi-dimensional Q-network\"-based training algorithm against the large action space containing 600+ permutations of green times for two phases. It can transform hundreds of combinatorial actions into tens of sub-actions. 3. We present \"aligned Q-fusion\" embedded in the algorithm to ensure that two concurrent lagging phases end simultaneously. This approach can fuse paired sub-action values and generate one remaining green time shared by the two lagging phases. The results of simulation experiments indicate that MD4-AQF can help an agent learn useful knowledge about signal control. Its best-performing policy achieves a median average vehicle delay of 47.89 s during training. The trained agent outperforms another MFDRL method by 14.4%, and a conventional method by 19.0%, in terms of the median delay. More interestingly, we believe MD4-AQF, in essence, provides universal solutions to some general control problems as follows. Determining the green time for two concurrent phases is essentially a multi-degree-of-freedom (DOF) control problem, where some DOFs operate in parallel but not strictly simultaneously. Our solution is to align the decision points of DOFs with the latest moment at which any DOF must make a decision. Then, the action should be defined as the move of each DOF after its own decision deadline. Therefore, an agent can control several concurrent DOFs at the same time. 2. Learning too many combinatorial actions for two phases is essentially a problem of the \"curse of dimensionality\" caused by controlling multiple DOFs at once. In such a case, the number of actions grows exponentially with the number of DOFs. Our solution is to perform dimension expansion on a DQN-style algorithm to produce multi-dimensional sub-action values. Each sub-action should correspond to the choice of a specific DOF. By doing so, the number of Q-network outputs can become only linear with the number of DOFs. 3. Ensuring simultaneous ends of two lagging phases is essentially a problem where all moves of DOFs are required to be identical in some states. Our solution is to align and fuse sub-action values. This is done by averaging the values of the same sub-action over different dimensions. Then, all DOFs should share and take the sub-action that maximizes the average, allowing them to move in sync. Although this study focuses primarily on MFDRL-based RTSC at a single intersection, the proposed MD4-AQF method can be seamlessly applied to multiple individually operating intersections within a road network. Given that our MD4-AQF targets the most complex eight-phase DRBPS at a four-leg intersection, it can also be adapted to simpler structures with fewer phases at T-intersections or other types of intersections. However, to achieve more effective coordinated control between multiple intersections, developing communication between the states and the actions of intersection agents may be necessary. Many challenging issues remain to be adequately addressed before MFDRL-based RTSC methods can be successfully applied in practice; for example, the \"reality gap\" between the dynamics of simulated and real intersections, as well as the reliable generalization of knowledge across varying intersection geometries and traffic demand patterns.