## 4 Action Definition with Decision Point Aligning  
To accommodate DRBPS, we improve the action defined in [25] for a fixed phase sequence, which is originally to choose the remaining green time for the current phase at the end of its minimum green. Figure 3: Typical DRBPS at a four-leg intersection. \(\Phi\) represents the phase number. During a phase, the traffic flows corresponding to the arrows shown in the figure are served. For example, when phase 2 is activated, the northbound ‘through’ vehicles and their adjacent parallel north–south pedestrians enjoy green lights. Phases (1 and 5) comprise a pair of concurrent leading phases, and so do phases (3 and 7). Phases (2 and 6) comprise a pair of concurrent lagging phases, and so do phases (4 and 8). We begin with the question of when the agent selects an action. This moment is termed \"decision point\". The question is non-trivial in DRBPS. If simply selecting an action whenever any phase reaches its minimum green like in [25], the action space can be inconsistent at different times, making it hard to determine the semantics of network outputs. This is because the two concurrent phases in a dual ring may reach their minimum greens either simultaneously or successively. Thus, the agent may sometimes need to make a decision for a single phase (corresponding to a small action space), but sometimes two decisions for two phases (a large combinatorial action space). We propose a \"decision point aligning\"-based action definition to address this challenge. It aims to enable the agent to control the two phases with a consistent action space. Meanwhile, it aims to postpone the moment of decision-making as much as possible, so that the agent can base the action selection on the latest state. Specifically, we align the decision points for two concurrent phases in the dual ring at the earliest moment when either or both phases can terminate. At this point (time step \(t\)) to determine the green times for the two phases at once, represented as a 2-tuple:  
\[A_{t}=\left\langle a_{t}^{1},a_{t}^{2}\right\rangle, \tag{1}\]  
\[\forall k\in\{1,2\},a_{t}^{k}\in\mathbb{Z}\text{ and }0\leq a_{t}^{k}\leq G_{ \text{max}}^{k}-G_{\text{min}}^{k}\,,\]  
where \(a_{t}^{k}\) denotes the remaining green time for the current phase in the \(k\)-th ring; and \(G_{\text{max}}^{k}\) and \(G_{\text{min}}^{k}\) are the maximum and minimum greens for the current phase in the \(k\)-th ring, respectively. The aligned decision point (ADP) and action definition can be best understood via the examples below (see Figure 4). 1. A case of two leading phases, \(\Phi 1\) and \(\Phi 5\). DRBPS allows an independent termination of each phase, subject to only its own minimum green. For these two phases, ADP is the moment that either phase's minimum green expires (i.e., the \"ADP1\" moment in the figure). It is the last possible moment to decide whether to give this phase the time beyond its minimum green. At this point, action \(A_{t}\) repres  
Figure 4: Relations between action definition, aligned decision point, interval parameter for dynamic bootstrapping, and DRBPS. Time-step size is one second. The colored blocks share the same explanations as in Figure 3. During a phase, the traffic flows corresponding to the arrows shown in the figure are served. For example, when phase 2 is activated, the northbound ‘through’ vehicles and their adjacent parallel north–south pedestrians enjoy green lights. times after the respective ends of the minimum green durations of the two phases. \(a_{t}^{1}\) and \(a_{t}^{2}\) can be unequal. They denote the lengths of time marked with \(a_{4}^{1}\) and \(a_{4}^{2}\) in the figure, respectively. 2. A case of two lagging phases, \(\Phi 2\) and \(\Phi 6\). DRBPS requires the simultaneous termination of the two phases, subject to both their minimum greens. For these two phases, ADP is the moment when the minimum greens of both phases expire (i.e., moment \"ADP2\" in the figure). It is the earliest possible moment at which the two phases are allowed to terminate simultaneously.