## 2 Literature Review
### Phase Structure: Incompatibility with DRBPS  
Almost all studies deployed an agent for an intersection, except for a few that involved a regional agent managing the signals across a road network [3; 4; 5; 6; 7; 8]. Researchers have started to train multiple intersection agents synchronously. But from the traffic engineering perspective, there was a remaining issue, that is, the phase structure used was often incompatible with DRBPS at each intersection. This incompatibility widened the gap between research and practical engineering applications. Two main causes of this incompatibility have been identified in the literature.  
(1) Some studies took a variable phase sequence [3; 4; 5; 6; 9; 10; 11; 12; 13; 14; 15; 16; 17; 18; 19], allowing agents to activate any preferred phase combination at every time step, as shown in Figure 2a. But to be frank, an unfixed phase sequence is generally not acceptable in traffic engineering. This is for the sake of safety, as detailed in Section 3.  
(2) Some studies maintained a fixed phase sequence [7; 8; 20; 21; 22; 23; 24; 25; 26; 27; 28; 29; 30; 31; 32; 33] but took a single-ring structure usually with four or two phases, as seen in Figure 2b. Such a structure is less flexible than DRBPS because it forces multiple vehicle movements with imbalanced demand to have the same green time. As a consequence, single-ring phase structures are not common in the real world [2].  
To the best of our knowledge, there were only three studies compatible with DRBPS.  
(1) [PERSON], [PERSON], and [PERSON] [26; 27] developed two MFDRL methods with the deep deterministic policy gradient algorithm. To accommodate DRBPS, a five-dimensional continuous action was defined. Its dimensions determined, respectively, the proportion in a cycle assigned to the phases between barriers, and the split for pairs of these phases. However, such actions required a pre-set cycle length. It could not change according to the variation  
Figure 2: Typical signal operations in previous studies. Solid one-way arrows represent vehicle movements, and two-way arrows represent adjacent parallel pedestrian movements. (**a**) Signal operation with variable phase sequence. An arbitrary phase or phase combination is chosen to be served every time step (usually a few or tens of seconds). (**b**) Signal operation in a single-ring phase structure. Four phases are served in cycles with a fixed sequence.  
of total traffic demand at an intersection. Furthermore, such actions caused signal timing to be adjusted only once per cycle, making it difficult to tackle the fluctuating demand on phases.  
(2) [PERSON] et al. [33] proposed an MFDRL method based on the advantage actor-critic algorithm. The authors presented a stage-based diagram with eight phases, defining an action as whether to switch to the next stage. However, such actions led to the signal operation within a defective DRBPS. The signals were forced to operate in a pre-set stage order. For example, the order might be \"east and west left-turn phases, east through and left-turn phases, east and west through phases\". Consequently, the east left-turn phase was always longer than the opposing west one in every cycle. This prevented the phases from timing flexibly based on their own demand.