Scaling Safe Multi-Agent Control for Signal Temporal Logic Specifications
Authors: Joe Eappen
Zikang Xiong
Dipam Patel
Aniket Bera
Suresh Jagannathan
Venue: CoRL 2024 (2024)
Keywords: Multi-Robot Systems
Path Planning for Multiple Mobile Robots or Agents
Collision Avoidance
Hybrid Logical/Dynamical Planning and Verification
Deep Learning Methods
TL;DR: Graphical Neural Networks (GNNs) can help scale Specification-driven control on Multi-agent systems beyond existing Mixed Integer Linear Programming (MILP)-based planners.

Abstract: Existing methods for safe multi-agent control using logic specifications like Signal Temporal Logic (STL) often face scalability issues. This is because they rely either on single-agent perspectives or on Mixed Integer Linear Programming (MILP)-based planners, which are complex to optimize. These methods have proven to be computationally expensive and inefficient when dealing with a large number of agents. To address these limitations, we present a new scalable approach to multi-agent control in this setting. Our method treats the relationships between agents using a graph structure rather than in terms of a single-agent perspective. Moreover, it combines a multi-agent collision avoidance controller with a Graph Neural Network (GNN) based planner, models the system in a decentralized fashion, and trains on STL-based objectives to generate safe and efficient plans for multiple agents, thereby optimizing the satisfaction of complex temporal specifications while also facilitating multi-agent collision avoidance. Our experiments show that our approach significantly outperforms existing methods that use a state-of-the-art MILP-based planner in terms of scalability and performance.

Scaling Safe Multi-Agent Control for Signal Temporal Logic Specifications
Joe Eappen, Zikang Xiong, Dipam Patel, Aniket Bera, Suresh Jagannathan
Purdue University
West Lafayette, IN 47907, United States { jeappen,xiong@ee xiong84, dipam, aniketbera, sj aganna) aganna)@purduee @purdue.
Existing methods for safe multi-agent control using logic specifications like Signal Temporal Logic (STL) often face scalability issues. This is because they rely either on single-agent perspectives or on Mixed Integer Linear Program- ming (MILP)-based planners, which are complex to optimize. These methods have proven to computationally expensive and inefficient when dealing with a large number of agents. To address these limitations, we present new scalable approach to multi-agent control in this setting. Our method treats the relationships between agents using graph structure rather than in terms of single-agent per- spective. Moreover, it combines a multi-agent collision avoidance controller with a Graph Neural Network (GNN) based planner, models the system in a decen- tralized fashion, and trains on STL-based objectives to generate safe and efficient plans for multiple agents, thereby optimizing the satisfaction of complex temporal specifications while also facilitating multi-agent collision avoidance. Our experi- ments show that our approach significantly outperforms existing methods that state-of-the the-art MILP-based planner in terms of scalability and performance.
Abstract:
Keywords: Multi-Robot Systems, Path Planning for Multiple Mobile Robots, Collision Avoidance, Specification- Guided Learning, Deep Learning Methods
Introduction
Learning-basee methods have shown promise in multi-agent systems (MAS) for tasks such as colli- sion avoidance, path planning, and task allocation [1, 2, 3, 4J. Extensions have also been developed to handle complex temporal tasks that may be described using formal languages such as Signal Temporal Logic (STL) [5, 6] and other temporal logies [7, 8. 9]; unfortunately, these methods have well-known limitations in terms of scalability and performance.
Signal Temporal Logic (STL) is a formal language for specifying complex temporal tasks that can be used to describe the behavior of agents in multi-agent system. In many settings, including autonomous vehicles [10]. drones [11], and robotic swarms [12]. it is essential to ensure that the agents satisfy complex temporal tasks such as sequentially visiting a series of locations while avoid- ing collisions with each other and the environment. Once the user has specified the task in STL, the task can be synthesized using formal methods [13, 14, 15] in certain environments; however, these methods often struggle to scale to complex specifications and environments. In response these challenges, Mixed Integer Linear Programming (MILP)-based planners [16, 17] have been devel- oped that can be used to plan over a range of STL specifications but still encounter difficulties with collision avoidance when a modest number such as 5 agents are considered (Table I).
Inspired by recent progress in learning-based planners [18, 19, 20), we propose novel approach to planning for multi-agent systems with STL specifications that can scale beyond these limitations demonstrated on up to 32 agents. More specifically, we introduce a Graph Neural Network (GNN) based planner using Neural Ordinary Difference Equations (ODEs) [18] (Fig. 1, Sec. 4.2) trained
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

Agent ②
Goal
kam
Sequence Spec.: Oxse Opapp Pn puaf
Ri Pookes taan Phatoe
G=(V.B)
fact
Radie
GNN
MLP
Update
Planner
ODE
(Left) GNN-ODE Planner Architecture for Multi-Agent Systems with STL Specifica- tions. The planner T generates a sequence of goals for agent given the initial state of the system G(O). The safety controller म; ensures that the agents do not collide while following the generated goals. A GNN encodes the graph representing the collective initial state of the system to yield an initial goal (0))) (red) for each agent i. This goal g, (0) is fed into Multi-Layer Perceptron (MIP) network to generate a new goal 9: (1) (blue) which is fed back into the MLP network in a feedback loop. This repeated for T 1 steps to generate a sequence of goals for the agent. The losses LSTL and Lach are detailed in Sec. 4.1 and are used to update our planner. (Middle) Real world experiments on N drones. (Right) An example trajectory for N agents for seq spec requiring agents to visit A then B and finally C in order.
Figure 1:
end-to-end on STL objective to generate safe and robust plans for multiple agents that can be realized using learnable MA collision avoidance controller ([21), Sec. 4.3). To scale up, we use the ODE-based component to plan general paths that satisfy the given task while using a GNN to model agent interactions in a scalable manner to achieve coordination between the agents as they determine which ODE-generated goal trajectory to follow. Our loss components (Sec. 4.1) allow the planner to find paths that satisfy the STL objective while also being achievable in the presence of collision avoidance maneuvers and agent-to-agent interactions.
Our contributions are as follows: 1) We propose novel scalable GNN-based planner (GNN-ODA
trained on an STL objective to generate safe and achievable plans for multiple agents. 2) We demon- strate the effectiveness of our approach on a range of STL specifications and show that our method can scale to a large number of agents and complex specifications beyond existing methods that use a state-of-the-an MILP-based planner with an average 65% improved success rate.
1.1 Related Work
Symbolic methods have been part of recent resurgence as neuro-symbolic algorithms [22, 23] which aspire combine the generalizability of neural methods with the ability of most symbolic systems to be interpretable and modifiable by human users. Notably, there have been efforts to in- tegrate temporal logic constraints within learning-enabll enabled controllers. In the field of Reinforcement Learning (RL), some examples of this are TLTL [24], which defines a reward function from a logic specification and reward shaping mechanisms, [25, 26] which create automata modeling a similar specification and augment RL-based algorithms used for control. This has been extended to the Multi-agem aueti- domain, which had recent work [27, 9] showing possibilities of coordinating multiple agents with diverging objectives, as well as the benefits of distributing specifications among agents in terms of scalability.
key aspect scaling control to higher-dimensiona. environments and robots involves efficiently incorporating a high-level planner. This involves decomposing complex logic planning from control tasks, allowing each component to focus on its specific role. The high-level planner focuses on logic-level planning, ensuring that the robot's actions adhere to complex specifications, such those defined by STL. In contrast, the low-level controller acts tracker, executing the high-level plan accurately. Modern control methods have demonstrated this benefit well from the burgeoning

---

progress in Hierarchical RL [28, 29, 30, 31, 25] methods as well as the successful integration of classical planners with advanced control schemes, including RI. controllers [32].
Symbolic techniques have appeared in robot motion planning as well with the use of Signal Tem- poral Logic (STL) to specify objectives for multi-robot systems, which can then be solved by MILP solvers [16]. traph-based algorithms [33] or sampling- -ased methods [34, 35]. Collision avoidance in these multi-robot systems is challenging problem since one must also achieve the underlying objectives as well and myriad of techniques [36, 37, 38, 21, 39] have attempted to handle this for general robot motion planning tasks. These existing methods, however, have not considered the gen- crality of symbolic methods in specifying these objectives or quickly fail to scale as the specification dimension, robot complexity and number of agents increases.
Background
Multi Agent Systems with Partial Observability We can represent a multi-agent system with N agents (1, N). Each agent has its own state si()) E S. R", can take action u,(t) E U, C RT, and the collective behavior of the agents is governed by dynamics func- tion (t + 1)
(t), (t)). For simplicity, we assume all agents have the same dynamics function Si = f, state space S. = S, and action space U; = U. A trajectory is a sequence of states T = (5(0), ((L)....... ((T))) where Th is the time horizon, s(t) = (t),..... SN (t)), a(t) = (w (I). (!)) and a policy क, is a function that maps the state agent to an ac- tion u, (s:). The state of the system is partially observable, meaning that each agent can only observe its own state and the states of other agents within its sensing range.
Signal Temporal Logic Signal Temporal Logic (STL) integrates both first-order logic and time- dependent modifications of linear temporal logic operators. The essential logical operators include A (and), - (not), V (or), and = (implies). Time-dependent operators are O (a..)] (eventually between times a and b), [a. (globally between times a and 0), and Uja.oj (until between times a and b). STL formulas are defined as:
= P & Ó à & + 0/a,1o ( | & M,,
where P is predicate function mapping states to real values. Quantitative semantics [13, 40] of STL evaluate robustness value, p( r), which measures how strongly a state trace satisfies or violates の. This robustness metric is differentiable, allowing for direct optimization of STL formulas through differentiable planners like neural networks.
Multi-agent Specification With regards multi multi-agent systems with N agents, the MA-STL spec- ification V is composed from N individual STL specifications A &s where di associates STL. specification for single agent with index i. The MA-STL specification y is satisfied if all individual STL specifications are satisfied and the agents do not collide.
Graphical Representation for Multi-Agent Systems Graph Neural Networks (GNNs) are adept at modeling multi-agent systems by representing agents and obstacles as vertices within a graph G = E). Each vertex in V = V, V, corresponds to either an agent or a static obstacle. Edges encapsulate direct interactions between vertices, with specific emphasis on agent-to-agent and obbsstl connections. We adopt distance-based adjacency criterion where edge (u;, E E exists if the Euclidean distance between vertices and Uj does not exceed predefined threshold R. for capturing the local topology of agents within this range [38]. A GNN processes the graph to produce a global embedding representing the collective state of the system. This global state further processed through specialized readout functions ri, tailored to extract and map the global embedding to specific set of actions u; for each agent [39, 21,38].
Barrier Certificates Barrier certificates [41] are a useful technique to avoid robot collisions in MA systems [42] by forcing the state of the entire system to stay within the safe region. For a state space S C 12" let S, C S be the unsafe set and S. = S²,S, the safe set, which contains the set
3

---

of initial conditions Su C S.. Also, define the space of control actions as 2 C R For dynamic
system s(t) f(s(t), u(t)), ) control barrier function h : R" + R satisfies:
h(s) 20 0 As Su, h(s) < O V' V.h- f(s, u) + a ah((s) 2 0 E 20. 0.
(I)
For control policy ( π 8 U and CBF (l), if (=(0) E 8 (s) 2 0) and the above conditions
are satisfied with (u = (2)), then (s(1) E & h(s) 2 0) for all E 0o). This implies that the state never enters the unsafe set (S.,) under (see [41].
Learning-baat braning--basdd approaches for barrier certificates [37, 39, 38, 21] have been shown scale in the number of agents beyond existing methods for known systems. Notably, a graphical perspective of the agents and their interactions can be used to model the system in a scalable manner (Sec. 4.3).
3 Problem Statement
Consider a MA-STL specification V N agents N (1. N). where each agent is position (t) E P C. I" with n being 2 or 3 for 2D or 3D environments respectively. Assume that each state (t) of agent i can be directly mapped to its position pt))), say the first n elements of s;(t) by a function filterp, based observation of Prays > 0 for each agent measuring the distance the environment with a sensing radius R > 0. The j-th ray of agent i is denoted as (t,, where (t) E R+ is the distance to the nearest obstacle the direction of the j-th ray at time t.
S + I.. Similar to Zhang et al. [21], we include a LiDAR the nearest obstacle in
The MA-STL motion planning problem We now establish the problem of motion planning for MA-STL in multi-agent systems. Essentially, the objective to identify a set of reference goals that when followed satisfy a given MA-STL specification, while ensuring that there are no collisions between the agents. Suppose there are N agents involved, and the time bound denoted by T₁ The planner x generates sequence of goals Tg. (0), ge (1)...... . 9: (T)) for agent i with a given plan length T Th- Each agent has size radius represented by r, where > 0. This means that when an agent is at position p W, it is entirely contained within ball of radius r centered at p, denoted as (p). With these considerations, we can define the planning problem as follows:
Definition 1 (Motion Planning in MA-STL) For a given MA-STL specification Ti = = Φs and a set of N agents N, the motion planning problem is finding distributed control policy Ti and planner zh for each agent i such that the following conditions are satisfied for closed-loop trajec- tories of agents in N with length Tu:
* (Safety- Agents) For all t E [0, TTl, for all i,j where i + j.
2 2r. (Safety- Obstacles) For all t E [w), and all i E N,, yt, (t) 2 2r for all j E [nrayel
(STL Satisfaction) There exists to, f1...., tr such that t; E {0. Th and to < t1 < < lT such that the closed-loop trajectories T (((1)), s((T)) of the agents satisfy the MA-STL specification Vi i.e. p(V,T) 2 0.
* (Achievability) For all E N, given the goal trajectory Th of length T from zg' the gap (s:tttr filter (t)(1) < cfor a small e E RR
Dr,, (T.) = Et=
Scaling STL for Multi-agent Systems Given an MA-STL specification V on a sys- tem of N-agents we would like to provide decentralized algorithm to execute a policy sat- isfying the specification with high probability. While we might assume a plan-then-execute technique [16] that finds a Piece-Wise Linear (PWL) path for each agent with K segments, such an approach quickly fails to scale with specification complexity and number of agents when considering collisions between agents at planning time. We posit this primarily due to its collision avoidance mechanism that introduces O(C)) K2) new variables, which quickly blows up (where C.¹² = N(N 1)/2). Consider two goal regions and B and sequential STL specification requiring agents to visit A (viz. 1 seq) to visit A then B (viz. 2 seq) while
Spec. 1/ Spec. 2 Planning Time (s)
3
seq 2 seq
11 / 211/-
1 seq / seq
Table 1: Planning when considering disjoint time or space [16]. PWL plan with K = 6 segments (1 seq ) / K = 10 segments (2 seq). The X seq spec. has X sequential waypoints.

---

avoiding collisions. Table demonstrates this by timing out (over 50 minutes) for a simple STL specification with N agents in a 2-D environment with Single Integrator dynamics as well as all specifications and number of agents considered in this work (Sec. 5, App- B).
Accounting for collision-avoidance independent of the objective is novel [37, 39], but, argue in this paper, in order to satisfy an STL specification, one must account for the temporal nature of the specification simultaneously with performing any collision-avoidance maneuvers. An alternative, as we propose, is to plan for the objectives while adjusting for collision avoidance by means of an iterative training procedure involving the safety controller (such as GCBF+) and the planner.
4 Approach
Our approach integrates planning, control, and safety mechanisms an end-to-end differentiable learning framework. We first introduce a differentiable STL framework using neural network planner to maximize STL robustness (Sec. 4.1). For efficient multi-agent planning, we employ GNNs to model agent relationships and generate decentralized goal sequences (Sec. 4.2). To ensure collision avoidance, we define a safe set states using GCBFs for robust control (Sec. 4.3). Finally, discuss the training of our integrated system (Sec. 4.4).
4.1 Differentiable Signal Temporal Logic for Planning
Signal Temporal Logic (STL) provides a robustness metric for a given trajectory that quantifies the level of satisfaction of specification & defined using the STL language (Sec. 2). Consider NN planner nh that takes as input the current state of the system and outputs sequence of goals "g To: (g, (0), g: (1), (I)..... 9: (T)) for agent with specification Øi. We can define loss function that attempts maximize the STL robustness score for the specification é, given the waypoints from the planner. Prior work [19, 40] has used the differentiability this score function to directly regularize planner's waypoints for use by a given low-level controller T (8.19))) which is goal-conditioned, Le. targeted to reach the goal 9, given the current state s; of agent i.
For the planner architecture, similar to Xiong et al. [19]. we consider using Th to predict the devi- ation between subsequent waypoints Agi. Based on this, to maximize the probability of satisfying the STL specification given controller ni we define the loss function as:
C Tg m5
E 8 So ~ay (ss), To ((
-ASTLL(OTTT)) A Auch Da, (Ta)
(2)
Esn
Cus
Here consider two loss components, the first being the STL robustness score p(o., of the planned waypoints Tg. and the second being the tracking error Dr., (Ti) of the controller Ti with respect to the planned waypoints. The coefficients ÀSTL, Aach are hyperparameters that control the relative importance of the two loss components (LSTI. and Lach) in the overall loss function. The STL Loss LSIL captures our objective, maximizing the STL robustness score of the planned waypoints Tg. with respect to the specification o; The achievable loss Lach the other hand ensures that the controller गy can track the planned waypoints Tg. using a distance metric D (Tt) (Defn. 1) that extracts the positions from T; using filterp, and minimizes normed distance between the two, i.e. Dr s. (Tt) = Ei ||filter,, (kt)) - filtee,, (1)( where k 0, E Z+ is fixed goal sampling rate during training such that kT = Th. In this paper, we consider the same specification Φ; o and use the same planner for all agents. This enables easy generalization to different numbers of agents during testing and allows for a more scalable approach to planning. We leave the question of how to support different specifications among agents for future work. This
leads to overall loss function for the planner and controller as Lee... MN L
4.2
GNNs for Planning in Multi-Agent Systems
Graphical models can be useful scale collision avoidance in multi-agent systems [39, 21, 38] by modeling the system in decentralized manner. Notably, by representing the agents nodes and
5

---

their interactions as edges, we can use Graph Neural Networks (GNNs) to process a graphical view of the system as described in Sec. (Fig. 1).
To handle the planning problem multi-agent systems we describe the planner zg. We choose a GNN-based planner that takes as input the initial state the system G(O) and outputs an initial goal g0) for agent taking into account the relative positions of the agents. Next we feed this goal go) into a 2-layer MLP to predict the deviation Ag. This process is repeated for T - steps to generate sequence of goals for the agent given the initial state. By using this GNN-based structure, can get this sequence of goals Tg. for each agent in a single forward pass of the planner Tg
As highlighted in Sec. 2, MA-STL can be thought of independent single-agent STL specifications on the agents, albeit with an additional constraint on avoiding collisions between the agents. While collision avoidance during planning time is expensive (Sec. 3), we attempt to plan for the objectives for a subset the agents and use this plan with a safety scheme during run-time. Along these lines, during deployment, that we sequentially visit decentralized manner using the GCBF+ controller (Sec. 4.3). One should note this would not be straightforward if had defined arbitrary STL specifications in the joint space of agents involving global coordination or synchronization of objectives [43, 9].
the GNN-ODE (Fig. 1) to generate sequence of waypoints
However, because this may detract from the overall objective due to collision avoidance maneuvers causing deadlocks, we update the planner iteratively by sampling the environment as detailed in Sec. 4.1 with the STL robustness score. In sense, "co-learn" the safety (GCBF+ controller, T;) and objective (GNN-ODE, () behavior which is a recurrent theme in recent work [19, 32] related to the safety of controllers in complex systems.
4.3 Collision Avoidance in MA Systems
Following [21], we define the safe set S, C SN of NVagent MAS as the MAS states 5 that satisfy the safety properties in Problem 1, i.e.
S, = S SN | (Ilssol > r, Vi E N Vjj Mrays) ^
lp: - -plll 2r)
min LJENN szy
(3)
Then, the unsafe, set of the MAS S,, SN \S is defined the complement of Sg. We now define the notion of GCBF[21]:
Definition 2 (GCBF) A continuously differentiable function h : S. + R is termed as a Graph CBF (GCBF) if there exists an extended class-K.s. function a and control policy म, SM U for each agent E Va of the MAS such that, for all $ E SN with N 2 M,
h(xx.)) + a(h(Ex))) 20, ViEV.
(4)
where for uj TTy (3N;)) and set of neighbours Nr of agent in the MAS within sensing radius R, we have
@h(5M.)
h(3w.)) Σ
f(s,, uz),
(5)
0sj
JEN:
From this definition, as a consequence of the results in Zhang et al. [21], if we find control policy πH and GCBF such that Eq. (4) holds for all agents and all states e S, then the MAS will never enter the unsafe set S,, under the control policy n.
4.4 End-to-End Differentiable Learning for MA-STI
By using the learning framework described in Sec. 4.2 and the safety mechanism Sec. 4.3, we can train the planner and controller in end-to-end differentiable manner using the loss function Eq. (2) (Sec. 4.1). We use iterative training loop to sample trajectories from the environment at different starting conditions and update the planner x for a trained common GCBF+ controller t, π using the loss C
Experiment Setup
Our experiments aim to validate the following two questions:

---

Metric
Planning Time (s)
(R))
Safetty (16)
Success (R)
(stepe) 4
Spec
Planner
N
GNN-ODD ODE
STLPY
GNN-ODE
STLPY
GNN-ODE
SILPY
GNN-ODE
STLPY
GNN-ODE
STLPY
Caver Branch
100,00
357.00
0.04
43.80
100.00
99.00
100,00
53.75
100.00
52.50
745.12
429.81
32
0.03
87.92
95.00
96.00
92.50
20.00
88.12
18.12
820.90
572.39
0.02
10.40
100.00
95.00
97.50
100,00
1062.00
429.76
78.00
96.25
1124.25
536.33
32
0.03
40.19
99.00
80.00
83,00
56.88
84.38
53.75
1252.59
708.22
0.02
26.16
100.00
98,00
100.000
82.50
TRI.00
S00R]
1874,00
10P5529
1927.50
1301.54
0.04
111.62
99.00
100.00
86.25
38.75
N5.62
38.75
2110.80
159K,19
8
0.07
3/16
95.00
98,00
95.00
100.00
95,00
97.50
998.64
637.11
Seg.
0.07
6,96
90.00
89.00
93.75
90.00
86.25
$5.00
1173.44
785.97
13.62
$4.00
64.38
66.88
1013.90
Table 2: Performance of the two planning schemes with the number of agents (N) and specification complexity for the DubinsCar Environment. We note average 65% improved success rate and highlight the best result in bold.
How scalable is neural STL planner over competing methods in terms of the number of agents and specification complexity?
How do the distinct components of our planner (GNN and ODE) help with scalability?
To demonstrate the robustness of our method various specifications and agent models we execute our experiments on the following robot benchmarks: 2D single integrator dynamics (App. C.I), 2D non-linear Dubins Car model (Table 2). 2D Double Integrator dynamics (App. C.3) and real-world 3D drone quadcopter setup moving in fixed 2D plane (App. C.4).
Our framework! was built using JAX [44] based off GCBF+ [21] (Sec. 4.3) with all comparisons using this underlying collision avoidance controller. To demonstrate the effectiveness of our method, compare it against a state-of-the-art MILP-based planner (STLPY [17], Table 2) and an ablation of our planner without the GNN component (labeled ODE, Table 3).
We evaluate the planner a range of specifications: seq, cover, loop and branch. These STL specifications can be drawn to parallels in the real-world. A seq task is akin to a set of drones that need to visit series of locations in specific order given time intervals for logging time-sensitive information. The cover task depicts a scenario where each drone measures a diffcrent sensor reading but must all cover the same locations within time interval to consolidate information. The loop task captures a set of surveillance drones patrolling the same areas. Lastly, consider a scenario where drones are grouped into two separate rooms with two goals present in each. Here branch task could represent a common specification applied to each agent that they must visit the goals of a particular room. For more formal description of the specifications, refer to Appendix B.
We sampled 30 random initial seeds for each experiment and report the mean planning time (in satisfied (Finish Rate), the percentage
seconds), the percentage of runs in which the specification of runs where the agent was safe (i.c. did not collide), the percentage of successful runs for cach specification where the STL specification was satisfied and no collisions occurred, and time-to-reach (TtR) in number of steps (i.e. how long it took for the successful runs to complete the task).
6 Results
Our results (Table 2) demonstrate how differentiable STL can be used to ensure agents achieve complex objectives while avoiding collisions in multi-agent systems.
Scalability in number of agents We first evaluate the scalability of approach in the number of agents (N) for the non-linear DubinsCar environment. From the results, we observe that the success rate decreases gradually the number of agents increases, which is expected as the number of agents increases the complexity of the problem. The results show that the single agent view of STLPY proves unsuccessful especially in the N 32 case where agent interactions are more prevalent. Notably our approach has a planning time that 70-1000x faster than the MIL---aased planner (STLPY) and does not blow up when considering larger number of agents N.
'Code: https///gthhbboommccapppaaassngs
7

---

Metric
Finish Rate (5)
Safety Rate (36)
Success (8)
(steps) 4
Percestage
Percentage
Percentage
Percentage
Spee N
Change
ODE
Change
ODE
Change
ODE
Change
ODE
Cover Branch
97.50
527.21
-1.00
96.00
-2.50
97.50
-625
93.25
-21.50
559.88
32
0.00
95.00
-8.78
84.38
-7.08
81.88
-18.18
671.66
0.00
100.00
0.00
100.00
0.00
100.00
762.57
100.00
802.95
0.00
99.00
7.35
91.25
7.40
90.62
-28.48
895.88
B
0.000
100.00
0.00
100.00
0.00
100.00
-66.3
1568.57
0.00
99.00
0.00
86.25
0.74
86.25
1818.59
8
-26.52
70,00
5.26
100.00
-26.32
70.00
31.31
1298,50
-32 22
61.00
-1.33
92.50
-28.
61.25
15.30
1352.94
-47.19
47.00
-28.98
47.50
7.54
1374.23
Table 3: Considering an ablation without the GNN module for the DubinsCar Environment at vari- ous scales and reporting the percentage change in values. Planning times are comparable.
Scalability in specification complexity For certain specifications such branch and loop, we observe that the MILP planner computation time is significant which can add up over different agent initializations. In contrast, our planner is able to generate a solution for all the specifications quickly and consistently for different agent initial positions, motivating our learning- ceanning-based approach. We further note the effect in TiR when using our algorithm. We rationalize this trade-off because our method finds longer paths that allow goals to be reached by the GCBF+ controller, which is trained to avoid other agents. This inherently reduces the number of collisions which is often greater priority. From ablation study (Table 3) we note the impact of the GNN module especially in terms of 28% impact in success rate for certain specifications such as seq which require increased coordination among agents, We can further reason that the impact in cover and loop of the GNN module is not as great since agents are not required to reach the goals within a strict order and thus require less coordination. With regards to the lower TtR of the successful runs, the lack of GNN module may yield plans that can satisfy the specification efficiently but fail in terms of coordination between agents (affecting the overall success rate).
Limitations
Model-based learning While model-free approach collision-avoidance [45, 32] would be more amenable to handle unknown environment dynamics, our approach is inherently model-based (as is GCBF [38, 21J, MACBF [37] and CAM [39]) This is primarily due the underlying con- troller and GCBF (akin to a barrier certificate), using the next state of the system while calculating the derivative for use in the loss function.
Map Complexity, Homogeneity Additionally, since the approach is decentralized, complex maps requiring communication and coordination between agents may cause safety issues. As mentioned in [21]. it may be hard in dense regions to act in a decentralized manner thus necessitating the use of inter-agent communication. We have considered the homogeneous case in this work, where all agents have the same dynamics and STL specifications. However, in the heterogeneous case, agents may have different dynamics and STL specifications thus needing a more complex controller and a planner capable of generalizing to multiple goal positions or STL specifications. Our planner does not consider obstacles directly, although as demonstrated in the Appendix (Tables 5, 6, 7), the GCBF+ controller to an extent provides inherent collision avoidance capabilities. Finally, the approach is limited by the complexity of the environment and the number of agents. While we have shown that the approach scales well with the number of agents, the complexity of the environment and the number of obstacles may cause the planner to fail find a achievable plan.
8 Conclusion
In this work, we have presented a novel approach to planning for agett systems with Sig- nal Temporal Logic specifications. Primarily we have shown that by using differentiable STL robustness metric, we can optimize for the satisfaction of complex temporal specifications given a controller with MA collision avoidance capabilities. We demonstrate that by training a GNN-ODE planner with a carefully constructed loss function we can overcome the limitations of the plan-then- execute approach and scale to complex specifications and large numbers of agents.
8

---

References
[1] K. Garg, S. Zhang, O. So, C. Dawson, and C. Fan. Learning Safe Control for Multi-Robot Systems: Methods, Verification, and Open Challenges, Nov. 2023. URL http://arxiv.
org/abs/2311. 13714 arXiv:2311.137 [es, eess, math].
[2] T. Huang, S. Koenig, and B. Dilkina. Learning resolve conflicts for multi-agent path finding with conflict-based search. Proceedings of the AAAI Conference on Artificial Intelligence, 35 (13):11246-) 11253, May 2021. 10t./00a..//aaai.... v35i13.17311. 1ttp::110.00.................. URL https://ojs aaai.
org/index: php/ http///rticlleeee////eee 173411
[3] M. Damani, Z. Luo, E. Wenzel, and G. Sartoretti. Primaly: Pathfinding via reinforcement and imitation multi-agent learning lifelong. JEEE Robotics and Automation Letters, 6(2): 2666 2673, 2021.
[4] Y. Li, X. Zhang, Т. Zeng. J. Duan, C. Wu, D. Wu, and X. Chen. Task placement and source allocation for edge machine learning: A gun-based multi-agent reinforcement learning paradigm. IEEE Transactions on Parallel and Distributed Systems, 34(12):3073-300 dec 2023. ISSN 1558-2183.
[5] Wang. S. Yang, Z. An, S. Han, Z. Zhang, Mangharam, M. Ma, and F. Miao. Multi- agent reinforcement learning guided by signal temporal logic specifications. arXiv preprint arXivv:2306. 06808, 2023.
[6] A. L. Forsberg, A. Nikou, A. V. Feljan, and J. Tumova. Multi-agent transformer--ccceeraaed rl for satisfaction of stl specifications, 2024.
[7] N. Zhang, W. Liu, and C. Belta. Distributed control using reinforcement learning with temporal-logic-bate reward shaping. In R. Firoozi, N. Mehr, E. Yel, R. Antonova, J. Bohg, M. Schwager, and M. Kochenderfer, editors, Proceedings of The 4th Annual Learning for Dy- namics and Control Conference, volume 168 of Proceedings of Machine Learning Research, pages 751-762. PMLR, 23-24 Jun 2022. URL https::/ mlr press/v168/ zhang22b.. hhmml.
[8] L. Hammond, A. Abate, Gutierrez, and M. Wooldridge. Multi-Agent Reinforcement Learning with Temporal Logic Specifications. arXtv:2102.00582 [cs], Feb. 2021. URL http://arxiv. org/ http:/ab//......22 00582. arXiv: 2102.00582.
[9] Eappen and S. Jagannathan. DistSPECTRL: Distributing specifications in multi-agent re- inforcement learning systems. In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery Databases (ECML-PKDD), 2022. ISBN 978-3-031- 26412-2.
[10] C. F. Tuncali, Fainekos, D. Prokhorov, Ito, and Kapinski. Requirements-driven test generation for autonomous vehicles with machine learning components. IEEE Transac- tions on Intelligent Vehicles, 5:265-280, 2019, URL httpst:::/ URRLAttp::/app..eeenttti semanticschooarr .rgg////rrorr/////ooooo/o CorpusID: 199442111.
[11] Y. V. Pant, H. Abbas, R. A. Quaye, and R. Mangharam. Fly-by-logic: Control of multi-drone fleets with temporal logic objectives. 2018 ACM/IEEE 9th International Conference on Cyber- Physical Systems (ICCPS), pages 186-197, 2018. URL https: / //ppi.pppp//p semanticscholar. org/ reg/CorpusIDD. 263896988.
[12] R. Yan, Z. Xu, and A. A. Julius. Swarm signal temporal logic inference for swarm behavior analysis. IEEE Robotics and Automation Letters, 4:3021-3028, 2019. URL https://api. semanticscholar. org/Corpussll 195832808.
[13] O. Maler and D. Nickovic. Monitoring temporal properties of continuous signals. In Formal Techniques, Modelling and Analysis of Timed and Fault-Tolerant Systems, pages 152-166. Springer, 2004.
[14] Gutierrez, L. Hammond, A. Lin, M. Najib, and M. Wooldridge. Rational Verification for Probabilistic Systems. In Proceedings of the 18th International Conference on Principles of Knowledge Representation and Reasoning, pages 312-322, 2021. 4o1::00..299333222222133 URL https: /oii org/10 wtgg/1..o 10. 24963/ kr 2021/30.

---

[15] J. Tumova and D. V. Dimarogonas. Multi-agent planning under local Itl specifications and event-based synchronization. Automatica, 70(C):239-24) 2422,, aug 2016. ISSN 0005-1098. doi: 101 ftff:.attoomaiic.200...... 04. 006.
URI. htttps://doi ptttp:://doi.or//10..oo 1016/ htt.://///....otttiitc........ aut.matica. 2016.
[16] D. Sun, J. Chen, S. Mitra, and C. Fan. Multi-agent motion planning from signal temporal logic specifications. IEEE Robotics and Aulomation Letters, 7(2):3451-3458 2022.
[17] V. Kurtz and H. Lin. Mixed-integer programming for signal temporal logic with fewer binary variables. IEEE Control Systems Letters, 2022.
[18] P. Das, A. Dasgupta, and D. Dalal. $ODESsolvers are also wayfinders: Neural ODEs for multi- agent pathplanning. In The Symbiosis of Deep Learning and Differential Equations III, 2023. URL
forumm ior
[19] Z. Xiong, D. Lawson, J. Eappen, A. H. Qureshi, and S. Jagannathan. Co-learning planning and control policies constrained by differentiable logic specifications. In 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024.
[20] F. Nawaz, T. Li, N. Matni, and N. Figueroa. Learning complex motion plans using neural odes with safety and stability guarantees. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 1-8. IEEE, 2024.
[21] S. Zhang. O. So. K. Garg, and C. Fan. GCBF+: A Neural Graph Control Barrier Function Framework for Distributed Safe Multi-Agent Control, Jan. 2024. URL http::/arxiv..o/ abs/2401. 14554. arXiv:2401.1455 145544 [cs, math].
[22] S. Chaudhuri, K. Ellis, O. Polozov, R. Singh, A. Solar-Lezama, and Y. Yue. Neurosym- bolic programming. Foundations and Trends Programming Languages, 7(3):158-22 243, 2021. ISSN 2325-1107. doi: 101:1001561//5500000999 URL http://dx doi. 20g//0.11/ 1561/ 2500000049.
[23] D. Xu, S. Nair, Zhu, J. Gao, A. Garg, L. Fei-Fei, and S. Savarese. Neural task programming: Learning to generalize across hierarchical tasks. In 2018 IEEE International Conference on Robotics and Aulomation (ICRA), pages 1-8. IEEE, 2018.
[24] X. Li, C.-I. Vasile, and C. Belta. Reinforcement learning with temporal logic rewards. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3834- 3839. IEEE, 2017.
[25] R. T. Icarte, T. Q. Klassen, R. Valenzano, and S. A. Mcllraith. Reward machines: Exploiting reward function structure in reinforcement learning. Journal of Artificial Intelligence Research, 73:173-208, 2022.
[26] K. Jothimurugan, R. Alur, and O. Bastani. A composable specification language for reinforce- ment learning tasks. In NeurIPS. 2019.
[27] C. Neary, Z. Xu, B. Wu, and U. Topcu. Reward machines for cooperative multi-agent rein- forcement learning. In AAMAS, 2021. ISBN 978 9781430383073.
[28] J. Yang, I. Borovikov, and H. Zha. Hierarchical cooperative multi-agent reinforcement learning with skill discovery. In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, pages 1566-1574, 2020.
[29] O. Nachum, S. Gu, H. Lee, and S. Levine. Data-efficient hierarchical reinffrcmmmmtt learning. In NeurIPS, 2018
[30] B. Haworth, G. Berseth, S. Moon, P. Faloutsos, and M. Kapadia. Deep integration of physical humanoid control and crowd navigation. In Motion, Interaction and Games, 2020. ISBN 978-1-4503-8811-0
[31] A. Vezhnevets, Y. Wu, M. Eckstein, R. Leblond, and J. Z. Leibo. Options as responses: Grounding behavioural hierarchies in multi-agent reinforcement learning. In International Conference Machine Learning, pages 9733-9742. PMLR, 2020.
10

---

[32] Z. Xiong, J. Eappen, A. H. Qureshi, and S. Jagannathan. Model-free neural Iyapunov control for safe robot navigation. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5572-5579, 2022. doi:10. 11011.0199//00547722222222009999 163.....
[33] A. T. Buyukkocak, D. Aksaray, and Y. Yazıcroğlu. Planning of heterogeneous multi-agent systems under signal temporal logic specifications with integral predicates. IEEE Robotics and Automation Letters, 632::1375-1382 2021. doi:
[34] Y. Kantaros and M. M. Zavlanos. Stylus A temporal logic optimal control synthesis algo- rithm for large-scale multi-robot systems. The International Journal of Robolics Research, 39(7):812-836, 2020. doi: 16:10.117002833422991222 URL https: http://oi.oogg////// 1177/ 0278364920913922.
[35] C. I. Vasile, X. Li, and C. Belta. Reactive sampling- based path planning with temporal logic specifications. The International Journal of Robotics Research, 39(8):0002-1023 2020. doi: 10. 10.1177/02773649229999999
URL https://dog org/ 10. 1177/0077836992011811
[36] J. Chen, J. Li, C. Fan, and B. C. Williams. Scalable and safe multi-agent motion planning with nonlinear dynamics and bounded disturbances. In Proceedings of the Thirty-Fifth AAAI Conference Artificial Intelligence (AAAI 2021), 2021.
[37] Z. Qin, K. Zhang. Y. Chen, J. Chen, and C. Fan. Learning safe multi-agent control with decen- tralized neural barrier certificates. In International Conference on Learning Representations, 2021. URL https: /ttp////ppeeeeeeeeeee/eeeeeeeeee met/forumiid=Pee __
[38] S. Zhang. K. Garg, and C. Fan. Neural Graph Control Barrier Functions Guided Distributed Collision-avoidance Multi-agent Control. Aug. 2023. URL //teee///perrevien /eteeeeeeeeeeeeeeeeeeeeeee net/ forumnP.e-vscddkkggdd.
[39] C. Yu, H. Yu, and S. Gao. Learning control admissibility models with graph neural networks for multi-agent navigation. In 6th Anmual Conference Robot Learning, 2022. URL https:
/openreview. net/forunt (d=
[40] K. Leung, N. Aréchiga, and M. Pavone. Backpropagation through signal temporal logic spec- ifications: Infusing logical structure into gradient-based methods. Int. Journal of Robotics Research, 2022.
[41] A. D. Ames, X. Xu, J. W. Grizzle, and P. Tabuada. Control barrier function based quadratic programs for safety critical systems. IEEE Transactions on Automatic Control, 62(8):3861- 3876, 2017. doi: 10. 10.11099AAC.2201226666666
[42] L. Wang, A. D. Ames, and M. Egerstedt. Safety Barrier Certificates for Collisions-Free Multi- robot Systems. IEEE Transactions on Robotics, 33(3):661-6)) 1-741,4,6,.66 June 2017. ISSN 1941-0468. doi: 1109/TO..001..666666
Conference Name: IEEE Transactions on Robotics.
[43] A. T. Buyukkocak, D. Aksaray, and Y. Yazicroğlu. Planning of heterogeneous multi-agent systems under signal temporal logic specifications with integral predicates. IEEE Robotics and Automation Letters, (12):1375-1382 2021. doi: 109/LRA.2022.3057000
[44] J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang. JAX: composable transfor- mations of Python+NumPy programs, 2018. URL http: // //tgtttttbb.t//////////// httgg//tb..ooogogggggeeeooo com/google//ar
[45] S. Wang, L. Fengb. X. Zheng, Y. Cao, O. O. Oseni, H. Xu, T. Zhang, and Y. Gao. A pol- icy optimization method towards optimal-time stability. In 7th Annual Conference on Robot Learning, 2023. URL httpps://openneveeee.eeee//// httt:/fooumiiddrOOHOO. rid=rOCWUMMBSSII
[46] C. Dawson and C. Fan. Robust counterexample-guidee optimization for planning from differ- entiable temporal logic. In 2022 IEFE/RSJ International Conferennce on Intelligent Robots and Systems (IROS), pages 7205-7212, 2022. 000::00...00///00777222........0
11

---

Contents
1
Introduction
1.1
|Related Work
2
Background
3
Problem Statement
4
Approach
4.1
Differentiable Signal Temporal Logic for Planning
4.2 GNNs for Planning in Multi-Agent Systems
4.3
Collision Avoidance in MA Systems
6
4.4
End-to-End Differentiable Learning for MA-STI.
5
Experiment Setup
Results
7 Limitations
8 Conclusion
A Implementation Details
13
A.I Environment Details
13
STL Specifications
14
Additional Experiments (including Obstacles)
14
C.I SingleIntegrator Environment
14
C.2 DubinsCar Environment
14
C.3 Doublelntegrator Environment
15
C,4 Real-world Drone Experiments
15
D Additional Baselines
16
D.I Following single common plan
16
D.2 Centralized Planner
17
D.3 Signal Specification (demonstrating Until, U)
17
12

---

A Implementation Details
Node features and edge features information specific to each node in our graph observation. Here, we set p. = 3 and use the node features v to one-hot encode the type of the node as either agent node, goal node or LiDAR ray hitting point node. The edge features esy E IRs,, where Pe > 0 is the edge dimension, are defined as the information shared from node j to the agent at node i, which depends i and j. Since the safety objective depends on the relative positions, one component of the edge features is Pij P3 Ps- The remaining edge features can be set depending on system dynamics, such as, relative velocities for double integrator dynamics.
Following Zhang et al. [21], the node features v; Re encode
the states of the nodes
Computation Resources All training procedures were ran on an AWS g4dn x1 xlagge instance or equivalent with 4 Intel Xeon-based CPU Cores and 16 GB of RAM with an Nvidia T4 GPU.
Evaluation Details Since we consider objectives that require agents navigate close one an- other at/near termination subsequently blocking the goal locations (A.B,C,D) C,D.D in Table 4), safety rates were reported until the point an agent had completed their plan. This can be thought of an al- ternative to the agents navigating to a 'safe' position upon completing their specification/plan drone setting, we captured this behavior by landing the drones at an agent-specific location upon completing their specification.
In
Planner Details For all plans, any time step t, planning step t', each agent proceeded to
the next waypoint (t'(' + 1) only when they reached goal 9:'))) within some threshold distance T Tgoot = 0.3 at a time 2 k=('' + +1] where k the goal sampling interval (Sec. 4.1). This allowed all agents to reach the waypoints in the plan without strict time restriction on the plan duration. The asynchronous nature of our plans (among agents) fits our problem description (Defn. 1), specifically the STL Satisfaction criteria. We leave the setting where agents follow synchronized plan to future work. For a given plan of length T with goal sample interval (values in Table 4), the maximum trajectory length horizon during evaluation T3 was 5kT.
A.1 Environment Details
Here, we provide the details of each experiment environment taken from Zhang et al. [21]. We used a common simulation time step 81 0.03 across all three environments.
SingleIntegrator We use single integrator dynamics as the base environment to verify the correct- ness of the implementation and to show the performance of the methods when there are no control
input limits. The dynamics is given as 2; = 04 where 2; = [p] p E I22 is the position of the i-th agent and e = [o e.' its velocity. In this environment, we use Eij = Iz 24 as the edge information.
DoubleIntegrator We use double integrator dynamics for this environment. The state of agent i is given by x, pp uf, " oflT, where [p[ p! is the position of the agent, and [uf, uf| is the velocity. The action of agent i is given by u, [a at) i.c., the acceleration. The dynamics function is given by:
t Juy of ag]
(6)
In this environment, we use Cij = Øÿ - x; as the edge information.
DubinsCar We use the standard Dubin's car model in this enviromment. The state of agent is given by x, = Ip." - f,, u,]" where [p]" the position of the agent, 6, is the heading, and v is the speed. The action of agent is given by u, [wi a, containing angular velocity and acceleration magnitude. The dynamics function is given by:
44 [uz cO,) Us sin( w;, oz]
We use eij ez(xg)
ex()) as the edge information, where es(x;)
[P] P., 14 cos(O,)) 14 sin(O.))
13

---

B STL Specifications
We formally define the Signal Temporal Logic (STL) specifications used in the experiments in Table 4. The specifications include sequential waypoint task (seg), a coverage task (cover). a loop task (loop), and a branching task (branch). The specifications are defined over a time horizon T and are satisfied if the agents satisfy the corresponding STL formula. We use four markers A. B, C, and D to represent rectangular predicates centered around x-y coordinates [0. 0). [2, 2]. [2, 0], and [0, 2]. respectively. The predicates are defined p; dist(s;; P:) S 1.0 where dist (s, Ps) is the L1-norm () (1) distance between the agent i's state s; and the predicate pi.
Spec.
Description
Formula
T
seq cover
Sequence of goals
310 A /3.27
(3 (B)
A (22/3a1(C)
15
20
Coverage over goals
010,Ty fo (B) A (C)
20
loop
Loop over goals
D (O.T/2) (910.T/a) (A) A (1,TT A (10TT/2(())
30
20
signal
Loop then move to final
(loopU fo.1 1) (1)
30
20
branch
Branching
(A) A 110. (B)) V (O)o,)(CC) (D)
20
10
Table 4: STL specifications used in the experiments. T and l are the specification lengths and goal sample intervals respectively.
C Additional Environments (including Obstacles)
In Tables 5, 6 and 7 we show results for the various environments and obstacle scenarios. While our GNN-ODE has an initial GNN module which can observe these obstacles, and is also trained to generate initial goals that 'achievable', the GNN-ODE is inherently limited to only consider obstacles within the sensing radius R of the agents at planning time (i.e. t = 0). As in Zhang al. [21], the GCBF+ controller is trained to avoid obstacles. Thus, with robust plan, we can achieve reasonably high success rates in this setting as well due to the run-time collision avoidance maneuvers. Planning times
nearly similar the results in Sec. 6 (Table 2) likely because our learning-based planners do not use environment dynamics at inference time and should have similar computation cost after training is complete. For this reason, avoid clutter, we omit this column in the following tables. We include results from the ODE ablation of our method as shown in Table under the column 'ODE'. Additional simulation videos are hosted online?
C.1 SingleIntegrator Environment
In Table 5 we contain the results for various combinations of specifications, 8 sampled obstacle positions (marked 'Y' if present, 'N' otherwise), and number of agents in the Singlelntegrator En- vironment. We observe that the GNN-ODE planner outperforms the other planners in terms of planning time and success rate across all the specifications and obstacles. We note the average im- provement in success rate of 10% for our GNN-ODE planner over the MILP planner which is not as large as the improvement in the non-linear DubinsCar environment (Table 2, 6). This is due to the Singlelntegrator environment being less constrained and the MILP planner being able to find feasible solution more easily.
C.2 DubinsCar Environment
In Table 6 we contain the results for various combinations of specifications, sampled obstacle po- sitions (marked" "Y" if present, 'N' otherwise), and number of agents the DubinsCar Environment. On average, with obstacles present as well we get 69% improvement in success rate for our GNN- ODE planner over the MILP planner, primarly due to the non-linear dynamics of the DubinsCar environment being challenging for the collision avoidance controller.
Site: htw.:pppww.giiiibbiooomsllggggge wtwp:stl//////mmma/eee
14

---

Meuri Planner
Finish Rose
Safety Rate
Success Rate
TIR.
Spec
Ubs
ODB
STLPY
GNN-ODE
STLPY
GNN
ODE
ODB
ODE
SILPY
Branch
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
1000.3 75
257.25
16
100000
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
1214.50
443.00
280.8
52
98.00
100.00
99.00
100.00
100.00
98.75
97.50
100.00
97.50
664.71
1005.81
320.23 23
100.00
93.00
98.00
100.00
100.00
95.00
100.00
92.50
92.50
1015.75
95
292.57
Y
16
99.00
96.00
94,00
97.50
100.00
95.00
96.25
96.25
88.75
1168.5 50
471.30
314.99
32
97.00
98.00
94.00
96.12
100,00
92.50
95,00
97.50
86.85
1812.59
1018,77
356.09 09
Cover
95.00
100.00
95.00
100.00
100.00
100.00
97.50
100,00
95,00
884.86
653.00
342.93
16
100.00
90.00
100.00
100.00
100.00
98.75
100.00
90.00
1024.40
758.25
364.56 56
30
98.00 00
97.50
1409
1237,41
95.00
100.00
97.50
1068.46
356. 17
96.00
91.00
100.00
100.00
97.90
96.25
50
91.25
1040
1043.66 66
386 12
32
96.00
98.00
91.00
100.00
99.38
96.88
96.25
97.50
88.75
1491.06
1297.69
488.79
Loop
100000
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
1890.25
2506.00
751.00
16
100000
100.90
100.00
100.00
100.00
100.00
100.00
100.00
100.00
1445 75
2120.12
855. 38
32
100.00
100,00
100.00
100.00
100.00
100.00
100.00
100.00
100.00
2332.19
2635.75
1062 25
100.00
100.03
90.00
95.00
100.00
92.50
95.00
100.00
82.50
1454.00
2112.25
841.32
16
99.00
96,00
93:00
96.25
100.00
96.25
95,00
96.25
88.75
2091 53
2315,44
927.83
96.00
96.25
2453.19
2769.78
1137.71
Sequone
100000
53,00
90.00
100.00
100.00
100.00
100.00
90.00
434.44
16
73.00
100.00
100.00
761.08
1119.07
470.
33.00
66.00
100.00
100.00
100.00
98.12
33.12
65.62
897.41
1464.39
661.21
45.00
88 00.00
160.00
100.00
100.80
97.50
80
731.43
1339.50
470.3 39
16
98.00
36.00
62.00
100.00
100.00
100.00
97.50
36.25
62.50
1030.83
1272.50
$18.39
98.00
28.00
74.00
100.00
100.00
99.38
98.12
28.12
73.12
1186.33
1649.19
797.86
Table 5: Performance of different planner modules with the scalability in the number of agents (N) and specification complexity for the Singlelntegrator Environment.
Motic
Finish Rate
Safety Rate
Success Rate
TIR. 1
Spec
Planner
USs
GNN
ODE
SILPY
GNN-ODE
ODE
STLPY
GNN ODE
ODE
STLPY
GNN-ODE
ODE
SILPY
Branch
100.00
100.00
100.00
85.00
1768.75
357.00
100000
100.00
57.50
53.75
92.75
52.50
1956.12
424.81
32
95.00
94.00
96.00
92.50
86.25
20:00
88.
18.12
870.00
674.52
572.30
100.00
100.00
95.00
97.50
67.50
97.50
50
728.50
562. 79
89
16
99.00
95.00
95.00
92.50
90.00
47.50
91.25
36.25
45.00
1828.95
595. 29
474.16
32
95.00
$5.00
90.00
86.88
74,38
32,50
81.58
63.75
25.00
841.66
708.9
$86.59
Cover
N
100.00
100.00
95.00
100.00
100.00
97.50
100.00
100.00
55.00
1062.00
754,57
429.76
16
100100
100.00
78.00
96.25
97.50
87.50
96.25
97.50
76.25
1127.00
802. 70
536.33
32
99.00
99.00
80.00
85.00
92.50
56.85
84.38
91.88
53.75
1252.59
883,31
708 22
Y
98.00
98.00
93:00
97.50
95.00
92,50
95.00
92.50
85,00
1094.07 07
821.71
460.30 30
16
113561
93.00
93.00
78.00
$1.25
83.12
53.75
75.62
49.38
1251220
972.33
674.09
Loop
100000
98.00
100.00
82.50
1874.00
1570.07
10W2.99
100.00
106.00
100.00
180.00
76.25
100.00
76.25
1963.10
1251
98.00
59.00
100.00
100.00
56.25
38.75
97.50
86.25
38.75
1936.27
1818.59
1598. 19
95.00
$8.00
78:00
100.00
60.00
92.50
95.00
55.00 00.00
70.00
1894.33
4554,25
1310.58
96.00
91.00
90.00
90:00
28.75
66.25
88.75
22.50
57.50
1969.3 37
4481.38
1378. 32
$9.00
91.00
88.00
$0.62
13,75
31.25
76.25
8.12
24.38
2138833
4274,29
1672.91
Seqpoce
N
95.00
70,00
98.00
97.50
100.00
100.00
95.00
70.00
97.50
1226.43
1298.50
637.11
16
95.00
70,00
89.00
96.25
100.00
90.00
92.50
70.00
85.00
1166% 14
1577 48
765.97
32
89.00
70.00
84.00
76.25
77.50
64.38
66.55
63.75
39,38
1277.91
1715.05
1013.90
95.00
67.00
100.00
95.00
1572.6
671 38
75.00
78.75
1175.86
839.37 1640.81
75.00
68.00
77.50
53.75
53.75
61.25
39.38
41.88
1293.39
1802.09
1042.54
Table 6: Performance of different planner modules with the scalability in the number of agents (N) and specification complexity for the DubinsCar Environment with obstacles.
C.3 DoubleIntegrator Environment
In Table 7 we contain the results for various combinations of specifications, 8 sampled obstacle (Obs) positions (marked 'Y' if present, 'N' otherwise), and number of agents (N) in the DoubleIntegrator Environment. The average improvement in success rate of 1 11% for our GNN-ODE planner over the MILP planner is similar to the Singlelntegrator environment (Table 5) due to GCBF+comtroller being more effective at collision avoidance with the linear dynamics of the Doublelntegrator environment.
C.4 Real-world Drone Experiments
The experimental validation of this methodology involved deploying a fleet of DJI Tello Ryze drones to track the trajectories generated via the Dubins Car model. The drones were configured in
15

---

Meuric Planner
Finish Rose
Safety Rate
Success Rate
TIR. 1
Spec
UBs
ODB
STLPY
UNN-ODD
STLPY
GNN
SILPY
Brinch
100.00
100.00
100.00
100.00
100.00
100.00
100.00
100,00
100.00
1384.7 75
536.00
379.30
16
100000
100.00
91.00
100.00
100.00
100.00
100.00
100.00
91.25
1768.38
1577.50
474.44
52
99.00
91.00
73.00
100.00
100.00
100.00
99.35
90.62
72.50
2510 85
2236644
517.02
100.00
100.00
98.00
97.50
100.00
100.00
97.50
100.00
97.50
2774 50
533.50
390
16
99.00
99.00
94,00
98.75
98.75
100.00
97.50
91.50
93.75
2923.95
1594.22
533.49
32
99.00
91.00
68.00
96.25
93.75
98.12
95,00
85.62
67.50
2543.79
2263,76
666.90 90
Cover
100.00
100.00
93.00
100.00
100,00
100.00
100.00
100,00
92.50
1681.00
738.00
572.20
16
100000
100.00
89.00
100.00
100.00
100.00
100.00
88.75
2127.43
645.93
1540.20
877.30
100.00
100.00
97.50
100.00
1649.5 50
767.50
498.07
100.00
89.00
98.75
100.00
97.50
98.75
2123.35
976-75
95.00
79.00
78.00
95.00
96.25
98.75
90.52
76.25
77.50
1630.34
1625.60
949.62
Loop
16
95.00
95.00
99.00
98.00
100.00
180.00
100.00
100.00
100.00
100.00
100.00
100.00
95.00
97.50
97.50
98.75
100.00
100.00
2612.70
1951.71
2716.54
$273.00
1219.75
1781 62
32
98.00
93.00
96.00
100.00
100.00
100.00
98.12
93.12
96.25
3271
$260.64
2703.45
95.00
96,00
100.00
100.00
95.00
100.00
95.00
92.50
100.00
2553. 52
27855.0 64
1229 75
95.00
59.00
96.00
100.00
96.25
47.50
96.25
95.00
95.00
2713.45
3431,33
1850 21
97.00
93.75
3353.944
2850.15
Seqjone
100000
90,00
85,00
100.00
100.00
100.00
1565.5
1162.35
693.67
100.00
100.00
1667
1472.54
926
81.00
19.00
28.00
100.00
100.00
100.00
1979.83
912.03
100.00
160.00
100.80
180.00
1952.75
1380.57
846
16
100.00
$4.00
60.00
100.00
98. 98.75
97.50
100.00
$2.50
60.00
1684.75
1574.84
960 90
52
84.00
51.00
34.00
99.38
93.12
93.75
85.12
49.38
33.12
1969.00 688
2003..88
930.35
Table 7: Performance of different planner modules with the scalability in the number of agents (N)
and specification complexity for the Doublelntegrator Environment.
WiFi mode to enable swarm behavior which was facilitated through the open-source DJITelloPy library.
Each Tello drone is equipped with an Inertial Measurement Unit (IMU), a forward-facing camera, and a downward-facing camera. The latter is useful for precise hovering and position estimation using the Vision Positioning System (VPS). However, this system inaccurate and unreliable as the drones do not possess other sensors like lidar depth cameras. To mitigate drift and correct the position estimate errors, ArUco tags were utilized to make the trajectory following robust for each drone. This ensured the swarm of drones could accurately follow the designated trajectory as evidenced in the simulation results.
D Additional Baselines
We focus on the most complex non-linear environment as presented in the main paper (DubinsCar) and have presented the results over 30 initial seeds in Table 8 and 9 and included bar charts with error bars for the main experiments in Figures 2 and 3. We chose this format as including all values in the tables created excessive clutter.
D.1 Following single common plan
Since the tasks are homogenous, one might attempt to follow common plan among agents with a formation. To compare this, we include results on a common shared plan generated by the STLPY solver [17] (marked STLPY (S)) for the specifications considered Table 8 and 9. The initial starting state of this plan was near the first predicate in each specification (viz. goal A).
An obvious benefit of single plan generated by the STLPY solver is a reduced planning time. Using a single common plan also enforced a rudimentary level of coordination among agents as evident in the branch task reducing the number of crossing paths (present in the earlier individually generated STLPY plans). However, in long horizon specifications such as loop and signal this effect detrimental causing increased safety violations (leading to a low overall success rate) due to lack of more informed coordination among agents. We posit the fixed predicate sizes further play a role in preventing this coordination scheme succeed.
16

---

Plazzing Tine (0
Secocs Role
TIR.
Parner
CE (40)
SILPY
SILPY (5)
(46)
GNN-CDE
46]
GNN-CDE
STLPY
9.28
9:11
0.01
22.50 05
1.51
K7.50
190.0
56.15
$5.20
1000
21
20k25
513.9
357J
S3593
16:75
0/1
0.03
65.01
1.55
80.00
18640
96.57
+2.50
98.1
63981
2403
156.6
S82.4
35.82
57.92
1.5
55.65
83.10
$1.35
18.12
51.33
$31.00
251.32
572-39
683 15
255
95.00
98:75
100.00
105-25
49.7%
36.73
8/82
0.63
20.14
2.58
76.25
97.50
99.28
76.25
83.31
115950
1113.7
795.99
536.33
959.38
$2.98
0.03
0.02
40.19
2.61
$2.50
$1.77
$9.36
$3.75
$4.90
1339.53
1241.22
$90.52
710
1218.11
$3.27
52.79
4.58
98.96
96.25
59.33
355.35
1819.1
1301.4
1379
107.17
0/43
0.03
111.63
4.41
0.25
97.60
12.50
36.75
26.35
3111.50
1951
6475.85
1996.19
1940 58
0.81
1-45
0.56
95.00
97.92
11.50
V1.30
100.00
104-00
1298.13
88274
83.99
0:83
6.85
15.22
0.88
48.12
40.35
59.76
03.23
130298
1236
1378.35
181590
1254.7
56,50
0,82
0.81
13.00
17.83
0.00
188.00
100,00
97.92
100.00
2538 50
4179.25
576.53
1150.25
18.23
196.00
91.25
1552.87
1290.72
1851.21
Table 8: Performance of STLPY [17]: (multi-agent), STLPY(S) [17] STLPY with a single com- mon plan, CE [46] Centralized Counterexample guided planner and our approaches (ODE, GNN- ODE). vs the number of agents (N) and specification complexity for the DubinsCar Environment. The results are averaged over 30 seeds and we highlight the best result in bold.
D.2 Centralized Planner
We found that the challenges in scalable collision avoidance modeling with the PWL MA-STL planner [16] (discussed in Section 3) were also present in other centralized STL planning strategies [46]. Namely, directly adding predicates for collision avoidance into the STL specification proved intractable for optimization over the long planning horizons considered due an O * K2) variable blowup for N agents and planning horizon K. To address this following proposed method, we attempt planning for the joint multi-agent task without considering collisions during planning, while introducing run-time collision avoidance schemes like GCBF+. As result, in the simulation step of [46] (Alg. 1, line 4), the GCBF+policy would be incorporated into the rollout. However, in our experiments, we found that the gradient vanishes too quickly (within 50 time steps) to meaningfully differentiate through the trajectory, which spans over 200 time steps for all our tasks (Table 4, Appendix B). Consequently, the algorithm in [46] cannot directly handle the longer-range STL tasks considered in our work. Additionally, we noted that the proposed approach was not robust to many initial starting positions, often leading to unfruitful plans.
To build centralized planner with [46), we opted to run the proposed method without incorporating collision avoidance during planning time simulations. This choice allowed us to achieve longer opti- mization horizons within a limited initial starting position range. The resulting plans were executed using the trained GCBF+ controller. The results, as shown marked CE in 8 and 9 for the DubinsCar Environment, indicate that the proposed approach performed demonstrably worse than ours and the other compared methods (note Success Rate). The planning time of this centralized approach scaled linearly with the number of agents since agent-agent interactions were not considered. These results further point to the importance of an achievability loss (Lach, Sec. 4.1) when planning in the pres- ence of run-time collision avoidance. This outcome reinforces the notion that current methods for centralized planning in multi-agent STL requires further advancements to effectively handle scalable collision avoidance.
D.3 Signal Specification (demonstrating Until, U)
In Table 8 we include signal specification demonstrating the Until operator. This is variant of our loop specification (Table 4, Appendix B) called signal with an additional predicate Vi, representing reaching goal A twice. If loop is represented as L1, and V represents reaching new goal D, the signal specification is (D.U%,.1) 1 A D2 i.c. loop A, B, and C until A is reached twice, then reach D.
17

---

Finish Rate ?
Safety Rate
Planner
CE (46)
GNN-ODE
ODE
STLPY
STLPY (S)
CE
GNN-ODE
ODE
STLPY
STLPY (S)
100,00
27.19
16
32
98.00
99.00
98.00
80.00
99.00
53.75
$1.99
91.04
56.88
35.21
12,00
100.00
100.00
95:00
100.00
100.00
100.00
97.
63.50
$5.83
16
10,00
99.00
100.00
99.00
99,00
53.75
100.00
96.25
67.50
$9.79
32
11.00
99.00
100.00
100.00
99.00
28:12
98.88
72.50
58.75
27.29
Sequeie
100.00
99.00
76.00
95.00
160.00
95.00
99.17
99.17
160,00
100,00
100,00
Table 9: Finish Rate and Safety Rate of STLPY [17]: (multi-agent), SSLPY(S) [17] STLPY with a single common plan, CE [46] : Centralized Counterexample guided planner and our approaches (ODE, GNN-ODE). Environment. Note that a successful trajectory is both finishing the task and being safe. While STLPY and other single agent planning schemes may reach the goal quickly, safety is violated (shown in the Safety Rate). The results are averaged over 30 seeds and we highlighted the best result bold.
the number of agents (N) and specification complexity for the DubinsCar
18

---

DubinsCar Success Rate
Tarmer
DabinsCar Safety W
1ell
Pan
GRO OCC
DubinsCar Planning Time (s)
CE 1481
100
Figure 2: We provide bar plots with error bars notating the standard deviations of the metrics considered. These are complementary Table 8 and Table
19

---

Dab TER
400
Ranner
CRN oct
2000
2000
Spenal
Coopr
Loop
DubinsCar Finish Rate M-
100
[48] Ga0--00
Vaa
STUPY
Figure 3: We provide bar plots with error bars notating the standard deviations of the metrics considered. These are complementary to Table 8 and Table 9.
20


Reviews:
[{"review_id": "3meFJZ0gfM", "reviewer": "Reviewer_HZ5o", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper proposes a multi-agent planning framework using Signal Temporal Logic. The proposed method relies on a graph neural network to generate plans in a decentralized manner. A comparison against mixed-integer linear programming-based planning is made.", "summary_of_recommendation": "The motivation of the paper is good and I appreciate the hardware experiments. Some technical details need to be addressed as it directly impacts the claim from the authors.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "(1) Why is safety treated separately from STL satisfaction? One can directly specify safety and liveness through an STL formula, which can be encoded as part of robustness, i.e., a measurement of how safe the agents are.\n\n(2) The \"co-learning\" of the safety (GCBF+) and objective controllers (GNN-ODE) is not clear to me. Are there cases where the GNN-ODE controller and GCBF+ controller cause the planner to be unable to find a solution? If yes, how do you address this?\n\n(3) In Table 2, the STLPY's safety rate is decreasing dramatically. Can you explain this in more detail? If your method is using GCBF+ and GNN-ODE, is the baseline algorithm also using some sort of CBF-based safety checking mechanism?\n\n(4) The STL formula used for scenarios (seq, cover, loop, branch) does not contain the \"Until\" operator. Can you show an example that includes it?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KRtIWAX7JT", "reviewer": "Reviewer_rCpB", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper concerns a problem of multi-agent planning where each agent shares the same dynamics and the STL specifications of their objectives. The problem is decoupled into two learning problems, one that learns the neural network planner that learns to achieve the STL objective, and the other that learns the Graph CBF to ensure that each agent does not collide to each other and the obstacles.", "summary_of_recommendation": "The method only works under strong assumptions, however, under this setting, I think the presentation of the paper and its originality are both very clear.", "technical_quality": 3, "clarity_of_presentation": 3, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "The author claims that by decoupling the problem of STL specification and safety, the proposed method does not suffer the poor scalability that other methods that solve STL suffer. However, in order to achieve this, the method operates under a strong assumption of homogeneous dynamics and objectives. Under this strong assumption, wouldn’t other methods’ scalability also improve?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IgKx55UEjT", "reviewer": "Reviewer_uqbs", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes a hierarchical architecture for multi-agent control with STL constraints, wherein a GNN and recurrent MLP predict a set of high-level waypoints that are tracked using a neural graph control barrier function controller. This combination is trained end-to-end on different environments and achieves better empirical scalability than MILP-based solution methods.", "summary_of_recommendation": "I think this paper would be well-received at CoRL, but there are a number of minor things the authors should be sure to address.", "technical_quality": 4, "clarity_of_presentation": 3, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "Please respond to my questions in the review above.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper received 3 confident reviews. It proposes a hierarchical architecture for multi-agent control with STL constraints, wherein a GNN and recurrent MLP predict a set of high-level waypoints that are tracked using a neural graph control barrier function controller. This combination is trained end-to-end on different environments and achieves better empirical scalability than MILP-based solution methods.

To address in rebuttal (in addition to reviewer comments):
- Discuss limitations on homogeneity, noted by several reviewers (dynamics, as well as STL specs)
- Address technical queries highlighted by reviewers
- Add training details
- Discuss insights obtained on real robot experiments

Post-rebuttal:
The reviewers provided key clarifications. The experimental analyses should ideally be extended in support of claims.
Decision: Accept