Neural Graph Control Barrier Functions Guided Distributed Collision-avoidance Multi-agent Control
Authors: Songyuan Zhang
Kunal Garg
Chuchu Fan
Venue: CoRL 2023 Poster (2023)
Keywords: Distributed control
Control barrier functions
Graph neural networks

Abstract: We consider the problem of designing distributed collision-avoidance multi-agent control in large-scale environments with potentially moving obstacles, where a large number of agents are required to maintain safety using only local information and reach their goals. This paper addresses the problem of collision avoidance, scalability, and generalizability by introducing graph control barrier functions (GCBFs) for distributed control. The newly introduced GCBF is based on the well-established CBF theory for safety guarantees but utilizes a graph structure for scalable and generalizable decentralized control. We use graph neural networks to learn both neural a GCBF certificate and distributed control. We also extend the framework from handling state-based models to directly taking point clouds from LiDAR for more practical robotics settings. We demonstrated the efficacy of GCBF in a variety of numerical experiments, where the number, density, and traveling distance of agents, as well as the number of unseen and uncontrolled obstacles increase. Empirical results show that GCBF outperforms leading methods such as MAPPO and multi-agent distributed CBF (MDCBF). Trained with only $16$ agents, GCBF can achieve up to $3$ times improvement of success rate (agents reach goals and never encountered in any collisions) on $<500$ agents, and still maintain more than $50\%$ success rates for $>\!1000$ agents when other methods completely fail.

Neural Graph Control Barrier Functions Guided Distributed Collision-avoidance Multi-agent Control
Songyuan Zhang Department Aeronautics and Astronautics Massachusetts Institute of Technology szhang210mit edu
Kunal Garg
Department of Aeronautics and Astronautics Massachusetts Institute of Technology kgarggmit. edu
Chuchu Fan
Department of Aeronautics and Astronautics Massachusetts Institute of Technology
chuchu@mit.ee edu
Abstract: We consider the problem of designing distributed collision-avoidancee multi-agent control large-scale environments with potentially moving obstacles, where large number of agents required maintain safety using only local information and reach their goals. This paper addresses the problem of collision avoidance, scalability, and generalizability by introducing graph control barrier functions (GCBFs) for distributed control. The newly introduced GCBF is based on the well-established CBF theory for safety guarantees but utilizes a graph structure for scalable and generalizable decentralized control. We use graph neural networks learn both neural GCBF certificate and distributed control. We also extend the framework from handling state-based models directly taking point clouds from LiDAR for more practical robotics settings. We demonstrated the efficacy of GCBF in variety of numerical experiments, where the number, density, and traveling distance of agents, as well as the number of unseen and uncontrolled obstacles increase. Empirical results show that GCBF outperforms leading methods such as MAPPO and multi-agent distributed CBF (MDCBF). Trained with only 16 agents, GCBF can achieve up 3 times improvement of success rate (agents reach goals and never encountered in any collisions) on 500 agents, and still maintain more than 50% success rates for 1000 agents when other methods completely fail.'
Keywords: Distributed control, Control barrier functions, Graph neural networks
Introduction
Multi-agent systems (MAS) can complete much more complex tasks efficiently as compared to single-agent systems such reconnaissance sensor coverage of large unexplored area. Safety of MAS, in terms of collision and obstacle avoidance, is a non-negotiable requirement in the numerous autonomous robotics applications (see [1] for an overview) such as a swarm of drones flying in a dense forest [2, 3]. multi-object configuration and manipulation warehouses [4, 5, 6] and autonomous driving [7, 8, 9]. In addition, the agents are required to either follow a pre-defined path or reach a destination for completing their individual or team objectives. With the increase in the number of robots in the MAS, it becomes difficult design control policies for all the agents for such multi-task problem as the computational complexity grows exponentially with the MAS scale [10].
Common ageet motion planning methods include solving mixed integer linear programs (MILP) for computing safe paths for agents [11, 12] and RRT-based methods [13]. However, they are not
Project website:
webssttt
7th Conference Robot Learning (CoRL. 2023), Atlanta, USA.

---

scalable to large-scale MAS. Multi-agent Reinforcement Learning (MARL) approaches, e.g., Multi- agent Proximal Policy Optimization (MAPPO) [14), have also been adapted to solve the problems. However, most of the MARL works model safety as a penalty rather than a hard constraint and thus, cannot guarantee safety. In recent years, safety constraints have been handled via control barrier functions (CBFs) [15]. Particular for MAS, generally CBF is assigned for each safety constraint, and then approximation method is used for accounting for the multiple constraints [16, 17, 18, 19]. Their issue is that it is very difficult to construct a handcrafted CBF for large-scale MAS consisting of highly nonlinear dynamics. The Multi-agent Decentralized CBF (MDCBF) framework in [20] uses neural network-based CBF designed for MAS, but they do not encode a method of distinguishing between other controlled agents and uncontrolled agents such as static and dynamic obstacles. This can lead to either conservative behavior if all the neighbors are treated as non-cooperative obstacles, or collisions if the obstacles are treated as cooperative controlled agents. Furthermore, they use discrete approximation of the time derivative of the CBF but do not account for changing graph topology, which can lead to wrong evaluation of the CBF constraints and consequently, failure. We provide further empirical evidence of a significant drop in the performance of MDCBF as the number of agents increases through various numerical experiments. The Control Admissiblity Models (CAM)-based framework in [21] also attempts to address a similar problem. However, it involves sampling control actions from a set defined by CAM and cannot always find feasible control input.
To overcome these limitations, in this paper, we present novel Graph CBF for large-scale MAS to address the problem of safety, scalability, and generalizability. We propose starning-based control policy to achieve higher safety rate in practice. We use graph neural networks (GNN) to better capture the changing graphical topology of distance-based inter-agent communications, We also use LiDAR-based observations for handling unseen and potentially nonstationary obstacles in real-world environments. With these technologies, our proposed framework can generalize well to many challenging settings, including more crowded environments and unseen obstacles.
We consider two 2D and one 3D environment in our experiments. In the obstacle-free case, we train with 16 agents and test with over 1000 agents. For < 500 agents, our method achieves a threefold improvement in safely reaching tasks, while for large-scale experiments (> 1000 agents) where the baselines achieve close to 0 success rate, our approach achieves 50% - 100% success rate. In the obstacle environment, we consider only 16 point-sized obstacles in training, while in testing, we consider up to 32 large obstacles. We see over 15% improvement in success rate over baselines. The experiments corroborate that the proposed method outperforms the existing methods in successfully completing the tasks in various 2D and 3D environments. Our contributions are summarized below:
We introduce Graph CBF (GCBF), a kind of barrier function for MAS to encode collision avoidance constraints and to handle different types of agents and obstacles.
We use GNNs to jointly learn a GCBF and a distributed controller which is robust to the changes of neighbors, and LiDAR-basee observation model for obstacles.
Empirical performance shows a significant improvement by our GCBF over other leading approaches, especially in difficult settings.
Related work Graph-based planning approaches such as prioritized multi-agent path finding [22], conffict-based search for MAPF [23] can be used for multi-agent path planning for known environ- ments. However, these offline path-planning methods cannot be used for dynamic environments or large-scale systems due to computational complexity. Another line of work for motion planning in ob- stacle environments is based on the notion of velocity obstacles [24, 25] defined using collision cones for velocity. Such methods can be used for large-scale systems with safety guarantees under mild assumptions. However, the current frameworks under this notion assume single or double integrator dynamics for agents. The work in [26] scales to large-scale systems, but it only considers discrete action space and hence does not apply to robotic platforms that use more general continuous input signals. Works such as [27, 28, 29] address this problem using GNNs for generalization to unseen environments and are shown work on teams of up hundred agents. However, they not scalable to very large-scale problems (c.g., .. a team of 1000 agents) due to the computational bottleneck.

---

In recent years, the most commonly employed method of solving safe motion planning problems involves neural CBF-based approaches [20, 30, 31, 32]. Machine learning (MI.)-based approaches have shown promising results in designing CBF- controllers for complex safety-critical systems [30, 31, 32). The NN-CBF framework consists of model-based learning [33, 32, 34, 35] or model-free learning [36, 37, 38). Our approach uses a model-based learning framework, and in contrast to the aforementioned works, applies to MAS. Utilizing the permutation-invariant methods have been employed for problems involving MAS [28, 27, 39, 21, 40, 41, 42, 29, 43). These prior work only consider static obstacles, or do not consider the presence of obstacles or uncontrolled agents at all. On the other hand, there is also lot of work on MARL-based approaches with focuses on motion planning [44, 45, 46, 47, 48, 49, 14, 50, 51, 52]. But these approaches cannot provide safety guarantees due to the reward structure and as argued in [53], MARL-based methods are still in the initial phase of development when it comes to safe multi-agent motion planning.
property, GNN-based
2 Problem formulation
We consider the problem of designing a distributed control framework to drive N agents V, == {1, ,....,, ]} to their goal locations while avoiding collisions. The motion of each agent is governed by general nonlinear dynamics i; (z,, ut), where x, R" and u, R" are the state, control input for the i-th agent, respectively and F; R - R is assumed to be locally Lipschitz continuous. Here, the vector 2; consists of the position p: along with other state variables such as speed, orientation, etc. Note that it is possible to consider heterogeneous MAS where the dynamics of agents are different. However, for simplicity, we restrict our paper to the case when all the agents have the same underlying dynamies, i.e., F; = F for all i. The environment also consists of stationary or dynamic obstacles O₃ for ls E (1.2..... .M) }. where O₂ represents the space occupied by obstacle k. The control objective for each agent is to navigate the obstacle environment to reach its goal location while maintaining safety. We use a LiDAR-based observation model similar to [33], which can be directly used for real-world robotic applications. The observation data consists of Plays evenly-spaced rays originating at the robot and measures the relative locations of objects in its sensing radius. The observation data for agent is denoted by y: E dimension of the environment. The collision avoidance requirement imposes that each pair of agents maintain a safety distance 2r where r > 0 is the radius of a circle that can contain the entire physical body of each agent. also requires that each agent maintains a safe distance from other obstacles in the environment. Furthermore, each agent has limited sensing radius R. We define the neighbor agents of agent i Noa {j Va [|p. pyll R. * i), and the neighbor obstacles of agent as N/O {k || < R}. Therefore, the agents can only sense other agents or obstacles in the set of their neighbors N₂- Nr U.N The formal statement of the problem is given below.
x where is the
Problem 1 Given set of N agents of safety radius r, sensing radius R and set of non-colliding goal locations {pgaa B design distributed control policy <T< Ti, Dt, x. 2000))) for each agent i, where I; is the conglomerated states of the neighbors j E Noo and Vs the conglomerated observations from No, such that the following holds for the closed-loop trajectories of the agents:
Collision avoidance: llu? (t)]l r, Vj. i.e., the agents do not collide with the obstacles; [p:(() - -r;(!)] > 2r for all 1 2 0, j # i.e, the agents do not collide with each other;?
* Liveness: inf [l):(t) goal = 0, i.e., each agent eventually reaches its goal location pooa
3 Methodology
Noticing that the agents, the hitting points of LiDAR rays, and the information flow between them can be naturally modeled as a graph, we propose a novel graph CBF (GCBF) which encodes the collision-avoidane constraint based on the graph structure of MAS. We use a nominal controller
In the rest of the paper, we omit the argument i for the sake of brevity.
3

---

for the liveness requirement and use GNNs to learn the GCBF jointly with the Dollision-aviidang controller. During application, the GCBF is used to detect unsafe scenarios and switch between the nominal controller and the learned controller for collision avoidance. Our GNN architecture is capable of handling a variable number of neighbors and so, it leads to a distributed and scalable solution to the safe MAS control problem.
We start by briefly reviewing the notion of CBF commonly used in literature for safety requirements [15]. For given closed safe set S C I"," an unsafe set Su C IR" such that S S = 0, a function h: IR" + TR is termed as a CBF if there exists a class-KC function a such that the following holds:
h(z) > 0 Vv E int(S), h(x) < Vr E Su: and sup Lyh(s, u) a(((x)) Vr E E,, (I)
where T,yh(z)= == 0h (ax) is the Lie derivative of the function h along J. and int(S) denotes the relative interior of closed set S. The existence of a CBF implies the existence of control input which keeps the system safe. Based on the notion of CBF, we define a new notion of graph CBF (GCBF) for encoding safety in MAS. Before formally introducing GCBF, we briefly review the basics of the graph structure. A directed graph is defined as G = (V, F) where V is the set of nodes and E= (i1, i is the set of edges representing the flow of information from node i2 to ij. For the considered MAS, the nodes consist of agents Va and the hitting points V, of LiDAR rays in their observations, and hence V = V. U V.- The edges are defined between each of the observed points and the observing agent when the distance between them is within the sensing radius R. Since the flow of information is from the observed point to the observing agent, the set of edges E = V. x V. We use GNN to represent GCBF, so we first define node and edge features for GCBF.
de
Node features and edge features The nodes features v, in GCBF encode the type of the agent with 2/ 0 for controlled agents (i.e., the agents that operate under the commanded controller) and v = 1 for uncontrolled agents (i.e., the hitting points for LiDAR rays). The edge features Giy are defined as the information shared from node j to agent i, which depends on the states of node j and node i Since the safety objective depends on the relative positions, one of the edge features is the relative position Pij. The rest of the features can be chosen depending on the underlying system dynamics, e.g.. relative velocities for double integrators, and relative headings for Dubin's cars. For brevity, we use & = (ein Cijzs.... 1 (UNM! Cii mesyy)) with é; E IRP for some p > 0, and (+15 Uy to represent the collected edge and node features T = (U),, u....... VI 1 for agent i. Here, use égj for ş Nr and 8g for the rays l & Nr with constant values that the sizes of the vectors Ezg, U2 remain fixed. Now we are ready to introduce the notion of GCBF.
Definition 1 (GCBF) A function h : RP {0, NNN-- + R is termed as a Graph CBF (GCBF) if there exists a class-K. function a such that he,, 0.) > 0 Vx, E S., h(e, 0.) < V22 E Su,s, and h(e, 0z) 2 -a(h(e,, 0,))
V x E S,, (2)
where St - fax (l > r, Vk Brays) N min llp: - pyll > 2r) } is the safe set for agent i and Su,; the unsafe set such that S, n Su,s 0.
je jeV
Note that can have variable-size inputs. Since it is not possible to mathematically define function with variable-size input, we use fixed-size input (ē,, ūy) so that we can define the input domain of function h. Since the node features are constant, h is computed with respect to the edge features as
GNN to model GCBF with input (eij...... Cijyy and (Uj...... Uiiy, since GNN
al((e, U.) de: i(,,, 0.) de, dz,
F(x,, us) + ah(ei, 0z)
De:
F(E;, Uz).
de;
Dz,
JEN.
Note that while choosing the edge features é; it is important make sure that the time derivates of the features of agent include the control so that the input can help keep the system safe. Thus, we
(ak(E,,5.) DE, F(,,, ut)) * O. This is similar to assuming Lih & for CBF in (1) where
assume Du. de, dz.
A monotonically increasing continuous function R+ + IR+ with (0) is termed Class-K..

---

0024
Information
2016
Neighbers
0,17
0.000
3.00
Node features
Aftestionn
Coacetenntion
-0,008
Edge features
-0.036
11
-0,032
Figure 1: Left: the contours of the learned GCBF of the agent with diamond mark, where the blue boundary is the O-level set of the GCBF. Orange circles are agents and black dots are obstacles. The weights the edges show the attention values. Right: the overview of the proposed framework.
F(z, u) f()) + g(r)u, which is very common (see [15]). Under this assumption, we can state the following result on the safety of the system under GCBF (the proof is given in Appendix C.I).
Theorem Given set of N agents, assume that there exists a GCBF satisfying (2) for some class-K. function a. Then, the resulfing closed-loop trajectories of agents with non-colliding initial conditions under any smooth control input u, U,tufe = {u EU;; h(e,, 0:) + a(h(E, o,)) 2 0} satisfy xz (1) E for all i E Va and t 0.
Safe control policy For the multi-objective Problem 1, we use a hierarchical approach for the goal-reaching and the safety objectives. First, we design a nominal controller un!om = noom (x,, 2x? for the goal-reaching objective. In this work, we use LQR and PID-based nominal controllers. Next, using the nominal controller, we design minimum-nome controller that satisfies the safety constraint using an optimization framework. With GCBF h, a solution to the following optimization problem:
goal
min
|
u nom
(4a)
willl:
ah(a, ū2)
0c, dz.
ah(a,, 0.)
de: F(z,, uz) 2 -a(h(e, B,)),
s.t.
F(I,, u4)+ + Σ
(4b)
dE,
de, dx,
JEN,
keeps agent i in its safety region. Note that (4) is not distributed framework for finding the control policy, since the constraint for computing depends on uj. Thus. it is not straightforward to solve (4) in a distributed manner, although there is some work on addressing such problems [54]. To this end, we use an NN-based control policy that satisfies the safety constraint and does not require solving centralized non-convex optimization problem online. Next, we discuss the training setup for jointly learning both GCBF and distributed safe control policy (see Figure 1).
Neural GCBF and distributed control policy training We parameterize candidate GCBF hi as NNs with parameters 0. The NN contains a GNN backbone and a multilayer perceptron (MLP) head. In the backbone, each connected edge {i.j} first goes through an MLP fe., which encodes the edge feature eig and the node features vy to latent space, i.c., 9i = for (esg, u2). Then, we use attention [55] to aggregate the information of the neighbors, i.e., 45 = EjEN, softmax(fé, (qt)) (e,)), (4:)). where fo, and Sag are two NNs parameterized by O2 and 0g. Jog often called "gate" NN in literature [56), and the output of softmax(fa ff,, (q is called "attention" which is scatter value between 0 and 1 for each agent) j E N,; represents how critical agent j is to agent i. We discuss later the necessity of applying attention. After the backbone, aggregated information is processed by the head with parameters 04 to get the GCBF value for each agent, i.e., h; = fo, (4:)-
We design the neural distributed controller as u, NN (e, ūg T nom (I,, x goal)) The distributed control policy Ta is an NN with similar structure as GCBF, designed for collision and obstacle avoidance. The GNN backbone of ng the same as the GNN backbone of he. except that n also uses nom as its feature. This helps the NN controller leam how to modify the agent's behavior given a nominal policy to avoid collisions with the neighbors and obstacles. Thus, we concatenate the nominal control signal u.om with the output of the GNN component as the input to the MLP component of T (see Figure 1). Note that the input to the control policy is only the local information
5

---

(ēg, B.), and unlike (4), it does not require knowledge of neighbors' inputs. In this way, the controller is fully distributed, and thanks to GNN's ability handle variable sizes of inputs, Te generalizes to larger graphs with much more neighbors. We train the GCBF and the distributed controller by minimizing the empirical loss C = Liev. Li, where L; is the loss for agent i defined as
0) -E
1 ha(es, 0.) alhaa(es, a:)]
b Eb
ha D;)]*
(5)
Σ by ho(e,, Bi)l + [o(es ue, Tnom (1x, (I,, 2.500))) Toom (Tz, x Roal #1ESu,
+
where [:] max(:,)) 0), and > is hyper-parameter to encourage strict inequalities. The training data { collected over multiple scenarios and the loss calculated by evaluating the CBF conditions on each sample point. We use the on-policy strategy to collect data by executing the learned controller ne with probability
6, and the nominal controller Tnom with probability c, where decreases linearly from to 0 during the training. In this way, we avoid the low-quality data generated by u, N at the beginning of the training and match the train-test distributions at the end of the training. The first three terms in (5) correspond to the GCBF conditions in (2), while the last term encourages small control deviation from nuom so that Te can have better goal-reaching performance with 1) 0 balance the weight of the GCBF constraint losses and the norm of the resulting input.
One of the challenges of evaluating the loss function L is estimating he. Similar to [21], we estimate he by (t+(2) 0s (f++1)) (t, (x(t)))) /5t, where St = + f& is the simulation timestep. However, the discretized approximation may cause issues if the graph connections change between any two consecutive time steps. Fortunately, the attention mechanism we use naturally addresses this problem. During training, the agents learn to pay more attention (i.e., close to 1) to nodes that are near and less attention (i.e., close to 0) to the nodes that are at the boundary of the sensing region. Therefore, if an edge breaks in between time steps and node gets out of the sensing radius, the CBF value does not change significantly. In this manner, the estimation of he does not encounter large errors due to edge changes between time steps. Note that hy includes the inputs from agent i as well as the neighbor agents j E M, Therefore, during training, when descent and backpropagate (, o). the gradients are passed to not only the controller of agent i but also the controllers of all neighbors in N₁ For the class- function a, we simply use a:(h) where a > 0 is a constant. More details on the training process are provided in Appendix A. and a discussion on the generalization result of neural GCBF provided in Appendix C.2.
gradient
h,
GCBF detector and online policy refinement When the training finishes, we can execute our controller in fully distributed manner. To achieve better goal-reaching performance, we use the learned GCBF as a detector to detect unsafe scenarios and use a switching control policy to reduce potential conservatism due to only using learned policy gg. In particular, we define the control assignment for each agent as u, u. 2.om if wuu E Lt and u, uNN, otherwise. Namely, at each time step, the system uses the nominal controller if the GCBF conditions (2) are satisfied with the nominal controller ueem. If not, it switches to the learned policy uNN for collision avoidance.
While the control policy Ty is designed satisfy the GCBF conditions (2), the GCBF conditions can still be violated because of various reasons, such as distribution shift in testing and difficulty in exploring the state-space in high-dimensions and large-scale MAS during training. To this end, similar to [20], we use an online policy refinement technique to make the learned policy safer. At given time instant, if the learned policy nH does not satisfy the GCBF conditions (2), we compute the
residue 6(, NN max (0.7 7 - ho 0/)) - ahho(E;, 0z)) and use gradient descent to update the control policy ne until 6(uNN) = or the maximum iteration is reached.
Experiments
Environments We conduct experiments three different environments consisting of a SimpleCar modeled under double-integrator dynamics, a DubinsCar model, and a Drone modeled under lin- earized drone dynamics (see Appendix A.2 for more details). Both car environments
2D while the drone environment is 3D. The parameters are It = 1, r = 0.05, u. = 0.8 in the 2D environments,
We re-emphasize the fact that during testing, the neighbors' inputs not required for na.

---

GCBF
MDCBF
MAPPO
SimpleCar
Drone
DubinsCar
Increase density
0.5
Success rale
Keep
0.5
density
0
Keep distance
0.5
128
256
512
1024
512
1024
2048
4096
32
128
256
$12
1024
Number of aagnnss
Figure 2: Success rate of GCBF, MDCBE, and MAPPO algorithms across the three environments and the three sets of experiments, namely, increasing density of the agents in a fixed workspace, increasing the size of the workspace to keep the density same, and increasing the size of the workspace but limiting the average distance traveled by agents.
and R = r 0.05, = 0.6 for the 3D environment, where uss is the maximum speed of the agents. The workspace X' 1" of each of the environments is a hyper-rectangle of side-length 1 > 0. The total timesteps of experiments are 2500 for 2D environments and 2000 for 3D environments.
Evaluation criteria We use safety rate, reaching rate, and success rate as the evaluation criteria for the performance of a chosen algorithm. The safety rate is defined as the ratio of agents not colliding with either obstacles or other agents during the experiment period over all agents. The reaching rate is defined as the ratio of agents reaching their goal location by the end of the experiment period. The success rate is defined as the ratio of agents that are both safe and goal-reaching. We note that the safety metric in [21] is slightly misleading as they measure the portion of collision- free states for safety rate. For each environment, we evaluate the performance over 16 instances of randomly chosen initial and goal locations from the workspace X' for 3 policies trained with different random seeds. Here, we report the mean success rate and their standard deviations for the 16 instances for each of the 3 policies. We report the safety rate, reaching rate, and ablation results in Appendix B.
Baselines We use MDCBF [20] and MAPPO [14] the baselines for comparisons (see Appendix A.3 for implementation details of the baselines). MDCBF learns pair-wise CBFs between agents and takes the minimum on one agent as the CBF value of this agent. Furthermore, it considers each neighbor equally important without attention and does not use CBF as detector but directly uses the learned controller. MAPPO is MARL-based algorithm that learns be safe and goal-reaching by maximizing the expected reward. For fair comparisons, we re-implement MAPPO using GNN.
Experiment settings We conduct four sets of experiments demonstrate the scalability, gener- alizability, and reliability of the proposed method. First, we fix the workspace size X' where the agent trajectories evolve. In this experiment, we use X' 32 x 32 for 2D car environments and X' 16 x 16x x 16 for the 3D drone environment and perform experiments with up to 1024 agents for the 2D environments and up to 4096 agents for the 3D environment. In the second set of experiments, we keep the per-unit agent density constant. To this end, we increase the size of X' as the number of agents increases from 16 4096 (see Appendix A.2 for workspace sizes). In the third set of experiments, we further constrain the maximum traveling distance to 4.0 units for each agent while increasing the size of the workspace keep the per-unit agent density constant. In the fourth set of experiments, we introduce moving obstacles where we perform experiments in the DubinsCar environment with up to 32 obstacles and 64 agents in a workspace X' 12 x 12. The obstacles are assumed to be moving with a bounded, constant, unknown speed up to 0.2 units and the size of the obstacle varies between to 0.5 units. Agents use LiDAR to detect obstacles. Each agent generates equally-spacee 32 rays with maximum sensing radius R = 1.0 unit. For the first three experiments, we train all the algorithms with 16 agents, and for the fourth experiment, we train with 64 agents and with 16 randomly generated point- sized obstacles to model LiDAR observations.
7

---

1.0
Success rate
0.9
GCBF
0.8
MDCRF
0.7
8
32
Number of obstacles
Figure 3: Left: Illustration of the DubinsCar environment with obstacles. The green circles are goal points and the red rectangles are obstacles. The solid blue line shows the connection between agents and the orange lines show LiDAR rays. Right: Success rate plots for GCBF and MDCBF
Results Figure 2 shows the performance of the proposed framework (GCBF) against the baselines MDCBF and MAPPO. In all the experiments, the success rate of GCBF is higher than that of the considered baselines. Particularly as the number of agents increases, the decrement in the success rate of MAPPO and MDCBF is very high. For the SimpleDrone environment, we notice that there is almost no drop in the success rate with an increase in the number of agents. We speculate that this because the agents in 3D have more degrees of freedom move to avoid collisions and hence, achieve very high safety rate (see the individual safety and goal-reaching plots in Appendix B). For the first two of experiments, the success rate drop primarily because the inter-agent interactions are increasing. In the first set of experiments, it is clear with an increase in the density of agents for fixed workspace, the inter-agent interactions increase. For the second set of experiments, although the per-unit agent density is the same, with an increase in the workspace size, the average distances traveled by the agents randomly generated initial and goal location instances also increase. Thus, the inter-agent interaction increases. We designed the third set of experiments to further analyze effect of traveling distance on success rate. the third set of experiments, not only the density but also the average distance traveled by each agent is fixed, which keeps the number of inter-agent interactions constant. The success rate of GCBF remains very close to with minimum success rate of GCBF is 97.1% % for SimpleCar, 99.7% for DubinsCar, and 99.5% for SimpleDrone. Figure illustrates that the proposed method using GCBF achieves a higher success rate across obstacle environments compared to MDCBF since it can deal with different types of neighbors. The success rate MAPPO with obstacles is consistently lower than 0.1, so we do not include it in the plot.
Limitations
In the current framework, there is no cooperation among the controlled agents, which leads to conservative behaviors. In certain scenarios, this can also lead to deadlocks. Another limitation is the assumption of knowledge of the neighbors' velocities. From a practical point of view, measuring relative position is possible using LiDAR or other sensors, but accurate estimation of other agents' velocities and accelerations not possible. Similar to any other NN-based control policy, the proposed method also suffers from difficulty in providing formal guarantees of correctness. In particular, it is difficult, if not impossible, to verify that the proposed algorithm can always keep the system safe via formal verification of the learned neural networks (see Appendix in [57] on NP-completeness of NN-verification problem). These limitations inform our future line of work on relaxation of the assumption available information, introducing cooperation among agents to reduce conservatism, and looking into methods verification of the correctness of the control policy.
Conclusions
In this paper, introduce new notion of GCBF to encode inter-agent collision and obstacle avoidance in control for large-scale multi-agent systems with LiDAR-basee observations, and jointly learn it with a distributed controller using GNNs. The proposed control framework is completely distributed as each agent only uses local information in its sensing region, and thus, is scalable to large-scale problems. Experimental results demonstrate that even when trained on small-scale MAS, the proposed method can achieve higher success rates in completing goal- reaching tasks while maintaining safety for large-scale MAS even in the presence of dynamic obstacles.
8

---

Acknowledgments
This work was partly supported by the National Science Foundation (NSF) CAREER Award #CCF- 2238030 and the MIT-DSTA program. Any opinions, findings, conclusions or recommendations expressed in this publication are those of the authors and don't necessarily reflect the views of the sponsors.
References
[1] C. Ju, Kim, Seol, and Son. review on multirobot systems in agriculture. Computers and Electronics in Agriculure, 202:107336, 2022.
[2] Y. Tian, K. Liu, K. Ok, L. Tran, D. Allen, N. Roy, and J. P. How. Search and rescue under the forest canopy using multiple uavs. The International Journal of Robotics Research, 39010-11): 1201-1221, 2020.
[3] K. A. Ghamry, M. A. Kamel, and Y. Zhang. Multiple uavs in forest fire fighting mission using particle swarm optimization. 2017 International Conference on Unmanned Aiveraft Systems (ICUAS), pages 1404-1409. IEEE, 2017.
[4] B. Li and H. Ma. Double-deck multi-agent pickup and delivery: Multi-robot rearrangement in large-scale warchouses. IEEE Robotics and Automation Letters, 8()):3701-3708, 2023.
1 100.00010000.RA2222322277777
[5] A. Kattepur, Rath, A. Simha, and A. Mukherjee. Distributed optimization multi-agent robotics for industry 4.0 warchouses. In Proceedings of the 33rd Annual ACM Symposium Applied Computing, pages 808-815, 2018.
[6] A. Kmjaic, D. Thomas, G. Papoudakis, L. Schäfer, Börsting, and S. V. Albrecht. Scalable multi-agent reinforcement learning for warehouse logistics with robotic and human co-workers. arXiv preprint arXiv: 1498, 2022.
[7] L. M. Schmidt, J. Brosig, A. Plinge, B. M. Eskofier, and C. Mutschler. An introduction to multi-agent reinforcement learning and review of its application autonomous mobility. In 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), pages 1342-1349, IEBE, 2022.
[8] Palanisamy. Multi-agent connected autonomous driving using deep reinforcement learning. In 2020 International Joint Conference Neural Networks (IJCNN), pages 1-7. IEEE, 2020.
[9] M. Zhou, J. Luo, J. Villella, Y. Yang, D. Rusu, J. Miao, W. Zhang, M. Alban, I. Fadakar, Z. Chen, et al. Smarts: An open-source scalable multi-agent rl training school for autonomous driving. In Conference on Robot Learning, pages 264-285. PMLR, 2021.
[10] M. Brambilla, E. Ferrante, M. Birattari, and M. Dorigo. Swarm robotics: review from the swarm engineering perspective. Swarm Intelligence, 7:1-41, 2013.
[11] J. Chen, J. Li, C. Fan, and B. C. Williams. Scalable and safe multi-agent motion planning with nonlinear dynamics and bounded disturbances. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11237-11245. 2021.
[12] R. J. Afonso. M. R. Maximo, and R. K. Galvão. Task allocation and trajectory planning for multiple agents in the presence of obstacle and connectivity constraints with mixed-integer linear programming. International Journal of Robust and Nonlinear Control, 30(14):5464-544 2020.
[13] Netter, G. P. Kontoudis, and K. G. Vamvoudakis. Bounded rational rıt-qx: Multi-agent motion planning in dynamic human- like environments using cognitive hierarchy and q-learning. In 2021 60th IEEE Conference on Decision and Control (CDC), pages 3597-3602. IEEE, 2021.
[14] C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. Wu. The surprising effectiveness of ppo in cooperative multi-agent games. Advances Neural Information Processing Systems, 1-24624, 2022.

---

[15] A. D. Ames, S. Coogan, M. Egerstedt, G. Notomista, K. Sreenath, and P. Tabuada. Control barrier functions: Theory and applications. 2019 18th European Control Conference (ECC), pages 1420-3431. IEEE, 2019.
[16] Glotfelter, J. Cortés, and M. Egerstedt. Nonsmooth barrier functions with applications to multi-robot systems. IEEE Control Systems Letters, 1(2):310311,, 2017.
[17] M. Jankovic and M. Santillo. Collision avoidance and liveness of multi-agent systems with chf-based controllers. In 2021 60th IEEE Conference on Decision and Control (CDC), pages 6822-6828. IEFE, 2021.
[18] R. Cheng, M. Khojasteh, A. D. Ames, and J. W. Burdick. Safe multi-agent interaction through robust control barrier functions with learned uncertainties. In 2020 59th IEEE Conference on Decision and Control (CDC), pages 777-783. IEEE, 2020.
[19] K. Garg and D. Panagou. Robust control barrier and control дyaрunov functions with fixed-time convergence guarantees. In 2021 American Control Conference (ACC), pages 2292-2297. IEEE, 2021.
[20] Z. Qin, K. Zhang, Y. Chen, J. Chen, and C. Fan. Learning safe multi-agent control with decen- tralized neural barrier certificates. In International Conference on Learning Representations, 2021. URL https: / hptp///eppeeeeeeeeeeeeeeee net net/forumiid-PG _1Brxxx
[21] C. Yu, H. Yu, and S. Gao. Learning control admissibility models with graph neural networks for multi-agent navigation. In Conference on Robot Learning, pages 934 945. PMLR, 2023.
[22] H. Ma, D. Harabor, P. J. Stuckey, J. Li, and S. Koenig. Searching with consistent prioritization for multi-agent path finding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7643-7650, 2019.
[23] G. Sharon, R. Stern, A. Felner, and N. R. Sturtevant. Conflict-based search for optimal multi- agent pathfinding. Artificial Intelligence, 219:40 66, 2015.
[24] J. Van den Berg, M. Lin, and D. Manocha. Reciprocal velocity obstacles for real-time multi- agent navigation. In 2008 IEEE international conference on robotics and automation, pages 1928-1935. leee, 2008
[25] S. H. Arul and D. Manocha. V-rvo: Decentralized multi-agent collision avoidance using voronoi diagrams and reciprocal velocity obstacles. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8097-8100 IEEE, 2021.
[26] L. Zheng, J. Yang, H. Cai, M. Zhou, W. Zhang, Wang, and Y. Yu. Magent: A many-agent reinforcement learning platform for artificial collective intelligence. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
[27] E. Tolstaya, J. Paulos, V. Kumaa, and A. Ribeiro. Multi-robot coverage and exploration using spatial graph neural networks. In 2021 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages
IEEE, 2021.
[28] Q. Li, F. Gama, A. Ribeiro, and A. Prorok. Graph neural networks for decentralized multi-robe path planning. In 2020 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 11785-11792. IEFE, 2020.
[29] C. Yu and S. Gao. Reducing collision checking for sampling-based motion planning using graph neural networks. Advances Neural Information Processing Systems, 34:4274 2021.
[30] R. Cheng, G. Orosz, R. M. Murray, and J. W. Burdick. End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3387-3395, 2019.
[31] C. Dawson, Z. Qin, S. Gao, and C. Fan. Safe nonlinear control using robust neural Iyapunov- barrier functions. In Conference on Robot Learning, pages 1724-1735. PMLR, 2022.
10

---

[32] A. Robey, H. Hu, L. Lindemann, H. Zhang. D. V. Dimarogonas, S. Tu, and N. Matni. Learning control barrier functions from expert demonstrations. In 2020 59th IEEE Conference on Decision and Control (CDC), pages 3717-3724. IEFF, 2020.
[33] C. Dawson, B. Lowenkamp. D. Goff, and C. Fan. Learning safe, generalizable perception-based hybrid control with certificates. IEEE Robotics and Automation Letters, 7(2):1904-199) 2022.
[34] M. Saveriano and D. Lee. Learning barrier functions for constrained motion planning with dynamical systems. In 2019 TEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages I 112-119. IEEE, 2019.
[35] M. Srinivasan, A. Dabholkar, S. Coogan, and P. A. Vela. Synthesis of control barrier functions using a supervised machine learning approach. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7139-7145, IEEE, 2020.
[36] Z. Qin, D. Sun, and C. Fan. Sablas: Learning safe control for black-box dynamical systems. IEEE Robotics and Automation Letters, 7(2): 1935,, 2022.
[37] A. Taylor, A. Singletary, Y. Yue, and A. Ames. Learning for safety-critical control with control barrier functions. In Learning for Dynamics and Control, pages 708-717. PMLR, 2020.
[38] A. J. Taylor, D. Dorobantu, H. M. Le, Y. Yue, and A. D. Ames. Episodic learning with control Iyapunov functions for uncertain robotic systems. In 2019 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages
IEEE, 2019.
[39] Blumenkamp, S. Morad, J. Gielis, Q. Li, and A. Prorok. A framework for real-world multi- robot systems running decentralized gnn-based policies. In 2022 International Conference on Robotics and Automation (ICRA), pages 8772-8778. IEEE, 2022.
[40] X. Jia, L. Sun, H. Zhao, M. Tomizuka, and W. Zhan. Multi-agent trajectory prediction by combining egocentric and allocentric views. In Conference Robot Learning, pages 1434 1443. PMLR, 2022.
[41] X. Ji. H. Li. Z. Pan, X. Gao, and C. Tu. Decentralized, unlabeled multi-agent navigation in obstacle-rich environments using graph neural networks. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8936-8943. IEEE, 2021.
[42] A. Khan, A. Ribeiro, V. Kumar, and A. G. Francis. Graph neural networks for motion planning. arXiv preprint arXiv:2006. 06248, 2020.
[43] Q. Li, W. Lin, Z. Liu, and A. Prorok. Message-aware graph attention networks for large-scale multi-robot path planning. IEEE Robotics and Automation Letters, 6(3):5533-5540) 2021.
[44] X. Xiao, B. Liu, G. Warnell, and P. Stone. Motion planning and control for mobile robot navigation using machine learning: a survey, Autonomous Robots, 46(5):569-597, 2022.
[45] S. H. Semnani, H. Liu, M. Everett, A. De Ruiter, and J. P. How. Multi-agent motion planning for dense and dynamic environments via deep reinforcement learning. IEEE Roboties and Automation Letters, 5(2):3221 -2226, 2020.
[46] B. Wang, Z. Liu, Q. Li, and A. Prorok. Mobile robot path planning in dynamic environments through globally guided reinforcement learning. IEEE Robotics and Automation Letters, 5(4): 1932.6939, 2020.
[47] W. Zhang, O. Bastani, and V. Kumar. Mamps: Safe multi-agent reinforcement learning via model predictive shielding. arXiv preprint arXiv: 1019900.266. /2639, 2019.
[48] M. Everett, F. Chen, and J. How. Motion planning among dynamic, Docision-making agents with deep reinforcement learning. In 2018 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3052-3059, IEEE, 2018.
[49] Z. Dai, T. Zhou, K. Shao, D. H. Mguni, B. Wang, and H. Jianye. Socially-attentive policy optimization multi-agent self-driving system. In Conference on Robot Learning, pages 946-955. PMLR, 2023.
11

---

[50] X. Pan, M. Liu, F. Zhong, Y. Yang, S.-C. Zhu, and Y. Wang. Mate: Benchmarking multi reinforcement learning in distributed target coverage control. Advances in Neural Information Processing Systems, 33:27862-27879, 2022.
[51] Z. Cai, H. Cao, W. Lu, L. Zhang, and H. Xiong. Safe multi-agent reinforcement learning through decentralized multiple control barrier functions. arXiv preprint arXiv:2103.12552 2021.
[52] B. Wang, J. Xie, and N. Atanasov. DarlIn: Distributed multi-agent reinforcement learning with one-hop neighbors. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9003-9010. IEEE, 2022.
[53] Y. Wang, M. Damani, Wang, Y. Cao, and G. Sartoretti. Distributed reinforcement learning for robot teams: a review. Current Robotics Reports, 3(4):239-22) 20,399257,,0000
[54] M. A. Pereira, A. D. Saravanos, O. So, and E. A. Theodorou. Decentralized safe multi-agent stochastic optimal control using deep FBSDEs and ADMM. Robotics: Science and Systems, 2022.
[55] Y. Li. C. Gu, T. Dullien, O. Vinyals, and P. Kohli. Graph matching networks for learning the similarity of graph structured objects. In International Conference on Machine Learning, pages 3835-3845. PMLR, 2019.
[56] Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel. Gated graph sequence neural networks. arXiv preprint arXiv: 1.05493, 2015.
[57] G. Katz, C. Barrett, D. L. Dill, K. Julian, and M. J. Kochenderfer. Reluplex: An efficient smt solver for verifying deep neural networks. In Computer Aided Verification: 29th International Conference, CAV 2017, Heidelberg, Germuny, July 24-28, 2017, Proceedings, Part I 30, pages 97-117. Springer, 2017.
[58] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint ar 1r2iv:1412.6980, 2014.
[59] C. Budaciu, N. Botezatu, M. Kloetzer, and A. Burlacu. On the evaluation of the crazyflie modular quadcopter system. In 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), pages 1189-1195. IEEE, 2019.
[60] L. Lindeemann and D. V. Dimarogonas. Control barrier functions for signal temporal logic tasks. IEEE Control Systems Letters, 311; 2018
12

---

Appendix
A
Experiment details
In this section, we introduce the details of the experiments, including the implementation details of GCBF and the basclines, the dynamics of the agents, and the settings of the environments.
A.1 Implementation details
Our learning framework contains two neural network models: the GCBF hg and the controller Tg. both of which contain GNN component and an MLP component. The GNN component of ha consists of three NNs: the embedding NN fe. the NN used in the attention aggregation fez and fø.. fo, is a two-hidden-layer MLP with 2048 neurons each, which maps the input to a 256D information space. The gate NN fo, a two-hidden-laye MLP with 128 neurons each. fog two-hidden-laye MLP with 2048 neurons each, which maps the information to 1024D space. The output of the GNN component of hg goes through another MLP component fo,, which a seree-iddden-lyeer MLP with neurons 512, 128, 32 that generates the GCBF value. In summary,
h; = ha (1y5 englsSeN,) fo. Σ
softmax(Jo, (fo, (113;) So, (Jo, (1yys egg)))) (6)
JEN.
To make the training easier, define To TT + Tnom: where NN is the NN controller and nnom is the nominal controller. this way, n only needs learn the deviation from noo- The controller To NN uses the same GNN component as ha, but before passing the output of the GNN component to the MIP component, we concatenate the output of the GNN component with the output of the nominal controller nom and pass the concatenated vectors the MLP component to get the final output of the controller (see Figure 1).
We use Adam [58] with learning rate 3 10 4 for hg and 1 10-3 for T to optimize the NNs for 500, 000 steps in training. The training time is around 60 minutes on a 13th Gen Intel(R) Core(TM) i7- 17-13700KF CPU @ 3400MHz and an NVIDIA RTX 3090 GPU. In training, we time each loss term with a balancing coefficient, i.e.
(+, = nsafe
Σ a.ES.
max (0, ho(e,, B:)) + qunsafe
Σ
max (0,7 + ho(?,, 0c))
+ Aderiv
Σ
max (0, - ho(es, 0z) ((ho(E, (.)) + Tol De Toom (Zi, a gool))
))ll
29 ESS
and we choose the hyper-parameters following Table 1.
Table Hyper-parameterr used our training
Environment
7
T/aaeo
Tunsafe
Tdderiv
?
SimpleCar
1.0
0.02
1.0
0.5
0.05
Drone
1.0
0.02
1.0
1.0
0.5
0.05
DubinsCar
1.0
0.02
1.0
1.0
0.2
0.0001
In practice, we find that it benefits training if there is non-empty region between the safe and the unsafe regions as it provides the NNs with some flexibility fit the safe-unsafe boundary. Thus, we define the region to be unsafe if the distance between two agents is less than 2r, the agents hit the obstacle, and safe if the distance between two agents or an agent and an obstacle is more than 4r.
A.2 Environments
Table 2 provides the side length for the experiments with keeping density. Next, we provide the details of each experiment environment.
SimpleCar We use double integrator dynamics for the SimpleCar environment, The state of agent
i is given by F; [pf Px of of]] where [p] P is the position of the agent, and [of e.' is
13

---

Table 2: Side-length of the workspace A' = 1 for various N.
N
16
32
64
128
256
512
1024
2048
4096
2D
11.3
16
22.6
32
45.3
64
N/A
N/A
3D
6.35
8
10.1
12.7
20.2
25.4
32
40.3
the velocity. The action of agent is given by u, - [a,, a i.e., the acceleration. The dynamics function is given by:
2;
(8)
The simulation timestep is öt 0.03. In this environment, information. For training MAPPO, we design the reward in this way: in each timestep, the agents receive -0.01 0.0001 [.0001||uu]] ||u reward. The agents also receive 2 reward for collision each timestep and a +4 reward for reaching the goal.
6ij 23 2y the edge
DubinsCar We use the standard Dubin's 's model in this enviromment. The state of agent is given by Ig = [p p.. 0,, u;]" where [p7 p is the position the agent, 6g is the heading, and eş is the speed. The action agent is given by u, = [wr, a.]" containing angular velocity and longitudinal acceleration. The dynamics function is given by:
U cOs (.))
oe sin Wi
2
ag
The simulation timestep is 61 = 0.03. We use Bay = ex(;) - (x;;) as the edge information, where <<;x;) = [pf- p",, e2 (0,), n sin(O:), 0g] T For training MAPPO, we design the reward in this way: in each timestep. the agents receive -0.0001 0.01 |/ reward. The agents also receive -0.1 reward for collision at each timestep and a + +10 reward for reaching the goal.
SimpleDrone We use linearized model for drones in our experiments. The state of agent i is given by x, = [VL . Pi, uf, o², w7]] where 1p" . _r] is the 3D position, and [o] o. vf] the velocity. The control inputs are uı = [af, a. at] and the dynamics function is given by:
44
(10)
1.1 1.1
1. 1
1.10
6o 6a²
The simulation timestep is 81 0.03. We use egj = sy MAPPO, we design the reward this way: in each timestep, the agents receive a -0.01 - 0.001 [ reward. The agents also receive -1 reward for collision at each timestep and +10 reward for reaching the goal.
the edge information. For training
In each case, the simulation terminates when all the agents reach the goal.
A.3 Baseline methods
We implement MDCBF [20] based on the official implementation' and MAPPO [14] based on the official inplementation". For fair comparisons, changed the structure of the actor and the critic to GNNs. The training time of GCBF and MDCBF are similar, both about 60 minutes, and around hours for MAPPO, 1 13th Gen Intel(R) Core(TM) 17-13700KF CPU 3400MHz and an NVIDIA RTX 3090 GPU.
htap:://iitha..coo/MMMAAAAAA
Stugss//gthbbbco///eyyccohhapppp
14

---

GCBF
MDCBF
MAPPO
SimpleCar
Drone
DubinsCar
Increase density
0.5
Safaly rale
Keep
density
0.5
Keep distance
0.5
128
256
512
1024
16
128
256
512
1024
2048
4096
128
256
$12
1024
Number of agents
Figure 4: Safety rate of GCBF, MDCBF, and MAPPO algorithms across the three environments and the three sets of experiments, namely, increasing density of the agents in fixed workspace, increasing the size of the workspace to keep the density same, and increasing the size of the workspace but limiting the average distance traveled by agents. Note that there are several overlaps in this figure: MDCBF with GCBF in the left bottom figure, the middle figure, and the right bottom figure; and all three lines in the middle bottom figure.
GCBF
MDCBF
MAPPO
SimpleCar
Drone
DubinsCar
Increuse density
0.5
Reaching rate
Keep
density
0.5
Keep distance
0.5
32
128
256
512
1024
32
512
1024
2048
4096
16
32
64
128
256
512
1024
Number of alaagenss
Figure 5: Goal-reaching rate of GCBF, MDCBF, and MAPPO algorithms across the three environ-
ments and the three sets of experiments, namely, increasing density of the agents in a fixed workspace, increasing the size the workspace to keep the density same, and increasing the size the workspace but limiting the average distance traveled by agents. Note that there are several overlaps of the lines: GCBF and MAPPO in the left three figures and the right bottom figure.
B Supplementary Experimental Results
B.1 Safety and Goal-reaching rates
We include the plots of the safety rates and goal-reaching rates for the first three sets of experiments (increase density, keep density, and keep distance) in Figure 4 and Figure 5, respectively. In the SimpleCar environment, GCBF achieves a higher safety rate in all three sets of experiments, while in the Drone environment, the performance of the baselines and GCBF overlap for the third set of experiments where we keep average traveling distance of agents constant. However, As can be observed in Figure 5, the goal-reaching rate of GCBF is higher than MDCBF and MAPPO, which results in a higher success rate as depicted in Figure in the main paper. Similarly, for the DubinsCar environment, MDCBF achieves similar or better safety than GCBF but it has lower
15

---

GCBF
MDCBF
MAPPO
Suocess rate
1.00
1.00
Reaching rate
1.00
0.75
0.75
0.95
0.50
Safely
0.50
0.90
0.25
0.25
0.85
0.00
0.00
0.80
32
16
32
8
16
32
Number of obstacles
Number of obstacles
Number of obsttacles
Figure 6: Left: success rate plots for GCBF and MDCBF. Middle: safety rate plots for GCBF and MDCBF, Right: goal-reaching rate plots for GCBF and MDCBF. Note that MAPPO and GCBF overlap in the reaching rate plot on the right.
goal-reaching rate, resulting lower overall success rate. On the other hand, MAPPO achieves very high goal-reaching rates in both SimpleCar and DubinsCar environments but fails to achieve high safety rate. This corroborates the reasoning mentioned in the Introduction Section in the main paper that MARL-based methods use penalties for safety violations and thus, cannot achieve a high safety rate in practice. We emphasize that success rate better metric of performance evaluation since there is always trade-off between prioritizing goal-reaching objectives and safety objectives, and success rate provides fair method of evaluating both at the same time.
For the obstacle environment, we present the success, safety, and goal-reaching rates for all three algorithms, namely, GCBF, MDCBF, and MAPPO in Figure In the presence of obstacles also, the goal-reaching rate is very high for MAPPO, but the safety rate is close to zero. Furthermore, we can observe sharp drop in the performance of MDCBF with 32 obstacles, while that of GCBF does not change significantly. We remind the reader that the training is performed with 16 point-sized obstacles model LiDAR observations, and conduct experiments with 32 large-sized experiments. Thus, a high success rate in all the test cases illustrates the generalizability of GCBF.
Additionally, we conduct tests for GCBF with increasing number of agents as well as obstacles, while keeping the ratio of the number of agents to the number of obstacles constant. We fix this ratio as 4 and report the performance in Figure 7. It can be observed that the safety rate remains close to 1 for all cases, with the minimum safety rate being 99. 12%. The success rate drops to 84% for 128 agents. We believe that this is because the workspace becomes very crowded with 128 agents and 32 obstacles, and hence, the agents are unable to reach their goals. However, the high safety rate illustrates that the proposed method can keep the large-scale system safe even with a large number of obstacles.
B.2 Ablation studies
Ablation sensing region We perform ablation experiments study the effect of the sensing radius parameter R on the success rate. We train GCBF for the DubinsCar environment with R = and perform tests with R [0.05, 0.1, 0.2, 0.5, 0.75, 1] (note that the safety distance is r 0.05). We observe that the success rate is lower for smaller values of R, as expected since a smaller sensing radius implies that agents have less time to react the neighbors. Also, can be observed from the left plot in Figure 8 that the success rate saturates at R = 0.5, which implies that the trained GCBF is robust to the sensing radius and can have similar performance even with much smaller sensing region.
1.
1.0
1.0
Reaching rate
Success rate
Safety
0.8
0.8
0.6
0.6
0.6
32
64
128
16
32
64
128
16
32
64
128
Number of agents
Number of agents
Number agents
Figure 7: GCBF experiments in obstacle environment with an increasing number of agents. Left: success rate. Middle: safety rate plots. Right: goal-reaching rate plots.
16

---

1.00
1,0
Safdty rate
Safety rafe
0.75
N-16
0.9
Success
N-32
0.95
0.50
6-
agents
N-128
64 agents
0.7
0.25
0.00
0.50
90
0.003
0.03
0.3
Sensing radius
Maximum Iterations
Learning rate
Figure 8: Ablation study for sensing radius R (left), maximum iterations of online policy update (middle), and learning rate for online policy update (right).
1.00
1.00
1.00
0.75
0.75
Reaching rale
0.75
Safeey rate
0.50
0.50
50.50
0.25
0.25
0.00
0.00
0.00
16
32
64
128
16
32
64
128
16
32
64
128
Number of agents
Number agents
Number of agentts
a = 0.01
0.1
a
=
α
10
100
Figure 9: Ablation study for the class-IC parameter or
Ablation online policy refinement Similar to [20], perform ablation studies online policy refinement for improved safety. In the first set of experiments, we vary the maximum iteration for policy refinement from 0 to 90 for a fixed learning rate of (r 0.3 and plot the safety rates in the middle plot in Figure 8. It can be observed that for 16 agents, policy refinement is not needed as even without policy refinement, the safety rate is 1.0. For 64 agents, online policy refinement improves the safety rate from 0.977% to 0.984%. This illustrates that online policy refinement can help improve the safety rate, especially as the number of agents grows.
The effect of the learning rate on the online policy update is captured the right plot in Figure Here, we observe that for learning rate Ir < 3, the safety rate remains high, but it drops significantly for higher learning. Thus, in conclusion, there is a wide range of papar-parameters for which GCBF works with high success rate, and its performance is robust changes in these hyper-paramette parameters.
Ablation of class-K parameter a We also perform ablation studies on the class-K. parameter a in the CBF derivative condition (the third inequality in (2)). We train and test GCBF for various choices of the parameter a from the set [0.01, 0.1, 10, 100). The testing performance for different values of for various numbers of agents for the DubinsCar environment. It can be observed that the GCBF performance peaks for a = 1.0 and performs relatively well for a = 0.1 as well. As the parameter increases, the performance decays. The primary reason for that, according to our intuition, is that larger a in a discrete-time simulation can lead to larger control inputs that can lead to violation of safety even if the continuous- time CBF conditions satisfied. Furthermore, very small value of (in this case, a = 0.01) leads to a very conservative approach, and the goal-reaching rate drops.
Website We include other details on our project website https://mit-realm hithilmm.gthbbbbiii wwwpluubbo/gcbb--wwessie
B.3 Handcrafted CBF
We conducted further experiments with handcrafted CBF for the SimpleCar experiments is relatively easier to design a handcrafted CBF for a double-integrator system. We design the CBF between each pair of agents as
h(21, x2) 2(p p5) (of - + 2(p) p0) - [(p] pz)² -pp -2)) 4r
(11)
We design two frameworks:
Centralized CBF:
In this framework, inputs of all the agents are solved together by setting up centralized QP containing CBF constraints of all the agents.
17

---

Table 3: Safety rates and the computational time (second) per control step of the handcrafted CBF
Safety rate
Computational time Centralized Decentralized
Number of agents
Centralized Decentralized
0.945
0.009
0.00047
32
1.0
0.820
0,013
0.00064
1.0
0.734
0.039
0.00096
128
1.0
0.550
0.244
0.00156
1.000
1.000
1.000
Suevee mic
Sufely tale
Reaching male
0.975
0.950
0.950
0.950
0.925
0.900
16
12
64
128
0.900
32
128
0.900
16
52
128
Number of agents
Number agents
Number of agents
Figure 10: Success rate, safety rate, and goal reaching rate of GCBF in the CrazyFlie environment.
Decenteratized CBF: this framework, each agent computes its own control input through QP with CBF constraint for its safety and assumes the neighbors take the nominal control 7 nnom as it is not possible to obtain the current control action synchronous setup.
show the safety rate results and the computational time (second) per control step in Table 3. Our experiments illustrated that the centralized hand-crafted CBF-QP-based controller maintain safety rate of 1.0. However, it is computationally very costly to solve CBF-QP for large-scale MAS. In particular, found out that for up to 64 agents, the CBF-QP can be used for "real-time" control synthesis (discrete-ting time step of 0.03s). but for more agents, the CBF-QP too slow (0.244s per step for 128 agents) and cannot be used for "real-time" control synthesis. We also tried to increase the control time step to 0.25s for 128 agents so that the CBF-QP is "real-time", ; but the safety rate drops to 0.55. The decentralized QP, the other hand, is fast enough to be used for real-time control synthesis for a large number of agents. However, we suspect that for the decentralized CBF-QP, since the agents do not know how other agents behave according the current situation, the safety rate is low. Thus, we conclude that while centralized hand-crafted CBFs can maintain high safety rates, they cannot be used in practice for large-scale MAS due their computational complexity. Decentralized CBFs can be applied in large-scale scale MAS, but the safety rate is lower than our approach because the centralized training of our approach gives the agents some consensus on their potential behaviors. In addition, our approach can deal with more complex dynamics where it is difficult to design hand-crafted CBFs.
B.4 More Realistic Dynamics
One of the advantages of GCBF is that is model-agnostic. To show that GCBF also works for other more realistic dynamics, we test GCBF for CrazyFlie dynamics. The 6-DOF quadrotor dynamics given in [59] with I22 consisting of positions, velocities, angular positions and angular
18

---

Table 4: Experimental results in the CrazyFlie environment
Number of agents
Safety rate
Goal reaching rate
Success rate
16
1.000 + 0.000
0.996 + 0.015
0.996 + 0.015
32
1.000 + 0.000
+ 1 0.016
0.994 + 0.016
64
0.998 + 0.007
0.997 + 0.008
0.995 + 0.010
128
0.993 = 0.010
0.996 + 0.006
0.989 = 0.012
velocities, and IRd consisting of the thrust at each of four motors:
P. = ( (s(v)c(() - (())s())=(())) uo(
(12a)
bvy =
- -(+))(()))))) w ()()))(())
(12b)
P= = w - (u())))))
(12c)
(12d)
p w ru ----uggs000 (
(12c)
à qu - put U1
(
(12h)
m
à d6) (()
(0) + (())
(12g)
q (() - (()
(12h)
4 P + (()(())))))))))))))))))))))))()) (()((())
(12i)
1
= Es
(U2 p1 Ino)
(12j)
1
(Us- pr( II I==))
(12k)
IyN
1
U1 + qr(1e= - (wy))
(121)
Tze
where m. Luzi Agyy kr. Kt > 0 are system parameters, g = 9.8 is the gravitational acceleration, c(-),(( s())ss)))))))))( s-),t(()) denote cos(.), sin(())) tan(.). respectively, Pp..Py, P=) denote the position of the quadro- tor, (0, 0, ()) its Euler angles and u = (Un, U2 Us, U4) the input vector consisting of thrust U, and moments U2, Us, Ua.
The relation between the vector u and the individual motor speeds given as
U41
Cr
Cr
Cr
Cr
U2
dCrv2
-Crv22
dCrv2
dCrv2
S.S.SA
(13)
Us
dCrv2
dCrv2
dCrv2
dCrrv2
U4
-CD
CD
-CD
CD
where wi is the angular speed of the i-th motor for i E {1,2, 2,3, 3, 4}. Cp the drag coefficient and Cr is the thrust coefficient. These parameters are given as: Izx Iny 1.395 10-5 kg-m² I,z 2, 173 10-5 kg-m², N/rpm² and = 0.03973 m (see [59]).
0.0299 kg, Cy 3582 x 10 10-10 N/rpm². CD 7.9379 x 10 10-12
We use the same setting as the third set of environments introduced in Section 4, i.e., fix the density and the average traveling distance. The results are shown in Figure 10 and Table 4. We can observe that GCBF works well in the CrazyFlie dynamics, which supports the generalizability of GCBF in more complex realistic dynamics.
C Theoretical results
C.1 Safety result: proof of Theorem
The proof of Theorem based on the CBF-based forward invariance arguments for time-varying safe sets [19, 60]. Since the edge features eiy depend the states of the agent and its neighbor
19

---

the arguments of h can be re-arranged to obtain the following form:
h(a,(t), Bttt) h (t((t), (Th, Tia (t) B((t))
(14)
which can be compactly written as in(t, æ;) since the information of the neighbors is time-dependent. Now, the total time-derivative of h reads
2 h(t,x)) 0t
h(t,
(t, x.)) F( u.).
(15)
Ox;
Now, from [60, Theorem IJ. holds that the set S(t) = {a; hit, z;) 2 0} is forward invariant for the unique solutions of the dynamics 2; = Fx; uz) if h is valid CBF, i.e., if there exists a class-IC function a such that sup uh((,, 20)) Now, per Theorem 1, let u, be smooth control A
input chosen from the set u Then, it holds that 2 =)t, + Su. /t,,, x;);;;; 2 -a(h(t, æ;), and thus, a valid CBF. Finally, since u, is smooth and the dynamical function is assumed to be locally Lipschitz continuous, it holds that the closed-loop dynamics F,,, uu(+;)) is locally Lipschitz continuous, and hence, the closed-loop solutions exist and are uniquely determined. Thus, it can be concluded that the set S(t) is forward invariant for the closed-looy system under the conditions of Theorem I, which completes the proof.
C.2 Generalization result
Correctness of the learned control policy The generalization results for the learned control policy for safety objectives dictate that the learned controller can keep the system safe even under unseen scenarios with high probability. The generalization results in [20, Proposition 3] states that generalization error remains bounded with high probability, and under certain regularity assumptions (such Lipschitz continuity) functions drawn from Reproducing Kernel Hilbert Space, this error vanishes with the increase in the number samples. The same generalization error-bound result holds for our learned controller and for the sake of brevity, refer interested readers to [20].
20


Reviews:
[{"review_id": "snWZDv6kpB", "reviewer": "Reviewer_pBaq", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper presents a novel approach called graph control barrier functions (GCBFs) for addressing the challenges of distributed safe multi-agent control in large-scale environments with moving obstacles. The method combines control barrier functions for safe with a graph structure for scalable and generalizable decentralized control. The authors utilize graph neural networks to learn both the neural GCBF and the distributed control policy. They further extend the framework to handle LiDAR-based point cloud inputs, allowing for obstacle detection and thus making it more applicable to real-world robotics settings. The empirical results demonstrate the superiority of the presented approach over existing methods on $2$d navigation tasks, achieving significantly higher safety and goal-reaching rates when increasing numbers of agents during inference. The proposed approach is completely distributed and scalable, making it suitable for large-scale multi-agent systems.", "summary_of_recommendation": "The paper is enjoyable to read, demonstrates a strong foundation, and has a convincing experimental section. The availability of code and comprehensive experimental details in the supplementary material further contribute to its strength. A large part of the weaknesses identified can be addressed through revisions and clarifications during the rebuttal and revision process. Unless any major issues are discovered during the rebuttal, I recommend accepting the paper.\n\nEdit: I want to thank the reviewers for the detailed discussion and for clearing up a majority of my concerns. I will remain with my original recommendation for a weak accept, but am now more certain of the contributions of the paper and their significance for CoRL. I raised the confidence from a 3 to a 4 to represent this.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "- How exactly is the online policy refinement performed, and how does this affect e.g., the runtime of the algorithm?\n- How does the experiment with obstacles behave when the number of agents is increased during inference?\n- Some of the parts of the Appendix, e.g., the ablations in Appendix B are not referenced in the main paper. Referencing them would strengthen the claims of the main paper and make it more coherent.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Strengths\n\nThe paper demonstrates several notable strengths:\n\n- Synergy of control and machine learning: The integration of control barrier functions (CBFs) with graph neural networks (GNNs) combines control theory with machine learning. This combination is both intriguing and promising for safe multi-agent control.\n- Relevance of the topic: The problem of distributed safe multi-agent control in large-scale environments with moving obstacles is highly relevant in the context of multi-agent systems and has applications in a plethora of real-world problems.\n- Clear and well-written: The paper stands out for its clarity and readability. The inclusion of clear figures and comprehensive descriptions aids in understanding the proposed approach and the experimental results.\n- Code availability and experimental details: The authors provde code and extensive experimental details in the supplementary material. This transparency enables reproducibility and facilitates further exploration and validation of the proposed method.\n- Convincing experiments against strong baselines: The paper presents persuasive experimental results by comparing the proposed approach against strong baseline methods like MAPPO and multi-agent distributed CBF (MDCBF). This empirical evidence showcases the superior performance and effectiveness of the presented approach in a variety of scenarios.\n\nOverall, these strengths contribute to the significance and impact of the paper, establishing its merit within the field of multi-agent systems and machine learning.\n\n### Weaknesses\n\nThe weaknesses of the approach are mostly and adequately addressed in the limitations section of the paper. Beyond that, some additional concerns are listed below\n\n- Potential sensitivity to hyperparameters: The paper lacks sufficient justification for the choice of hyperparameters, such as the weights of the losses in the Appendix and the choice of the function $\\alpha(h(x))$. Conducting experiments with different parameter choices or explaining the choices that are made would provide insights into the method's behavior and robustness.\n- Inconsistent experimental setups: The third experiment, which involves obstacles, differs significantly from the first two experiments. Additionally, the third experiment does not increase the number of agents during inference. Providing a more consistent experimental setup and explaining the rationale behind these differences would improve the clarity of the results.\n- The connection between the fixed-size inputs for the mathematical definition on Page 4, and the input to the GNN could be stated more clearly. While this is briefly explained in Footnote 3, it would be appreciated if the authors could clearly distinguish between theory and practical application, and provide a brief explanation for how the two differ.\n- Additional visualizations: While videos are available in the supplementary material, incorporating additional \"strips\" of policies at critical timesteps in the main paper or appendix would enhance the visualization of the approach. Furthermore, including a visualization for the 3D drone task, such as e.g., a 2D image with depth colorization, would be insightful.\n- Typos:\n    - L. 275: “Detach” → “Detect”\n    - L. 94 “assumed to” → “assumed to be”", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "qxdtLGipZ5", "reviewer": "Reviewer_umVL", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper addresses the problem of controlling robots to get to their destination without collision, by proposing graph control barrier functions to encode safety constraints and a graph neural network to learn the corresponding control. Experiments are performed in simulation using different kinematic models for the robot, comparing with other two approaches. ", "summary_of_recommendation": "While the paper could be improved on the experimental side, the paper overall shows an interesting method based on GCBF and GNNs that significantly improves the performance in scenarios with a large number of robots. Please see the details in the \"Review\" box.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "No specific questions apart from the suggestions for improvement.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper addresses a relevant problem in multirobot control and proposes an interesting method with graph neural networks to achieve it within the control barrier function framework. The paper shows a significant increase over two recent methods with a very large number of robots. Also, the limitations section discusses reasonably the main points observed during the experiments, identifying potential future work.\n\nThere are some directions of improvement for the paper:\n- a popular type of approach for multi-robot collision avoidance is \"velocity obstacle\", which should be included in the discussion of the related work at least, and as part of the comparison. It would be also interesting to have a comparison with an optimal planning strategy.\n\n- the experiments include idealized kinematic models; it would be good to use a realistic simulator or enrich the current model with noise.\n\n- the statement that planning approaches \"do not generalize to new unseen environments.\" is not clear, as those approaches, given the map of an environment, will calculate the corresponding path -- unseen is typically interpreted as not seen perhaps during the experiments. This point should be clarified and rephrased.\n\n- the appendix could include the proof of Theorem 1.\n\nA couple of typos: derivates -> derivatives; \"LiDAR to detach obstacles\" -> \"... detect ...\"", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "pY3oaQcvsz", "reviewer": "Reviewer_QaEJ", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes a safety mechanism for multi-agent systems using Control Barrier Functions (CBF). The proposed framework trains a graph neural network to learn both the controller and a barrier certificate. The paper provides numerical simulations to assess the scalability of the proposed framework using 2D ground vehicles and 3D drones. ", "summary_of_recommendation": "In its current format, the paper proposes a heuristic to promote safe control actions during training. The paper does not provide any clear assumptions under which the proposed framework is guaranteed to provide safe control actions. Moreover, the numerical results lack comparison against the model-based design of CBFs. Given the criticality of safety, I do not recommend this paper for inclusion in CoRL 2023.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "(1) What is the effect of the data quality used during training on the correctness of the proposed framework?\n\n(2) Which assumptions on the robot dynamics/workspace are needed to ensure the safety of the multi-agent system? When can one trust the decisions taken by GCBF to be safe?\n\n(3) How does GCBF perform against manually designed CBFs for 2D ground vehicles?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "- The paper comes across as a heuristic for safety. It builds a false notion of safety without guaranteeing any actual safety. Calling the trained neural network a Control Barrier Function is misleading since the trained network is not guaranteed to satisfy the CBF conditions (presented in equation (1)).\n\n- The loss function depends on the empirical evaluation around the data points used for training. This raises the question of the effect of the data quality used during training on the correctness of the proposed framework.\n\n- Which assumptions on the robot dynamics/workspace are needed to ensure the safety of the multi-agent system? When can one trust the decisions taken by GCBF to be safe?\n\n- The numerical results section is inconclusive. In particular, the paper lacks to compare against model-based (manually designed) CBF-based methods. The dynamics of the 2D vehicle used in simulation have been widely studied, and manually designed CBFs for multi-agents of these robots have been presented in the literature.\n\n- The limitations section claims that it is hard, if not impossible, to ensure that the proposed method ensures the system's safety. First, this claim was not supported by any evidence. Second, if the claim is true, this points out that the proposed approach may not be the right one to handle a crucial and consequential issue like safety.", "extra_scores": {"impact": "2: Unsure, difficult to assess."}, "extra_text": {}}, {"review_id": "n0XdI1XITa", "reviewer": "Reviewer_dLpN", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes to use Graph Neural Networks (GNNs) to learn a safety score and safe action policy. The main idea is to formulate the safety score as a control barrier function and use it to switch between a nominal and a learned-based controller. ", "summary_of_recommendation": "The paper is well-written and technically sound. The major weakness is the contribution which seems to be minor and incremental wrt to reference [20] in the paper. Moreover, there are no real robotics experiments that could be a major contribution wrt the literature.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "1 - Can you please clearly state the contribution of this article wrt [20]?\n\n2 - Liveness seems to be not well defined. Would a better definition of liveness be that there is a t such that ||p_i(t)-p_i^{goal}|| < \\epsilon, where \\epsilon is a distance threshold? \\epsilon can be the same as r. Can the authors explain the proposed definition?\n\n3 - How does the proposed approach compare performance with an optimization-based control barrier function? It would be good to present results for a baseline following one of the approaches as proposed in reference [15] of the paper.\n\n4 - In the training loss, there is only one objective the affect the safe policy weights, which is the normal distance to the nominal control action. Can the authors explain how such an objective ensures that we are learning a safe policy action?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n\nOverall the paper is well-written and well-motivated. The provided code seems to implement exactly the proposed method. \n\nWeaknesses:\n\n1- The contribution seems incremental wrt to the reference [20] of the paper. The main difference wrt [20] is that this article proposes using the GNN to learn the safe action policy and use the learned control barrier function as a switching policy between the nominal and learned safe controller. I would recommend the authors revise their claimed contributions.\n\n2 - There are only simulation results. Adding real robot experiments would increase this paper's contribution.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "koSrYEYoTlG", "reviewer": "Reviewer_fBfM", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper proposes to train a graph Neural Network (GNN) to control mobile robots. Every robot is equipped with a GNN and receives information from its neighbour environment, making the method distributed. A loss function based on Control Barrier Functions (CBF) is used to avoid inter-agent collisions between the robots and collisions of the robots with static obstacles. The GNN formulation makes the proposed method scalable with the size of the swarm and the paper shows that with its method more than 1000 agents can be controlled although the network was trained on only 16 agents.", "summary_of_recommendation": "While the paper is written well and the method is applicable for huge swarms, its methods lacks of a theoretical analysis. The statements in the paper about the method being safe should be reformulated to account for this. Furthermore, see main strengths and weaknesses as listed above. ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "1. The wording in title, abstract and main body of the paper refers to the learned controller as being safe. This is slightly missleading, as the controller has no theoretical guarantees. While it is motivated out of CBFs there are no guarantees given weather there exists a CBF and weather the training converges to it in final time. Maybe rename the \"safe controller\" to \"collision avoidance controller\" or something similar.\n2. Formula 5 is not clear to me. Where does the sum come from? Is it a batch (the same agent simulated multiple times in different settings at once)?.\n3. Page 6 describes two different ways to calculate \\dot{h}. (Lines 207/208 and the paragraph beginning with Line 212). Please clarify this.\n4. In the appendix the training time is given (1 hour on described computer). Please state this in the main text and also give the training times of MDCBF and MAPPO too. For me just 1 hour training time sound very short so this could also be an additional contribution.\n5. In Line 295 it is said that the success rate is very close to 1. Please use exact numbers. \n6. The experimental section compares the algorithm to two other learning based methods. It would be also interesting to see how it compares to classic methods like potential fields to see the benefits of learning.\n7. A short section in the video describing the overall method and approach could be added.\n8. In the video, it takes a long time for a few agents in the experiments to reach their goal. Maybe show the beginning in normal speed and once the majority of agents have reached their goal speed up the video.\n9. How often does the policy refinement strategy have to be used in the experiments?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Strengths\nS1) Interesting deep learning approach combining multiple different methods like CBFs, GNNs and attention mechanism.\n\nS2) Highly scalable.\n\nS3) Training can be done in a relatively short amount of time (only 1 hour).\n\n### Weaknesses\nW1) The paper states that its controller is safe, although it has no safety guarantees (as stated by the paper itself in the limitations).\n\nW2) While the paper derives a new type of CBFs called Graph CBF, the theoretical analysis is incomplete. The proof for the given theorem (if the policy satisfies CBF properties, then there are no collisions) is skipped and there is no further analysis (like existence results).\n\nW3) It is not clear how the method in this paper compares to classical non-learning methods. \n\nW4) The paper mentions a policy refinement strategy in their method, but does not investigate it further in the experiments section.\n\n### Evaluation\nThe proposed learning method is interesting and the experimental results are impressive. \nHowever, the paper uses the word safe in title, abstract and main text although the proposed method is not inherently safe, which the paper itself states in the limitation section.\nThe wording in the paper should be changed to account for this. \nIt is also not clear how the method in this paper compares to classical non-learning methods, which should be addressed.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a method for distributed control of multiple agents with graph barrier functions encoding safety constraints and a graph neural network to learn the controller. Experiments were performed in simulation for both 2D and 3D cases. 

The approach is novel and connects geometry deep learning with multiple methods such as CBFs, GNNs and attention mechanisms. It is also highly scalable and fast to train. 

Reviewers were mostly positive about the paper but there were also a few concerns, for example:
- Calling the trained neural network a Control Barrier Function is misleading since the trained network is not guaranteed to satisfy the CBF conditions (presented in equation (1)); which the authors agree with.
-  Comparison against model-based (manually designed) CBF-based methods. 
- The policy refinement strategy in the method is not discussed in the experiments section.

The authors have addressed these and other issues adequately in the rebuttal and provided an improved version of the paper with a different title, clarifying the most critical issue.
Decision: Accept (Poster)