SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[1] Maryam Fazel, Rong Ge, Sham Kakade, and Mehran Mesbahi. Global convergence of policy gradient methods for the linear quadratic regulator. In International Conference on Machine Learning, pages 1467-1476. PMLR, 2018.
[2] Yingying Li, Tianpeng Zhang, Subhro Das, Jeff Shamma, and Na Li. Safe adaptive learning for linear quadratic regulators with constraints. Technical report, Technical report, 2023.
[3] Samuel Coogan and Murat Arcak. A dissipativity approach to safety verification for interconnected systems. IEEE Transactions on Automatic Control, 60(6):1722-1727, 2014.
[4] Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, pages 321-384, 2021.
[5] Guannan Qu, Yiheng Lin, Adam Wierman, and Na Li. Scalable multi-agent reinforcement learning for networked systems with average reward. Advances in Neural Information Processing Systems, 33:2074-2086, 2020.
[6] Jianhong Wang, Wangkun Xu, Yunjie Gu, Wenbin Song, and Tim C Green. Multi-agent reinforcement learning for active voltage control on power distribution networks. Advances in Neural Information Processing Systems, 34:3271-3284, 2021.
[7] Tianshu Chu, Sandeep Chinchali, and Sachin Katti. Multi-agent reinforcement learning for networked system control. In International Conference on Learning Representations, 2019.
[8] Wenqi Cui, Yan Jiang, and Baosen Zhang. Reinforcement learning for optimal primary frequency control: A Lyapunov approach. IEEE Transactions on Power Systems, 2022.
[9] Yuanyuan Shi, Guannan Qu, Steven Low, Anima Anandkumar, and Adam Wierman. Stability constrained reinforcement learning for real-time voltage control. American Control Conference (ACC), 2022.
[10] Jie Feng, Yuanyuan Shi, Guannan Qu, Steven H Low, Anima Anandkumar, and Adam Wierman. Stability constrained reinforcement learning for real-time voltage control in distribution systems. IEEE Transactions on Control of Network Systems, 2023.
[11] Charles Dawson, Sicun Gao, and Chuchu Fan. Safe control with learned certificates: A survey of neural lyapunov, barrier, and contraction methods for robotics and control. IEEE Transactions on Robotics, 2023.
[12] Guannan Qu, Yuanyuan Shi, Sahin Lale, Anima Anandkumar, and Adam Wierman. Stable online control of linear time-varying systems. In Learning for Dynamics and Control, pages 742-753. PMLR, 2021.
[13] Christian Fiedler, Carsten W Scherer, and Sebastian Trimpe. Learning-enhanced robust controller synthesis with rigorous statistical and control-theoretic guarantees. In 2021 60th IEEE Conference on Decision and Control (CDC), pages 5122-5129. IEEE, 2021.
[14] Changhong Zhao, Enrique Mallada, and Florian Dörfler. Distributed frequency control for stability and economic dispatch in power networks. In Proc. of American Control Conference, pages 2359-2364, July 2015. doi: 10.1109/ACC.2015.7171085.
[15] Martin Andreasson, Dimos V Dimarogonas, Henrik Sandberg, and Karl Henrik Johansson. Distributed control of networked dynamical systems: Static feedback, integral action and consensus. IEEE Transactions on Automatic Control, 59(7):1750-1764, 2014.
[52] T Athay, Robin Podmore, and Sudhir Virmani. A practical method for the direct analysis of transient stability. IEEE Transactions on Power Apparatus and Systems, PAS-98(2):573-584, 1979.
[53] Ján Drgona, Aaron Tuor, and Draguna Vrabie. Learning constrained adaptive differentiable predictive control policies with guarantees. arXiv preprint arXiv:2004.11184, 2020.
[54] Erik Weitenberg, Claudio De Persis, and Nima Monshizadeh. Exponential convergence under distributed averaging integral frequency control. Automatica, 98:103-113, Dec. 2018.
[55] Erik Weitenberg, Yan Jiang, Changhong Zhao, Enrique Mallada, Claudio De Persis, and Florian Dörfler. Robust decentralized secondary frequency control in power systems: Merits and tradeoffs. IEEE Transactions on Automatic Control, 64(10):3967-3982, Oct. 2019.
[56] Florian Dörfler and Sergio Grammatico. Gather-and-broadcast frequency control in power systems. Automatica, 79:296-305, May 2017. ISSN 0005-1098. doi: https://doi.org/10.1016/j.automatica.2017.02.003.
[57] Peter W Sauer, Mangalore A Pai, and Joe H Chow. Power system dynamics and stability: with synchrophasor measurement and power system toolbox. John Wiley & Sons, 2017.
We leverage the results in [49, 50] that the structure (5) with ReLU activation is a universal approximation for any convex function. However, ReLU activations are not strictly convex, and thus we design the Softplus-  $\beta$  activation. By showing that the structure (5) with Softplus-  $\beta$  activation can approximate neural networks with the ReLU activations arbitrarily closely when  $\beta$  is sufficiently large, we then prove that the structure (5) with Softplus-  $\beta$  can universally approximate any strictly convex function.
To prepare for the proof of Theorem 1, we first derive the following Lemma about the difference between ReLU activations and Softplus-  $\beta$  activation.
Lemma 3. Consider the ReLU activation  $\sigma_l^{ReLU}(x) \coloneqq \max(x,0)$ . For all  $x \in \mathbb{R}$  and  $\Delta > 0$ , we have  $0 < \left(\sigma_l^{\text{Softplus}}(\Delta + \Delta) - \sigma_l^{ReLU}(x)\right) < \Delta + \frac{1}{\beta} \log(2)$ .
$$
\left(\sigma_ {l} ^ {\text {S o f t p l u s} \beta} (x + \Delta) - \sigma_ {l} ^ {\text {R e L U}} (x)\right) = \left(\sigma_ {l} ^ {\text {S o f t p l u s} \beta} (x + \Delta) - \sigma_ {l} ^ {\text {S o f t p l u s} \beta} (x)\right) + \left(\sigma_ {l} ^ {\text {S o f t p l u s} \beta} (x) - \sigma_ {l} ^ {\text {R e L U}} (x)\right),
$$
we prove the lemma by deriving the bound for  $\left(\sigma_l^{\mathrm{Softplus}\beta}(x + \Delta) - \sigma_l^{\mathrm{Softplus}\beta}(x)\right)$  and  $\left(\sigma_l^{\mathrm{Softplus}\beta}(x) - \sigma_l^{\mathrm{ReLU}}(x)\right)$  as follows
(i) Since  $\frac{\mathrm{d}\sigma_l^{\mathrm{Softplus}}(x)}{\mathrm{d}x} = e^{\beta x} / (1 + e^{\beta x})\in (0,1)$ , then  $0 < \left(\sigma_{l}^{\mathrm{Softplus}}(\boldsymbol {x} + \Delta) - \sigma_{l}^{\mathrm{Softplus}}(\boldsymbol {x})\right) < \Delta$  for  $\Delta >0$
(ii) Explicitly represent  $\sigma_l^{\mathrm{Softplus}\beta}(x) - \sigma_l^{\mathrm{ReLU}}(x)$  yields
$$
\sigma_ {l} ^ {\text {S o f t p l u s} \beta} (x) - \sigma_ {l} ^ {\text {R e L U}} (x) = \left\{ \begin{array}{c c} \frac {1}{\beta} \log (1 + e ^ {\beta x}) - x, & x \geq 0 \\ \frac {1}{\beta} \log (1 + e ^ {\beta x}), & x <   0 \end{array} \right.
$$
$$
\frac {\mathrm {d} \left(\sigma_ {l} ^ {\text {S o f t p l u s} \beta} (x) - \sigma_ {l} ^ {\text {R e L U}} (x)\right)}{\mathrm {d} x} := \left\{ \begin{array}{l l} - 1 / (1 + e ^ {\beta x}), & x \geq 0 \\ e ^ {\beta x} / (1 + e ^ {\beta x}), & x <   0 \end{array} \right.
$$
and therefore  $\left(\sigma_l^{\mathrm{Softplus}\beta}(x) - \sigma_l^{\mathrm{ReLU}}(x)\right) \leq \left(\sigma_l^{\mathrm{Softplus}\beta}(0) - \sigma_l^{\mathrm{ReLU}}(0)\right) = \frac{1}{\beta} \log(2)$ . Note that  $\frac{1}{\beta} \log (1 + e^{\beta x}) > \frac{1}{\beta} \log (e^{\beta x}) = x$ , then  $0 < \left(\sigma_l^{\mathrm{Softplus}\beta}(x) - \sigma_l^{\mathrm{ReLU}}(x)\right) \leq \frac{1}{\beta} \log(2)$ .
Combining (i) and (ii),  $0 < \left(\sigma_{l}^{\mathrm{Softplus}\beta}(x + \Delta) - \sigma_{l}^{\mathrm{ReLU}}(x)\right) < \Delta +\frac{1}{\beta}\log (2).$
The proof of Theorem 1 is given below.
Proof. Previous works [49, 50] have shown that the structure (5) with ReLU activation is a universal approximation for any convex function. Hence, for any  $q(z): \mathcal{Z} \mapsto \mathbb{R}$ , there exists  $g(z; \theta)^{\mathrm{ReLU}}$  constructed by (5) where the activation function is ReLU and satisfying  $|g(z; \theta)^{\mathrm{ReLU}} - q(z)| < \frac{1}{2}\epsilon$ . Note that
$$
\left| g (\boldsymbol {z}; \boldsymbol {\theta}) ^ {\text {S o f t p l u s} \beta} - q (\boldsymbol {z}) \right| \leq \left| g (\boldsymbol {z}; \boldsymbol {\theta}) ^ {\text {S o f t p l u s} \beta} - g (\boldsymbol {z}; \boldsymbol {\theta}) ^ {\text {R e L U}} \right| + \left| g (\boldsymbol {z}; \boldsymbol {\theta}) ^ {\text {R e L U}} - q (\boldsymbol {z}) \right|,
$$
then it suffices to prove  $|g(z; \pmb{\theta})^{\mathrm{Softplus}\beta} - q(z)| < \epsilon$  by showing the existence of  $g(z; \pmb{\theta})^{\mathrm{Softplus}\beta}$  such that  $|g(z; \pmb{\theta})^{\mathrm{Softplus}\beta} - g(z; \pmb{\theta})^{\mathrm{ReLU}}| < \frac{1}{2}\epsilon$  for all  $z \in \mathcal{Z}$ .
Next, we compute the difference of the structure (5) with softplus-  $\beta$  and ReLU activations by tracing through the first layer to the last layer, under the same weights  $\pmb{\theta} = \left\{W_{0:k - 1}^{(z)},W_{1:k - 1}^{(o)},b_{0:k - 1}\right\}$ .
The difference between the output of the first layer in  $g(z; \theta)^{\mathrm{Softplus}\beta}$  and  $g(z; \theta)^{\mathrm{ReLU}}$  is
$$
\boldsymbol {o} _ {1} ^ {\text {S o f t p l u s} \beta} - \boldsymbol {o} _ {1} ^ {\text {R e L U}} = \sigma_ {1} ^ {\text {S o f t p l u s} \beta} \left(W _ {0} ^ {(z)} z + b _ {0}\right) - \sigma_ {1} ^ {\text {R e L U}} \left(W _ {0} ^ {(z)} z + b _ {0}\right), \tag {10}
$$
which yields  $\mathbb{0} < o_1^{\mathrm{Softplus}\beta} - o_1^{\mathrm{ReLU}} \leq \frac{1}{\beta} \log(2) \mathbb{1}$  by Lemma 3.
The difference of the second layer is
$$
\begin{array}{l} \boldsymbol {o} _ {2} ^ {\text {S o f t p l u s} \beta} - \boldsymbol {o} _ {2} ^ {\text {R e L U}} = \sigma_ {2} ^ {\text {S o f t p l u s} \beta} \left(W _ {1} ^ {(o)} \boldsymbol {o} _ {1} ^ {\text {S o f t p l u s} \beta} + W _ {2} ^ {(z)} z + b _ {2}\right) - \sigma_ {2} ^ {\text {R e L U}} \left(W _ {1} ^ {(o)} \boldsymbol {o} _ {1} ^ {\text {R e L U}} + W _ {2} ^ {(z)} z + b _ {2}\right) \\ = \sigma_ {2} ^ {\text {S o f t p l u s} \beta} \left(W _ {1} ^ {(o)} \left(\boldsymbol {o} _ {1} ^ {\text {S o f t p l u s} \beta} - \boldsymbol {o} _ {1} ^ {\text {R e L U}}\right) + W _ {1} ^ {(o)} \boldsymbol {o} _ {1} ^ {\text {R e L U}} + W _ {2} ^ {(z)} z + b _ {2}\right) \\ - \sigma_ {2} ^ {\text {R e L U}} \left(W _ {1} ^ {(o)} o _ {1} ^ {\text {R e L U}} + W _ {2} ^ {(z)} z + b _ {2}\right). \tag {11} \\ \end{array}
$$
Since all the element of  $W_1^{(o)}$  is positive, we have  $\mathbb{0} < W_1^{(o)}\left(\pmb{o}_1^{\mathrm{Softplus}\beta} - \pmb{o}_1^{\mathrm{ReLU}}\right) \leq \frac{1}{\beta}\log (2)W_1^{(o)}\mathbb{1}$ .
Applying Lemma 3 element-wise yields  $0 \leq \pmb{o}_2^{\mathrm{Softplus}\beta} - \pmb{o}_2^{\mathrm{ReLU}} \leq \frac{1}{\beta} \log(2) W_1^{(o)} \mathbb{1} + \frac{1}{\beta} \log(2) \mathbb{1}$ .
Similarly  $\mathbb{0} \leq o_3^{\mathrm{Softplus}\beta} - o_3^{\mathrm{ReLU}} \leq \frac{1}{\beta} \log(2) W_2^{(o)} W_1^{(o)} \mathbb{1} + \frac{1}{\beta} \log(2) W_2^{(o)} \mathbb{1} + \frac{1}{\beta} \log(2) \mathbb{1}$ . By induction
$$
\mathbb {0} \leq \boldsymbol {o} _ {l + 1} ^ {\text {S o f t p l u s} \beta} - \boldsymbol {o} _ {l + 1} ^ {\text {R e L U}} \leq \frac {1}{\beta} \log (2) \left(\mathbb {1} + \left(\sum_ {i = 1} ^ {l} \prod_ {j = i} ^ {l} W _ {j} ^ {(o)} \mathbb {1}\right)\right) \tag {12}
$$
$$
0 \leq g (\boldsymbol {z}; \boldsymbol {\theta}) ^ {\text {S o f t p l u s} \beta} - g (\boldsymbol {z}; \boldsymbol {\theta}) ^ {\text {R e L U}} \leq \frac {1}{\beta} \log (2) \left(1 + \left(\sum_ {i = 1} ^ {k - 1} \prod_ {j = i} ^ {k - 1} W _ {j} ^ {(o)} \mathbb {1}\right)\right), \tag {13}
$$
where  $\prod_{j = i}^{k - 1}W_j^{(o)}\coloneqq W_{k - 1}^{(o)}W_{k - 2}^{(o)}\dots W_i^{(o)}$
Let  $\beta >\frac{2}{\epsilon}\log (2)\left(1 + \left(\sum_{i = 1}^{k - 1}\prod_{j = i}^{k - 1}W_{j}^{(o)}\right)\mathbb{1}\right)$ , then  $0\leq g(\pmb {z};\pmb {\theta})^{\mathrm{Softplus}\beta} - g(\pmb {z};\pmb {\theta})^{\mathrm{ReLU}}\leq \frac{1}{2}\epsilon$ . We complete the proof using
$$
| g (\boldsymbol {z}; \boldsymbol {\theta}) ^ {\text {S o f t p l u s} \beta} - q (\boldsymbol {z}) | \leq | g (\boldsymbol {z}; \boldsymbol {\theta}) ^ {\text {S o f t p l u s} \beta} - g (\boldsymbol {z}; \boldsymbol {\theta}) ^ {\text {R e L U}} | + | g (\boldsymbol {z}; \boldsymbol {\theta}) ^ {\text {R e L U}} - q (\boldsymbol {z}) | <   \frac {1}{2} \epsilon + \frac {1}{2} \epsilon = \epsilon .
$$
Proof. At an equilibrium, the right side of (4b) equals to zero gives  $\pmb{y}^{*} = \bar{\pmb{y}}$  and the corresponding set of equilibrium  $\mathcal{E} = \{x^{*}, s^{*} | f(x^{*}, r(s^{*})) = 0, y^{*} = \bar{y}, h(x^{*}) = y^{*}\}$ . We construct a Lyapunov function to prove that if there is a feasible equilibrium in  $\mathcal{E}$ , then the system is locally asymptotically stable around it.
We construct a Lyapunov function using the storage function in Assumption 2 and the Bregman distance in Lemma 2 as
$$
V (\boldsymbol {x}, \boldsymbol {s}) | _ {\boldsymbol {x} ^ {*}, \boldsymbol {s} ^ {*}} = S (\boldsymbol {x}, \boldsymbol {x} ^ {*}) + B (\boldsymbol {s}, \boldsymbol {s} ^ {*}; \boldsymbol {\theta} ^ {(I)}, \boldsymbol {\beta} ^ {(I)}), \tag {14}
$$
where the functions by construction satisfy  $S(\pmb{x}, \pmb{x}^{*}) \geq 0$ ,  $B(\pmb{s}, \pmb{s}^{*}; \pmb{\theta}^{(I)}, \pmb{\beta}^{(I)}) \geq 0$  with equality holds only when  $\pmb{x} = \pmb{x}^{*}$  and  $s = s^{*}$ , respectively. Hence,  $V(\pmb{x}, s)|_{\pmb{x}^{*}, \pmb{s}^{*}}$  is a well-defined function that is positive definite and equals to zero at the equilibrium.
To prepare for the calculation of the time derivative of the Lyapunov function, we start by calculating the time derivative of the function  $B(\pmb{s},\pmb{s}^{*};\pmb{\theta}^{(I)},\pmb{\beta}^{(I)})$ :
$$
\begin{array}{l} \dot {B} (\boldsymbol {s}, \boldsymbol {s} ^ {*}; \boldsymbol {\theta} ^ {(I)}, \boldsymbol {\beta} ^ {(I)}) = \left(\nabla_ {\boldsymbol {s}} g ^ {(I)} (\boldsymbol {s}; \boldsymbol {\theta} ^ {(I)}, \beta^ {(I)}) - \nabla_ {\boldsymbol {s}} g ^ {(I)} \left(\boldsymbol {s} ^ {*}; \boldsymbol {\theta} ^ {(I)}, \beta^ {(I)}\right)\right) ^ {\top} \dot {\boldsymbol {s}} \tag {15} \\ \stackrel {1} {=} \left(\boldsymbol {r} (\boldsymbol {s}) - \boldsymbol {r} \left(\boldsymbol {s} ^ {*}\right)\right) ^ {\top} \left(- (\boldsymbol {y} - \boldsymbol {y} ^ {*})\right), \\ \end{array}
$$
where ① follows from  $\nabla_{\pmb{s}}g^{(I)}(\pmb {s};\pmb{\theta}^{(I)},\beta^{(I)}) = \pmb {r}(\pmb {s})$  and  $\dot{\boldsymbol{s}} = (-(\boldsymbol {y} - \bar{\boldsymbol{y}})) = (-(\boldsymbol {y} - \boldsymbol{y}^{*}))$
The time derivative of the Lyapunov function is
$$
\begin{array}{l} \dot {V} (\boldsymbol {x}, \boldsymbol {s}) | _ {\boldsymbol {x} ^ {*}, \boldsymbol {s} ^ {*}} = \dot {S} (\boldsymbol {x}, \boldsymbol {x} ^ {*}) + \dot {B} (\boldsymbol {s}, \boldsymbol {s} ^ {*}; \boldsymbol {\theta}, \beta) \\ \overset {1} {\leq} - \rho \| \boldsymbol {y} - \boldsymbol {y} ^ {*} \| ^ {2} + (\boldsymbol {y} - \boldsymbol {y} ^ {*}) ^ {\top} (\boldsymbol {u} - \boldsymbol {u} ^ {*}) + (\boldsymbol {r} (\boldsymbol {s}) - \boldsymbol {r} (\boldsymbol {s} ^ {*})) ^ {\top} (- (\boldsymbol {y} - \boldsymbol {y} ^ {*})) \\ \overset {(2)} {=} - \rho \| \boldsymbol {y} - \boldsymbol {y} ^ {*} \| ^ {2} + (\boldsymbol {y} - \boldsymbol {y} ^ {*}) ^ {\top} (\boldsymbol {p} (- \boldsymbol {y} + \bar {\boldsymbol {y}}) - \boldsymbol {p} (- \boldsymbol {y} ^ {*} + \bar {\boldsymbol {y}})) \tag {16} \\ \stackrel {3} {\leq} - \rho \| \boldsymbol {y} - \boldsymbol {y} ^ {*} \| ^ {2} \\ \end{array}
$$
where ① follows from the strict EIP property and equations derived in (15). The equality ② is derived by plugging in the controller design in (4a) where  $\pmb{u} = \pmb{p}(-\pmb{y} + \bar{\pmb{y}}) + \pmb{r}(\pmb{s})$  and  $\pmb{u}^{*} = \pmb{p}(-\pmb{y}^{*} + \bar{\pmb{y}}) + \pmb{r}(\pmb{s}^{*})$ . The inequality ③ uses strictly monotone property of  $\pmb{p}(\cdot)$ .
Therefore,  $\dot{V}(\boldsymbol{x},\boldsymbol{s})|_{\boldsymbol{x}^*,\boldsymbol{s}^*}\leq 0$  with equality only holds at the equilibrium. By Lyapunov stability theory in Proposition 1, the system is locally asymptotically stable around the equilibrium.
We demonstrate the effectiveness of the proposed neural-PI control in two dynamical systems: vehicle platooning and power system frequency control. All experiments are run with an NVIDIA Tesla T4 GPU with 16GB memory. For completeness, the figures highlighted in Section 6 are also shown below with more thorough discussions. Code is available at this link.
B.1 Vehicle platooning
B.1.1 Problem statement
The first experiment is the vehicle platoon control in [3, 23] with  $m$  vehicles, where  $\pmb{u} \in \mathbb{R}^{m}$  is the control signal to adjust the velocities of vehicles and the output  $\pmb{y} \in \mathbb{R}^{m}$  is their actual velocities. The state is  $\pmb{x} = (\zeta, \pmb{y})$ , where  $\zeta \in \mathbb{R}^{m}$  is the relative position of vehicles with  $\zeta(0) \perp Im(\mathbb{1}_{m})$  (namely, the vehicles are not in the same position at the time step 0). The dynamic model is
$$
\dot {\boldsymbol {\zeta}} = \boldsymbol {\Gamma} \boldsymbol {y},
$$
$$
\dot {\boldsymbol {y}} = \hat {\kappa} \left(- \left(\boldsymbol {y} - \boldsymbol {\lambda} ^ {0}\right) + \hat {\rho} \left(\boldsymbol {u} - \boldsymbol {E} \hat {\boldsymbol {D}} \boldsymbol {E} ^ {\top} \boldsymbol {\zeta}\right)\right), \tag {17}
$$
where  $\hat{\kappa} = \mathrm{diag}(\kappa_1,\dots ,\kappa_m),\hat{\rho} = \mathrm{diag}(\rho_1,\dots ,\rho_m)\in \mathbb{R}^{m\times m}$  are constant diagonal matrices and  $\kappa_{i} > 0,\rho_{i} > 0$  for all  $i = 1,\dots ,m$ . The vector  $\pmb{\lambda}^{0} = (\lambda_{1}^{0},\dots ,\lambda_{m}^{0})\in \mathbb{R}^{m}$  reflects the default velocity of vehicles. The matrix  $\pmb {E}\in \mathbb{R}^{m\times e}$  is the incidence matrix that indicates the neighbouring relations for  $e$  pairs of neighbouring vehicles and satisfy  $ker(E^{\top}) = Im(\mathbb{1}_m)$ . The matrix  $\Gamma \coloneqq I_m - \frac{1}{m}\mathbb{1}_m\mathbb{1}_m^\top$  extracts the relative velocities of vehicles by  $\Gamma y$ . The diagonal matrix  $\hat{D}\in \mathbb{R}^{m\times m}$  represents the sensitivity to the relative distance of neighbouring vehicles.
B.1.2 Verification of Assumption 2
We start by verifying the uniqueness of  $\pmb{x}^*$  for any  $\pmb{u}^* \in \mathbb{R}^m$ . At the equilibrium, the right side of (17) equals zero gives
$$
\left(- \left(\boldsymbol {y} ^ {*} - \boldsymbol {\lambda} ^ {0}\right) + \hat {\rho} \left(\boldsymbol {u} ^ {*} - \boldsymbol {E} \hat {\boldsymbol {D}} \boldsymbol {E} ^ {\top} \boldsymbol {\zeta} ^ {*}\right)\right) = \mathbb {0} _ {m} \text {a n d} \boldsymbol {\Gamma} \boldsymbol {y} ^ {*} = \mathbb {0} _ {m}. \tag {18}
$$
For a given  $\pmb{u}^{*}\in \mathbb{R}^{m}$ , suppose there exists  $\pmb{x}_a^* = (\zeta_a^*,\pmb{y}_a^*)$  and  $\pmb{x}_b^* = (\zeta_b^*,\pmb{y}_b^*)$ ,  $\pmb{x}_a^*\neq \pmb{x}_b^*$  such that (18) holds. Plugging in (18) gives
$$
\left(\boldsymbol {y} _ {a} ^ {*} - \boldsymbol {y} _ {b} ^ {*}\right) + \hat {\rho} \boldsymbol {E} \hat {\boldsymbol {D}} \boldsymbol {E} ^ {\top} \left(\boldsymbol {\zeta} _ {a} ^ {*} - \boldsymbol {\zeta} _ {b} ^ {*}\right) = \mathbb {O} _ {m} \tag {19a}
$$
$$
\boldsymbol {\Gamma} \left(\boldsymbol {y} _ {a} ^ {*} - \boldsymbol {y} _ {b} ^ {*}\right) = \mathbb {0} _ {m}. \tag {19b}
$$
Note that  $\pmb{\Gamma}\pmb{E} = \pmb{E}$ . Left multiplying (19a) with  $(\pmb{E}\hat{\pmb{D}}\pmb{E}^{\top}(\zeta_{a}^{*} - \zeta_{b}^{*}))^{\top}\pmb{\Gamma}$  yields  $(\pmb{E}\hat{\pmb{D}}\pmb{E}^{\top}(\zeta_{a}^{*} - \zeta_{b}^{*}))^{\top}\hat{\rho}\pmb{E}\hat{\pmb{D}}\pmb{E}^{\top}(\zeta_{a}^{*} - \zeta_{b}^{*}) = 0$ , which holds if and only if  $\pmb{E}\hat{\pmb{D}}\pmb{E}^{\top}(\zeta_{a}^{*} - \zeta_{b}^{*}) = \mathbb{0}_{m}$  since  $\hat{\rho} \succ 0$ .
Hence,  $(\pmb{y}_a^* - \pmb{y}_b^*) = -\hat{\rho} \pmb{E} \hat{\pmb{D}} \pmb{E}^\top (\zeta_a^* - \zeta_b^*) = 0_m$  and therefore  $\pmb{y}_a^* = \pmb{y}_b^*$ . Note that  $k e r(\pmb{E} \hat{\pmb{D}} \pmb{E}^\top) = I m(\mathbb{1}_m)$  and  $I m(\Gamma) \perp I m(\mathbb{1}_m)$ , thus  $(\zeta_a^* - \zeta_b^*) \perp I m(\mathbb{1}_m)$ . Hence,  $\pmb{E} \hat{\pmb{D}} \pmb{E}^\top (\zeta_a^* - \zeta_b^*) = 0_m$  if and only if  $\zeta_a^* = \zeta_b^*$ . Therefore, for every equilibrium  $\pmb{u}^* \in \mathbb{R}^m$ , there is a unique  $\pmb{x}^* = (\zeta^*, \pmb{y}^*) \in \mathbb{R}^n$  such that  $\pmb{f}(\pmb{x}^*, \pmb{u}^*) = 0_n$ .
Let the storage function be  $S(\pmb{x},\pmb{x}^{*}) = \frac{1}{2} (\pmb{y} - \pmb{y}^{*})^{\top}\hat{\pmb{\kappa}}^{-1}\hat{\pmb{\rho}}^{-1}(\pmb{y} - \pmb{y}^{*}) + \frac{1}{2}\pmb{\zeta}^{\top}\pmb{E}\hat{\pmb{D}}\pmb{E}^{\top}\pmb{\zeta}$ . Then
$$
\begin{array}{l} \dot {S} (\boldsymbol {x}, \boldsymbol {x} ^ {*}) = (\boldsymbol {y} - \boldsymbol {y} ^ {*}) ^ {\top} \hat {\boldsymbol {\rho}} ^ {- 1} \hat {\boldsymbol {\kappa}} ^ {- 1} \dot {\boldsymbol {y}} + \zeta^ {\top} E \hat {D} E ^ {\top} \dot {\zeta} \\ = \left(\boldsymbol {y} - \boldsymbol {y} ^ {*}\right) ^ {\top} \hat {\rho} ^ {- 1} \left(- \left(\boldsymbol {y} - \boldsymbol {\lambda} ^ {0}\right) + \hat {\rho} \left(\boldsymbol {u} - \boldsymbol {E} \hat {\boldsymbol {D}} \boldsymbol {E} ^ {\top} \boldsymbol {\zeta}\right)\right) + \boldsymbol {\zeta} ^ {\top} \boldsymbol {E} \hat {\boldsymbol {D}} \boldsymbol {E} ^ {\top} \boldsymbol {\Gamma} \boldsymbol {y} \\ \stackrel {1} {=} - \hat {\rho} ^ {- 1} \| \boldsymbol {y} - \boldsymbol {y} ^ {*} \| _ {2} ^ {2} + (\boldsymbol {y} - \boldsymbol {y} ^ {*}) ^ {\top} (\boldsymbol {u} - \boldsymbol {u} ^ {*}) \\ \stackrel {2} {\leq} - \left(\min  _ {i} \rho_ {i} ^ {- 1}\right) \| \boldsymbol {y} - \boldsymbol {y} ^ {*} \| _ {2} ^ {2} + \left(\boldsymbol {y} - \boldsymbol {y} ^ {*}\right) ^ {\top} \left(\boldsymbol {u} - \boldsymbol {u} ^ {*}\right) \\ \end{array}
$$
where ① follows from  $\left(-(\pmb{y}^{*} - \pmb{\lambda}^{0}) + \hat{\rho}\left(\pmb{u}^{*} - \pmb{E}\hat{\pmb{D}}\pmb{E}^{\top}\pmb{\zeta}^{*}\right)\right) = \mathbb{0}_{m}$  and  $\pmb{E}^{\top}\pmb{y}^{*} = \pmb{E}^{\top}\pmb{\Gamma}\pmb{y}^{*} = \mathbb{0}_{e}$  by definition of equilibrium. The inequality ② follows from  $\hat{\rho} > 0$ . Therefore, the dynamics in (17) satisfy conditions in Assumption 2.
B.1.3 Simulation and Visualization
Simulation and training setup We adopt the model setup and parameters in [3, 23]. The number of vehicles is  $m = 20$ . The sensitivity parameter is  $\kappa_{i} = 1$  for all vehicles. The parameters  $\lambda_{i}^{0}$  and  $\rho_{i}$  are randomly generated by  $\lambda_{i}^{0} \sim \text{uniform}[5,6]$  and  $\rho_{i} \sim \text{uniform}[1,2]$ , respectively. We train for 400 epochs, where each epoch trains with 300 trajectories with initial velocities  $y_{i}(0) \sim \text{uniform}[5,6]$  and initial integral variable  $s_{i}(0) = 0$ . The stepsize in time is set as  $\Delta t = 0.02s$  and for  $K = 300$  steps in a trajectory (Namely, each trajectory evolves 6s).
We implement control law in  $\mathbf{u}$  to realize a specific output agreement at  $\bar{\mathbf{y}}$  and reduce the transient cost. The transient cost is set to be  $J(\mathbf{y},\mathbf{u}) = \sum_{k = 1}^{K}||\mathbf{y}(k\Delta t) - \bar{\mathbf{y}} ||_1 + \hat{c} ||\mathbf{u}(k\Delta t)||_2^2$ , where  $\hat{c} = \mathrm{diag}(c_1,\dots ,c_m)$  with  $c_{i}\sim \mathrm{uniform}[0.025,0.075]$ . The loss function in training is set to be the same as  $J(\mathbf{y},\mathbf{u})$ , such that neural networks are optimized to reduce transient cost through training. The neural PI controller can be trained by most model-based or model-free algorithms, and we use the model-based framework in [8, 53] by embedding the system dynamic model in the computation graph shown in Figure 4 and training Neural-PI by gradient descent through  $J(\mathbf{y},\mathbf{u})$ .