## Appendix D Proof of Theorem 2  
In the network, each generator \(i\in\mathcal{G}\) will have \(\hat{\tau}\) virtual buffer units denoted as \(i^{1},i^{2},...,i^{\hat{\tau}}\) among its neighbors. Any information sent from generator \(j\) to \(i\) is first stored in these buffers, which retain it for the specified traffic delay time. These buffers are purely for storage, have no computational role, and each generator has an equal in-degree and out-degree of one.  
Generators are numbered from \(1\) to \(N\), while the mentioned buffers are numbered from \(N+1\) to \(N(\hat{\tau}+1)\) to distinctly index the generators and buffers. A set of buffers is assigned for each \(i\)-th generator in the network, represented as \(N+i,...,N\hat{\tau}+i\). These buffers correspond to the delay of \(1,...,\hat{\tau}\) step for the \(i\)-th generator, where \(i\in\{1,...,N\}\).  
Additionally, variables \(\hat{\lambda}(k)\) and \(\hat{g}(k)\) are introduced for all agents \(i\in\{1,...,N(\hat{\tau}+1)\}\) within the augmented directed graph. The update rule takes the following form:  
\[\hat{\lambda}_{i}(k+1)=\sum_{h=1}^{N(\hat{\tau}+1)}\hat{w}_{ih}\lambda_{h}(k) -\begin{cases}\gamma_{i}(k)\frac{\
abla\Phi_{i}\big{(}\lambda_{\hat{\tau}_{i} }(k)\big{)}+\
u_{i}(k)}{y_{ii}(k)},&\text{if }i\in O_{c},\\ 0,&\text{if }i\in O_{s,w},\\ \gamma_{i}(k)\frac{\
abla\Phi_{i}\big{(}\lambda_{\hat{\tau}_{i}}(k)\big{)}+ \
u_{i}(k)}{y_{ii}(k)},&\text{if }i\in O_{b},\end{cases}\]\[P_{i}(k+1)=\begin{cases}\varphi_{c_{i}}\Big{(}\
abla C_{c_{i}^{-1}}\left(\lambda_{ c_{i}}(k+1)\cdot\mu_{c_{i}^{-1}}(k)\right)\Big{)},&\text{if}i\in O_{c},\\ \varphi_{s,w_{i}}\big{(}P_{s,w_{i}}\cdot\mu_{s,w_{i}}(k)\big{)},&\text{if}i\in O _{s,w},\\ \varphi_{b_{i}}\Big{(}\Upsilon\big{(}E_{b_{i}}(t)\big{)}\cdot\mu_{b_{i}}(k) \Big{)},&\text{if}i\in O_{b},\end{cases}\]  
where for the index \(h\in\{1,...,m(\tau+1)\}\) holds  
\[\hat{w}_{ih}=\begin{cases}w_{ij},&\text{if}\quad h=j+\tau_{ji}N,\\ 0,&\text{otherwise}.\end{cases} \tag{109}\]  
The parameter \(w_{ij}\) denotes the weight assigned to the generator \(i\in\{1,...,N\}\) calculated according to the procedure given in **Definition 2**. For the update rule of each member of the buffer \(i\in\{N+1,...,N(\tau+1)\}\) applies the following:  
\[\hat{\lambda}_{i}(k+1) =\hat{\lambda}_{i-N}(k),\] \[\hat{y}_{i}(k+1) =\hat{y}_{i-N}(k).\]  
The initialization of variables will be done as follows:  
\[\hat{\lambda}_{i}(0)=0,\quad\text{and}\quad\hat{y}_{i}(0)=0,\quad\forall i\in \{N+1,...,N(\tau+1)\}.\]  
Then, for the members of the weight matrix \(\hat{W}\) the following applies:  
\[\hat{w}_{ih}=\begin{cases}1,&\text{for}\quad h=i-N,\\ 0,&\text{otherwise}.\end{cases}\]  
The weight matrix \(\hat{W}\) for the extended directed graph is row stochastic and strongly connected, satisfying the condition in **Lemma 1**. The remainder of the proof aligns with **Theorem 1** and leads to the supermartingale convergence Theorem [62]. This is because there is no perturbation for the buffer members in the extended graph, and it handles values exceeding the maximum, resulting in a drop-off packet and the use of the last valid value. Additionally, the calculation of the noisy gradient remains uniformly bounded, eliminating the need for an exact gradient calculation in the proof of **Theorem 1**.