# Distributed Online Multi-Label Learning with Privacy Protection in Internet of Things
## 4 Distributed OMLC Algorithm
### Distributed Discrete-Time Update  
To make it easier to handle the problem state in the last section, according to graph theory [49], the network connection is described in a matrix form to better express the constraints in (3). We define \(W\) to be the adjacency matrix for the graph \(\mathbb{G}\) and the \(ij\)-th entry of \(W\) is \(w_{ij}\), representing the weighting parameter for the edge between the \(i\)-th and \(j\)-th node, which should be a positive real number or zero. The range of index numbers \(i\) and \(j\) starts from \(1\) and ends with the number of nodes in the network. Zero of \(w_{ij}\) means no connection between the node \(i\) and \(j\). Let \(D\) denote the degree matrix with each \(i\)-th entry equal to the sum of the \(i\)-th row of \(W\), where \(d_{i}=\sum_{j=1}^{m}w_{ij}\). Then, the Laplacian matrix \(L\) of the graph \(\mathbb{G}\) is given by \(L=D-W\). Note that \(L\) is symmetric and positive semi-definite. To describe the constraints of Equation (3) in terms of the Laplacian matrix, \(\bar{L}=L\otimes I_{d}\) is defined, where \(\otimes\) denotes the Kronecker product. To adapt \(\bar{L}\), it could be defined that \(\bar{P}=\text{col}\left\{P_{1},P_{2},\ldots,P_{m}\right\}\), which summarized the projection matrices for all nodes, and \(\bar{X}=\text{diag}\{X_{1},X_{2},\ldots,X_{m}\}\), which represents a block diagonal matrix with the \(i\)th diagonal block equal to \(X_{i}\). Then, the original problem can be transferred to an equivalent global form as  
\[ \begin{split} P^{*}&=\arg\min_{P_{i}\in\mathbb{R}^{d \times q}}\frac{1}{2}\|\bar{X}\bar{P}-Y\|_{F}^{2}\\ & s.t.\ \bar{L}\bar{P}=0\end{split}  \tag{4}\]We use the Lagrange multiplier method to solve the problem state in Equation (4). The Lagrangian function is defined as  
\[G(\bar{P},B)=\frac{1}{2}\|\bar{X}\bar{P}-Y\|_{F}^{2}I_{q}+B^{\prime}\bar{L}\bar{P} \tag{5}\]  
where \(B\in\mathbb{R}^{md\times q}\), which is the Lagrange multiplier. Setting the partial derivatives of \(G(\bar{P},B)\) with respect to the elements of \(\bar{P}\) and \(B\) to zero, respectively, gives  
\[ \begin{split}\frac{\partial G(\bar{P},B)}{\partial\bar{P}}& =\bar{X}^{\prime}(\bar{X}\bar{P}-Y)+\bar{L}^{\prime}B=0\\ \frac{\partial G(\bar{P},B)}{\partial B}&=\bar{L} \bar{P}=0\end{split}  \tag{6}\]  
Since \(L\) is symmetric, it could be deduced that \(L=L^{\prime}\). In order to break up the whole system into parts, Equation (6) is disassembled into blocks as nodes. Introduce an additional matrix \(\beta_{i}\) for each node to split \(B\) as \(B=\text{col}\ \{\beta_{1},\beta_{2},\ldots,\beta_{m}\}\); then, the following equivalent equations could be obtained:  
\[ \begin{split}(X_{i}^{\prime}X_{i}P_{i}-X_{i}^{\prime}Y)+\sum_{j \in\mathcal{N}_{i}}w_{ij}(\beta_{i}-\beta_{j})&=0\\ \sum_{j\in\mathcal{N}_{i}}w_{ij}(P_{i}-P_{j})&=0\end{split}  \tag{7}\]  
where \(\mathcal{N}_{i}\) stands for the neighbours of node \(i\). In order to solve discrete-time updates between multiple nodes in the system, a common way is to find the change of the state matrix, \(\hat{P}_{i}\) and \(\hat{\beta}_{i}\). Then, replace them with \(\frac{P_{i}(t+1)-P_{i}(t)}{\Delta t}\) and \(\frac{\delta_{i}(t+1)-\beta_{i}(t)}{\Delta t}\), respectively. \(t\) stands for the round of update. Motivated by the discrete update ideas of [PERSON] et al. [50], we define that  
\[ \begin{split}\hat{P}_{i}(t)&=-(X_{i}^{\prime}X_{i}P _{i}(t)-X_{i}^{\prime}Y)\\ &\quad-\sum_{j\in\mathcal{N}_{i}}w_{ij}(\hat{P}_{i}(t)-\beta_{j}( t))\\ \hat{\beta}_{i}(t)&=\sum_{j\in\mathcal{N}_{i}}w_{ij} (P_{i}(t)-P_{j}(t))\end{split}  \tag{8}\]  
with the mix use of \(\hat{P}_{i}(t+1)\), \(\hat{\beta}_{i}(t+1)\) and \(\hat{P}_{j}(t)\), \(\hat{\beta}_{j}(t)\), for \(j\in\mathcal{N}_{i}\) on the right-hand side of Equation (8). In addition, we replace \(\hat{P}_{i}\) and \(\hat{\beta}_{i}\) with \(\frac{P_{i}(t+1)-P_{i}(t)}{\Delta t}\) and \(\frac{\delta_{i}(t+1)-\beta_{i}(t)}{\Delta t}\), respectively, where \(\Delta t=\frac{1}{\hat{\alpha}_{i}}\).