# Distributed Online Multi-Label Learning with Privacy Protection in Internet of Things
## 2 Related Work
### Distributed Least-Squares Iterative Methods  
In traditional distributed algorithms such as federated learning, a centralized server is necessary to collect and aggregate models from each independent node. Because of this, a single node in such a setting can often only access its own database. However, this is often inefficient in real-world scenarios considering various factors such as bandwidth, time and privacy. A distributed algorithm that does not require a centralized server can solve this problem significantly. In the field of multi-label classification problems, many algorithms involve solving linear least-squares systems to be used to determine differences between samples. Therefore, the study of distributed least-squares iterative methods is a major point of our research.  
The distributed least-squares iterative methods can be divided into four methods: Distributed Multi-Splitting method, Distributed Modified Conjugate Gradient Least-Squares method, Distributed Least Mean Squares method, and Distributed Recursive Least-Squares method. Each of these algorithms has advantages and disadvantages in different aspects due to the differences in the calculation methods [44]. The original least-squares problem is defined as \(\mathbf{Ax}=\mathbf{b}\), and the matrix \(\mathbf{A}\) and vector \(\mathbf{b}\) are split and distributed to multiple nodes. The following descriptions all refer to this definition.  
**Distributed Multi-Splitting Method** works by parallelising the stationary iterative method based on a well-known single splitting. Parallelisation uses a space decomposition method that splits the matrix \(\mathbf{A}\) into blocks to split the original problem into more minor local problems [45; 46]. As the algorithm splits matrix \(\mathbf{A}\) and assigns computational tasks to different nodes, each iteration needs to pass through all the nodes in the network.  
**Distributed Modified Conjugate Gradient Least-Squares method** is based on a distributed variant of the Modified Conjugate Gradient Least-Squares method. A common approach to solving the least-squares problem is to minimize it by solving the normal equation. The resulting method, the Conjugate Gradient Least-Squares method, is often used as the basic iterative method for solving least-squares problems. [PERSON] and [PERSON] [47] improved the Conjugate Gradient Least-Squares method and described an improved conjugate gradient least-squares method to reduce inner product global synchronization points and improve parallel performance. The method can also be applied to distributed scenarios and is called the Distributed Modified Conjugate Gradient Least-Squares method.  
**Distributed Least Mean Squares Method** allows each node to make an estimation based on local data and calculate the optimal global solution by exchanging the estimation results only with its neighbours. The advantage of this method is that only the exchange of local data is necessary. However, as a cost, it has a relatively slow convergence rate.  
**Distributed Recursive Least-Squares Method** was developed by [PERSON] and [PERSON] [48]. This method achieves an exact recursive solution by appealing to collaborative techniques. It requires circular paths in the network to be computed node-by-node. This method has the advantage of a fixed number of iterations, but its drawback is the need to exchange large dense matrices between nodes.