# Global-Entropy Driven Exploration with Distributed Models under Sparsity Constraints
## 2 Signal Models for Static Process Estimation
### Distributed Signal Models
#### 2.2.3 The SOE versus the SOF Approach  
Both SOE and SOF formulations attempt to find a solution to the optimization (2) using different data distribution approaches. Both realizations with the ADMM algorithm show similar approaches to the optimization: there is a consensus step, where a certain quantity is cooperatively computed by a multi-agent system, followed by several local optimizations or calculations. It is this cooperative step that eventually determines the requirement on the communication bandwidth to implement the scheme. Thus, we summarize the sizes of the vectors, which have to be communicated, in Table 1.  
In particular, we see that in the case of SOE, the agents have to agree on the variable \(\overline{\mathbf{z}}^{[i+1]}\) in (8), which has dimensionality of \(N\)--the number of support points in \(\mathbf{X}_{N}\)--which defines the discretization of the exploration area \(\mathcal{X}\). This can be large and potentially exceed the locally available number of measurements, i.e., \(M_{k}\ll N\). Instead, in SOF, the consensus is performed over an \(M\)-dimensional vector \(\overline{\mathbf{\Phi}\mathbf{w}}^{[i+1]}\) in (16).  
Furthermore, we see that the complexity of the local optimization problem (7) in SOE is essentially dictated by the matrix inversion in (7). Applying the inversion naively, the complexity is \(\mathcal{O}(N^{3})\), the dimensionality of the estimated vector. In [9], Section 4, the authors discussed general patterns and provided methods for the reduction of the complexity, e.g., the matrix inversion can be reduced to \(\mathcal{O}(NM_{k}^{2})\). Instead, the complexity of (15) in SOF is dictated by the dimensionality of \(\mathbf{w}_{k}\), the number of features collected by each agent. Thus, when \(M\ll N_{k}\), SOF can be significantly more efficient as compared to SOE. Additionally, the agents keep their model size \(N_{k}\) low, which is in SOF permitted due to the individual models. However, the SOF approach requires a more involved communication protocol that makes measurements globally available. Depending on a particular application, this may pose a challenge for a practical realization of a distributed system.  
Generally speaking, the complexity of SOE is mainly driven by the number of features \(N\); whereas, the complexity of SOF is determined by the number of measurements \(M\).  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & SOE & SOF \\ \hline Estimation & & & \\  & \(\bullet\) & Distribute \(\mathbf{w}\in\mathcal{R}^{N}\) & \(\bullet\) & Distribute \(\mathbf{\Phi}_{k}\mathbf{w}_{k}\in\mathcal{R}^{M}\) and \(\{\mathbf{x}_{m},y_{m}\}\) \\  & \(\bullet\) & Static number of features & & for each new measurement \\  & \(\bullet\) & More intuitive to implement & \(\bullet\) & Permits pruning unused kernels \\  & & \(\bullet\) & Estimation is faster \\ \hline Exploration & & & \\  & \(\bullet\) & Estimate \(\mathbf{A}\in\mathbb{R}^{N\times N}\), \(\mathbf{c}(\mathbf{x})\in\mathbb{R}^{N\times 1}\) and \(b(\mathbf{\hat{x}})\) by means of a consensus. Each & \(b(\mathbf{\hat{x}})\) by means of a consensus. \\  & \(\bullet\) & \(\bullet\) & The agents have to communicate the proposed positions \(\mathsf{X}_{k}\). \\  & \(\bullet\) & Agents can place features everywhere and can use any kernel. \\  & \(\bullet\) & Agents can place features only at previously defined positions and are restricted to a predefined kernel. \\ \hline \hline \end{tabular}
\end{table}
Table 1: Differences between the distribution paradigms.