# Global-Entropy Driven Exploration with Distributed Models under Sparsity Constraints
## 3 Distributed Cooperative Exploration
### Exploration for Splitting-over-Features  
The SOF differs from the SOE case in several respects that have an impact on computing (28). First, since each agent is able to construct its own model, there is no need to keep track of inactive features; in fact, in SOF, only the active features are kept. Despite this, we will still keep the subscript notation \(\mathcal{A}\) for consistency with the SOE case. Another pleasing aspect of SOF is that the selection of possible features \(\bar{\mathbf{x}}\) is a completely local task solved by an individual agent: an agent decides independently which new feature will be added to the model. Therefore, here, we decide to place features at the selected measurement location, i.e., we let \(\bar{\mathbf{x}}=\mathbf{x}\). This additionally simplifies the optimization (28). A careful analysis of (28) shows that in the SOF case, except for \(\varphi(\mathbf{x},\mathbf{x})\), \(\varphi(\mathbf{X}_{M},\mathbf{x})\) and \(b(\mathbf{x})\), the values \(\mathbf{A}\), \(\mathbf{c}(\mathbf{x})\) and \(\psi_{\mathcal{A}}(\mathbf{x})\) are distributed quantities in SOF. In contrast to SOE, they cannot be computed directly via consensus. Instead, we will show how the products \(\mathbf{\psi}_{\mathcal{A}}(\mathbf{x})^{T}\mathbf{A}^{-1}\mathbf{\psi}_{\mathcal{A}}(\mathbf{x})\), \(\mathbf{c}_{\mathcal{A}}(\mathbf{x})^{T}\mathbf{A}^{-1}\mathbf{\psi}_{\mathcal{A}}(\mathbf{x})\) and \(q(\mathbf{x})\) can be efficiently computed cooperatively. To this end, consider \(\mathbf{A}\) in (23). Using the matrix inversion lemma, we can compute:  
\[\mathbf{A}^{-1}=(\lambda\mathbf{\Psi}_{\mathcal{A}}^{T}\mathbf{\Psi}_{\mathcal{A}}+\widehat {\mathbf{\Delta}}_{\mathcal{A}})^{-1}=\widehat{\mathbf{\Delta}}_{\mathcal{A}}^{-1}- \widehat{\mathbf{\Delta}}_{\mathcal{A}}^{-1}\mathbf{\Psi}_{\mathcal{A}}^{T}(\lambda^ {-1}\mathbf{I}+\mathbf{\Psi}_{\mathcal{A}}\widehat{\mathbf{\Delta}}_{\mathcal{A}}^{-1}\bm {\Psi}_{\mathcal{A}}^{T})^{-1}\mathbf{\Psi}_{\mathcal{A}}\widehat{\mathbf{\Delta}}_{ \mathcal{A}}^{-1}. \tag{32}\]  
The quantity \(\mathbf{\Psi}_{\mathcal{A}}\widehat{\mathbf{\Delta}}_{\mathcal{A}}^{-1}\mathbf{\Psi}_{ \mathcal{A}}^{T}\) can be computed cooperatively. Indeed,  
\[\mathbf{D}\triangleq\mathbf{\Psi}_{\mathcal{A}}\widehat{\mathbf{\Delta}}_{\mathcal{A}}^{- 1}\mathbf{\Psi}_{\mathcal{A}}^{T}=\sum_{k=1}^{K}\mathbf{\Psi}_{\mathcal{A},k}\widehat {\mathbf{\Delta}}_{\mathcal{A},k}^{-1}\mathbf{\Psi}_{\mathcal{A},k}^{T}, \tag{33}\]  
where \(\widehat{\mathbf{\Delta}}_{\mathcal{A},k}=\text{diag}\{\widehat{\delta}_{1,k}, \ldots,\widehat{\delta}_{N_{k},k}\}\) are the regularization parameters of the local weight vector \(\mathbf{w}_{k}\). As we see, (33) can be computed in a distributed fashion using the averaged consensus algorithm [25; 26]. Let us point out that since in SOF, only active features are kept, \(\widehat{\mathbf{w}}_{k}\) is not sparse, and thus, \(\widehat{\mathbf{\Delta}}_{\mathcal{A},k}\) is always bounded, i.e., (32) is a stable inversion. Now, we can compute the Schur complement \(q(\mathbf{x})=b(\mathbf{x})-\mathbf{c}(\mathbf{x})^{T}\mathbf{A}^{-1}\mathbf{c}(\mathbf{x})\).