# Global-Entropy Driven Exploration with Distributed Models under Sparsity Constraints
## 3 Distributed Cooperative Exploration
### Centralized Exploration  
We begin with the LASSO optimization (2). On the one hand, the presence of the \(\ell_{1}\) penalty leads to a sparse estimate of the parameter vector \(\mathbf{w}\); on the other hand, this term makes the resulting objective function non-smooth in \(\mathbf{w}\). This makes computation of the D-optimality difficult, because the estimator's covariance is then not defined. As a consequence, the D-optimality criterion cannot be computed. In order to circumvent this difficulty, we approximate the estimator's covariance using the following approach. Let \(\delta_{i}\triangleq 2\delta/|w_{i}|\), \(i=1,\ldots,N\). This allows us to rewrite (2) as:  
\[\underset{\mathbf{w}_{k}}{\text{minimize}} \frac{\lambda}{2}\left\|\mathbf{\
abla}(\mathbf{X}_{M},\mathbf{X}_{N})\mathbf{w}- \mathbf{y}\right\|^{2}+\frac{1}{2}\sum_{i=1}^{N}\delta_{i}|w_{i}|^{2},\] subject to \[\delta_{i}=2\delta/|w_{i}|. \tag{19}\]  
It is easy to see that, for a fixed \(\delta_{i}\), the resulting objective function becomes smooth in \(\mathbf{w}\). Now, consider the state of the algorithm, when a sparse estimate \(\widehat{\mathbf{w}}\) that solves (3.2) is computed. Thus, we can approximate the covariance of the estimator in (3.2) by using the covariance of the estimator in (3.2). To this end, we use the latter to compute the inverse of the second order derivative evaluated at \(\widehat{\mathbf{w}}\) with \(\widehat{\delta}_{i}=2\delta/|\widehat{w}_{i}|\), which is the estimators' covariance. Thus, the second order derivative of (3.2) is \(\mathbf{C}=(\mathbf{\
abla}^{T}\mathbf{\
abla}+\widehat{\mathbf{\Delta}})^{-1}\), where \(\widehat{\mathbf{\Delta}}=\text{diag}\{\widehat{\delta}_{1},\ldots,\widehat{ \delta}_{N}\}\). Let us point out that since \(\widehat{\mathbf{w}}\) is sparse, \(\widehat{\delta}_{i}\rightarrow\infty\) when \(\widehat{w}_{i}\to 0\). This allows us to partition the columns of \(\mathbf{\
abla}(\mathbf{X}_{M},\mathbf{X}_{N})\) and a diagonal matrix \(\widehat{\mathbf{\Delta}}\) into two sub-matrices:  
1. Active features \(\mathbf{\
abla}_{\mathcal{A}}\triangleq\mathbf{\
abla}_{\mathcal{A}}(\mathbf{X}_{M},\mathbf{ X}_{N})\), which collects the columns of \(\mathbf{\
abla}(\mathbf{X}_{M},\mathbf{X}_{N})\) that correspond to non-zero weight estimates and finite regularization coefficients \(\widehat{\mathbf{\Delta}}_{\mathcal{A}}\). 2. Inactive features \(\mathbf{\
abla}_{\mathcal{I}}\triangleq\mathbf{\
abla}_{\mathcal{I}}(\mathbf{X}_{M},\mathbf{ X}_{N})\), which are associated with zero elements in \(\widehat{\mathbf{w}}\) and correspond to inactive regularization parameters \(\widehat{\mathbf{\Delta}}_{\mathcal{I}}=\infty\mathbf{I}\). Clearly, \(\mathbf{\
abla}(\mathbf{X}_{M},\mathbf{X}_{N})=[\mathbf{\
abla}_{\mathcal{A}},\mathbf{\
abla}_{ \mathcal{I}}]\) and \(\widehat{\mathbf{\Delta}}=\text{diag}\{\widehat{\mathbf{\Delta}}_{\mathcal{A}}, \widehat{\mathbf{\Delta}}_{\mathcal{I}}\}\) (with an appropriate permutation of columns in \(\mathbf{\
abla}(\mathbf{X}_{M},\mathbf{X}_{N})\)). Obviously, inactive features \(\mathbf{\
abla}_{\mathcal{I}}\) do not contribute to the variance of the weight estimate: the corresponding entries in \(\mathbf{C}\) tend to zero as \(\mathbf{\Delta}_{\mathcal{I}}\rightarrow\infty\mathbf{I}\). The covariance of active elements can thus be approximated by \(\mathbf{C}_{\mathcal{A}}\triangleq(\lambda\mathbf{\
abla}_{\mathcal{A}}^{T}\mathbf{\
abla }_{\mathcal{A}}+\widehat{\mathbf{\Delta}}_{\mathcal{A}})^{-1}\), i.e., using the columns of the active components only. Now, consider a potential measurement location \(\mathbf{x}\in\mathcal{X}\), and let us study how \(\mathbf{C}_{\mathcal{A}}\) changes when this measurement is to be taken into account. To simplify the notation, we neglect now the dependency of \(\mathbf{\
abla}\) on \(\mathbf{X}_{M}\) and \(\mathbf{X}_{N}\). The new measurement will result in an \"addition\" of a new row vector to the design matrix \(\mathbf{\
abla}\).