# Global-Entropy Driven Exploration with Distributed Models under Sparsity Constraints
## 2 Signal Models for Static Process Estimation
### Centralized Signal Model  
Let us consider an agent equipped with a sensor for measuring a spatial process in the environment, which we consider to be two-dimensional. Our goal is to reconstruct a spatial function \(f:\mathbb{R}^{2}\mapsto\mathbb{R}\) that models this process using collected measurement samples. Formally, we assume that the agent collects \(M\) samples \(y[m]\in\mathcal{Y}\subset\mathbb{R}\), \(m=1,\ldots,M\), of the process at the positions \(\mathbf{x}[m]\in\mathcal{X}\subset\mathbb{R}^{2}\), \(m=1,\ldots,M\). The notation \([m]\) denotes a measurement index taken by the agent. The positions are estimated using some localization system. We assume that the localization problem has been solved using, e.g., SLAM algorithms [17], GNSS or any other localization system. Thus, a measurement is represented with a pair \(\{\mathbf{x}[m],y[m]\}\). We will assume the function \(f\) to be sufficiently smooth and model it as a weighted superposition of known and fixed kernels \(\varphi(\mathbf{x},\mathbf{x}^{\prime}):\mathcal{X}\times\mathcal{X}\mapsto\mathbb{R}\). To be able to formulate the problem in a convenient matrix-vector form, we collect all measurements \(\{y[m],\mathbf{x}[m]\}\), \(m=1,\ldots,M\), as follows: \(\mathbf{y}\widehat{=}[y[1],\ldots,y[M]]^{T}\) and \(\mathbf{X}_{M}\widehat{=}[\mathbf{x}[1],\ldots,\mathbf{x}[M]]^{T}\). Now, we define a kernel vector: \(\mathbf{\varphi}(\mathbf{X}_{M},\mathbf{x})\widehat{=}[\mathbf{\varphi}(\mathbf{x}[1],\mathbf{x}), \ldots,\mathbf{\varphi}(\mathbf{x}[M],\mathbf{x})]^{T}\), which collects the values of the kernel function \(\varphi(\mathbf{x},\mathbf{x}^{\prime})\) evaluated at all measurement locations \(\mathbf{X}_{M}\). The kernel vectors \(\mathbf{\varphi}(\mathbf{X}_{M},\mathbf{x})\) can then be evaluated at different locations \(\mathbf{x}\in\mathcal{X}\). Considering \(N\) arbitrary locations \(\mathbf{X}_{N}=[\mathbf{x}_{1},\ldots,\mathbf{x}_{N}]^{T}\), we define a design matrix \(\mathbf{\Psi}(\mathbf{X}_{M},\mathbf{X}_{N})=[\mathbf{\varphi}(\mathbf{X}_{M},\mathbf{x}_{1}),\ldots, \mathbf{\varphi}(\mathbf{X}_{M},\mathbf{x}_{N})]\in\mathbb{R}^{M\times N}\). The spatial measurements made by the agents are then approximated as a linear combination of the columns of the design matrix \(\mathbf{\Psi}(\mathbf{X}_{M},\mathbf{X}_{N})\) using the following model:  
\[\mathbf{y}=\mathbf{\Psi}(\mathbf{X}_{M},\mathbf{X}_{N})\mathbf{w}+\mathbf{e}, \tag{1}\]  
where \(\mathbf{w}\in\mathbb{R}^{N}\) is a weight vector and \(\mathbf{e}\in\mathbb{R}^{M}\) is additive noise. In the following, we will assume that elements of \(\mathbf{e}\) are iid samples from a zero-mean normal distribution with precision \(\lambda\), i.e., \(\mathbf{e}\sim\mathcal{N}(0,\lambda^{-1}\mathbf{I})\). In the considered inference framework, we assume the weights \(\mathbf{w}\) to be sparse. In other words, we assume that many elements in \(\mathbf{w}\) are zero. The inclusion of this constraint serves the following purposes. First of all, it acts as a problem regularization, especially when the number of measurements is small, i.e., \(M\ll N\). Second, the sparsity constraints naturally permits us to keep only relevant columns in the design matrix \(\mathbf{\Psi}(\mathbf{X}_{M},\mathbf{X}_{N})\) even when \(M>N\), i.e., the resulting model effectively \"compresses\" the measured data and represents it in a compact form. The sparse estimation of the weights \(\mathbf{w}\) under the measurement model (1) can be solved optimally by means of a LASSO optimization problem [18]:  
\[\widehat{\mathbf{w}}=\operatorname*{arg\,min}_{\mathbf{w}}\frac{\lambda}{2}\left\|\bm {\Psi}(\mathbf{X}_{M},\mathbf{X}_{N})\mathbf{w}-\mathbf{y}\right\|^{2}+\delta\|\mathbf{w}\|_{1}, \tag{2}\]  
with \(\|\cdot\|_{1}\) denoting the \(\ell_{1}\)-norm and \(\delta>0\) being the regularization parameter. LASSO problems, also known as basis pursuit denoising, have been intensively investigated in the literature in the context of compressed sensing and sparse signal reconstruction (see, e.g., [19; 20; 21]).