## 2 Standard Projection-Based Approach for Data Reconstruction
OUTPUT:
4:- Approximation of \(\widehat{\mathbf{y}}_{g}\)
5:Create snapshot matrix : \(\mathbf{Y}=[\mathbf{y}_{1},\ldots,\mathbf{y}_{n_{s}}]\in\mathbb{R}^{n\times n_ {s}}\)and let \(r=rank(\mathbf{Y})\). 6: Construct basis \(\mathbf{V}\) of rank \(k\leq r\) for \(\mathbf{Y}\). 7: Find coefficient vector \(\mathbf{a}\) from \(\widehat{\mathbf{y}}_{c}\) using least-squares problem in (1): \(\min_{\mathbf{a}\in\mathbb{R}^{k}}\|\widehat{\mathbf{y}}_{c}-\mathbf{V}_{c} \mathbf{a}\|_{2}^{2}\). 8: Compute the approximation \(\widehat{\mathbf{y}}_{g}~{}\approx~{}\mathbf{V}_{g}\mathbf{a}\). ```
**Algorithm 1** Standard POD-LS approach for approximating missing data

## 3 Optimal Basis for Data Set

Consider a set of snapshots \(\{\mathbf{y}_{1},\mathbf{y}_{2},\ldots,\mathbf{y}_{n_{s}}\}\) where \(\mathbf{y}_{j}\in\mathbb{R}^{n}\), \(j=1,2,\ldots,n_{s}\). Suppose we want to approximate a snapshot \(\mathbf{y}_{j}\) by using a set of orthonormal vectors \(\{\phi_{1},\phi_{2},\ldots,\phi_{k}\}\subset\mathbb{R}^{n}\), where \(k<n\). Then, the approximation is in the form \(\mathbf{y}_{j}\approx\sum_{i=1}^{k}w_{i}\phi_{i}\), for some constant \(w_{i}\), \(i=1,2,\ldots,k\). Alternatively, we can write this approximation in matrix form as \(\mathbf{y}_{j}\approx\mathbf{\Phi}_{k}\mathbf{w}\), where \(\mathbf{\Phi}_{k}=[\phi_{1},\phi_{2},\ldots,\phi_{k}]\in\mathbb{R}^{n\times k}\) is a matrix of basis vectors and \(\mathbf{w}=[w_{1},w_{2},\ldots,w_{k}]^{T}\in\mathbb{R}^{k}\) is the vector of unknown coefficients. To find \(\mathbf{w}\), we use the fact that \(\mathbf{\Phi}\) has orthonormal columns, i.e., \(\mathbf{\Phi}^{T}\mathbf{\Phi}=\mathbf{I}\), and the minimum error occurs when the residual is orthogonal to the column span of \(\mathbf{\Phi}_{k}\), i.e., \(\mathbf{\Phi}_{k}^{T}(\mathbf{y}_{j}-\mathbf{\Phi}_{k}\mathbf{w})=0\), which implies that \(\mathbf{w}=\mathbf{\[PERSON]}^{T}\mathbf{y}_{j}\) and the approximation becomes

\[\mathbf{y}_{j} \approx \mathbf{\[PERSON]}_{k}\mathbf{\[PERSON]}_{k}^{T}\mathbf{y}_{j}. \tag{3}\]

The goal is to find the orthonormal basis that minimizes this approximation error in 2-norm for a given basis rank \(k\leq\text{rank}(\{\mathbf{y}_{1},\mathbf{y}_{2},\ldots,\mathbf{y}_{n_{s}}\})\)

\[\min_{\mathbf{\[PERSON]}_{k}\in\mathbb{R}^{n\times k}}\sum_{j=1}^{n_{s}}\|\mathbf{ y}_{j}-\mathbf{\Phi}_{k}\mathbf{\Phi}_{k}^{T}\mathbf{y}_{j}\|_{2}^{2}\quad \text{ such that }\quad\mathbf{\Phi}_{k}^{T}\mathbf{\Phi}_{k}=\mathbf{I}_{k}, \tag{4}\]

where \(\mathbf{I}_{k}\in\mathbb{R}^{k\times k}\) is the identity matrix. This optimal basis is often called proper orthogonal decomposition (POD), which is also known by other names, for example, Karhunen-Loeve decomposition (KLD), principal component analysis (PCA), or singular value decomposition (SVD). It can be shown [44; 45] that the POD basis defined above can be obtained from the left singular vector of the snapshot matrix \(\mathbf{Y}=[\mathbf{y}_{1},\mathbf{y}_{2},\ldots,\mathbf{y}_{n_{s}}]\). Let \(\mathbf{Y}=\mathbf{U}\Sigma\mathbf{V}^{T}\) be the singular value decomposition of \(\mathbf{Y}\), where matrices \(\mathbf{U}=[\mathbf{u}_{1},\ldots,\mathbf{u}_{r}]\in\mathbb{R}^{n\times r}\) and \(\mathbf{V}=[\mathbf{v}_{1},\ldots,\mathbf{v}_{r}]\in\mathbb{R}^{n_{s}\times r}\) are matrices with orthonormal columns and \(\Sigma=diag(\sigma_{1},\ldots,\sigma_{r})\in\mathbb{R}^{r\times r}\) is a diagonal matrix with \(\sigma_{1}\geq\sigma_{2}\geq\cdots\geq\sigma_{r}>0\). Then the POD basis matrix of dimension \(k\) is \(\mathbf{U}_{k}=[\mathbf{u}_{1},\ldots,\mathbf{u}_{k}]\in\mathbb{R}^{n\times k}\), \(k\leq r\), i.e. \(\mathbf{U}_{k}=\operatorname*{arg\,min}_{\mathbf{\Phi}_{k}\in\mathbb{R}^{n \times k}}\sum_{j=1}^{n_{s}}\|\mathbf{y}_{j}-\mathbf{\Phi}_{k}\mathbf{\Phi}_{k}^ {T}\mathbf{y}_{j}\|_{2}^{2}\quad\text{with}\quad\mathbf{U}_{k}^{T}\mathbf{U}_{ k}=\mathbf{I}_{k}\).