## 2 Background
### OMP (Orthogonal Matching Pursuit)  
Orthogonal matching pursuit (OMP) is a greedy algorithm that selects relevant features iteratively. In each iteration, the OMP algorithm chooses the feature that has the highest correlation with the current residuals, then updates the residuals and repeats the process until it obtains a small number of relevant features. This method is directly related to CS in that it attempts to solve a sparsity problem [24; 25]. OMP can be used to select a small subset of relevant features from a large dataset, which provides an approximate solution to sparse problems rather than solving a direct optimization problem, as in the case of the LASSO algorithm. OMP is computationally efficient and can provide fast approximate solutions, making it more suitable for problems where features are few and the data are large. Mathematically, the goal of the OMP algorithm is to iteratively select the features that best explain the current residuals. Thus, the steps of the algorithm are as follows [24; 25]:  
1. At each step, compute the correlations \(\rho_{j}=x_{j}^{\mathsf{T}}r\), where \(\mathsf{r}=\) the residuals;
2. Select feature \(\mathsf{j}*\) with \(\max\lvert\rho_{j}\rvert\);
3. Update the coefficients and residuals: \[\beta=\underset{\beta}{\text{argmin}}\big{\|}X_{\beta}\beta-y\big{\|}_{2}^{2} \,,\ r=y-X_{\beta},\] where \(\beta\) is the set of selected features;
4. The process continues until the maximum number of features is selected (criterion set by the user), or until the norm of the residuals falls below a predetermined threshold. Thus, OMP does not explicitly minimize a cost function with the norm \(\ell\)1, but iteratively tries to approximate the solution using a constructive approach. At each step, a feature is selected and added to the model, and the coefficients are recalculated to minimize the residuals. In classification problems, \(\mathsf{y}\) is a target variable with discrete values (e.g., binary classes 0/1). OMP can be used in two ways, namely: (i) Features are selected to discriminate between classes by maximizing their separation (in terms of correlation or other similar measures). (ii) The algorithm is applied either directly to a linear classification model (e.g., logistic regression) or to reduce the dimension of the data before applying another classifier (e.g., SVM, k-NN). Thus, in classification, OMP has the advantage that it selects a relevant subset of features, which improves the interpretability of data. It reduces the complexity of the model, which helps to avoid overfitting and is effective for sparse data [26; 27]. Besides these advantages, OMP still has some limitations, namely sensitivity to noise (OMP may choose features that are only coincidentally correlated with \(\mathsf{y}\), especially in the presence of a low signal-to-noise ratio), dependence on the training data in that its results can vary significantly depending on the data used for training, and the fact that it does not handle strong correlations between features well, i.e., if two features are highly correlated, OMP may favor one and ignore the other, even if both are relevant. Table 1 summarizes the important aspects of the previously presented feature selection algorithms that formed the basis of the experiment presented in this paper. \begin{table}
\begin{tabular}{c c c c c c} \hline
**Citteria** & **BP (Basis Pursuit)** & **BP,ADMM** & **LASSO** & **Group LASSO** & **OMP** \\ \hline Algorithm type & Convex optimization (\(f1\)) & BP variant with ADMM for distributed optimization. & Convex optimization (\(f1\) with regularization). & Convex optimization with membership (\(f1/2\)) & Iterative greedy method \\ \hline \multirow{2}{*}{Cott function} & \multirow{2}{*}{\(\text{min}|\beta|_{1},\,\text{s.t.