# Aiding Dictionary Learning Through Multi-Parametric Sparse Representation
## 5 Results
### Synthetic Data  
In our first experiment, we use synthetic signals. We generate a random dictionary of \(m\times n\) elements drawn from a zero mean and unit variance Gaussian distribution. After normalizing the dictionary, we randomly choose \(s\) atoms to generate \(r\) sparse signals whose coefficients are also picked at random. Our goal is to generate \(N\) total signals that are perturbed, dense versions of \(r\) clean, sparse signals. We call the \(r\) signals seeds and we proceed to perturb each seed \(N/r\) times with different additive white Gaussian noise:  
\[\mathbf{y}_{i}=\mathbf{D}_{0}\mathbf{x}+\mathbf{v}_{i},\quad i=1:N/r, \tag{30}\]where \(\mathbf{D}_{0}\) is the random dictionary, \(x\) the sparse seed signal and \(\mathbf{v}_{i}\) the additive noise. Signal \(\mathbf{y}_{i}\) is thus a dense signal; all its elements are non-zero. We repeat (30) \(r\) times to generate \(N\) synthetic signals that we store in \(\mathbf{Y}\), which is thus known to have a sparse structure.  
In the following we generate \(n=1500\) signals, of \(m=20\) elements each, from \(r=\{15,150,1500\}\) seeds built as the linear combination of \(s=\{3,4,5\}\) vectors from a dictionary with \(n=50\) atoms.  
We proceed with the construction shown in Algorithm 2 where the critical regions and associated optimal laws are computed as-needed. As stated earlier, this is justified by the excessive number of potentially feasible sets of active constraints (26).  
Note that we use the real dictionary \(\mathbf{D}_{0}\). For a full analysis (not the object of this paper), we may repeat the procedure for all the intermediate dictionaries obtained along the dictionary learning procedure (step 4 in Algorithm 1).  
In Figures 4 and 5, we depict the plots in red,  
Figure 4 shows that the number of regions (23) is significantly lower than the number of iterations of the parameter vector \(\mathbf{y}\) for all combinations of parameters \(s\) (sparsity) and \(r\) (seed).  
As expected, the most relevant factor in the synthetic examples is the \"randomness\" of the data: the best cases are those for which \(n=1500\) vectors \(\mathbf{y}\) are the result of only \(r=15\) seeds; still, even in the worst case, where each \(\mathbf{y}\) is associated with a single seed (\(n=r=1500\)), we have a significant difference between the number of regions and of iterations.  
Figure 5 details the distribution of the inclusion occurrences (i.e., how many vector instances \(\mathbf{y}\) stay in a certain critical region). Beyond the expected fact that to the cases with lower randomness correspond more inclusions, we also note that, in general, the plots have a steep descent. Table 2 shows the number of regions as a percentage of the total number of signals.  
Out of the many stored regions (23), only relatively few contain multiple vectors \(\mathbf{y}\). This is of interest in cases of limited storage (see Remark 8).  
The computation times show empirically the effect of having a large number of stored regions. While the signal shown in Figure 6 is noisy, a clear ascending trend is visible (at each iteration step, there are more regions to be searched--step 3 of Algorithm 2).  
Disregarding memory considerations, the continued storage of critical regions makes sense until the point where searching through the existing regions takes longer than solving the online optimization problem.