# Adaptive-Size Dictionary Learning Using Information Theoretic Criteria
## 4 Numerical Results
### Experiments with Unknown Sparsity Level  
Using the same setup as in Section 4.1, we run ITC-ADL with several values of \(s\), not only the true one, and choose the sparsity level that gives the best ITC value. Otherwise, the algorithm is unchanged and estimates the dictionary size as usual. Table 9 presents the average values of the sparsity level and of the dictionary size given by the above procedure for \(n_{\text{true}}=128\).  
The variance of the estimated \(s\) is very low: typically, only two values are obtained. For example, when \(s=8\), for SNR = 10 dB, only values of 6 and 7 are obtained; when SNR\(=\)40 dB, only values of 8 and 9 are obtained; etc. One can see that the sparsity level estimations are rather accurate; the only deviation is at low SNR, when \(s\) is underestimated. The dictionary size is well estimated, in accordance with the previous results.  
A competing algorithm is Adaptive ITKrM [9] (Matlab sources available at [[https://www.uibk.ac.at/mathematik/personal/schmass/code/adl.zip](https://www.uibk.ac.at/mathematik/personal/schmass/code/adl.zip)]([https://www.uibk.ac.at/mathematik/personal/schmass/code/adl.zip](https://www.uibk.ac.at/mathematik/personal/schmass/code/adl.zip))), reported to give very good results in DR problems. However, it seems that this algorithm needs more signals than ours. In our setup with \(N=4000\), A-ITKrM gives very poor results. Table 10 gives results for \(N\) = 20,000, where A-ITKrM becomes competitive. Due to time constraints, the results are averaged over only 20 runs. The table shows the obtained average sparsity level, dictionary size, and ratio RMSE/RMSEi, where RMSE is the error of our algorithm (using ERNML\({}_{1}\)) and RMSEi is the error of A-ITKrM. Since A-ITKrM severely underestimates \(s\), we have computed RMSEi (and RMSE) using OMP with the true \(s\). Even so, our algorithm finds a more accurate version of the dictionary. While ITKrM gives very good estimates  
Figure 1: Evolution of the dictionary size (**left**) and of the RMSE (**right**) during 10 runs of ITC-ADL, with \(n=128,s=10\).  
\begin{table}
\begin{tabular}{c c c c c c c|c c c c c} \hline \hline
**SNR ( dB)** & **s = 4** & **6** & **8** & **10** & **12** & **SNR ( dB)** & **s = 4** & **6** & **8** & **10** & **12** \\ \hline \multirow{2}{*}{10} & 4.00 & 5.10 & 6.90 & 7.88 & 8.24 & \multirow{2}{*}{10} & 3.98 & 4.64 & 5.00 & 6.00 & 8.00 \\  & 128.38 & 128.14 & 128.04 & 128.02 & 128.04 & & 128.78 & 128.10 & 128.10 & 128.12 & 127.88 \\ \hline \multirow{2}{*}{20} & 4.00 & 6.00 & 8.00 & 10.00 & 11.86 & \multirow{2}{*}{20} & 4.00 & 6.00 & 8.00 & 9.76 & 11.10 \\  & 128.14 & 128.02 & 128.12 & 128.16 & 128.00 & & 128.28 & 128.02 & 128.10 & 128.14 & 128.16 \\ \hline \multirow{2}{*}{30} & 4.02 & 6.00 & 8.06 & 10.18 & 12.16 & \multirow{2}{*}{30} & 4.04 & 6.00 & 8.08 & 10.12 & 12.30 \\  & 128.04 & 128.14 & 128.52 & 128.46 & 129.16 & & 128.14 & 128.04 & 128.56 & 128.78 & 129.56 \\ \hline \multirow{2}{*}{40} & 4.04 & 6.10 & 8.16 & 10.42 & 12.40 & \multirow{2}{*}{40} & 4.00 & 6.04 & 8.14 & 10.48 & 12.60 \\  & 128.22 & 128.72 & 129.36 & 132.66 & 132.52 & & 128.12 & 128.66 & 130.18 & 130.34 & 132.76 \\ \hline \hline \end{tabular}
\end{table}
Table 9: Average values of \(s\) (top) and \(n\) (bottom), when \(n_{\text{true}}=128\). Left: ERNML\({}_{1}\). Right: EBIC.  
of \(n\) and a good approximation of the dictionary, it seems unable to refine the dictionary as well as our algorithm.  
In the current unoptimized implementations, the execution times of ITC-ADL and A-ITKrM are comparable, our algorithm being faster for small \(s\) and slower for large \(s\). Limited trials with \(N=50\),000 suggest that the above remarks continue to stand true.