# Adaptive-Size Dictionary Learning Using Information Theoretic Criteria
## 4 Numerical Results
### Execution Times and Discussion of Parameter Values
\begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline
**SNR ( dB)** & \(\mathbf{s=4}\) & **6** & **8** & **10** & **12** & **SNR ( dB)** & \(\mathbf{s=4}\) & **6** & **8** & **10** & **12** \\ \hline \multirow{3}{*}{\(10\)} & 256 & 256 & 256 & 254 & 330 & & 256 & 256 & 222 & 206 & 225 \\  & 258.74 & 256.88 & 256.64 & 346.60 & 444.28 & 10 & 258.46 & 256.32 & 253.40 & 246.30 & 308.10 \\  & 268 & 270 & 261 & 566 & 616 & 277 & 260 & 257 & 257 & 386 \\ \hline \multirow{3}{*}{\(20\)} & 255 & 256 & 256 & 256 & 256 & & 256 & 256 & 256 & 259 \\  & 259.80 & 256.54 & 256.50 & 256.62 & 267.36 & 20 & 259.30 & 256.84 & 256.36 & 256.42 & 256.04 \\  & 288 & 261 & 260 & 260 & 444 & & 275 & 261 & 260 & 260 & 273 \\ \hline \multirow{3}{*}{\(30\)} & 256 & 256 & 256 & 256 & 256 & & 256 & 256 & 256 & 256 & 236 \\  & 258.90 & 261.98 & 260.32 & 269.66 & 325.76 & 30 & 258.84 & 261.02 & 258.12 & 261.86 & 263.68 \\  & 312 & 335 & 357 & 374 & 477 & & 312 & 329 & 267 & 287 & 291 \\ \hline \multirow{3}{*}{\(40\)} & 256 & 256 & 256 & 256 & 256 & & 256 & 256 & 256 & 256 \\  & 267.02 & 261.50 & 271.12 & 304.62 & 342.72 & 40 & 265.12 & 260.12 & 260.80 & 275.56 & 268.28 \\ \cline{1-1}  & 343 & 343 & 357 & 422 & 489 & & 343 & 336 & 283 & 320 & 322 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Minimum (top), average (middle) and maximum (bottom) sizes computed when \(n_{\text{true}}=256\). Left: ERNML\({}_{1}\). Left: ERNML\({}_{1}\). Right: EBIC. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**\#** & **Algorithm** & \(\mathbf{s=4}\) & **6** & **8** & **10** & **12** \\ \hline \multirow{3}{*}{\(128\)} & ITC-ADL & 39.2 & 52.1 & 66.6 & 83.7 & 104.6 \\  & AK-SVD & 10.3 & 14.2 & 17.9 & 22.0 & 26.4 \\ \hline \multirow{3}{*}{\(192\)} & ITC-ADL & 75.0 & 101.8 & 133.2 & 165.9 & 212.6 \\  & AK-SVD & 20.3 & 28.5 & 36.3 & 44.2 & 52.6 \\ \hline \multirow{3}{*}{\(256\)} & ITC-ADL & 121.9 & 162.2 & 212.1 & 267.0 & 374.2 \\  & AK-SVD & 36.0 & 48.5 & 63.0 & 75.9 & 90.7 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Average execution times (in seconds) of ITC-ADL (with ERNML\({}_{1}\)) and AK-SVD. \(s\in\{6,10\}\), SNR\(=30\) dB; the ITC is ERNML\({}_{1}\). Since we have run again the algorithm, some results may be different from those from the previous section, due to the random factors involved. We gave the size steps \(\delta_{+}\) and \(\delta_{-}\) values between 2 and 8. For \(n=128\), the results are very similar. For \(n=256\), a trend is (barely) visible in Table 6: larger size steps lead to poorer results. This is natural, since a large dictionary increase is a perturbing factor when the algorithm is near convergence. On the other hand, a small size step is not useful in the first iterations of the algorithm; if the initial size is far from the true one, the convergence can be very slow. Thus, although we have obtained good results with constant \(\delta_{+}=\delta_{-}=5\), some refinements are certainly possible. It makes sense to decrease \(\delta_{+}\), \(\delta_{-}\) as the algorithm evolves. This allows fine tuning of the size when the algorithm approaches convergence. The number of candidates \(n_{\text{cand}}\) has more influence on the results but again this is visible especially for \(n=256\). Table 7 shows the average dictionary sizes given by ITC-ADL for \(n_{\text{cand}}\in\{10,15,20,25\}\). It is clear that a larger number of candidates is beneficial; since this leads to a larger execution time, a compromise is necessary. This was our reason for taking \(n_{\text{cand}}=20\). However, for \(n=128\), even \(n_{\text{cand}}=10\) seems sufficient, as the sizes (not shown here) are virtually the same for all considered \(n_{\text{cand}}\) values. We turn now to the parameter \(c\), the number of iterations between size changes. Intuitively, small values of \(c\) lead to better results, but with more computational effort. Table 8 shows the results for \(n=256\), for \(c\in\{3,4,5,6,8,10\}\). The effect of \(c\) is visible for the more difficult problems.