# Fast and Efficient Union of Sparse Orthonormal Transforms via DCT and Bayesian Optimization +
## 3 Proposed Algorithm
### Parameter Setting via Bayesian Optimization  
It is important to specify \(\lambda\) in our method as well as in other sparse coding methodologies. The performance of sparse coding, which is quite sensitive to \(\lambda\), specified suitable target sparsity. In other words, we determined the optimal value by target sparsity. Figure 2 shows an example of the performance for different \(\lambda\). In our previous work [12], we found the best value in an exhaustive method with a step size of 0.001 yet did not mention how the optimal \(\lambda\) can be obtained. In this study, to find the optimal value for each target sparsity, we used a Bayesian optimization method [13; 14]. Bayesian optimization is a global optimization method for finding a global optimal point, even if the objective is not convex. Neural networks highly use Bayesian optimization for hyperparameter tuning. It requires less time to find optimal values than that required by grid search and random search. It also assumes that the uncertainty of an objective function follows a Gaussian process as a prior and then samples the next point with the maximum value of the acquisition function. The acquisition function determines the location of the next sampling point. Among several types of acquisition functions, this study exploited the expected improvement. Further details are beyond the scope of this paper; readers are referred to [13; 14]. Figure 2 shows an example of the performance for different \(\lambda\),which indicates that the performance varies greatly with \(\lambda\) and is not convex. The objective function for Bayesian optimization is defined as the difference between the original data and the compressed data. The equation is as follows. \[f(\lambda_{s})=\sum\limits_{i=1}^{L}\Bigl{\{}\left\|X_{i}-\hat{X}_{i}\right\| _{F}^{2}\Bigr{\}} \tag{12}\]  
Here, \(s\) is the target sparsity level and \(\hat{X}_{i}\) is the compressed data reconstructed by \(s\) largest elements in \(A_{i}\) and its corresponding atoms of \(H_{i}\), \(A_{i}\), and \(H_{i}\) are computed by Equations (10) and (11) with \(\lambda_{s}\), which is the best parameter for the target sparsity \(s\). Figure 2: PSNR(dB) for different values of \(\lambda\). This resulted from the case of three orthonormal dictionaries and target sparsity 3. Figure 3 shows the sampled points and the estimated values of the objective function via Bayesian optimization and estimates the same variable as in Figure 2. PSNR (dB) represents the objective function in Equation (12) by multiplication with \(-1\). In proposed method, the final procedure is follows: (1) Calculate the posterior distribution of the objective function in Equation (12) using the pre-observed \(\lambda\) value. (2) Choose \(\lambda\), in which the expected implementation acquisition function points to the maximum. (3) Conduct the sparse coding optimization algorithm using Equation (9). (4) If the number of evaluations is not the maximum, go to step (1). Algorithm 2 provides a detailed description of the overall proposed algorithm. ```  Given the dataset \(X=\{x_{1},x_{2},\ldots,x_{m}\}\in\mathbb{R}^{n\times N}\), the number of dictionaries, \(L\), the target sparsity, s, and the maximum evaluation number, \(N_{\lambda}\). Initialization: \(X\)s are transformed into the DCT domain. Classify the transformed \(\hat{X}_{i}\)s via Equation (4). while\(N_{\lambda}\) iterationsdo  Calculate the posterior distribution of the objective function via GP regression. Choose \(\lambda_{s}\) using the maximum value of the EI acquisition function: \(EI(\lambda)=\mathbb{E}[\max(f(\lambda_{best})-f(\lambda),0)]\). \(\lambda_{s}=\arg\max_{\lambda}EI(\lambda)\). whileStoppingCondition is not metdo  Update the coefficients:  For \(i=1,\ldots,L\), \(A_{i}=\mathcal{T}(H_{i}^{T}\hat{X}_{i},\lambda_{s}^{1/2})\). Find the optimal dictionary:  For \(i=1,\ldots,L\), (a) Compute the SVD: \(\hat{X}_{i}A_{i}^{T}=U_{i}\Sigma_{i}V_{i}^{T}\). (b) Update dictionary: \(H_{i}=U_{i}V_{i}^{T}\).