# A Cascaded Ensemble of Sparse-and-Dense Dictionaries for Vehicle Detection
## 3 Training of CESDD
### SDD Ensemble Learning
#### 3.3.1 Basic SDD Learning and Coding  
For the task of learning SDD for a single subcategory, the SLR-decomposition-based method [11] is proposed. Given a training sample collection \(\mathbf{D}\in\mathbb{R}^{k\times m}\) from a certain subcategory, with \(m\) samples of \(k\)-dimension arranged as column vectors, we decompose it into three parts as:  
\[\mathbf{D}=\mathbf{A}\mathbf{X}+\mathbf{B}\mathbf{Y}+\mathbf{R} \tag{1}\]  
where the first term \(\mathbf{A}\mathbf{X}\) preserves subcategory-specific intrinsic patterns existing throughout all samples (\(\mathbf{A}\in\mathbb{R}^{k\times n_{A}}\), \(\mathbf{X}\in\mathbb{R}^{n_{A}\times m}\) denote the intrinsic sparse dictionary and the corresponding sparse coding, respectively), the second term \(\mathbf{B}\mathbf{Y}\) preserves non-intrinsic patterns (\(\mathbf{B}\in\mathbb{R}^{k\times n_{B}}\), \(\mathbf{Y}\in\mathbb{R}^{n_{B}\times m}\) denote the non-intrinsic dense dictionary and the corresponding dense coding, respectively), the third term \(\mathbf{R}\in\mathbb{R}^{k\times m}\) represents residues like noises, and \(n_{A},n_{B}\) denote the atom number of \(\mathbf{A},\mathbf{B}\), respectively. Taking into consideration that samples within certain subcategory share similarities to some extent, low-rank property of \(\mathbf{D}\) and \((\mathbf{A},\mathbf{B})\) could be reasonably expected. In addition, with the assumption that \(\mathbf{A}\) is responsible for capturing subcategory-intrinsic patterns while \(\mathbf{B}\) for non-subcategory-intrinsic patterns, for a sample of the same subcategory, its relatively stable subcategory-intrinsic pattern is expected to be captured by one or a few atoms in \(\mathbf{A}\) with each atom being a distinct mode, while the relatively unstable non-subcategory-intrinsic part should be allowed more freedom to choose linear combination coefficients over atoms in \(\mathbf{B}\) to better deal with the variances. Thus, the coding \(\mathbf{X}\) over \(\mathbf{A}\) is expected to be sparse, while the coding \(\mathbf{Y}\) over \(\mathbf{B}\) is expected to be relatively dense. With all these considerations, the desired objective for the proposed SDD learning could be expressed as:  
\[ \begin{split}\min_{\mathbf{A},\mathbf{X},\mathbf{B},\mathbf{Y}, \mathbf{R}}&\|\mathbf{A}\|_{*}+\omega\|\mathbf{X}\|_{1}+\lambda \|\mathbf{B}\|_{*}+\tau\|\mathbf{Y}\|_{F}^{2}+\eta\|\mathbf{R}\|_{1}\\ &\text{s.t.}\ \mathbf{D}=\mathbf{A}\mathbf{X}+\mathbf{B}\mathbf{Y}+ \mathbf{R}\end{split}  \tag{2}\]  
The low-rank property of \((\mathbf{A},\mathbf{B})\) could be measured with nuclear norm; the sparsity of codings \(\mathbf{X}\) could be measured with \(\ell_{1}\) norm; since there is no requirement of sparsity for codings \(\mathbf{Y}\), only its magnitude is to be minimized, measured with Frobenius norm; the residues \(\mathbf{R}\), possibly noises, should exhibit sparsity and is thus measured with \(\ell_{1}\) norm. At the same time, the reconstruction quality is required to be maintained, acting as a restriction term. Once the dictionary pair \((\mathbf{A},\mathbf{B})\) is learned through optimizing the SDD learning objective proposed above, they could be used to represent incoming samples. Given a single input sample \(\mathbf{d}\in\mathbb{R}^{k}\), its representation with \((\mathbf{A},\mathbf{B})\) could be expressed as:  
\[\mathbf{d}=\mathbf{A}\mathbf{x}+\mathbf{B}\mathbf{y}+\mathbf{r} \tag{3}\]  
where \(\mathbf{x}\in\mathbb{R}^{n_{A}}\) is the coding over \(\mathbf{A}\), \(\mathbf{y}\in\mathbb{R}^{n_{B}}\) is the coding over \(\mathbf{B}\), and \(\mathbf{r}\in\mathbb{R}^{k}\) is the residue. The expected properties of the coding result should follow the analysis of the SDD learning objective above. Thus, the objective of coding over \((\mathbf{A},\mathbf{B})\) is directly inferred as below:  
\[ \begin{split}\underset{\mathbf{x},\mathbf{y},\mathbf{r}}{\min}& \|\mathbf{x}\|_{1}+\gamma\|\mathbf{y}\|_{2}^{2}+\beta\|\mathbf{r}\|_{1}\\ \text{s.t.}&\mathbf{d}=\mathbf{A}\mathbf{x}+ \mathbf{B}\mathbf{y}+\mathbf{r}\end{split}