# Fusion of Bilateral 2 DPCA Information for Image Reconstruction and Recognition
## 3 Methodology
### Direct Bilateral 2 DPCA  
In this paper, we propose a novel, bilateral 2 DPCA called DB2 DPCA by fusing the two unilateral algorithms, i.e., 2 DPCA and A2 DPCA, in a direct fashion--that is, DB2 DPCA takes the projection matrices \(\mathbf{U}\) and \(\mathbf{V}\) from the two unilateral algorithms, respectively, the same as (2D)\({}^{2}\)PCA. The difference between the two bilateral algorithms is that (2D)\({}^{2}\)PCA projects an image onto two lateral projection matrices simultaneously while DB2 DPCA projects an image onto a single lateral projection matrix each time and then combines the results from two laterals together.  
For a test image sample \(\widetilde{\mathbf{X}}\), its feature matrix by DB2 DPCA is calculated by concatenating the feature matrices of 2 DPCA and A2 DPCA as  
\[\left[vec\left((\widetilde{\mathbf{X}}-\overline{\mathbf{X}})\mathbf{U} \right);vec\left(\mathbf{V}^{T}(\widetilde{\mathbf{X}}-\overline{\mathbf{X}}) \right)\right]\in\mathbb{R}^{(h\times k+l\times w)\times 1}, \tag{16}\]  
where \(vec(\cdot)\) means to transform a matrix into a column vector by taking elements from the matrix in a column-wise manner. Figure 1 shows an illustration of this operation that transforms a matrix \(\mathbf{A}\) into a column vector \(\mathbf{b}\), i.e., \(\mathbf{b}=vec(\mathbf{A})\).  
The reconstruction image of sample \(\widetilde{\mathbf{X}}\) by DB2 DPCA is  
\[\frac{1}{2}\left[(\widetilde{\mathbf{X}}-\overline{\mathbf{X}})\mathbf{U} \mathbf{U}^{T}+\overline{\mathbf{X}}\right]+\frac{1}{2}\left[\mathbf{V} \mathbf{V}^{T}(\widetilde{\mathbf{X}}-\overline{\mathbf{X}})+\overline{ \mathbf{X}}\right]\in\mathbb{R}^{h\times w}. \tag{17}\]  
Figure 1: An illustration of transforming a matrix into a column vector by taking elements from the matrix in a column-wise manner.  
Here, we use arithmetic mean to combine the reconstruction results of the two lateral 2 DPCAs. However, geometric mean or other kinds of means can also be tried. In general, for two positive scalars \(a\) and \(b\), the generalized mean [58] is defined as  
\[\left(\frac{a^{r}+b^{r}}{2}\right)^{\frac{1}{l}}, \tag{18}\]  
when \(r\) is a real number. Two special cases of generalized means are the arithmetic mean when \(r=1\) and the geometric mean when \(r=0\). In this paper, we focus on \(r=1\), i.e., the arithmetic mean, in consideration of computational complexity and reconstruction performance.  
The algorithm procedure of DB2 DPCA is shown in Algorithm 1. First, we calculate projection matrices \(\mathbf{U}\) and \(\mathbf{V}\) by solving the optimization problems of 2 DPCA and A2 DPCA, i.e., Equations (6) and (10), respectively. Then, we calculate the feature matrix of a test image sample by Equation (16) which concatenates the feature matrices of the two unilateral 2D algorithms. The extracted features can be used to evaluate the performance of DB2 DPCA in image classification. And we calculate the reconstruction image of a test image sample by Equation (17), which averages the reconstruction results of the two unilateral 2D algorithms. The obtained reconstruction image can be used to evaluate the performance of DB2 DPCA in image reconstruction.  
```
Input: training image samples \(\mathbf{X}_{i}\), \(i=1,2,...,n\); feature numbers \(k\) and \(l\). Output: the feature matrix and the reconstruction image. 1. Calculate projection matrix \(\mathbf{U}\) by solving Equation (6). 2. Calculate projection matrix \(\mathbf{V}\) by solving Equation (10). 3. Calculate the feature matrix by Equation (16). 4. Calculate the reconstruction image by Equation (17).
```  
**Algorithm 1**Algorithm procedure of DB2 DPCA  
By concatenating the feature matrices of 2 DPCA and A2 DPCA, DB2 DPCA extracts more features that are useful for image recognition. By averaging the reconstruction results of 2 DPCA and A2 DPCA, DB2 DPCA strengthens their common parts and weakens their differences; thus, it is expected to obtain better performance in image reconstruction.