# Depth Image Super Resolution Based on Edge-Guided Method
## 2 Proposed Method
### Edge Recovery Using Sparse Coding
#### 2.1.1 Sparse Dictionary Training  
The LR and HR images are represented as \(\mathbf{z}_{l}\in\mathbf{R}^{N_{l}}\), and \(\mathbf{y}_{h}\in\mathbf{R}^{N_{h}}\), where \(N_{h}=s^{2}\cdot N_{l}\), and \(s>1\) is some integer scale-up factor. The blur operator is denoted by \(\mathbf{H}:\mathbf{R}^{N_{h}}\rightarrow\mathbf{R}^{N_{l}}\), and the decimation operator for a factor \(s\) is denoted by \(\mathbf{D}:\mathbf{R}^{N_{h}}\rightarrow\mathbf{R}^{N_{l}}\). The acquisition model of how to generate an LR image from an HR image can be described as:  
\[\mathbf{z}_{l}=\mathbf{D}\mathbf{H}\mathbf{y}_{h}+\mathbf{v} \tag{1}\]  
where \(\mathbf{v}\) is an additive noise in the acquisition process. Given \(\mathbf{z}_{l}\), the problem is to find \(\mathbf{\hat{y}}\in\mathbf{R}^{N_{h}}\) such that \(\mathbf{\hat{y}}\approx\mathbf{y}_{h}\). That is, \(\left\|\mathbf{\hat{y}}-\mathbf{y}_{h}\right\|_{2}\) tends to zero. To avoid the complexities caused by the different resolutions between \(\mathbf{z}_{l}\) and \(\mathbf{y}_{h}\), it is assumed that the image \(\mathbf{z}_{l}\) is scaled-up by a simple interpolation operator \(\mathbf{Q}:\mathbf{R}^{N_{l}}\rightarrow\mathbf{R}^{N_{h}}\) (e.g., bicubic interpolation) that fills out the missing pixels between the original pixels in the input LR image. The scaled-up image shall be denoted by \(\mathbf{y}_{l}\) and it satisfies the relation:  
\[\mathbf{y}_{l}=\mathbf{Q}\mathbf{z}_{l}=\mathbf{Q}(\mathbf{D}\mathbf{H} \mathbf{y}_{h}+\mathbf{v})=\mathbf{Q}\mathbf{D}\mathbf{H}\mathbf{y}_{h}+ \mathbf{Q}\mathbf{v}=\mathbf{U}\mathbf{y}_{h}+\overline{\mathbf{v}} \tag{2}\]  
The reconstruction problem now is cast to process \(\mathbf{y}_{l}\in\mathbf{R}^{N_{h}}\) and produce a result \(\mathbf{\hat{y}}_{h}\in\mathbf{R}^{N_{h}}\), which will get as close as possible to the original HR image, \(\mathbf{y}_{h}\in\mathbf{R}^{N_{h}}\). The algorithm we propose operates on patches extracted from \(\mathbf{y}_{l}\), aiming to estimate the corresponding patch from \(\mathbf{y}_{h}\). Let \(\mathbf{p}^{k}=\mathbf{R}_{n}^{k}\mathbf{y}\) be an image patch of size \(n\times n\) centered at location \(k\) and extracted from the image \(\mathbf{y}\) by the linear operator \(\mathbf{R}\). The stride \(d\) is used for spatially shifting of image patches. Hence, the LR and HR patches are extracted as:  
\[\mathbf{p}_{l}^{k}=\mathbf{R}_{n}^{k}\mathbf{y}_{l},\,\mathbf{p}_{h}^{k}= \mathbf{R}_{n}^{k}\mathbf{y}_{h} \tag{3}\]  
It shall be further assumed that \(\mathbf{p}_{l}^{k}\) and \(\mathbf{p}_{h}^{k}\) can be represented sparsely by coefficients \(\mathbf{q}^{k}\) over the dictionary pair \(\mathbf{A}_{l}\) and \(\mathbf{A}_{h}\), respectively, namely:  
\[\mathbf{p}_{l}^{k}=\mathbf{A}_{l}\mathbf{q}^{k},\,\mathbf{p}_{h}^{k}=\mathbf{ A}_{h}\mathbf{q}^{k} \tag{4}\]  
To acquire such a dictionary pair \(\mathbf{A}_{l}\) and \(\mathbf{A}_{h}\), we choose to apply jointly dictionary training method proposed in ref. [7]. The flow of training dictionary pair is summarized in Algorithm 1. The first step is to construct the training set. A set of HR training images \(\left\{\mathbf{y}_{h}^{j}\right\}_{j}\) are collected, LR images \(\left\{\mathbf{y}_{l}^{j}\right\}_{j}\) are constructed using scale-down operator \(\mathbf{U}\) and pairs of matching patches that form the training database \(\left\{\mathbf{p}_{h}^{k}\mathbf{p}_{l}^{k}\right\}_{k}\), are extracted. After finishing training database preparation, we can enter into dictionary learning stage.