# Image Error Concealment Based on Deep Neural Network
## 3 Our Proposal
### Similar Data Collection  
Error concealment is an ill-posed inverse problem, as the true values of missing pixels are unavailable in practice. Thus, the prediction error, which will reduce the reconstruction performance, is inevitable. In the proposed method, we utilized the AE-P network to reconstruct the missing pixels. Considering that the AE-P network has similar outputs for similar inputs, we assumed that the prediction errors for similar inputs were similar. Based on this assumption, we searched for the data similar to the current input of the AE-P network. Then, the network prediction errors of those data could be used to correct the prediction of the current input. This can reduce the prediction error and improve the reconstruction performance. Many methods such as those in [8; 28] collected similar samples in pixel space. However, it is not reliable to measure the similarity of samples in pixel space,  
Figure 8: The procedure of transforming the shape from _up-left_ into the standard shape _down-left_. Figure 7: The red square stands for the current missing pixel. Four support area can be found for the currently missing pixel. since the pixel value in the pixel space represents the gray value. In pixel space, the collected similar samples may only be similar in pixel value, but not in the feature, especially in the texture region. To avoid the drawbacks of collecting similar samples in pixel space, we defined the feature space and collected similar samples in this space. The feature space is composed of the sample features, which are determined through the bottleneck layer of the trained AE-P network. In feature space, the feature similarity between samples is easy to measure, so we can collect samples with more similar features. In addition, the computational complexity is greatly reduced, since the dimension of feature space is much lower than that of pixel space. The Euclidean distance is used to measure the similarity between samples and the current support area in feature space. We defined \(\mathbf{p}\) as the current support area of the current missing pixel, and \(\mathbf{q}\) as a collected sample with the same shape as \(\mathbf{p}\). \(f_{\mathbf{p}}\) and \(f_{\mathbf{q}}\) are corresponding feature representations determined through the bottleneck layer of the trained AE-P network; then, the Euclidean distance \(D_{f}(\mathbf{p},\mathbf{q})\) is used to measure the similarity of \(\mathbf{p}\) and \(\mathbf{q}\) as:  
\[D_{f}(\mathbf{p},\mathbf{q})=\sqrt{\sum_{i=1}^{k}\left(f_{\mathbf{p}}^{i}-f_{ \mathbf{q}}^{i}\right)^{2}} \tag{5}\]  
where \(f_{\mathbf{p}}^{i}\) and \(f_{\mathbf{q}}^{i}\) are the \(i^{\text{th}}\) values of \(f_{\mathbf{p}}\) and \(f_{\mathbf{q}}\), respectively. \(k\) is the dimension of the feature space. The formula shows that the smaller the distance, the more similar \(\mathbf{p}\) and \(\mathbf{q}\) are in the feature space. Formula (5) may fail at measuring the sample similarity in pixel space, since some samples are similar in feature space but very different in pixel space. Therefore, we required these collected similar samples as determined by Formula (5) to be similar to \(\mathbf{p}\) in pixel space. We also used the Euclidean distance to measure the similarity between the collected samples and the current support area in pixel space. In order to make the measurement of the similarity in pixel space more reliable, some adjacent pixels of the missing pixel were added in the similarity calculation. As illustrated in Figure 9, the green pixels are the added ones. The similarity between \(\mathbf{p}\) and \(\mathbf{q}\) in pixel space is defined as follows:  
\[D_{\mathbf{p}}(\mathbf{p},\mathbf{q})=\sqrt{\sum_{i=1}^{n}\left(\overline{ \mathbf{p}}^{i}-\overline{\mathbf{q}}^{i}\right)^{2}} \tag{6}\]  
where \(\overline{\mathbf{p}}\) and \(\overline{\mathbf{q}}\) are the augmented pixel sets corresponding to \(\mathbf{p}\) and \(\mathbf{q}\), respectively. \(n\) is the pixel number of \(\overline{\mathbf{p}}\).