# Image Error Concealment Based on Deep Neural Network
## 3 Our Proposal
### Design of AE-P Neural Network  
In this paper, a deep neural network named the AE-P network is designed to achieve the EC purpose. The network can accomplish two tasks. One is to predict the missing pixels through using the local correlation between the missing pixels and the available pixels. The other is to extract the image features so as to search for feature similar samples. The second task is accomplished through AE, which is short for auto-encoder, as proposed by [PERSON] (1996) in 2006. The AE is an unsupervised learning method that reconstructs the input signal at the output side so as to realize the dimensionality reduction of complex data. A typical AE network consists of an encoder and a decoder. Between the encoder and decoder is the bottleneck layer, which we focus on in our method, since it is the most compact feature representation of input data. The encoder is from the input layer to the bottleneck layer, and the decoder is from the bottleneck layer to the output layer. This bottleneck layer is exactly the data representation after dimensionality reduction. Compared with the traditional dimension reduction methods such as principal component analysis (PCA), the AE network can extract high-level features of the data due to the excellent non-linear representation power of the deep neural network. To train the AE-P network, we firstly use the subset \(\mathbf{T}^{\mathrm{s}}\) to train the AE part network through unsupervised learning. Suppose that the samples in \(\mathbf{T}^{\mathrm{s}}\) are \(N\)-dimensional data; then, the dimension of the input layer and the output layer of the AE network are \(N\), correspondingly. Then, we need to determine the dimension of the bottleneck layer, which is a significant issue in the design of the AE network. In general, the lower dimension of the bottleneck layer, the higher the efficiency of coding at the expense of more information loss. So, it is necessary to balance the coding efficiency and information loss. In our method, we set the dimension of the bottleneck layer to be half that of the input layer. The structure of the deep AE network in our method is shown in Figure 3a, and the optimization of AE network is defined as:  
\[\mathrm{argmin}{\left\|{\mathrm{t}^{\mathrm{s}}-\mathrm{t}^{\mathrm{s}}} \right\|^{2}} \tag{2}\]  
where \(\mathrm{t}^{\mathrm{s}}\) is the one of sample of subset \(\mathbf{T}^{\mathrm{s}}\). \(\mathrm{t}^{\mathrm{s}}\) is the corresponding output of the AE network. As can be seen from the formula, the objective of the optimization of the AE network is to minimize the mean square error between the input and the output. Next, we use the subset \(\mathbf{T}^{\mathrm{u}}\) as the corresponding label set of \(\mathbf{T}^{\mathrm{s}}\) to train the prediction part network through supervised learning. Figure 3b is the structure of the prediction network. The input of the prediction network \(\mathrm{t}^{\mathrm{s}}\) is the available part of the training samples, which are the same as the input of the AE network. The output are the predictions of the unavailable part pixels of the corresponding samples. Since we reconstruct only one missing pixel at a time, the output layer dimension of the  
Figure 3: Structure of the utilized neural networks. (**a**) The structure of auto-encoder part network. (**b**) The structure of the prediction part network. prediction network is one, representing the prediction of the missing pixel. The optimization of AE network is defined as:  
\[\mathrm{argmin}\|\mathrm{t}^{\mathrm{u}}-\mathrm{\hat{t}}^{\mathrm{u}}\|^{2} \tag{3}\]  
where \(\mathrm{t}^{\mathrm{u}}\) represents the true values of the missing pixels, and \(\mathrm{\hat{t}}^{\mathrm{u}}\) represents the corresponding predictions of the prediction network. Consider that the inputs of the two aforementioned networks are exactly the same; therefore, the front parts of the two networks are designed to be the same. Thus, a new network, which we call the AE-P network, is formed through combining the two networks and sharing the same part.