# Image Error Concealment Based on Deep Neural Network
## 1 Introduction  
Reliably delivering high-quality multimedia data is a significant task for applications such as television broadcasting. However, the transmission channel is not always satisfactory. When multimedia data are transmitted over error-prone channels or bandwidth-limited channels, packet loss will greatly reduce the quality of the received multimedia data. A straightforward way to alleviate this problem is to retransmit that multimedia data. However, retransmission is unavailable in many practical applications, especially under certain real-time constraints such as live broadcast and multicast. Therefore, it is crucial to develop error concealment techniques to reconstruct the received error multimedia data in order to guarantee the quality of transmission. Image error concealment (EC), as a post-processing method, reconstructs the missing pixels without the need to modify the encoder or change the channel conditions [1]. The basic idea behind EC is to predict the missing pixels by using the correctly received ones in the current frame or adjacent frames based on the spatial or temporal correlations. According to which kind of correlation is utilized, EC methods can be classified into three categories: spatial EC (SEC) [2; 3; 4; 5; 6; 7; 8; 9; 10; 11; 12; 13; 14; 15], temporal EC (TEC) [16; 17; 18; 19; 20; 21; 22], or spatial-temporal EC (STEC) [23; 24; 25; 26]. In the case that the neighbor frames are not available, the SEC methods only use the information extracted from the neighboring pixels of the missing ones in the current frame. Here, the TEC methods purely take advantage of the temporal correlation. The missing blocks are replaced with similar areas in the previously decoded frames. STEC methods can be considered as a combination of SEC and TEC, which exploits the correlation inboth spatial and temporal domains. Considering that the temporal correlation or information does not always exist (for example, the corrupted images are still images), therefore, we focus on the SEC methods in this paper and reconstruct the missing pixels using only the spatial information. SEC methods reconstruct the missing pixels by utilizing the correctly received ones in adjacent regions based on the natural images that have local correlation. [PERSON] and [PERSON] [PERSON] and [PERSON] (2018) reconstructed the missing blocks by utilizing the spatially correlated edge information, which was extracted from a large local neighborhood of surrounding pixels. This method also can be viewed as an alternating projection onto convex sets (POCS) method. It performs well in reconstructing the major edges, but may incur objectionable false edges in smooth areas. [PERSON] and [PERSON] [PERSON] and [PERSON] (2018) proposed a sequential recovery method based on orientation adaptive interpolation (OAI). In this method, the previously recovered pixels can be used in the recovery afterward. This sequential pixel-wise manner can improve the capability of capturing important edge features from the surroundings. [PERSON] et al. [PERSON] et al. (2018) concealed missing image blocks based on the concept of visual clearness (VC) of edge. They used Hough transform to find the relevant edges and employed the visually clearest ones for interpolating. As several directional interpolations are combined through visual clearness, more complex textures can be reconstructed. However, it is hard to accurately determine the visual clearness. [PERSON] et al. [PERSON] et al. (2019) treated natural images as Markov random fields (MRF). They reconstructed the missing pixels by exploiting the information from wide neighborhoods. This method gained a visually comfortable but sometimes may blur details. In [PERSON] et al. (2018), [PERSON] et al. suggested using multivariate kernel density estimation (MKDE) to conceal the corrupted images. Furthermore, a minimum mean square error (MMSE) estimator was exploited to recover missing pixels in [PERSON] and [PERSON] (2018).