# A Review of Image Inpainting Methods Based on Deep Learning
## 4 Deep-Learning-Based Image Inpainting Algorithm
### Deep-Learning-Based Image Restoration Process
(2019) classified image inpainting methods proposed in classic papers into three categories from a global perspective: sequence-based methods, CNN-based methods, and GAN-based methods. [PERSON] et al. (2019) summarized recent major deep-learning-based image drawing methods and classified existing methods into three types of network structures: convolutional autoencoders, generative adversarial networks, and recurrent neural networks. [PERSON] et al. (2019) summarized deep-learning-based image inpainting methods and divided them into single-stage methods and progressive methods. [PERSON] et al. (2020) summarized traditional image inpainting methods and deep-learning-based methods, dividing the deep-learning-based methods into those based on autoencoders, generative models and those based on network structure optimization. [PERSON] et al. (2020) divided existing image inpainting methods into methods based on CNN, methods based on GAN, and methods based on improving the GAN loss function. Previous works categorized deep learning models without considering that an image inpainting model can integrate CNN structures, attention mechanisms, and GAN-based adversarial loss. They also provided an overview of improvements achieved in prior research. In this article, we review deep-learning-based image inpainting methods based on different improvement optimization perspectives of existing methods, such as different computing components in the neural network models, convolution calculations, and attention calculations. The improvements related to computing components, such as convolution and attention calculations, are categorized under model computation component optimization. Improvements based on different network structures, such as multistage inpainting \begin{table}
\begin{tabular}{p{113.8 pt} p{227.6 pt}} \hline \hline
**Concept** & **Definition and Description** \\ \hline CNN (convolutional neural network) & A deep learning model particularly suited for processing image data. Through convolution operations, it captures local features in images. \\ \hline Attention mechanism & Used to allocate higher weights to important information in data, allowing the model to focus more on significant details, thereby achieving efficient resource allocation. \\ \hline Transformer & A deep learning model that abandons RNN and CNN structures, adopting a full attention mechanism. It is adept at handling long-sequence data dependencies. \\ \hline Convolution & A key operation in neural networks for processing of image data, allowing the network to understand related pixel values in images. \\ \hline Dilated convolution & A specialized convolution operation that adjusts the dilation rate to alter the receptive field size, enabling the model to “see” a larger area of the input image. \\ \hline Partial convolution & A specialized convolution operation that convolves only conditionally valid pixels, enabling image restoration models to repair any irregular area. \\ \hline Gated convolution & Similar to partial convolution but using a soft filtering mechanism. Its parameters can be learned from the data. It learns a dynamic selection mechanism acting on the feature map for each channel and spatial position. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Key concepts in deep learning for image restoration. and single-stage inpainting, are classified as network structure optimization methods. Improvements based on different training methods for image inpainting are categorized as training methods. Overall, the existing works are classified as follows: image inpainting model component optimization methods, model structure improvement methods, and various training methods. The overall classification framework is shown in Figure 2.