# Parallel Image Completion with Edge and Color Map
## 3 Materials and Methods
### Architecture of PIC-EC
#### 3.2.1 Generators  
The generators in PIC-EC follow an encoder-decoder architecture similar to the network proposed by [PERSON] et al. [40], which has achieved impressive results for super-resolution, style transfer and image-to-image translation problems. This architecture allows for reducing the memory usage and computational time by initially decreasing the resolution before further processing the image. Specifically, the generator consists of encoders that down-sample twice, followed by eight residual blocks and decoders that up-sample images back to the same resolution as the input. The residual blocks make it easy for the network to learn the identify function. It is an appealing property for image completion task, since the output image should share structure with the input image. As listed in Table 1, the encoder includes one \(7\times 7\) convolutional layer with stride 1 and 3 pixels reflection padding and two \(4\times 4\) convolutional layers with stride 2 and 1 pixel reflection padding to reduce the spatial size by half while doubling the number of feature map channels. The reduced spatial dimensions allow convolution kernels to have a larger receptive field in the input image. The dilated convolution is used in the first layer of each residual block with dilation factor of 2. By employing dilated convolution, the generators can effectively enlarge the receptive field without increasing the amount of parameters and computational power. It plays an important role in the image completion task, as the context is critical for realism. The configuration of the generator is designed mainly according to the receptive field. We hope the receptive field of each neuron in the last layer can be as large as the input image size of \(256\times 256\). This means that the neurons in the last layer can see the entire input image and more information can be used to generate contents. It is essential for the image completion problem. The receptive field of the configuration listed in Table 1 is \(248\times 248\); it is very close to the input image size. In this case, the receptive field of neurons inside the generators is relatively large. In general, the kernel size corresponding to the large feature maps should be larger  
Figure 1: Overview of the PIC-EC (Parallel Image Completion with Edge and Color) framework for image completion. Masked edge map \(E_{gt}\), masked grayscale image \(I_{gray}\) and the mask \(M\) are the inputs of \(G_{1}\) to predict the full edge map \(E_{comp}\). Masked color map \(\mathcal{C}_{gt}\), masked RGB image \(I_{gt}\) and the mask \(M\) are the inputs of \(G_{2}\) to generate the full color map \(C_{comp}\). Generated edge map, generated color map and masked RGB image are then concatenated with the mask, and fed to \(G_{3}\) to get the completed image \(I_{comp}\). because the features in these feature maps tend to be larger. Therefore, the generator is designed to fulfill these conditions. In order to avoid the notorious checkerboard artifact problem, the deconvolution layers in decoders were replaced with nearest neighbor interpolation upsampling layer and convolutional layer, and the two types of layer alternated. In the image completion generator \(G_{3}\), the upsampling layers also concatenated with the edge and color information generated by \(G_{1}\) and \(G_{2}\), as shown in Figure 1. It can be seen as edge and color map insertion that effectively propagates the prior information throughout the image completion generator \(G_{3}\) to make the image completion results more realistic and accurate. It should be noted that the edge and color maps should be resized to the same size as the corresponding layers in the decoders before concatenating with them. After each convolutional layer, except the last one, there is a ReLU function. The final layer varies depending on the generator. In the generator \(G_{1}\) and \(G_{2}\), this layer has the channel size of 1 and 3 with sigmoid function, respectively.