# Parallel Image Completion with Edge and Color Map
## 3 Materials and Methods
### Loss Function
#### 3.3.1 Edge Path  
The edge path was designed for solving the constrained optimization problem in Equation (3) to obtain the edge prior. For clarity, the letter \(E\) was used to represent the edge map. The input of edge generator \(G_{1}\) was the masked edge map \(\tilde{E}_{gt}=E_{gt}\odot(1-M)\) conditioned with its masked grayscale image \(\tilde{I}_{gray}=I_{gray}\odot(1-M)\) and the mask \(M\) (1 for the missing region, 0 for background), illustrated in Figure 1. The edge map was predicted as:  
\[E_{pred}=G_{1}\left(\tilde{E}_{gt},\tilde{I}_{gray},M\right). \tag{6}\]  
The composite edge map \(E_{comp}\) was constructed by combining the background region of ground truth edge map \(E_{gt}\) with the generated edge map \(E_{pred}\) in the missing region:  
\[E_{comp}=E_{gt}\odot(1-M)+E_{pred}\odot M. \tag{7}\]  
Then, \(E_{gt}\) and \(E_{comp}\) are fed to the discriminator \(D_{1}\) for adversarial training. If the input of \(D_{1}\) is only the edge map (\(E_{gt}\) or \(E_{comp}\)), the adversarial loss is hard to optimize and the training tends to fail. This is due to the sparse property of the edge map. Unlike the natural images that have an understandable distribution on each local region, the pixels in the edge map are sparsely distributed and contain less information for the discriminator to judge whether the generated distribution is close to the real distribution or not. To address this problem, the edge maps \(E_{gt}\) and \(E_{comp}\) conditioned on \(I_{gray}\) were adopted as the inputs of discriminator \(D_{1}\). With this setup, the composite edge map \(E_{comp}\) is not only required to be similar to the ground truth edge map \(E_{gt}\), but also required to align with the edge information of the ground truth grayscale \(I_{gray}\). The discriminator then obtained adequate knowledge to tell the difference between the fake distribution and the real distribution, and the training process became stable. The network was trained with a joint loss that contained a feature matching loss \(\mathcal{L}_{fm,1}\), a content loss \(\mathcal{L}_{con}\), and an adversarial loss \(\mathcal{L}_{adv}^{G_{1}}\), and the overall loss was computed below:  
\[\mathcal{L}_{G_{1}}=\lambda_{fm,1}\mathcal{L}_{fm,1}+\lambda_{con}\mathcal{L }_{con}+\lambda_{adv,1}\mathcal{L}_{adv}^{G_{1}}, \tag{8}\]  
\[\mathcal{L}_{D_{1}}=\mathcal{L}_{adv}^{D_{1}}, \tag{9}\]  
where \(\lambda_{fm,1}\), \(\lambda_{con}\) and \(\lambda_{adv,1}\) were the regularization parameters. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Type** & **Kernel** & **Dilation** & **Stride** & **Channels** \\ \hline Convolution & \(4\times 4\) & \(1\) & \(2\times 2\) & 64 \\ Convolution & \(4\times 4\) & \(1\) & \(2\times 2\) & 128 \\ Convolution & \(4\times 4\) & \(1\) & \(2\times 2\) & 256 \\ Convolution & \(4\times 4\) & \(1\) & \(1\times 1\) & 512 \\ Convolution & \(4\times 4\) & \(1\) & \(1\times 1\) & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Detailed architecture of the discriminators. The output layer produces a score map instead of a single score, so as to tell the realism of different local regions of the input images. The feature matching loss \(\mathcal{L}_{fm,1}\) compared the feature maps in the intermediate layers of the discriminator for the real and fake inputs. It stabilized the training process by forcing the generator to produce edge maps with representations that were similar to real ones. This is similar to the perceptual loss [40] where the feature maps are compared with those from the pre-trained VGG (Visual Geometry Group) network. However, since the VGG network was not trained on the edge dataset, it failed to capture the features that were in the edge model.