## 3 Proposed Method
### Preprocessing for Probability Map Distribution Matching  
The SSIN output probability map \(\widetilde{\mathbf{x}}\) and the ground-truth static probability map \(\mathbf{y}\) have a noticeable difference in the probability distribution. More specifically, \(\widetilde{\mathbf{x}}\) has a data distribution from 0 to 1 because the last layer activation function in the SSIN is the softmax function which presents the probability of each class. On the other hand, one-hot encoded \(\mathbf{y}\) includes only two numbers, 1 representing the label and 0 for the other non-representing labels. The purpose of the discriminator is to learn and compete with the generator for the adversarial unsupervised training of the inpainting segmentation map. However, the SSIN output \(\widetilde{\mathbf{x}}\) and the static map \(\mathbf{y}\) have definite differences in data distributions. Such mismatch  
Figure 3: Overview of our architecture for the semantic segmentation inpainting network (SSIN). bothers the discriminator from learning structural features for the segmentation map and consequently lowers the performance of SSIN, concentrating on learning how to match the data distribution. Therefore, it is important to match the probability distribution of \(\widetilde{\mathbf{x}}\) and \(\mathbf{y}\) to make the SSIN produce static label maps that cannot be distinguished from ground-truth maps. We introduce a new preprocessing method for the probability map by matching the probability distribution. Our basic idea is to match the distribution of \(\mathbf{y}\) to \(\widetilde{\mathbf{x}}\) by copying both the maximum value of \(\widetilde{\mathbf{x}}\) and the non-representing values to \(\mathbf{y}\). In detail, we limit the \(\widetilde{\mathbf{x}}\) with a maximum value and fix the summation probability to 1 for each pixel. Then, by swapping the maximum position as \(\mathbf{y}\), we obtain preprocessed label map \(\overline{\mathbf{y}}\). To prevent the representative label from having a maximum value of 1 and make the other classes have a zero value, \(\tau\) is set to limit the maximum value \(\widetilde{\mathbf{x}}\), and the residuals are dispersed partially to other non-maximum classes to fool the discriminator about the probability distribution. Mathematically, by setting an upper bound \(\tau\) for the SSIN output tensor \(\widetilde{\mathbf{x}}\), the values of non-true labels are shifted close to one, and backpropagation becomes active. The output tensor \(\widetilde{\mathbf{x}}\) is clipped with a maximum value of \(\tau\) as in (1):  
\[\widetilde{\mathbf{x}}_{min}=\min(\tau,\widetilde{\mathbf{x}}), \tag{1}\]  
where each pixel-wise component of \(\widetilde{\mathbf{x}}\) has upper limitation of \(\tau\). This hyperparameter value is set near 1 to limit only the representative components. The summation of the label map per pixel should equal 1 because it represents the probability map. Therefore, the residual should be distributed to other class labels after clipping the segmentation map. Before spreading the residual to other nonrepresentative classes, the residual value can be defined as follows in (2):  
\[\widetilde{x}_{res}=\widetilde{\mathbf{x}}_{il}-(\widetilde{\mathbf{x}}_{min})_{il}, \tag{2}\]  
where \(\mathbf{i}\in N^{H\times W}\)implies the spatial position, given its representative label \(\mathbf{l}\), where \(\widetilde{x}_{res}\) is a constant residual value. If the maximum value of \(\widetilde{\mathbf{x}}_{i}\) is smaller than \(\tau\), \(\widetilde{x}_{res}\) becomes 0. After calculating the residual, it spreads proportionally, which means a larger label value has residuals, and a smaller label value has fewer.