Original: M ODEL -B ASED R OBUST A DAPTIVE S EMANTIC S EGMENTATION
Revised: M ODEL -B ASED R OBUST A DAPTIVE S EMANTIC S EGMENTATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Semantic image segmentation enjoys a wide range of applications such as autonomous vehicles and medical imaging while it is typically accomplished by deep neural networks (DNNs).
Revised: Semantic image segmentation enjoys a wide range of applications such as autonomous vehicles and medical imaging while it is typically accomplished by deep neural networks (DNNs).

Original: Nevertheless, DNNs are known to be fragile to input perturbations that are adversarially crafted or occur due to natural variations such as changes in weather or lighting conditions
Revised: Nevertheless DNNs are known to be fragile to input perturbations that are adversarially crafted or occur due to natural variations, such as changes in weather or lighting conditions

Original: This issue of lack of robust - ness prevents the application of learning based semantic segmentation methods on safety-critical applications.
Revised: This issue of lack of robustness prevents the application of learning-based semantic segmentation methods on safety-critical applications.

Original: To mitigate this challenge, in this paper, we propose a new robust training algorithm, called MRTAdapt, for Model-based Robust
Revised: To mitigate this challenge, in this paper, we propose a new robust training algorithm, called MRTAdapt, for Model-based Robust

Original: Adaptive training, to enhance the robustness of DNN-based semantic segmentation methods against natural variations that leverages model-based robust training algorithms and generative adversarial networks.
Revised: Adaptive training, to enhance the robustness of DNN-based semantic segmentation methods against natural variations that leverages model-based robust training algorithms and generative adversarial networks.

Original: Natural variation effects are minimized from both image and label sides.
Revised: Natural variation effects are minimized from both image and label sides.

Original: We provide extensive experimental results on both real-world and synthetic datasets demonstrating that model-based robust adaptive training algorithm outperforms multiple state-of-the-art models under various natural variations.
Revised: We provide extensive experimental results on both real-world and synthetic datasets demonstrating that the proposed training algorithm result in robust models that outperform multiple state-of-the-art models under various natural variations.

Original:  INTRODUCTION
Revised:  INTRODUCTION

Original: In recent years, computer vision has become one of the most promising research areas in deep learning because it has empowered a great amount of industry-level applications.
Revised: In recent years, computer vision has become one of the most promising research areas in deep learning because it has empowered a great amount of industry-level applications.

Original: In the context of computer vision, semantic segmentation is a core task formulated as a dense labeling problem
Revised: In the context of computer vision, semantic segmentation is a core task formulated as a dense labeling problem

Original: Luc et al.
Revised: Luc et al.

Original: (2016), targeting to allocate every pixel with a label Hsin et al.
Revised: (2016), targeting to allocate every pixel with a label Hsin et al.

Original: (2019) of what is being represented.
Revised: (2019) of what is being represented.

Original: There are many applications now being used have made great progress with the help of semantic segmentation, such as medical image processing Xue et al.
Revised: There are many applications now being used have made great progress with the help of semantic segmentation, such as medical image processing Xue et al.

Original: (2018), autonomous vehicles
Revised: (2018), autonomous vehicles

Original: Zhao et al.
Revised: Zhao et al.

Original: and
Revised: and

Original: robotics Wang et al.
Revised: robotics Wang et al.

Original: However, many applications of semantic segmentation are life-critical, which means that low model accuracy may pose direct threat to human safety Oakden-Rayner et al.
Revised: However, many applications of semantic segmentation are life-critical, which means that low model accuracy may pose direct threat to human safety Oakden-Rayner et al.

Original: Therefore, it is critical to design training algorithms that can enhance robustness of segmentation methods against input perturbations.
Revised: Therefore, it is critical to design training algorithms that can enhance robustness of segmentation methods against input perturbations.

Original: A signiﬁcant number of researches focusing on the robustness of computer vision have been conducted in recent years Arnab et al.
Revised: A signiﬁcant number of researches focusing on the robustness of computer vision have been conducted in recent years Arnab et al.

Original: Kamann & Rother (2020)
Revised: Kamann & Rother (2020)

Original: Robey et al.
Revised: Robey et al.

Original: Tramer et al.
Revised: Tramer et al.

Original: However, the majority of existing works focus on image classiﬁcation tasks Hendrycks &
Revised: However, the majority of existing works focus on image classiﬁcation tasks Hendrycks &

Original: Dietterich (2019).
Revised: Dietterich (2019).

Original: Robust semantic segmentation methods against adversarial attacks have been proposed in Hsin et al.
Revised: Robust semantic segmentation methods against adversarial attacks have been proposed in Hsin et al.

Original: Xue et al.
Revised: Xue et al.

Original: Hung et al.
Revised: Hung et al.

Original: Xie et al.
Revised: Xie et al.

Original: Work conducted by Goodfellow et al.
Revised: Work conducted by Goodfellow et al.

Original: (2015) proved that neural networks can be easily misled by some intentionally designed yet imperceptible perturbations to generate an incorrect answer with high conﬁdence.
Revised: (2015) proved that neural networks can be easily misled by some intentionally designed yet imperceptible perturbations to generate an incorrect answer with high conﬁdence.

Original: In computer vision, adversarial attacks apply pixel-level changes onto the image that lead the model to wrong predictions, where the attacked image looks perceptually similar to the original one Ma et al.
Revised: In computer vision, adversarial attacks apply pixel-level changes onto the image that lead the model to wrong predictions, where the attacked image looks perceptually similar to the original one Ma et al.

Original: The perturbation-based robust training algorithms have already tackled this problem.
Revised: Perturbation-based robust training algorithms have already tackled this problem.

Original: But in real life, there may be changes that raised by some out-of-distribution variations such as snow weather or extreme brightness that can not be represented using several pixel-level changes.
Revised: Nevertheless, in real life, there may be changes that raised by some out-of-distribution variations such as snow weather or extreme brightness that can not be represented using small pixel-level changes.

Revised: Current works on robustness mainly focus on adapting the domain gap on a single side, i.e., either from image or label side.

Revised: For instance, Robey et al.

Revised: (2020) proposed a model-based robust learning architecture which is applied on Convolutional Neural Networks(CNN) to maintain high prediction accuracy under natural variations for image classiﬁcation tasks.

Revised: Yet it is highly dependent on the performance of the natural variation model that capture the changes from source to target domain, which makes the algorithm less robust and less scalable in semantic segmentation task.

Original:  In this paper, we build upon Robey et al.
Revised: Meanwhile, AdaptSegNet proposed by Tsai et al. (2020) minimizes the distribution gap on the output side. In this paper, we build upon Robey et al.

Original: to design a new robust training algorithm for semantic segmentation tasks.
Revised: to design a new robust training algorithm for semantic segmentation tasks.

Original: The objective is to build a segmentation model that generates high accuracy predictions under natural variation effects.
Revised: The objective is to build a segmentation model that generates high accuracy predictions under natural variation effects.

Original: Inspired by Tsai et al.
Revised: Our method also utilize the idea from Tsai et al.

Original: (2020) that images from different domains with great appearance difference may share some similarity on the label side such as spatial layout and local context, we showed that  semantic feature map of an image under any natural variations remains unchanged which we refer to as semantic meaning invariance.
Revised: (2020) that images from different domains with great appearance difference may share some similarity on the label side such as spatial layout and local context. We showed that the semantic feature map of an image under any natural variations remains unchanged which we refer to as semantic meaning invariance.

Original: Our proposed training algorithm is minimizing the gap on both image and label side.
Revised: Our proposed training algorithm minimizes the gap on both image and label side.

Original: On image side, the model-based robust training algorithm is applied to train  model using the simulated target domain images to enhance robustness.
Revised: On image side, the model-based robust training algorithm is applied to train a model using the simulated target domain images to enhance robustness.

Original: On label side, we apply generative adversarial network (GAN) to minimize the feature map gap between simulated natural variation images and target domain images.
Revised: On label side, we apply generative adversarial networks (GANs) to minimize the feature map gap between simulated natural variation images and target domain images.

Revised: We have also included extensive comparisons showing that our method outperforms related state-of-the-art works in domain adaptation.

Original: Our contributions are: 1) We propose MRTAdapt, a new model-based training algorithm to enhance robustness of DNN-based semantic segmentation methods against natural variations.
Revised: Our contributions are: 1) We propose MRTAdapt, a new model-based training algorithm to enhance robustness of DNN-based semantic segmentation methods against natural variations.

Original: We build on top of generative adversarial network and model-based robust training algorithm to minimize the gap on both image and label side to enhance robustness.
Revised: We build on top of generative adversarial networks and model-based robust training algorithms to minimize the gap on both image and label side to enhance robustness.

Original: Our results on Cityscapes Cordts et al.
Revised: Our results on Cityscapes Cordts et al.

Original: (2016) and Synthia Ros et al.
Revised: (2016) and Synthia Ros et al.

Original: datasets show that our method outperforms multiple state-ofthe-art domain adaptation techniques, namely AdaptSegNet Tsai et al.
Revised: datasets show that our method outperforms multiple state-ofthe-art domain adaptation techniques, such as AdaptSegNet Tsai et al.

Original: (2020), ADVENT Vu et al.
Revised: (2020), ADVENT Vu et al.

Original: (2019) and FDA Yang & Soatto (2020).
Revised: (2019) and FDA Yang & Soatto (2020).

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Semantic Segmentation.
Revised: Semantic Segmentation.

Original: In the past decades, Convolution Neural Networks(CNN) are widely used in semantic segmentation.
Revised: In the past decades, Convolution Neural Networks(CNN) are widely used in semantic segmentation.

Original: Current state-of-the-art semantic segmentation frameworks are mostly developed from Fully Convolutional Network (FCN) by Long et al.
Revised: Current state-of-the-art semantic segmentation frameworks are mostly developed from Fully Convolutional Network (FCN) by Long et al.

Original: ResNet proposed by He et al.
Revised: ResNet proposed by He et al.

Original: (2015) used a residual block to sum the nonlinear activation output and identity mapping, which is proved to improve the gradient propagation and increase the accuracy of semantic segmentation.
Revised: (2015) used a residual block to sum the nonlinear activation output and identity mapping, which is proved to improve the gradient propagation and increase the accuracy of semantic segmentation.

Original: DenseNetby Huang et al.
Revised: DenseNet Huang et al.

Original: (2018a) is regarded as the extension of ResNet, uses the concatenation of previous feature maps called dense block.
Revised: (2018a) builds upon ResNet and uses the concatenation of previous feature maps called dense block.

Original: This gives each layer in DenseNet information from all preceding layers.
Revised: This gives each layer in DenseNet information from all preceding layers.

Original: Jégou et al.
Revised: Jégou et al.

Original: (2017) extended DenseNet into FCN architecture.
Revised: (2017) extended DenseNet into FCN architecture.

Original: SegNet proposed by Badrinarayanan et al.
Revised: SegNet proposed by Badrinarayanan et al.

Original: (2017) introduces the deep convolution encoder-decoder architecture to the ﬁeld of semantic segmentation.
Revised: (2017) introduces the deep convolution encoder-decoder architecture to the ﬁeld of semantic segmentation.

Original: In 2015, Zhou et al. (2015) proved that empirical size of receptive ﬁeld is much smaller than the theoretical size.
Revised: Also, in Zhou et al. (2015), it is proved that empirical size of receptive ﬁeld is much smaller than the theoretical size.

Original: Chen et al.
Revised: Chen et al.

Original: and Yu & Koltun (2016) used dilated convolution to enlarge the receptive ﬁeld.
Revised: and Yu & Koltun (2016) used dilated convolution to enlarge the receptive ﬁeld.

Original: ParseNet by Liu et al.
Revised: ParseNet by Liu et al.

Original: (2015) adds global context to CNN for semantic segmentation.
Revised: (2015) adds global context to CNNs for semantic segmentation.

Original: He et al.
Revised: He et al.

Original: (2014) introduced spatial pyramid pooling in DCNN.
Revised: (2014) introduced spatial pyramid pooling in DCNN.

Original: PSPNet introduced by Zhao et al.
Revised: PSPNet introduced by Zhao et al.

Original: (2017) uses a novel global pyramid pooling module to capture both global context information.
Revised: (2017) uses a novel global pyramid pooling module to capture both global context information.

Original: Duta et al.
Revised: Duta et al.

Original: (2020) extended the idea by combining both local and global Pyramidal Convolution blocks in the neural network.
Revised: (2020) extended the idea by combining both local and global Pyramidal Convolution blocks in the neural network model.

Original: Chen et al.
Revised: Chen et al.

Original: (2017b) introduced atrous spatial pyramid pooling.
Revised: (2017b) introduced atrous spatial pyramid pooling.

Original: Depth image is also used along with original RGB information for semantic segmentation by Wang et al.
Revised: Depth image is also used along with original RGB information for semantic segmentation by Wang et al.

Original: For actual applications, Zhao et al.
Revised: For actual applications, Zhao et al.

Original: (2018) proposed ICNet for real-time semantic segmentation accomplishing fast inference without sacriﬁcing too much quality left behind.
Revised: (2018) proposed ICNet for real-time semantic segmentation accomplishing fast inference without sacriﬁcing too much quality left behind.

Original: Azimi et al.
Revised: Azimi et al.

Original: (2020) proposed aerial perspective dataset for dense semantic segmentation.
Revised: (2020) proposed aerial perspective dataset for dense semantic segmentation.

Original: Domain Adaptation.
Revised: Domain Adaptation.

Original: Combining with the techniques of Generative Adversarial Network (GAN)
Revised: Combining with the techniques of Generative Adversarial Network (GAN)

Original: Goodfellow et al.
Revised: Goodfellow et al.

Original: , Ganin & Lempitsky (2015) proposed DANN to reduce the distribution gap between different domains by using discriminator to make the prediction cannot be identiﬁed between source and target domain.
Revised: , Ganin & Lempitsky (2015) proposed DANN to reduce the distribution gap between different domains by using discriminator to make the prediction cannot be identiﬁed between source and target domain.

Original: Pan et al.
Revised: Pan et al.

Original: ﬁrst separate target domain into splits based on entropy-based ranking and then deploy self-supervised adaptation technique to reduce the domain gap between synthetic data and real images.
Revised: ﬁrst separate target domain into splits based on entropy-based ranking and then deploy self-supervised adaptation technique to reduce the domain gap between synthetic data and real images.

Original: PIT proposed by Lv et al.
Revised: PIT proposed by Lv et al.

Original: (2020) constructs pivot information shared across domains.
Revised: (2020) constructs pivot information shared across domains.

Original: Chen et al.
Revised: Chen et al.

Original: (2019) used depth image as guided information to build adaptation method from synthetic to real dataset.
Revised: (2019) used depth image as guided information to build adaptation method from synthetic to real dataset.

Original:  MUNIT proposed by Huang et al.
Revised: In the ﬁeld of utilizing synthetic datasets to auxiliate the training process of real-world images, MUNIT proposed by Huang et al.

Original: (2018b) learn conditional distribution of target domain which can separate domain-invariant semantic content of an image from domain-speciﬁc properties.
Revised: (2018b) learn conditional distribution of target domain which can separate domain-invariant semantic content of an image from domain-speciﬁc properties.

Original: Vu proposed ADVENT model which maximize prediction certainty in target domain by introducing entropy loss
Revised: Vu proposed ADVENT model which maximize prediction certainty in target domain by introducing entropy loss

Original: Vu et al.
Revised: Vu et al.

Original: Zhu proposed CycleGAN Zhu et al.
Revised: Zhu proposed CycleGAN Zhu et al.

Original: (2020) which was trying to learn a mapping such that the distribution of generated image is indistinguishable from the target domain distribution using adversarial loss.
Revised: (2020) which was trying to learn a mapping such that the distribution of generated image is indistinguishable from the target domain distribution using adversarial loss.

Original: AdaptSegNet proposed by Tsai et al.
Revised: AdaptSegNet proposed by Tsai et al.

Original: (2020) aimed to reduce the gap between the outputs from source and target domain given that images might be very different in appearance.
Revised: (2020) aimed to reduce the gap between the outputs from source and target domain given that images might be very different in appearance.

Original: FDA proposed by Yang & Soatto (2020) uses Fast Fourier Transform(FFT) to adapt source and target domain.
Revised: FDA proposed by Yang & Soatto (2020) uses Fast Fourier Transform(FFT) to adapt source and target domain.

Original: Robustness.
Revised: Robustness.

Original:  Hendrycks & Dietterich (2019) established rigorous benchmarks for image classiﬁcation and proposed a series of image corruption examples.
Revised: As for robustness against corrupted images, Hendrycks & Dietterich (2019) established rigorous benchmarks for image classiﬁcation and proposed a series of image corruption examples.

Revised: Kamann & Rother (2020) showed that robustness increases with the performance of the semantic segmentation model and dense prediction cell was only designed to improve performance on clean data.

Original:  Arnab et al.
Revised: Regarding robustness on semantic segmentation, Arnab et al.

Original: (2018) did the ﬁrst evaluation of adversarial attacks on semantic segmentation and analyzed multi-scale processing.
Revised: (2018) did the ﬁrst evaluation of adversarial attacks on semantic segmentation and analyzed multi-scale processing.

Original: Kamann &

Original: Rother (2020) showed that robustness increases with the performance of the semantic segmentation model and dense prediction cell was only designed to improve performance on clean data.

Original: Robey et al.
Revised: Robey et al.

Original: (2020) proposed model-based robust training architecture focusing on the topic of image classiﬁcation.
Revised: (2020) proposed model-based robust training architecture focusing on the topic of image classiﬁcation.

Original: Wong & Kolter (2020) bridged the gap between real-world perturbations and adversarial defenses by learning perturbation sets from data, through common image corruptions.
Revised: Wong & Kolter (2020) bridged the gap between real-world perturbations and adversarial defenses by learning perturbation sets from data, through common image corruptions.

Original:  M ETHOD
Revised:  M ETHOD

Original: In this section we propose a new robust training algorithm for semantic segmentation tasks.
Revised: In this section we propose a new robust training algorithm for semantic segmentation tasks.

Original: Particularly, in section 3.1, we deﬁne the semantic segmentation task and the natural variations.
Revised: Particularly, in section 3.1, we deﬁne the semantic segmentation task and the natural variations.

Original: In section 3.2, we give the outline of our proposed algorithm.
Revised: In section 3.2, we give the outline of our proposed algorithm.

Original: In section 3.3, we deﬁne semantic meaning invariance and clarify its connection to our algorithm.
Revised: In section 3.3, we deﬁne semantic meaning invariance and clarify its connection to our algorithm.

Original: In section 3.4, we introduce the detailed training procedure of our model-based robust adaptive training algorithm(MRTAdapt).
Revised: In section 3.4, we introduce the detailed training procedure of our model-based robust adaptive training algorithm(MRTAdapt).

Original: 3. S EMANTIC S EGMENTATION AND N ATURAL V ARIATION
Revised: 3. S EMANTIC S EGMENTATION AND N ATURAL V ARIATION

Original: Consider an input x ∈ R H × W × 3 representing an RGB image with three channels and its corresponding label map
Revised: Consider an input x ∈ R H × W × 3 representing an RGB image with three channels and its corresponding label map

Original: y ∈ R H
Revised: y ∈ R H

Original: × W which is annotated for every pixel in the image.
Revised: × W which is annotated for every pixel in the image.

Original: A semantic segmentation task is to train a neural network that generates pixel-wise segmentation feature map ˆ y ∈ R H × W .
Revised: A semantic segmentation task is to train a neural network that generates pixel-wise segmentation feature map ˆ y ∈ R H × W .

Original: We assume the dataset is drawn i.i.d.
Revised: We assume the dataset is drawn i.i.d.

Original: from distribution ( x, y ) ∼ P .
Revised: from distribution ( x, y ) ∼ P .

Original: The optimization objective in semantic segmentation is to ﬁnd the best weight w that minimizes the loss function
Revised: The optimization objective in semantic segmentation is to ﬁnd the best weight w that minimizes the loss function

Original: L ( x, y ; w ) with respect to input x , label y and weight w , which can be written as:
Revised: L ( x, y ; w ) with respect to input x , label y and weight w , which can be written as:

Original: [Equation]
Revised: [Equation]

Original: Next we denote by δ ∈ ∆ a natural variation applied to x that is derived from a nuisance space ∆ .
Revised: Next we denote by δ ∈ ∆ a natural variation applied to x that is derived from a nuisance space ∆ .

Original: A model for the natural variation can be learned from data.
Revised: A model of natural variation can be learned from the difference of source and target domain images.

Original: Particularly, we denote the natural variation model that transforms the input image x from the source domain into an image in the (perturbed) target domain as V ( x, δ ) .
Revised: Particularly, we denote the natural variation model that transforms the input image x from the source domain into an image in the (perturbed) target domain as V ( x, δ ) .

Original: Note that, in general, it is hard to write a natural variation model learned from data into an analytical form because there is no closed form expression capturing the natural perturbation.
Revised: Note that, in general, it is hard to write a natural variation model learned from data into an analytical form because there is no closed form expression capturing the natural perturbation.

Original: Natural variation model can be customized regarding the needs of each dataset.
Revised: Natural variation model can be customized regarding the needs of each dataset.

Original: In this paper, our goal is to design a semantic segmentation method that is robust to natural variations δ .
Revised: In this paper, our goal is to design a semantic segmentation method that is robust to natural variations δ .

Original: 3. A LGORITHM O VERVIEW
Revised: 3. A LGORITHM O VERVIEW

Original: Our proposed model-based robust adaptive training algorithm (MRTAdapt) shown in Figure 1 has two modules: a segmentation network G using a model-based robust training algorithm to enhance robustness on image side, and a discriminator network D to enhance robustness on label side.
Revised: Our proposed model-based robust adaptive training algorithm (MRTAdapt) shown in Figure 1 has two modules: a segmentation network G using a model-based robust training algorithm to enhance robustness on image side, and a discriminator network D to enhance robustness on label side.

Original: Two sets of images from source and target domain will have the notation of I s and I t , while the image generated using natural variation model will be denoted as I nv .
Revised: Two sets of images from source and target domain will have the notation of I s and I t , while the image generated using natural variation model will be denoted as I nv .

Original: After using source domain images to train the model, we transform images from source domain with natural variation model.
Revised: After using source domain images to train the model, we transform images from source domain with natural variation model.

Original: Then we use simulated natural variation images to ﬁnd the highest segmentation loss by applying the model-based robust training algorithm to optimize the segmentation network.
Revised: Then we use simulated natural variation images to ﬁnd the highest segmentation loss by applying the model-based robust training algorithm to optimize the segmentation network.

Original: Then we predict the segmentation softmax output for the unlabeled target domain images I t by forwarding the target images into the same segmentation network.
Revised: Then we predict the segmentation softmax output for the unlabeled target domain images I t by forwarding the target images into the same segmentation network.

Original: To minimize the gap between the output of natural variation images and target domain images, we use these two predictions as the input to the discriminator to distinguish whether the feature map generated by segmentation network is from natural variation model or target domain.
Revised: To minimize the gap between the output of natural variation images and target domain images, we use these two predictions as the input to the discriminator to distinguish whether the feature map generated by segmentation network is from natural variation model or target domain.

Original: Using an adversarial loss for discriminator, the loss will back-propagate to segmentation network to help segmentation network to generate similar distributions in target domain to natural variation.
Revised: Using an adversarial loss for discriminator, the loss will back-propagate to segmentation network to help segmentation network to generate similar distributions in target domain to natural variation.

Original: Detailed information will be covered in the following sections.
Revised: Detailed information will be covered in the following sections.

Original: 3. S EMANTIC M EANING I NVARIANCE
Revised: 3. S EMANTIC M EANING I NVARIANCE

Original: Images under different kinds of natural variations have huge visual perspective difference.
Revised: Images under different kinds of natural variations have huge visual perspective difference.

Original: For domain adaptation problem in computer vision, the motivation of most works is based on the fact that annotating images in a new domain setting is time-consuming and extremely expensive.
Revised: For domain adaptation problem in computer vision, the motivation of most works is based on the fact that annotating images in a new domain setting is time-consuming and extremely expensive.

Original: In this paper, we are trying to simulate the real-world setting when the model is trained with annotated images under normal condition in the source domain but natural variation conditions from target domain occur during inference time.
Revised: In this paper, we are trying to simulate the real-world setting when the model is trained with annotated images under normal condition in the source domain but natural variation conditions from target domain occur during inference time.

Original: Most of the current works focus on improving the performance from the image side.
Revised: Most of the current works focus on improving the performance from the image side.

Original: AdaptSegNet tries to solve this problem from the prospective of the label side.
Revised: AdaptSegNet tries to solve this problem from the prospective of the label side.

Original: The experiment conducted in this work has proved that spatial layout and local context of labels are similar for different datasets for semantic segmentation problem Tsai et al.
Revised: The experiment conducted in this work has proved that spatial layout and local context of labels are similar for different datasets for semantic segmentation problem Tsai et al.

Original: Inspired by the idea of minimizing the gap on label size, we found the semantic information will remain the same from the label perspective regardless of the types of natural variation on the image.
Revised: Inspired by the idea of minimizing the gap on label size, we found the semantic information will remain the same from the label perspective regardless of the types of natural variation on the image.

Original: For example, the car in the image under strong brightness will remain the semantic meaning as a car compared to normal setting.
Revised: For example, the car in the image under strong brightness will remain the semantic meaning as a car compared to normal setting.

Original: Take Figure 2 as an example, left side is the same image with different brightness condition compared to the right side.
Revised: Take Figure 2 as an example, left side is the same image with different brightness condition compared to the right side.

Original: Since the image is only changed into a brighter version while all the objects remain the same, the semantic information will be unchanged under natural variation.
Revised: Since the image is only changed into a brighter version while all the objects remain the same, the semantic information will be unchanged under natural variation.

Original: In the context of semantic segmentation, unchanged semantic information means labels will be the same.
Revised: In the context of semantic segmentation, unchanged semantic information means labels will be the same.

Original: Thus, labels in the feature map will remain unchanged when we apply natural variation model on source domain image to transform into target domain.
Revised: Thus, labels in the feature map will remain unchanged when we apply natural variation model on source domain image to transform into target domain.

Original: 3. M ODEL -B ASED R OBUST A DAPTIVE T RAINING A LGORITHM
Revised: 3. M ODEL -B ASED R OBUST A DAPTIVE T RAINING A LGORITHM

Original: Different from AdaptSegNet where the goal was to minimize the gap between the label from source domain and target domain, we build on top of model-based robust training algorithm with different optimization goal.
Revised: Different from AdaptSegNet where the goal was to minimize the gap between the label from source domain and target domain, we build on top of model-based robust training algorithm with different optimization goal.

Original: Using the idea of semantic meaning invariance, we treat the generated natural variation image generated as "source" domain image.
Revised: Using the idea of semantic meaning invariance, we treat the generated natural variation image generated as "source" domain image.

Original: For the reason that the generated natural variation image is chosen by model-based robust training algorithm is aiming to represent the worst-case scenario of natural variation.
Revised: For the reason that the generated natural variation image is chosen by model-based robust training algorithm is aiming to represent the worst-case scenario of natural variation.

Original: The objective of our proposed model-based robust adaptive training algorithm is trying to reduce the gap on both image and label side.
Revised: The objective of our proposed model-based robust adaptive training algorithm is trying to reduce the gap on both image and label side.

Original: Here we formulate the optimization task as a min-max problem, which will be minimizing the segmentation loss for source and natural variation images, while maximizing the probability of natural variation predictions being treated as target domain feature map.
Revised: Here we formulate the optimization task as a min-max problem, which will be minimizing the segmentation loss for source and natural variation images, while maximizing the probability of natural variation predictions being treated as target domain feature map.

Original: Our proposed algorithm is an integration of model-based robust training algorithm and generative adversarial network.
Revised: Our proposed algorithm is an integration of model-based robust training algorithm and generative adversarial network.

Original: [Equation]
Revised: [Equation]

Original: 3.4. M ODEL -B ASED R OBUST T RAINING F ORMULATION
Revised: 3.4. M ODEL -B ASED R OBUST T RAINING F ORMULATION

Original: The objective of the model-based robust semantic segmentation algorithm is to train the model so that it can be robust to natural variations such changes in the weather or lightning conditions.
Revised: The objective of the model-based robust semantic segmentation algorithm is to train the model so that it can be robust to natural variations such changes in the weather or lightning conditions.

Original: Given that we only have labeled data for source domain images while labels for target domain are unavailable, our goal is to simulate and transform the normal condition training data into natural variation cases and then, the model can be trained until it generates high accuracy prediction under abnormal conditions.
Revised: Given that we only have labeled data for source domain images while labels for target domain are unavailable, our goal is to simulate and transform the normal condition training data into natural variation cases and then, the model can be trained until it generates high accuracy prediction under abnormal conditions.

Original: In general, the concept of this algorithm can be treated as a data augmentation method that widens the range of conditions under which the model can attain high performance.
Revised: In general, the concept of this algorithm can be treated as a data augmentation method that widens the range of conditions under which the model can attain high performance.

Original: The objective of the robust model-based training algorithm can be formulated as Robey et al.
Revised: The objective of the robust model-based training algorithm can be formulated as Robey et al.

Original: [Equation]
Revised: [Equation]

Original: The inner maximization represents the loss inferred by the natural variation model.
Revised: The inner maximization represents the loss inferred by the natural variation model.

Original: Given an input image x from source domain, we need to search in the nuisance space ∆ for a local optimal parameter δ that generates the worst-case scenario of natural variation.
Revised: Given an input image x from source domain, we need to search in the nuisance space ∆ for a local optimal parameter δ that generates the worst-case scenario of natural variation.

Original: The optimal nuisance parameter is denoted as δ ∗ that infers biggest loss between prediction and original label map.
Revised: The optimal nuisance parameter is denoted as δ ∗ that infers biggest loss between prediction and original label map.

Original: The objective of the inner maximization problem is to maximize the segmentation loss of the model under natural variation.
Revised: The objective of the inner maximization problem is to maximize the segmentation loss of the model under natural variation.

Original: The objective of outer minimization problem is to train the weights of the model w to be robust against worst-case natural variation condition upon convergence, which is represented in the form of minimizing the sum of the loss in this case.
Revised: The objective of outer minimization problem is to train the weights of the model w to be robust against worst-case natural variation condition upon convergence, which is represented in the form of minimizing the sum of the loss in this case.

Original: During actual implementation, we assume the number of datapoints being trained is a ﬁnite number N .
Revised: During actual implementation, we assume the number of datapoints being trained is a ﬁnite number N .

Original: This means a dataset being recognized as training set will contain N image-label pairs: { x i , y i } Ni =1 .
Revised: This means a dataset being recognized as training set will contain N image-label pairs: { x i , y i } Ni =1 .

Original: Also, the natural variation model being applied in the algorithm is learned from data.
Revised: Also, the natural variation model being applied in the algorithm is learned from data.

Original: Model-based Robust Training generates random domain-speciﬁc style code for input images from source domain and ﬁlters the style codes which infer maximal loss out for weights update.
Revised: Model-based Robust Training generates random domain-speciﬁc style code for input images from source domain and ﬁlters the style codes which infer maximal loss out for weights update.

Original: That means, for every mini-batch size of input images ( x, y ) from source domain, we apply the natural variation model to transform the image into target domain, where we will treat the transformed image and original label as a new image-label pair denoted as ( V ( x, δ ) , y ) .
Revised: That means, for every mini-batch size of input images ( x, y ) from source domain, we apply the natural variation model to transform the image into target domain, where we will treat the transformed image and original label as a new image-label pair denoted as ( V ( x, δ ) , y ) .

Original: Then we utilize both pairs of datapoints to train the network for the outer minimization problem.
Revised: Then we utilize both pairs of datapoints to train the network for the outer minimization problem.

Revised: 3.4. N ETWORK T RAINING P ROCEDURES

Revised: For the segmentation network G , we deﬁne the segmentation loss as a weighted sum from both the loss inferred from source domain images as well as the highest loss derived from natural variation images using model-based robust training algorithm.

Revised: Two losses are connected via λ nv and λ ori , which represent the contributions of generated natural variation images and source domain images to the segmentation network respectively:

Revised: [Equation]

Original: Algorithm
Revised: Algorithm

Original: 1: δ ∗ ← 0 , L max ←2: foreach Batch( x, y ) do 3: for ksteps do 4:
Revised: 1: δ ∗ ← 0 , L max ←2: foreach Batch x, y ) do 3: for k do 4:

Original: Sampleδ from∆ uniformly random 5:
Revised: Sample from uniformly random 5:

Original: L current ← L ( V ( x, δ ) , y ; w ) 6: if L current > L max then 7:
Revised: L current ← L ( V ( x, δ ) , y ; w ) 6: if L current > L max then 7:

Original: L max ← L current 8: δ ∗ ← δ 9: end if 10: end for 11: ∇ ← ∇ w ( λ nv L ( V ( x, δ ∗ ) , y ; w ) + λ ori L ( x, y ; w )) 12: end
Revised: L max ← L current 8: δ ∗ ← δ 9: end if 10: end for 11: ∇ ← ∇ w ( λ nv L ( V ( x, δ ∗ ) , y ; w ) + λ ori L ( x, y ; w )) 12: end

Original: 3.4. N ETWORK T RAINING P ROCEDURES

Original: For the segmentation network G , we deﬁne the segmentation loss as a weighted sum from both the loss inferred from source domain images as well as the highest loss derived from natural variation images using model-based robust training algorithm.

Original: Two losses are connected via λ nv and λ ori , which represent the contributions of generated natural variation images and source domain images to the segmentation network respectively:

Original: [Equation]

Original: For target domain images, we feed them into segmentation network and obtain an adversarial loss, which aims to fool the discriminator and maximize the probability of target domain predictions P t being considered natural variation predictions.
Revised: For target domain images, we feed them into segmentation network and obtain an adversarial loss, which aims to fool the discriminator and maximize the probability of target domain predictions P t being considered natural variation predictions.

Original: For the reason that we do not have the ground truth feature map of target domain images, segmentation loss will not be calculated.
Revised: For the reason that we do not have the ground truth feature map of target domain images, segmentation loss will not be calculated.

Original: [Equation]
Revised: [Equation]

Original: For the discriminator training, we forward a given segmentation softmax output P to a fullyconvolutional discriminator using cross-entropy loss for two classes (natural variation and target domain).
Revised: For the discriminator training, we forward a given segmentation softmax output P to a fullyconvolutional discriminator using cross-entropy loss for two classes (natural variation and target domain).

Original: The loss can be written as:
Revised: The loss can be written as:

Original: [Equation]
Revised: [Equation]

Original: So the min-max problem can be rewritten into the following form, where segmentation loss L seg and adversarial loss L adv is summed by applying corresponding weights representing the contribution to the network:
Revised: So the min-max problem can be rewritten into the following form, where segmentation loss L seg and adversarial loss L adv is summed by applying corresponding weights representing the contribution to the network:

Original: [Equation]
Revised: [Equation]

Original:  I MPLEMENTATION 4. D ATASET P REPROCESSING
Revised:  I MPLEMENTATION 4. D ATASET P REPROCESSING

Original: Brightness Characteristic of an image can be represented using Hue-Saturation-Value (HSV), where hue, saturation and value represent the color, grayness and brightness respectively
Revised: Brightness Characteristic of an image can be represented using Hue-Saturation-Value (HSV), where hue, saturation and value represent the color, grayness and brightness respectively

Original: Latifah et al.
Revised: Latifah et al.

Original: In this paper, we use natural variation model to simulate target domain images.
Revised: In this paper, we use natural variation model to simulate target domain images.

Original: To achieve a better simulation result, we arrange our data preprocessing for brightness condition into two phase:
Revised: To achieve a better simulation result, we arrange our data preprocessing for brightness condition into two phase:

Original: Phase 1: divide original dataset into three subsets: bright, medium and dark by calculating the average Hue-Saturation-Value of each image.
Revised: Phase 1: divide original dataset into three subsets: bright, medium and dark by calculating the average Hue-Saturation-Value of each image.

Original: In this paper, we divide the three subsets by their HSV values with 72 and 90 representing the boundary of dark-medium and medium-bright on Cityscapes dataset, using 75 and 130 for Synthia dataset.
Revised: In this paper, we divide the three subsets by their HSV values with 72 and 90 representing the boundary of dark-medium and medium-bright on Cityscapes dataset, using 75 and 130 for Synthia dataset.

Original: For training set, use bright subset as source domain and dark subset as target domain to simulate the fact that most current dataset is constrained in daytime condition.
Revised: For training set, use bright subset as source domain and dark subset as target domain to simulate the fact that most current dataset is constrained in daytime condition.

Original: Dark subset is referring to the case when a vehicle is moving at nighttime under low brightness condition.
Revised: Dark subset is referring to the case when a vehicle is moving at nighttime under low brightness condition.

Original: There are 931 adjusted source domain images representing bright condition, and 486 target domain images representing dark condition for Cityscapes dataset.
Revised: There are 931 adjusted source domain images representing bright condition, and 486 target domain images representing dark condition for Cityscapes dataset.

Original: For Synthia dataset, bright and dark images are 1200 and 1174 respectively.
Revised: For Synthia dataset, bright and dark images are 1200 and 1174 respectively.

Original: Phase 2: adjust the overall brightness of bright subset by applying OpenCV library to enlarge the difference of brightness between training set and testing set.
Revised: Phase 2: adjust the overall brightness of bright subset by applying OpenCV library to enlarge the difference of brightness between training set and testing set.

Original: Snow For snow weather as natural variation, we eliminate the effect of brightness and rearrange the training set with different conﬁguration.
Revised: Snow For snow weather as natural variation, we eliminate the effect of brightness and rearrange the training set with different conﬁguration.

Original: For Cityscapes, we have source domain subset withimages and target domain subset with 1064 images.
Revised: For Cityscapes, we have source domain subset withimages and target domain subset with 1064 images.

Original: For Synthia, source domain and target domain have 1000 and 1170 images respectively.
Revised: For Synthia, source domain and target domain have 1000 and 1170 images respectively.

Original: Both subsets contain images in bright and dark condition, which allows the network to be exposed to images from different brightness condition.
Revised: Both subsets contain images in bright and dark condition, which allows the network to be exposed to images from different brightness condition.

Original: So the natural variation of brightness effect discussed in previous section is minimized.
Revised: So the natural variation of brightness effect discussed in previous section is minimized.

Original: We remained unchanged for source domain images, referring to the case that we train our model in normal condition.
Revised: We remained unchanged for source domain images, referring to the case that we train our model in normal condition.

Original: Then we apply the snow corruption operation being used in ImageNet-C with severity of 1 to target domain images, referring to the case that our model would be exposed to natural variation images during inference.
Revised: Then we apply the snow corruption operation being used in ImageNet-C with severity of 1 to target domain images, referring to the case that our model would be exposed to natural variation images during inference.

Original: Due to the limitation of computing resources, the image size being trained in this paper is relatively small.
Revised: Due to the limitation of computing resources, the image size being trained in this paper is relatively small.

Original: So we apply a proper level of severity for snow effect so that the performance will not break down completely.
Revised: So we apply a proper level of severity for snow effect so that the performance will not break down completely.

Original: 4. I MPLEMENTATION D ETAILS
Revised: 4. I MPLEMENTATION D ETAILS

Original: In this paper, natural variation model is powered by MUNIT Huang et al.
Revised: In this paper, natural variation model is powered by MUNIT Huang et al.

Original: (2018b)
Revised: (2018b)

Original: on Cityscapes dataset and RAIN proposed by Luo et al.
Revised: on Cityscapes dataset and RAIN proposed by Luo et al.

Original: (2020) on Synthia dataset.
Revised: (2020) on Synthia dataset.

Original: PSPNet With the natural variation model derived from either known a priori or model learning from data.
Revised: PSPNet With the natural variation model derived from either known a priori or model learning from data.

Original: For the latter one, we apply MUNIT on Cityscapes dataset and RAIN on Synthia dataset, we use PSPNet Zhao et al.
Revised: For the latter one, we apply MUNIT on Cityscapes dataset and RAIN on Synthia dataset, we use PSPNet Zhao et al.

Original: (2017) to do semantic segmentation.
Revised: (2017) to do semantic segmentation.

Original: In PSPNet, we use ResNet He et al.
Revised: In PSPNet, we use ResNet He et al.

Original: (2015) model as backbone with the dilated network method Yu & Koltun (2016), then plug in the robust training module to generate new datapoints and obtain feature map using the weighted combination.
Revised: (2015) model as backbone with the dilated network method Yu & Koltun (2016), then plug in the robust training module to generate new datapoints and obtain feature map using the weighted combination.

Original: Pyramid pooling module with 4 level representing different size of the sub-regions in the feature map is applied to gather both local and global region information.
Revised: Pyramid pooling module with 4 level representing different size of the sub-regions in the feature map is applied to gather both local and global region information.

Original: Then we concatenate the feature map obtained by pyramid pooling module with original feature map to generate ﬁnal feature map.
Revised: Then we concatenate the feature map obtained by pyramid pooling module with original feature map to generate ﬁnal feature map.

Original: MUNIT For MUNIT Huang et al.
Revised: MUNIT For MUNIT Huang et al.

Original: (2018b) model that captures natural variation model, we set the training iterations into 300000.
Revised: (2018b) model that captures natural variation model, we set the training iterations into 300000.

Original: Weight decay is set to 0.0001 with image size ﬁxed at 200 ∗ 200 .
Revised: Weight decay is set to 0.0001 with image size ﬁxed at 200 ∗ 200 .

Original: The architecture of MUNIT is consisted of two auto-encoders, one for each domain.
Revised: The architecture of MUNIT is consisted of two auto-encoders, one for each domain.

Original: For each autoencoder, it is composed by one content encoder, one style encoder and one decoder.
Revised: For each autoencoder, it is composed by one content encoder, one style encoder and one decoder.

Original: Content encoder is consisted of strided convolutional layers for down-sampling purposes followed by residual blocks.
Revised: Content encoder is consisted of strided convolutional layers for down-sampling purposes followed by residual blocks.

Original: Instance normalization is attached at the end of each convolutional layer in content encoder.
Revised: Instance normalization is attached at the end of each convolutional layer in content encoder.

Original: Style encoder ﬁrst down-samples the images by convolutional layer,
Revised: Style encoder ﬁrst down-samples the images by convolutional layer,

Original: then global pooling and fully connected layer are applied.
Revised: then global pooling and fully connected layer are applied.

Original: For the decoder, style code being generated is fed to a multilayer perceptron to generate parameters for Adaptive Instance Normalization (AdaIN).
Revised: For the decoder, style code being generated is fed to a multilayer perceptron to generate parameters for Adaptive Instance Normalization (AdaIN).

Original: The AdaIN is applied to the residual blocks which are processing the content code.
Revised: The AdaIN is applied to the residual blocks which are processing the content code.

Original: The output will then be up-sampled and serve as the ﬁnal reconstructed image.
Revised: The output will then be up-sampled and serve as the ﬁnal reconstructed image.

Original: LSGAN is serving as discriminator while VGG is applied for the domain-invariant perceptual loss calculation.
Revised: LSGAN is serving as discriminator while VGG is applied for the domain-invariant perceptual loss calculation.

Original: RAIN For RAIN Luo et al.
Revised: RAIN For RAIN Luo et al.

Original: (2020) model proposed by Luo.
Revised: (2020) model proposed by Luo.

Original: We set the training iterations to 100K and learning rate as 0.0001.
Revised: We set the training iterations to 100K and learning rate as 0.0001.

Original: This model is the composition of AdaIN Huang & Belongie (2017) and auto-encoder.
Revised: This model is the composition of AdaIN Huang & Belongie (2017) and auto-encoder.

Original: For AdaIN, VGG model is applied as encoder.
Revised: For AdaIN, VGG model is applied as encoder.

Original:  E XPERIMENT
Revised:  E XPERIMENT

Original: Due to computation resources limitation, the experiments in this paper are done on Google Cloud
Revised: Due to computation resources limitation, the experiments in this paper are done on Google Cloud

Original: Platform (GCP) using 16 GB of memory and one NVIDIA Tesla T4 GPU with 16 GB GDDR6 of graphic memory.
Revised: Platform (GCP) using 16 GB of memory and one NVIDIA Tesla T4 GPU with 16 GB GDDR6 of graphic memory.

Original: The operating system being used in the experiment is Ubuntu 18.04.
Revised: The operating system being used in the experiment is Ubuntu 18.04.

Original: The code is implemented based on Python 3.6 and PyTorch 1.5.1.
Revised: The code is implemented based on Python 3.6 and PyTorch 1.5.1.

Original: For fair comparison, all the images being trained in this paper will have the same size of 201 ∗ 201 with batch size of 8 for all the methods being applied in the experiment.
Revised: For fair comparison, all the images being trained in this paper will have the same size of 201 ∗ 201 with batch size of 8 for all the methods being applied in the experiment.

Original: For evaluation metrics, we use pixel-wise accuracy and mean of class-wise intersection over union (mIoU) to deﬁne evaluation metrics.
Revised: For evaluation metrics, we use pixel-wise accuracy and mean of class-wise intersection over union (mIoU) to deﬁne evaluation metrics.

Original: Pixel-wise accuracy is calculated using the number of pixels common between prediction output and ground truth divided by the number of pixels in ground truth feature map.
Revised: Pixel-wise accuracy is calculated using the number of pixels common between prediction output and ground truth divided by the number of pixels in ground truth feature map.

Original: The Intersection over Union metric is also named as Jaccard index to calculate the overlap between prediction and ground truth.
Revised: The Intersection over Union metric is also named as Jaccard index to calculate the overlap between prediction and ground truth.

Original: To be speciﬁc,
Revised: To be speciﬁc,

Original: IoU (IoU = Area of Overlap Area of Union ) calculates the number of pixels common between prediction output and ground truth divided by number of pixels across both feature maps.
Revised: IoU (IoU = Area of Overlap Area of Union ) calculates the number of pixels common between prediction output and ground truth divided by number of pixels across both feature maps.

Original: Inspired by Zhao et al.
Revised: Inspired by Zhao et al.

Original: , we use ResNet-101 to generate basic feature map.
Revised: , we use ResNet-101 to generate basic feature map.

Original: ResNet-101 keeps a proper balance between high performance and proper complexity.
Revised: ResNet-101 keeps a proper balance between high performance and proper complexity.

Original: Serving as baseline of the experiment, we train and test
Revised: Serving as baseline of the experiment, we train and test

Original: PSPNet using source domain images as training set and test the performance in target domain.
Revised: PSPNet using source domain images as training set and test the performance in target domain.

Original: Source domain only PSPNet tends to present how natural variation affects the performance in normal conﬁguration.
Revised: Source domain only PSPNet tends to present how natural variation affects the performance in normal conﬁguration.

Original: To compare our model in a comprehensive way, we also evaluate the performance of PSPNet when it is trained using labeled images from target domain (dark condition) and test the performance using images from target domain.
Revised: To compare our model in a comprehensive way, we also evaluate the performance of PSPNet when it is trained using labeled images from target domain (dark condition) and test the performance using images from target domain.

Original: We train all models for 200 epochs with basic learning rate of 0.01.
Revised: We train all models for 200 epochs with basic learning rate of 0.01.

Original: For the MUNIT network being trained in this experiment, we use batch size of 1, weight decay of 0.0001 and learning rate of 0.0001 with step side of 10000 referring to the frequency of learning rate decaying.
Revised: For the MUNIT network being trained in this experiment, we use batch size of 1, weight decay of 0.0001 and learning rate of 0.0001 with step side of 10000 referring to the frequency of learning rate decaying.

Original: Comparison Methods We compare our method with several recent domain adaptation methods, including AdaptSegNet Tsai et al.
Revised: Comparison Methods We compare our method with several recent domain adaptation methods, including AdaptSegNet Tsai et al.

Original: (2020), ADVENT Vu et al.
Revised: (2020), ADVENT Vu et al.

Original: (2019), and FDA Yang & Soatto (2020).
Revised: (2019), and FDA Yang & Soatto (2020).

Original: AdaptSegNet introduces generative adversarial network technique to reduce the gap on the label side of source domain and target domain.
Revised: AdaptSegNet introduces generative adversarial network technique to reduce the gap on the label side of source domain and target domain.

Original: Two methods are proposed in ADVENT, namely MinEnt and
Revised: Two methods are proposed in ADVENT, namely MinEnt and

Original: AdvEnt.
Revised: AdvEnt.

Original: MinEnt maximize prediction certainty in the target domain by using a proposed entropy loss L ent .
Revised: MinEnt maximize prediction certainty in the target domain by using a proposed entropy loss L ent .

Original: AdvEnt shares similarity with the concept of AdaptSegNet and minimize the entropy by making target entropy distribution similar to the source domain.
Revised: AdvEnt shares similarity with the concept of AdaptSegNet and minimize the entropy by making target entropy distribution similar to the source domain.

Original: We evaluate the performance of both methods being proposed by ADVENT in this paper.
Revised: We evaluate the performance of both methods being proposed by ADVENT in this paper.

Original: FDA utilize Fast Fourier Transform (FFT) and replace the low-level frequencies of target domain images into source domain images.
Revised: FDA utilize Fast Fourier Transform (FFT) and replace the low-level frequencies of target domain images into source domain images.

Original: From the original literature of these proposed methods, the semantic segmentation baseline being applied is DeepLabV2 Chen et al. (2017a) with ResNet101 using SGD.
Revised: From the original literature of these proposed methods, the semantic segmentation baseline being applied is DeepLabV2 Chen et al. (2017a) with ResNet101 using SGD.

Original: To compare the performance from the same backbone of segmentation network, we replace the DeepLabV2 into PSPNet in this paper.
Revised: To compare the performance from the same backbone of segmentation network, we replace the DeepLabV2 into PSPNet in this paper.

Original: All the conﬁgurations including initial learning rate of 0.1, momentum of 0.9 and weight decay of 0.0001 keep unchanged.
Revised: All the conﬁgurations including initial learning rate of 0.1, momentum of 0.9 and weight decay of 0.0001 keep unchanged.

Original: Cityscapes
Revised: Cityscapes

Original: [Table] From Table 1, segmentation model that only trained with source domain images will have huge performance drop during the inference time under natural variations.
Revised: From Table 1, segmentation model that only trained with source domain images will have huge performance drop during the inference time under natural variations.

Original: Model-based robust training algorithm (MRT) has 11% and 7% of improvement in terms of IoU compared to the model only trained with source domain images.
Revised: Model-based robust training algorithm (MRT) has 11% and 7% of improvement in terms of IoU compared to the model only trained with source domain images.

Original: The other domain adaptation methods such as AdaptSegNet,
Revised: The other domain adaptation methods such as AdaptSegNet,

Original: AdvEnt and MinEnt have stable and better performance compared to the model-based robust training algorithm.
Revised: AdvEnt and MinEnt have stable and better performance compared to the model-based robust training algorithm.

Original: FDA method has 15% of improvement in mIoU compared to source only PSPNet for brightness case yet not promising result on snow effects.
Revised: FDA method has 15% of improvement in mIoU compared to source only PSPNet for brightness case yet not promising result on snow effects.

Original: Our proposed method has 16% and 12% of mIoU increase compared to baseline, which outperforms all other comparison methods.
Revised: Our proposed method has 16% and 12% of mIoU increase compared to baseline, which outperforms all other comparison methods.

Original: For brightness condition, our proposed MRTAdapt even has higher accuracy than segmentation model trained with target domain images.
Revised: For brightness condition, our proposed MRTAdapt even has higher accuracy than segmentation model trained with target domain images.

Original: Synthia For Synthia dataset, we apply RAIN model in stead of MUNIT model to capture natural variation effect.
Revised: Synthia For Synthia dataset, we apply RAIN model in stead of MUNIT model to capture natural variation effect.

Original: As results shown in Table 2, comparing with all other methods, our proposed
Revised: As results shown in Table 2, comparing with all other methods, our proposed

Original: MRTAdapt outperforms other state-of-the-art domain adaptation models and also reach similar performance to the case when we have the ground truth label of target domain.
Revised: MRTAdapt outperforms other state-of-the-art domain adaptation models and also reach similar per-

Original: Meanwhile, our method shows high robustness in different abnormal conditions.
Revised: [Table]

Original: [Table]
Revised: [Table]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we address the problem that current  semantic segmentation methods are vulnerable to natural variation effects, which are very common in real-world conditions.
Revised: In this paper, we addressed the problem of fragility in current learning-based semantic segmentation methods against natural variation effects that typically occur in real-world conditions.

Original: Using industry-level segmentation architecture PSPNet as backbone network, we adopt a model-based robust training algorithm that was originally proposed for image classiﬁcation problem to design a robust semantic segmentation method.
Revised: Using industry-level segmentation architecture PSPNet as backbone network, we adopted a model-based robust training algorithm that was originally proposed for image classiﬁcation problem to design a robust semantic segmentation method.

Original: By applying a model which can capture the changes that lie within source and target domain, we challenge the network with the worst-case scenario natural variation images by ﬁnding the biggest loss inferred.
Revised: By applying a model which can capture the changes that lie within source and target domain, we challenge the network with the worst-case scenario natural variation images by ﬁnding the biggest loss inferred.

Original: Later, we extend based on the AdaptSegNet which was trying to minimize the distribution gap of source and target domain from the label perspective, we discover that natural variation will not alter the semantic meaning of the label.
Revised: Also, building upon the AdaptSegNet that minimizes the distribution gap of source and target domain from the label perspective, we showed that natural variation will not alter the semantic meaning of the label.

Original: Based on this idea, we propose model-based robust adaptive training algorithm and achieve higher performance compared to several domain adaptation methods.
Revised: Based on this idea, we proposed a model-based robust adaptive training algorithm that achieves higher performance than several domain adaptation methods do.

Original: Indeed, our methodis dependent on the performance of the model that captures natural variation effects, so research on reducing and eventually eliminating this boundary could be expected in the near future.
Revised: Our future research directions will focus on reducing and eventually eliminating the dependence of the proposed method on the performance of the model that captures natural variation effects  A PPENDIX [Table] [Table]

Revised: [Table]

Revised: [Table]