Original: S EMI - SUPERVISED REGRESSION WITH SKEWED DATA VIA ADVERSARIALLY FORCING THE DISTRIBUTION OF PREDICTED VALUES
Revised: S EMI - SUPERVISED REGRESSION WITH SKEWED DATA VIA ADVERSARIALLY FORCING THE DISTRIBUTION OF PREDICTED VALUES

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Advances in scientiﬁc ﬁelds including drug discovery or material design are accompanied by numerous trials and errors.
Revised: Advances in scientiﬁc ﬁelds including drug discovery or material design are accompanied by numerous trials and errors.

Original: However, generally only representative experimental results are reported.
Revised: However, generally only representative experimental results are reported.

Original: Because of this reporting bias, the distribution of labeled result data can deviate from their true distribution.
Revised: Because of this reporting bias, the distribution of labeled result data can deviate from their true distribution.

Original: A regression model can be erroneous if it is built on these skewed data.
Revised: A regression model can be erroneous if it is built on these skewed data.

Original: In this work, we propose a new approach to improve the accuracy of regression models that are trained using a skewed dataset.
Revised: In this work, we propose a new approach to improve the accuracy of regression models that are trained using a skewed dataset.

Original: The method forces the regression outputs to follow the true distribution; the forcing algorithm regularizes the regression results while keeping the information of the training data.
Revised: The method forces the regression outputs to follow the true distribution; the forcing algorithm regularizes the regression results while keeping the information of the training data.

Original: We assume the existence of enough unlabeled data that follow the true distribution, and that the true distribution can be roughly estimated from domain knowledge or a few samples.
Revised: We assume the existence of enough unlabeled data that follow the true distribution, and that the true distribution can be roughly estimated from domain knowledge or a few samples.

Original: During training neural networks to generate a regression model, an adversarial network is used to force the distribution of predicted values to follow the estimated ‘true’ distribution.
Revised: During training neural networks to generate a regression model, an adversarial network is used to force the distribution of predicted values to follow the estimated ‘true’ distribution.

Original: We evaluated the proposed approach on four real-world datasets (pLogP, Diamond, House, Elevators).
Revised: We evaluated the proposed approach on four real-world datasets (pLogP, Diamond, House, Elevators).

Original: In all four datasets, the proposed approach reduced the root mean squared error of the regression by around 55 percent to 75 percent compared to regression models without adjustment of the distribution.
Revised: In all four datasets, the proposed approach reduced the root mean squared error of the regression by around 55 percent to 75 percent compared to regression models without adjustment of the distribution.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Advances in scientiﬁc ﬁelds including drug discovery or material design are accompanied by numerous trials and errors.
Revised: Advances in scientiﬁc ﬁelds including drug discovery or material design are accompanied by numerous trials and errors.

Original: However, generally only representative experimental results are chosen to be reported.
Revised: However, generally only representative experimental results are chosen to be reported.

Original: As a consequence of this reporting bias, the distribution of the reported results can differ from the true distribution.
Revised: As a consequence of this reporting bias, the distribution of the reported results can differ from the true distribution.

Original: For this reason, when data from the literature are used to train a regression model, predictions from the regression model may differ from the true distribution because the model is derived using biased data (Lin et al., 2002; Galar et al., 2011).
Revised: For this reason, when data from the literature are used to train a regression model, predictions from the regression model may differ from the true distribution because the model is derived using biased data (Lin et al., 2002; Galar et al., 2011).

Original: In particular, pharmaceutical development is often affected by this problem.
Revised: In particular, pharmaceutical development is often affected by this problem.

Original: Quantitative structure-activity relationship (QSAR), including drug-target interaction (DTI), is consistently affected by the bias in the reported experimental data, because usually the targeted range of molecular property is clearly deﬁned (Liu et al., 2015; Chen & Zhang, 2013).
Revised: Quantitative structure-activity relationship (QSAR), including drug-target interaction (DTI), is consistently affected by the bias in the reported experimental data, because usually the targeted range of molecular property is clearly deﬁned (Liu et al., 2015; Chen & Zhang, 2013).

Original: When regression is performed using such skewed data, it often erroneously predicts that the target properties are satisﬁed.
Revised: When regression is performed using such skewed data, it often erroneously predicts that the target properties are satisﬁed.

Original: As a consequence, it is difﬁcult to discover molecules that have the desired properties (Peng et al., 2017).
Revised: As a consequence, it is difﬁcult to discover molecules that have the desired properties (Peng et al., 2017).

Original: Active learning applications also have a similar problem.
Revised: Active learning applications also have a similar problem.

Original: Many active learning methods repeat the selection of new data by applying certain criteria and retraining the surrogate model (Lookman et al., 2019; RouetLeduc et al., 2016; Yuan et al., 2018).
Revised: Many active learning methods repeat the selection of new data by applying certain criteria and retraining the surrogate model (Lookman et al., 2019; RouetLeduc et al., 2016; Yuan et al., 2018).

Original: During this process, the data can be skewed according to the criteria
Revised: During this process, the data can be skewed according to the criteria

Original: (de Mello, 2013; Prabhu et al., 2019).
Revised: (de Mello, 2013; Prabhu et al., 2019).

Original: However, despite this problem, few studies have tried to improve the accuracy of regression models that have been trained on skewed data.
Revised: However, despite this problem, few studies have tried to improve the accuracy of regression models that have been trained on skewed data.

Original: In this work, we propose a new approach to improve the accuracy of a regression model that is trained using skewed data.
Revised: In this work, we propose a new approach to improve the accuracy of a regression model that is trained using skewed data.

Original: We assume the presence of enough unlabeled data which follow the true distribution, and that the true distribution can be roughly estimated using domain knowledge or a few examples.
Revised: We assume the presence of enough unlabeled data which follow the true distribution, and that the true distribution can be roughly estimated using domain knowledge or a few examples.

Original: We use a semi-supervised learning framework with an adversarial network to force the distribution of the regression output to resemble the assumed true distribution (Figure 1).
Revised: We use a semi-supervised learning framework with an adversarial network to force the distribution of the regression output to resemble the assumed true distribution (Figure 1).

Original: At the same time, by sharing the front part of the regression model with the encoder of an adversarial autoencoder (AAE), the process of forcing the distribution of output values is regularized in a way that the information of the labeled data is represented stably.
Revised: At the same time, by sharing the front part of the regression model with the encoder of an adversarial autoencoder (AAE), the process of forcing the distribution of output values is regularized in a way that the information of the labeled data is represented stably.

Original: We created skewed datasets by selecting data that exceeded a certain threshold from four real-world datasets (pLogP, Diamond, House, Elevators), then evaluated the proposed approach using these skewed datasets.
Revised: We created skewed datasets by selecting data that exceeded a certain threshold from four real-world datasets (pLogP, Diamond, House, Elevators), then evaluated the proposed approach using these skewed datasets.

Original: The proposed approach reduced the root mean squared error (RMSE) of the regression model derived using each of the four datasets, compared to the regression model that had been trained using only the skewed datasets.
Revised: The proposed approach reduced the root mean squared error (RMSE) of the regression model derived using each of the four datasets, compared to the regression model that had been trained using only the skewed datasets.

Original: We also veriﬁed that the proposed approach is feasible even when the estimate of the true distribution is not perfect.
Revised: We also veriﬁed that the proposed approach is feasible even when the estimate of the true distribution is not perfect.

Original: [Figure]
Revised: [Figure]

Original: 𝑦
Revised: 𝑦

Original: [Figure]
Revised: [Figure]

Original: Adversarial network for forcing output distribution
Revised: Adversarial network for forcing output distribution

Original: [Figure]
Revised: [Figure]

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Semi-Supervised Learning (SSL) is a machine-learning strategy to learn using partially-labeled datasets (Chapelle et al., 2009).
Revised: Semi-Supervised Learning (SSL) is a machine-learning strategy to learn using partially-labeled datasets (Chapelle et al., 2009).

Original: In the ﬁeld of SSL, various methods have been developed, including those using generative models (Kingma et al., 2014), graphs (Goldberg & Zhu, 2006), self-training (Rosenberg et al., 2005) and consistency regularization (Sohn et al., 2020).
Revised: In the ﬁeld of SSL, various methods have been developed, including those using generative models (Kingma et al., 2014), graphs (Goldberg & Zhu, 2006), self-training (Rosenberg et al., 2005) and consistency regularization (Sohn et al., 2020).

Original: SSL can improve classiﬁcation and regression models by using information in a large set of unlabeled data to train a relatively small set of labeled data (Xie et al., 2019;
Revised: SSL can improve classiﬁcation and regression models by using information in a large set of unlabeled data to train a relatively small set of labeled data (Xie et al., 2019;

Original: Creswell et al., 2018; Dimokranitou, 2017; Rezagholiradeh & Haidar, 2018).
Revised: Creswell et al., 2018; Dimokranitou, 2017; Rezagholiradeh & Haidar, 2018).

Original: These approaches generally assume that labeled and unlabeled datasets are well distributed without distortion.
Revised: These approaches generally assume that labeled and unlabeled datasets are well distributed without distortion.

Original: Several methods were tried to train a model on data that have biased distribution.
Revised: Several methods were tried to train a model on data that have biased distribution.

Original: A method proposed by (Zhu et al., 2003) incorporated class prior knowledge to adjust the class distribution.
Revised: A method proposed by (Zhu et al., 2003) incorporated class prior knowledge to adjust the class distribution.

Original: Another method proposed by (Kim et al., 2020) used a model trained with skewed labeled data to generate pseudo-labels of unlabeled data for retraining by considering the entropy of prediction.
Revised: Another method proposed by (Kim et al., 2020) used a model trained with skewed labeled data to generate pseudo-labels of unlabeled data for retraining by considering the entropy of prediction.

Original: Positive Unlabeled (PU) learning uses data with only positive and unlabeled samples for binary classiﬁcations (Elkan & Noto, 2008); PU learning has been studied intensively (Hsieh et al., 2019; Youngs et al., 2015).
Revised: Positive Unlabeled (PU) learning uses data with only positive and unlabeled samples for binary classiﬁcations (Elkan & Noto, 2008); PU learning has been studied intensively (Hsieh et al., 2019; Youngs et al., 2015).

Original: However, to the best of our knowledge, few methods used skewed labeled datasets for regression.
Revised: However, to the best of our knowledge, few methods used skewed labeled datasets for regression.

Original: Although other methods for learning with imbalanced data have been developed, such as the Synthetic
Revised: Although other methods for learning with imbalanced data have been developed, such as the Synthetic

Original: Minority Oversampling Technique (SMOTE) for regression (Torgo et al., 2013), these approaches can only be applied if even some of the data are in the sparse area.
Revised: Minority Oversampling Technique (SMOTE) for regression (Torgo et al., 2013), these approaches can only be applied if even some of the data are in the sparse area.

Original:  M ETHOD
Revised:  M ETHOD

Original: Our proposed approach is aimed to improve accuracy of regression models that are trained on a dataset that is skewed according to labels.
Revised: Our proposed approach is aimed to improve accuracy of regression models that are trained on a dataset that is skewed according to labels.

Original: D l denotes a skewed labeled dataset with n samples that satisﬁes
Revised: D l denotes a skewed labeled dataset with n samples that satisﬁes

Original: [Equation]
Revised: [Equation]

Original: where x i ∈ R
Revised: where x i ∈ R

Original: d is d -dimensional input, y i ∈ R is ground truth target and θ is the bias threshold above which the labeled data are present.
Revised: d is d -dimensional input, y i ∈ R is ground truth target and θ is the bias threshold above which the labeled data are present.

Original: We assume that we have an estimate of the true distribution of targets y , p ( y ) , and that we have an unbiased unlabeled dataset D u
Revised: We assume that we have an estimate of the true distribution of targets y , p ( y ) , and that we have an unbiased unlabeled dataset D u

Original: = { x n +1 , x n +2 , . . .
Revised: = { x n +1 , x n +2 , . . .

Original: , x m } that has sufﬁciently large m (cid:29) n and in which the distribution of ground truth targets follows p ( y ) .
Revised: , x m } that has sufﬁciently large m (cid:29) n and in which the distribution of ground truth targets follows p ( y ) .

Original: In this section, we describe our approach to make the regression outputs follow the true distribution as closely as possible by using D l and D u in semi-supervised fashion.
Revised: In this section, we describe our approach to make the regression outputs follow the true distribution as closely as possible by using D l and D u in semi-supervised fashion.

Original: The proposed approach consists of two parts:
Revised: The proposed approach consists of two parts:

Original: “Adversarial network for forcing output distribution”
Revised: “Adversarial network for forcing output distribution”

Original: and “Regularization using AAE”.
Revised: and “Regularization using AAE”.

Original: In the ﬁrst part, we present a method that uses an adversarial network to force the output distribution of the regression model to be similar to p ( y ) .
Revised: In the ﬁrst part, we present a method that uses an adversarial network to force the output distribution of the regression model to be similar to p ( y ) .

Original: Then, we describe a method to properly regularize this forcing process by implementing an AAE.
Revised: Then, we describe a method to properly regularize this forcing process by implementing an AAE.

Original: 3. A DVERSARIAL NETWORK FOR FORCING OUTPUT DISTRIBUTION
Revised: 3. A DVERSARIAL NETWORK FOR FORCING OUTPUT DISTRIBUTION

Original: Let ˆ y be the output of a regressor that aims to predict ground truth targets y , and q (ˆ y | x ) be the regression distribution.
Revised: Let ˆ y be the output of a regressor that aims to predict ground truth targets y , and q (ˆ y | x ) be the regression distribution.

Original: Then for given data distribution p d ( x ) , a distribution of predicted regressor output values q (ˆ y ) can be deﬁned as:
Revised: Then for given data distribution p d ( x ) , a distribution of predicted regressor output values q (ˆ y ) can be deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: We use ˆ y u to denote the predicted values for D u , and ˆ y l to denote the predicted values for D l .
Revised: We use ˆ y u to denote the predicted values for D u , and ˆ y l to denote the predicted values for D l .

Original: Then the distributions of predicted values are denoted as q (ˆ y u ) and q (ˆ y l ) .
Revised: Then the distributions of predicted values are denoted as q (ˆ y u ) and q (ˆ y l ) .

Original: The proposed approach is to regularize the regressor to have q (ˆ y u ) close to p ( y ) .
Revised: The proposed approach is to regularize the regressor to have q (ˆ y u ) close to p ( y ) .

Original: To do so, an adversarial network is placed at the end of the regressor (Figure 1).
Revised: To do so, an adversarial network is placed at the end of the regressor (Figure 1).

Original: The model is concurrently trained on D l for regression, and on D u to force the regression outputs to have similar distribution to p ( y ) .
Revised: The model is concurrently trained on D l for regression, and on D u to force the regression outputs to have similar distribution to p ( y ) .

Original: The discriminator distinguishes outputs of the regressor from randomly-sampled values in p ( y ) , and the regressor learns to deceive the discriminator.
Revised: The discriminator distinguishes outputs of the regressor from randomly-sampled values in p ( y ) , and the regressor learns to deceive the discriminator.

Original: Suppose we divide D u into two subsets, D above that consists of x that have ground truth target values > θ , and D below which consist of x that have ground truth target values < θ .
Revised: Suppose we divide D u into two subsets, D above that consists of x that have ground truth target values > θ , and D below which consist of x that have ground truth target values < θ .

Original: If the regressor is well trained on D l , then prediction on D above is expected to be relatively accurate because the distributions of x of D l and D above are similar.
Revised: If the regressor is well trained on D l , then prediction on D above is expected to be relatively accurate because the distributions of x of D l and D above are similar.

Original: In contrast, for D below , the regressor should predict values that have not ever been seen; therefore, we hypothesized that during adversarial training on D u , the prediction on D above should stay relatively still, whereas prediction on D below is ﬂexible.
Revised: In contrast, for D below , the regressor should predict values that have not ever been seen; therefore, we hypothesized that during adversarial training on D u , the prediction on D above should stay relatively still, whereas prediction on D below is ﬂexible.

Original: If this hypothesis holds, then the portion of q (ˆ y u ) that is > θ should be ﬁlled with prediction outputs on D above , and prediction outputs on D below should be < θ to deceive the discriminator.
Revised: If this hypothesis holds, then the portion of q (ˆ y u ) that is > θ should be ﬁlled with prediction outputs on D above , and prediction outputs on D below should be < θ to deceive the discriminator.

Original: These responses result in the regressor having q (ˆ y u ) close to p ( y ) while maintaining accurate prediction for outputs > θ .
Revised: These responses result in the regressor having q (ˆ y u ) close to p ( y ) while maintaining accurate prediction for outputs > θ .

Original: 3. R EGULARIZATION BASED ON AAE
Revised: 3. R EGULARIZATION BASED ON AAE

Original: Even if prediction on D above is intact and the predicted output distribution of D u ﬁts the assumed true distribution, the prediction on D below will still have a large error in most cases.
Revised: Even if prediction on D above is intact and the predicted output distribution of D u ﬁts the assumed true distribution, the prediction on D below will still have a large error in most cases.

Original: To guide the prediction output on D below correctly, the information from above θ must be conveyed to below θ .
Revised: To guide the prediction output on D below correctly, the information from above θ must be conveyed to below θ .

Original: We used the AAE to regularize the process of forcing the output distribution to achieve appropriate propagation of the information.
Revised: We used the AAE to regularize the process of forcing the output distribution to achieve appropriate propagation of the information.

Original: We set the encoder of an AAE to share the front part of the regression network (Figure 1).
Revised: We set the encoder of an AAE to share the front part of the regression network (Figure 1).

Original: We denote the shared part of the regression network as R enc and rest of the network as R post .
Revised: We denote the shared part of the regression network as R enc and rest of the network as R post .

Original: R enc works as the encoder of the AAE and the front part of the regression network at the same time.
Revised: R enc works as the encoder of the AAE and the front part of the regression network at the same time.

Original: Therefore, latent vectors of the
Revised: Therefore, latent vectors of the

Original: AAE should act as a useful feature for the R post which functions to predict y from the latent vectors.
Revised: AAE should act as a useful feature for the R post which functions to predict y from the latent vectors.

Original: To fulﬁl its function, R enc must consider knowledge from the labeled dataset.
Revised: To fulﬁl its function, R enc must consider knowledge from the labeled dataset.

Original: Also, during the process of forcing the output distribution, R post also transforms the given distribution of the latent vectors to p ( y ) .
Revised: Also, during the process of forcing the output distribution, R post also transforms the given distribution of the latent vectors to p ( y ) .

Original: To be a useful feature for R post , latent vectors should be arranged in a similar way to p ( y ) , which possesses information about how the labeled dataset is skewed.
Revised: To be a useful feature for R post , latent vectors should be arranged in a similar way to p ( y ) , which possesses information about how the labeled dataset is skewed.

Original: This implies that latent vectors become more useful if R enc is aware of the skewed distribution of the labeled dataset.
Revised: This implies that latent vectors become more useful if R enc is aware of the skewed distribution of the labeled dataset.

Original: To facilitate this process, we shaped the distribution of the latent vectors to have similar characteristics to p ( y ) by adjusting the target distribution of AAE.
Revised: To facilitate this process, we shaped the distribution of the latent vectors to have similar characteristics to p ( y ) by adjusting the target distribution of AAE.

Original: Consequently, we hypothesized that the regularization process of AAE can be well guided by this information from labeled data and p ( y ) .
Revised: Consequently, we hypothesized that the regularization process of AAE can be well guided by this information from labeled data and p ( y ) .

Original: This good guidance results in well-controlled propagation of information from the portion > θ to the prediction on D below .
Revised: This good guidance results in well-controlled propagation of information from the portion > θ to the prediction on D below .

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we present results of experiments on regression of artiﬁcially-skewed datasets.
Revised: In this section, we present results of experiments on regression of artiﬁcially-skewed datasets.

Original: For a given dataset, we ﬁrst randomly sampled a sufﬁcient amount of data to construct an unlabeled train dataset.
Revised: For a given dataset, we ﬁrst randomly sampled a sufﬁcient amount of data to construct an unlabeled train dataset.

Original: Then we divided the rest of the data arbitrarily into three datasets to form a labeled train dataset, a validation dataset and a test dataset.
Revised: Then we divided the rest of the data arbitrarily into three datasets to form a labeled train dataset, a validation dataset and a test dataset.

Original: To induce bias into the labeled dataset fairly, only data with labels that had values more than one standard deviation above the mean of all labels were selected randomly until the labeled dataset was sufﬁciently large.
Revised: To induce bias into the labeled dataset fairly, only data with labels that had values more than one standard deviation above the mean of all labels were selected randomly until the labeled dataset was sufﬁciently large.

Original: For the experiments, we used four datasets: moses (QSAR on calculated pLogP), diamond, house and elevators (Polykovskiy et al., 2018; Wickham, 2016; Alcala-Fdez et al., 2011).
Revised: For the experiments, we used four datasets: moses (QSAR on calculated pLogP), diamond, house and elevators (Polykovskiy et al., 2018; Wickham, 2016; Alcala-Fdez et al., 2011).

Original: Since QSAR on small molecules suffers from skewed datasets in many practical applications, our experiments are focused on the QSAR on calculated pLogP task.
Revised: Since QSAR on small molecules suffers from skewed datasets in many practical applications, our experiments are focused on the QSAR on calculated pLogP task.

Original: Other datasets were used to demonstrate the general applicability of the proposed approach.
Revised: Other datasets were used to demonstrate the general applicability of the proposed approach.

Original: We focused on comparing regression accuracy of the models that were trained using our approach or without it.
Revised: We focused on comparing regression accuracy of the models that were trained using our approach or without it.

Original: Further, to explore the effectiveness at which the adversarial network forced output distribution and
Revised: Further, to explore the effectiveness at which the adversarial network forced output distribution and

Original: AAE based regularization separately, we conducted ablation studies.
Revised: AAE based regularization separately, we conducted ablation studies.

Original: As the proposed approach requires an estimate of the true distribution, we tested the sensitivity of the model to the quality of the estimate.
Revised: As the proposed approach requires an estimate of the true distribution, we tested the sensitivity of the model to the quality of the estimate.

Original: In each test, the true distribution of the data was estimated to be Gaussian distribution that has a mean and standard
Revised: In each test, the true distribution of the data was estimated to be Gaussian distribution that has a mean and standard

Original: [Table]
Revised: [Table]

Original: deviation of 1) full data or 2) 20 randomly-sampled data.
Revised: deviation of 1) full data or 2) 20 randomly-sampled data.

Original: We set the assumed true distribution as Gaussian distribution for experiments on all four datasets regardless of their real distributions.
Revised: We set the assumed true distribution as Gaussian distribution for experiments on all four datasets regardless of their real distributions.

Original: These experiments should not consider the label of unlabeled dataset until the ﬁnal accuracy test.
Revised: These experiments should not consider the label of unlabeled dataset until the ﬁnal accuracy test.

Original: Therefore, to tune hyperparameters, we only considered accuracy of the regression on labeled dataset and distribution mismatch between predicted values and the assumed true distribution.
Revised: Therefore, to tune hyperparameters, we only considered accuracy of the regression on labeled dataset and distribution mismatch between predicted values and the assumed true distribution.

Original: 4. E XPERIMENTS ON QSAR TASK
Revised: 4. E XPERIMENTS ON QSAR TASK

Original: The agreement between ground truth and predicted pLogP were affected by the method used.
Revised: The agreement between ground truth and predicted pLogP were affected by the method used.

Original: The model trained using fully-labeled data accurately predicted pLogP (Figure 2a).
Revised: The model trained using fully-labeled data accurately predicted pLogP (Figure 2a).

Original: However, the model trained with skewed data without the proposed approach, showed decrease in accuracy as the ground truth value declined below the bias threshold (Figure 2b); this result indicates that the model’s extrapolation ability is not sufﬁcient for this task.
Revised: However, the model trained with skewed data without the proposed approach, showed decrease in accuracy as the ground truth value declined below the bias threshold (Figure 2b); this result indicates that the model’s extrapolation ability is not sufﬁcient for this task.

Original: In contrast, the model trained with the proposed approach followed the data well (Figure 2c).
Revised: In contrast, the model trained with the proposed approach followed the data well (Figure 2c).

Original: The RMSE (Table 1) also show that about 70 % of error in the model without the proposed approach is reduced using the proposed approach.
Revised: The RMSE (Table 1) also show that about 70 % of error in the model without the proposed approach is reduced using the proposed approach.

Original: This result suggests that information learned from skewed labeled data can guide the outputs of the regression model to properly ﬁt in the assumed true distribution during the training process; otherwise, the regression model with proposed approach would show predictions that have a distribution that matches the assumed true distribution, but would not be accurate.
Revised: This result suggests that information learned from skewed labeled data can guide the outputs of the regression model to properly ﬁt in the assumed true distribution during the training process; otherwise, the regression model with proposed approach would show predictions that have a distribution that matches the assumed true distribution, but would not be accurate.

Original: 4. A BLATION STUDY
Revised: 4. A BLATION STUDY

Original: To address the separate inﬂuence of the adversarial network to force output distribution and AAE for regularization, we conducted an ablation study that excluded each component.
Revised: To address the separate inﬂuence of the adversarial network to force output distribution and AAE for regularization, we conducted an ablation study that excluded each component.

Original: When only the forcing output distribution part was excluded, the model showed almost uniform values outside of the labeled region (Figure 3a).
Revised: When only the forcing output distribution part was excluded, the model showed almost uniform values outside of the labeled region (Figure 6a).

Original: The result indicates that regularization alone is not sufﬁcient for extrapolation in this task.
Revised: The result indicates that regularization alone is not sufﬁcient for extrapolation in this task.

Original: Comparison with (Figure 2b) demonstrates that the regularization of the predicted values reduces the variation, but that the effect of skewed dataset holds the predicted values to near the lowest labeled data point.
Revised: Comparison with (Figure 2b) demonstrates that the regularization of the predicted values reduces the variation, but that the effect of skewed dataset holds the predicted values to near the lowest labeled data point.

Original: As a result, the RMSE of the model is not meaningfully improved from that of the model without the proposed approach (Table 1).
Revised: As a result, the RMSE of the model is not meaningfully improved from that of the model without the proposed approach (Table 1).

Original: When only AAE for regularization was excluded, the prediction values were relatively accurate but the trend of prediction according to ground truth ﬂuctuated slightly (Figure 3b).
Revised: When only AAE for regularization was excluded, the prediction values were relatively accurate but the trend of prediction according to ground truth ﬂuctuated slightly (Figure 6b).

Original: The ﬂuctuation suggests a decrease in the ability to convey information from labeled data.
Revised: The ﬂuctuation suggests a decrease in the ability to convey information from labeled data.

Original: Although the RMSE of the regression model was
Revised: Although the RMSE of the regression model was

Original: P r e d i c t e d v a l u e
Revised: P r e d i c t e d v a l u e

Original: Ground truth (
Revised: Ground truth (

Original: [Figure]
Revised: [Figure]

Original: P r e d i c t e d v a l u e
Revised: P r e d i c t e d v a l u e

Original: Ground truth
Revised: Ground truth

Original: Bias threshold (b) - - -    - - -
Revised: Bias threshold (b) - - -    - - -

Original: P r e d i c t e d v a l u e
Revised: P r e d i c t e d v a l u e

Original: Ground truth
Revised: Ground truth

Original: Bias threshold (c) - - -    - - -
Revised: Bias threshold (c) - - -    - - -

Original: [Figure]
Revised: [Figure]

Original: r
Revised: r

Original: [Figure]
Revised: [Figure]

Original: qu e
Revised: qu e

Original: [Figure]
Revised: [Figure]

Original: F r e qu e n c y (
Revised: F r e qu e n c y (

Original: [Figure]
Revised: [Figure]

Original: F r e qu e n c y ( A . U . )
Revised: F r e qu e n c y ( A . U . )

Original: [Figure]
Revised: [Figure]

Original: threshold (f)
Revised: threshold (f)

Original: [Figure]
Revised: [Figure]

Original: ∼ 40 % lower than that of the model without the proposed approach (Table 1), strong regularization seems wise, because in practice, we cannot evaluate the error of the model if fully-labeled data are not available.
Revised: ∼ 40 % lower than that of the model without the proposed approach (Table 1), strong regularization seems wise, because in practice, we cannot evaluate the error of the model if fully-labeled data are not available.

Original: 4. G ENERAL APPLICABILITY OF PROPOSED APPROACH
Revised: 4. G ENERAL APPLICABILITY OF PROPOSED APPROACH

Original: RMSE were also collected for the models for datasets other than QSAR (Table 1).
Revised: RMSE were also collected for the models for datasets other than QSAR (Table 1).

Original: For the other three datasets, RMSE was reduced by 55 % to 75 % compared to the RMSE of the regression model trained with the skewed dataset and without the proposed approach.
Revised: For the other three datasets, RMSE was reduced by 55 % to 75 % compared to the RMSE of the regression model trained with the skewed dataset and without the proposed approach.

Original: The result was consistent even for the difﬁcult task that shows high RMSE (e.g., house).
Revised: The result was consistent even for the difﬁcult task that shows high RMSE (e.g., house).

Original: This result demonstrates that even slight information from the labeled data may guide the process of forcing output distribution.
Revised: This result demonstrates that even slight information from the labeled data may guide the process of forcing output distribution.

Original: However, in practice, tasks may be performed on datasets for which the true distribution cannot be estimated accurately even when the dataset is known to be skewed.
Revised: However, in practice, tasks may be performed on datasets for which the true distribution cannot be estimated accurately even when the dataset is known to be skewed.

Original: The proposed approach relies strongly on the assumed true distribution, so we tested sensitivity of the proposed approach to the assumed true distribution.
Revised: The proposed approach relies strongly on the assumed true distribution, so we tested sensitivity of the proposed approach to the assumed true distribution.

Original: For this purpose, we estimated the true distribution of the datasets by using only 20 random samples instead of the full data.
Revised: For this purpose, we estimated the true distribution of the datasets by using only 20 random samples instead of the full data.

Original: Interestingly, the degree of RMSE improvement over the model without the proposed approach was reduced by only 10 % to 20 % .
Revised: Interestingly, the degree of RMSE improvement over the model without the proposed approach was reduced by only 10 % to 20 % .

Original: Considering that the true distributions of the datasets are not exactly Gaussian, this result indicates that the proposed approach is not sensitive to error in the assumed true distribution, and is appropriate for practical applications.
Revised: Considering that the true distributions of the datasets are not exactly Gaussian, this result indicates that the proposed approach is not sensitive to error in the assumed true distribution, and is appropriate for practical applications.

Original: P r e d i c t e d v a l u e
Revised: P r e d i c t e d v a l u e

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: P r e d i c t e d v a l u e
Revised: P r e d i c t e d v a l u e

Original: Ground truth
Revised: Ground truth

Original: Bias threshold (b) - - -    - - -
Revised: Bias threshold (b) - - -    - - -

Original: F r e qu e n c y ( A . U . )
Revised: F r e qu e n c y ( A . U . )

Original: pLogP value (c)
Revised: pLogP value (c)

Original: [Figure]
Revised: [Figure]

Original: F r e qu e n c y ( A . U )
Revised: F r e qu e n c y ( A . U )

Original: [Figure]
Revised: [Figure]

Original: (d)
Revised: (d)

Original: [Figure]
Revised: [Figure]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We proposed a new approach to improve a regression model that is trained on skewed dataset.
Revised: We proposed a new approach to improve a regression model that is trained on skewed dataset.

Original: The method uses adversarial forcing to make the output distribution follow the assumed true distribution.
Revised: The method uses adversarial forcing to make the output distribution follow the assumed true distribution.

Original: The adversarial network to force the output distribution restrains a regression model to have the same distribution as the output distribution, and the AAE regularizes the model in the appropriate way.
Revised: The adversarial network to force the output distribution restrains a regression model to have the same distribution as the output distribution, and the AAE regularizes the model in the appropriate way.

Original: We evaluated the proposed approach on four datasets.
Revised: We evaluated the proposed approach on four datasets.

Original: The proposed approach increased the accuracy of the regression model on all datasets.
Revised: The proposed approach increased the accuracy of the regression model on all datasets.

Original: Further assessment showed that the regression quality is maintained even when the proposed approach is applied with rough estimation of the true distribution. 
Revised: Further assessment showed that the regression quality is maintained even when the proposed approach is applied with rough estimation of the true distribution. A A PPENDIX A. E XPERIMENTS ON OTHER TYPES OF BIASES F r e qu e n c y ( A . U . ) pLogP value - - -    - - -

Revised: P r e d i c t e d v a l u e

Revised: Ground truth - - -    - - -

Revised: F r e qu e n c y ( A .

Revised: U )

Revised: pLogP value - - -    - - -

Revised: P

Revised: [Figure]

Revised: e d i c t e d v a l u e

Revised: Ground truth

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: F r e qu e n c y ( A . U . )

Revised: pLogP value - - -    - - -

Revised: P r e d i c t e d v a l u e

Revised: Ground truth - - -    - - -

Revised: F r e qu e n c y ( A .

Revised: U )

Revised: pLogP value - - -    - - -

Revised: P r e d i c t e d v a l u e

Revised: Ground truth

Revised: [Figure]

Revised: F r e qu e n c y ( A . U . )

Revised: pLogP value - - -    - - -

Revised: P r e d i c t e d v a l u e

Revised: Ground truth - - -    - - -

Revised: F r e qu e n c y ( A .

Revised: U )

Revised: pLogP value - - -    - - -

Revised: P r e d i c t e d v a l u e

Revised: Ground truth

Revised: [Figure]