Title: LEARNING TO AUGMENT INFLUENTIAL DATA

Abstract: Data augmentation is a technique to reduce overfitting and to improve generalization by increasing the number of labeled data samples by performing label preserving transformations; however, it is currently conducted in a trial and error manner. A composition of predefined transformations, such as rotation, scaling and cropping, is performed on training samples, and its effect on performance over test samples can only be empirically evaluated and cannot be predicted. This paper considers an influence function which predicts how generalization is affected by a particular augmented training sample in terms of validation loss. The influence function provides an approximation of the change in validation loss without comparing the performance which includes and excludes the sample in the training process. A differentiable augmentation model that generalizes the conventional composition of predefined transformations is also proposed. The differentiable augmentation model and reformulation of the influence function allow the parameters of the augmented model to be directly updated by backpropagation to minimize the validation loss. The experimental results show that the proposed method provides better generalization over conventional data augmentation methods.

1. INTRODUCTION:

In supervised learning, deep neural networks generally require large amounts of labeled data for training. Insufficient labeled data will lead to poor generalization due to overfitting. One simple method to reduce overfitting and to improve generalization is to perform data augmentation, which involves creating additional labeled data by transforming each training sample with label preserving transformations. Different augmentation can bring about huge differences in test performance depending on the task (Dosovitskiy et al. (2016) ; Graham (2014) ; Cui et al. (2015) ; Zhang et al. (2015) ). Unfortunately, it is an area which has not been extensively explored. Even on a well-studied image classification task (Simonyan & Zisserman (2015) ; He et al. (2016) ; Xie et al. (2017) ), training data is often augmented in a manner similar to that performed in training the AlexNet (Krizhevsky et al. (2012) ). A composition of predefined transformations, such as rotation, translation, cropping, scaling and color perturbation, is a popular choice; however, determining the strength of each transformation that will lead to the best performance is an art and is often set in an empirical manner with trial and error by observing validation loss (Simard et al. (2003) ; Ciregan et al. (2012) ).

An alternative to tuning a composition of predefined transformations is to learn a strategy for composing the transformations (Sixt et al. (2018) ; Ratner et al. (2017) ; Lemley et al. (2017) ; Peng et al. (2018) ; Cubuk et al. (2018) ). To learn a strategy, various criteria have been considered. Sixt et al. (2018) and Ratner et al. (2017) considered a strategy for augmenting realistic samples without considering the classification model. Given a classification model, Lemley et al. (2017) and Peng et al. (2018) considered antithetical strategies for augmenting samples that respectively minimize (Lemley et al. (2017) ) or maximize (Peng et al. (2018) ) training loss. It is not fully understood why these antithetical studies that relate data augmentation to training loss are effective in improving performance on test samples. Cubuk et al. (2018) considered small child models to compute a validation loss for augmenting samples to improve generalization. Here, learning requires a reinforcement learning framework with the validation loss as a reward signal. It is not clear how well the validation loss of the small child model approximates the validation loss of the final classification model which may have little relevance to the child model. This paper proposes a data augmentation method that is able to directly trace the impact of the augmentation model on generalization. For this, we considered an influence function (Cook & Weisberg (1980) ; Koh & Liang (2017) ) to compute how the validation loss is affected by a particular augmented training sample. The influence function approximates the change in validation loss due to inclusion or exclusion of the augmented training sample without a leave-one-out retraining process. We also propose a differentiable augmentation model that generalizes the conventional augmentation method by the composition of predefined transformations. Reformulation of the influence function and the differentiable augmentation model enable a gradient of the validation loss to flow through the augmented samples and augmentation model; thus, the augmentation model can learn to augment samples to directly improve generalization by backpropagation.

The remainder of this paper is organized as follows. Section 2 briefly reviews some of the most relevant studies in the literature related to the proposed method, and Sections 3 and 4 describe the details of the proposed method. The experimental and comparative results are reported in Section 5. The conclusions are presented in Section 6.

2. RELATED WORK:



2.1. DATA AUGMENTATION METHODS:

In this subsection, we provide a brief overview of data augmentation methods in the following three categories: (i) unsupervised methods that do not consider the classification model during learning1 , (ii) adversarial methods that maximize classification loss and (iii) supervised methods that minimize classification loss.

Unsupervised methods Unsupervised methods include the conventional data augmentation method, which is a composition of predefined transformations, such as rotating, translating, cropping, scaling and color perturbation (Simonyan & Zisserman (2015) ; He et al. (2016) ; Xie et al. (2017) ; Krizhevsky et al. (2012) ). The transformations are manually chosen in an empirical way with trial and error by observing the validation loss (Simard et al. (2003) ; Ciregan et al. (2012) ). Ratner et al. (2017) considered a generator that generates a sequence of predefined transformations. Given the sequence, the training sample is augmented by applying predefined transformations in a consecutive way. The generator is learned in the generative adversarial network (GAN) framework (Goodfellow et al. (2014) ) that the generated sequence produces a realistic augmented sample; however, the classifier is not considered during the learning process and the effect of data augmentation can only be observed by trial and error.

Adversarial methods Adversarial methods include hard sample mining that collects or augments samples that are misclassified by the current classification model. It has been used in training SVM models (Dalal & Triggs (2005) ), boosted decision trees (Dosovitskiy et al. (2016) ), shallow neural networks (Rowley et al. (1998) ) and deep neural networks (Shrivastava et al. (2016) ). Peng et al. (2018) and Wang et al. (2017a) considered generating hard samples by adversarially updating ranges of predefined transformations, such as occluding (Peng et al. (2018) ; Wang et al. (2017a) ), scaling and rotating (Peng et al. (2018) ). In these methods, flexible and complex transformation models cannot be used because adversarial update on these models may generate adversarial examples (Peng et al. (2018)) . Adversarial examples are perturbed or transformed samples by a small amount, but they result in the classification model predicting an incorrect answer with high confidence (Szegedy et al. (2014) ). Training classification model with adversarial examples provides the robustness on adversarial examples but often degenerate performance on clean test samples (Tramèr et al. (2018) ). Recent studies have shown that adversarial updates in convolution based transformations (Baluja & Fischer (2018) ) and spatial transformations (Xiao et al. (2018) ) may generate adversarial examples. Lemley et al. (2017) designed an augmentation model that learns how to augment samples while learning a classification model in a way that reduces the training classification loss, but its effect on performance over test samples can only be empirically evaluated and cannot be predicted. Cubuk et al. ( 2018) considered small child models to compute validation loss to evaluate several augmentation policies over predefined transformations; however, learning requires a reinforcement learning framework because a gradient cannot flow from the validation loss to training samples, and non-differentiable predefined transformations stop the gradient. Furthermore, the validation loss of the small child model may be different from the validation loss of the final classification model. Goodfellow et al. (2014) proposed a generative adversarial network (GAN) that is a framework for training a deep generative network in an adversarial process. The framework considers the simultaneous training of two networks: a generator that generates a sample from random noise and a discriminator that estimates the probability that a sample came from the training data rather than the generator. The adversarial process is formed by a minimax two-player game between a discriminator that learns to distinguish a source of the sample and a generator that is trained to generate an indistinguishable sample by the discriminator.

Supervised methods:



2.2. GENERATIVE ADVERSARIAL NETWORKS:

Based on the GAN framework, several studies have shown their potential to use on data augmentation by generating class-conditional images (Odena et al. (2017) 

2.3. INFLUENCE FUNCTIONS:

The influence function is a classic technique from robust statistics (Cook & Weisberg (1980) ) that estimates how the model parameters change due to up-weighting a particular training sample by a small amount. Cook & Weisberg (1980) focused on removing training points from linear models and Cook (1986) ; Thomas & Cook (1990) ; Wei et al. (1998) studied them with a wider variety of perturbations. Koh & Liang (2017) scaled up the influence function to non-convex and highly-complex models, including deep neural networks, by using efficient approximations based on Hessian-vector products (HVPs; Pearlmutter (1994)), conjugate gradients (Martens (2010) ) and stochastic estimation (Agarwal et al. (2016) ). They also considered the influence on the validation loss as up-weighting a particular training sample that can be computed without a retraining process and can be used for multiple purposes: debugging models, detecting dataset errors and even creating training-set attacks.

3. AUGMENTED DATA EVALUATION:

Tuning or learning an augmentation model involves evaluating an augmented sample (Figure 1 ). Given the classification model, this is often performed by computing the training loss of the sample; however, the effect on test samples cannot be predicted and can only be computed in a trial and error manner. Evaluating an augmented sample using validation loss requires learning two classifiers (including and excluding the sample during the learning process), which is computationally expensive because the models need to be fully trained and because of the cost to compute the loss over validation samples. Rather than repeat this prohibitive process, we propose a method to approximate the validation loss change due to a particular augmented sample without the retraining process.

3.1. PROBLEM SET UP:

In a classification task, given an input space X , an output space Y and parameter space Θ, a learner aims to learn a classification model F that maps X → Y and is parameterized by θ. Given a sample z = (x, y) ∈ X × Y and parameters θ ∈ Θ, let l(z, θ) be the loss evaluated on the sample z at parameters θ. With training data z tr = {z i } N i=1 , an empirical risk and an empirical risk minimizer are given as follows:

EQUATION

where θ(z tr ) = arg min θ∈Θ L(z tr , θ).

(2)

To measure the generalization of the classification model F , validation data z val = {z j } M j=1 is often considered, and generalization is approximated by an average of loss values over validation data z val at parameter θ(z tr ):

EQUATION

As shown in Figure 1 , consider a label preserving transformation G that maps X → X , and let τ be the control parameters, x = G(x, τ ) be an augmented input, z = (x, y) be an augmented sample, and ztr = {z i } N i=1 be an augmented training dataset. In addition, consider a learnable network E parameterized by φ. It estimates control parameters τ given the input x. Thus, x = G(x, E(x, φ)).

Given the transformation model G, the goal is then to find the optimal τ for each input x-otherwise, to find optimal parameters φ of E-that minimizes a classification loss over the validation data z val when the classification model is learned using ztr :

EQUATION

θ(z tr ) = arg min θ∈Θ L(z tr , θ).

(5)

Here, ztr = {x, y} N i=1 = {G(x, E(x, φ)), y} N i=1 . Solving Equations 4-5 requires a bi-level optimization process such that Equation 4 can only be solved after Equation 5 is optimized.

3.2. INFLUENCE BY UPWEIGHTING A TRAINING SAMPLE:

Consider a change in model parameters θ due to removing a particular training sample z i . Formally, this change is θ(z tr \z i )-θ(z tr ). Influence functions (Cook & Weisberg (1980) ; Koh & Liang (2017)) provide an efficient approximation without a retraining process to obtain θ(z tr \z i ). Let us consider the change in model parameters due to upweighting z i by an amount of l(z i , θ) in the loss function:

EQUATION

Then, from Cook & Weisberg (1980) , the following approximation can be derived:

EQUATION

where

EQUATION

Here, H(θ)

1 N N i=1 ∇ 2 θ l(z i , θ

) is the Hessian evaluated at θ. Using Equation 9 and applying the chain rule, the influence of upweighting z i ∈ z tr on the validation loss at z j ∈ z val can be approximated (Koh & Liang (2017)) as shown below:

EQUATION

where

I up, loss (z i , z j ) dl(z j , θ(z tr ∪ z i )) d =0 (11) = ∇ θ l(z j , θ(z tr )) d θ(z n ∪ z i ) d =0 (12) = -∇ θ l(z j , θ(z tr )) H( θ(z tr )) -1 ∇ θ l(z i , θ(z tr )).

(13) For the validation dataset z val , Equation 12 is simply expanded by:

EQUATION

Equation 11 describes a gradient of l(z j , θ(z tr )) w. r. t. at nearby = 0, and the influence of removing z i can be approximate by Equation 10.

3.3. INFLUENCE BY AUGMENTATION:

With a training sample z i and the corresponding augmented training sample zi , let θ(z tr ∪ zi \ z i ) be the estimate of θ with downweighting z i and with upweighting zi by the amount, and let θ(z tr ∪ zi \z i ) be the estimate of θ by replacing z i with zi . An analogous approximation of Equations 10-13 yields:

-

EQUATION

where the influence function I aug, loss (z i , zi , z val ) is:

EQUATION

= -∇ θ L(z val , θ(z tr )) H( θ(z tr )) -1 ∇ θ l( zi , θ(z tr )) -∇ θ l(z i , θ(z tr )) (18)

= I up, loss (z i , z val ) -I up, loss (z i , z val ).

(19) The influence function I aug, loss (z i , zi , z val ) predicts the difference in influences of the augmented sample and the original sample. In addition, the influence function can be used to evaluate the effectiveness of the augmented data zi in comparison to its original. Because the augmentation model is deterministic, this influence by replacement reflects the augmentation process during learning final classification model; however, it does not mean that original samples are excluded during learning the classifier. If the original sample has the most influence than other transformed samples, then E learns identity transformation for that sample.

3.4. REFORMULATION OF INFLUENCE FUNCTIONS:

To compute influence functions efficiently, we adopt techniques such as conjugate gradients and stochastic estimation used in Koh & Liang (2017) . Given G and F , Hessian-vector products (HVPs) s val = H( θ(z tr )) -1 ∇ θ L(z val , θ(z tr )) is precomputed and cumulated for validation samples using conjugate gradients and stochastic estimation techniques. The HVPs are fixed during learning E and are used to compute influence functions of augmented samples. We further approximate the influence function by only considering the top fully connected layer of F2 , thus the remaining ∇ θ l(z i , θ(z tr )) can be represented by a simple closed form. This makes it possible to represent s val ∇ θ l(z i , θ(z tr )) as a closed form by regarding s val as a fixed vector. The gradient from this flows through fixed F and G, and is then used to update E by the chain rule. 

4. TRANSFORMATION MODELS:

Transformations for augmenting images can be categorized as either spatial transformations or appearance transformations. In this section, the proposed transformation models that generalize both transformations are described. The models are differentiable, so a gradient from the influence function can be propagated to E during learning. 

EQUATION

Here, i, j denotes the source coordinate, s w (i, j), and s b (i, j) denote the multiplication factor and bias, respectively. Note that when we perform global average pooling on s w and s b , the transformation is reduced to an affine transform. In this formulation, the spatial transformation is fully defined by s w and s b and these parameters are what we want to generate from τ g .

Spatial transformation model G g takes an input τ g and then generates s w and s b . Flow field s denotes the coordinates to be transformed and is obtained by operations like the point-wise affine transform by s w and s b . The warp operation in Figure 2 -(a) transforms x to x based on the bilinear interpolation indexed by s. The model is designed by a stacked transposed convolutional network with its final layer having 4 + 2 channels. After we obtain s, average pooling is applied to smoothing s, and then the image is warped by bilinear interpolation, which is differentiable (Jaderberg et al. (2015) ). All computations in this formulation can be implemented by a feed-forward neural network.

4.2. APPEARANCE TRANSFORMATION MODEL:

Transformations in appearance include enhance contrast, brightness, color and hue shift. These transformations can be formed by 1 × 1 spatial dimension filters. Thus, to formulate the appearance transformation model, we focused on generating 1 × 1 spatial filters. The proposed transformation model for the appearance transformation is illustrated in Figure 2 filtering x with filter weights c w and bias c b . The model is designed by a transposed convolutional network with its final layer having 1 × 1 + 1 channels for grey images and 3 × 3 + 3 channels for RGB images.

4.3. LEARNING TRANSFORMATION MODELS:

During learning, G is trained based on the GAN framework and is fixed thereafter. Then E is trained to predict τ , which maximizes the influence function for each x and x = G(x, E(x)) pair. In practice, we combine the spatial and appearance transformation model by concatenating (τ s , τ a ) and (s w , s b , c w , c b ). The WGAN-GP (Gulrajani et al.) is used for training G with modifications. First, two discriminators over the image space and flow field s space are considered. The discriminator over the image space is updated to distinguish x and x, and the discriminator over flow field s space is updated to distinguish s = G s (τ ) and manually generated flow fields that will result in random affine transforms. Moreover, to prevent the identity transformation, the inverse of the mean-square error on the flow field space and image space are additionally considered when updating G. For the details of hyperparameters and architectures, refer to Appendix A and B.

5. EXPERIMENTS:



5.1. TOY 2D DATASET:

To visualize the proposed method, we conduct experiments on a toy 2D dataset. Transitions along the x-axis and y-axis are considered to the augmentation model, and the influence is computed by validation samples. A simple 3-layer neural network is considered for classification. In Figure 3 , red and blue dots represent data points of class red and blue, and a contour plot of predictive probability is overlaid.

In the first row of Figure 3 , the influence value of augmented samples near the mis-classified validation sample is high, so the augmentation model learns transitions toward the mis-classified validation samples. In the second row of Figure 3 , we generate training and validation data and then move the validation data to the downside to make a mismatch with the training data. In this setting, the influence value of the downside transition is high, so the augmentation model learns downside transition. 

5.3. CIFAR-10 DATASETS:

We ran experiments on the CIFAR-10 using a subset of the class labels to train the classification models and treating the rest as unlabeled data. We used a ResNet-56 (He et al. (2016) ). In Table 2 , we list classification accuracies for various data augmentation methods from Ratner et al. (2017) . We used a 128 dimensional τ and four-layered convolutional neural network with additional fully connected layer for E and a symmetry transposed convolutional neural network for G. For the proposed method, we additionally use the random cropping and horizontal flip technique as in Ratner et al. (2017) . For the details, refer to Appendix A and B.

6. CONCLUSION:

Data augmentation is a technique for avoiding overfitting and improving generalization by increasing the size of labeled datasets; however, it is currently conducted in a trial and error manner. Composition of predefined transformations, such as rotation, scaling and cropping, are performed on training samples, and its effect on performance over test samples can only be empirically evaluated and cannot be predicted. This paper considered the influence function, which predicts how generalization in terms of a validation loss is affected by a particular augmented training sample without comparing the performances that include and exclude it in the training process. We also proposed a differentiable augmentation model that generalizes the conventional composition of predefined transformations. The differentiable augmentation model and reformulation of the influence function allowed the augmented model parameters to be updated by backpropagation to minimize the validation loss. Our results confirmed that the proposed method provides better generalization than conventional data augmentation methods. 

:

Unsupervised methods may observe validation loss; however, they are referred to as unsupervised because the augmentation model is learned without label supervision.

This approximation is only considered during learning E. All parameters of F are updated during learning F

Figures and Tables:

Figure 1: A generic data augmentation framework for the classification task. An input training sample x is transformed to x by a transformation model which is parameterized by τ . The conventional method usually defines G as a composition of predefined transformations with randomly sampled corresponding ranges τ (solid line path). In this paper, G is a differentiable parametric model, and a learnable network E for estimating τ from x is also considered to obtain the transformed sample x that maximizes generalization of the classification model (dashed line path).

;Nguyen et al. (2017);Mirza & Osindero (2014)) or improving the realism of synthetic samples(Shrivastava et al. (2017);Sixt et al. (2018);Wang et al. (2017b)); however, difficulties in learning generative models (relative to a classification model) require additional unlabeled(Odena et al. (2017);Nguyen et al. (2017);Mirza & Osindero (2014)) or synthetic(Shrivastava et al. (2017);Sixt et al. (2018);Wang et al. (2017b)) samples to improve performance.

Figure 2: The proposed transformation models for spatial and appearance transformation. (a) Spatial transformation model G s takes an input τ g and then generates s w and s b . Flow field s denotes the coordinates to be transformed and is obtained by operations like the point-wise affine transform by s w and s b . The warp operation transforms x to x by interpolating x based on the coordinates in s. (b) Appearance transformation model G a takes an input τ a and then generates c w and c b . x is obtained by filtering x by a filter with weights c w and bias c b .

SPATIAL TRANSFORMATION MODEL Spatial transformations include random flip, crop, scaling, rotation, shearing, translation, and affine transformation. These transformations can be defined by the coordinate change in pixel locations. The proposed transformation model for spatial transformation is illustrated in Figure 2-(a). We define spatial transformation-in a way similar to Jaderberg et al. (2015); Xiao et al. (2018)-by

-(b). The appearance transformation model G a takes an input τ a and then generates c w and c b . c w and c b are average pooled as the spatial transformation model. Transformed image x is obtained by x + δx, where δx is obtained by

Figure 3: Toy example on a synthetic binary classification dataset. Red and blue dots represent data points of class red and blue. A contour plot of predictive probability is overlaid. Each column represents: (i) training samples and the predictive probability contour plot from model parameters learned using training samples, (ii) validation samples and the predictive probability contour plot from model parameters learned using training sample, (iii) augmented samples to maximize influence and the predictive probability contour plot from model parameters learned using augmented samples, (iv) validation samples with the predictive probability contour plot from model parameters learned using augmented samples.

Figure 4: Images are transformed by applying linearly interpolated ten τ 's to the same training sample in CIFAR-10 dataset. Each row represents: (i) the training image, (ii) spatial transformation model outputs, (iii) spatially transformed images, (iv) appearance transformation model outputs and (v) final transformed images.

We ran experiments on the MNIST using only a subset of the class labels to train the classification models and treating the rest as unlabeled data. We used a four-layer all-convolutional CNN for classification. Experiments were conducted under the same setting asRatner et al. (2017). In Table1, we list classification accuracies for various data augmentation methods fromRatner et al. (2017). None indicates that no augmentation is applied, and Heur is the standard heuristic approach of applying random compositions of the given set of transformation operations. Ratner denotes the results fromRatner et al. (2017). We used a 128 dimensional τ and four-layered convolutional neural network with additional fully connected layer for E and a symmetry transposed convolutional neural network for G. For the proposed method, we additionally use the random cropping technique as inRatner et al. (2017). For the details, refer to Appendix A and B.

Table content: <table><tr><td colspan="3">: MNIST accuracies of various data</td><td colspan="3">Table 2: CIFAR-10 accuracies of various</td></tr><tr><td colspan="2">augmentation methods.</td><td/><td colspan="2">data augmentation methods.</td><td/></tr><tr><td/><td colspan="2">The number of labeled data</td><td/><td colspan="2">The number of labeled data</td></tr><tr><td>Method</td><td>1%</td><td>10%</td><td>Method</td><td>10%</td><td>100%</td></tr><tr><td>None</td><td>90.2%</td><td>97.3%</td><td>None</td><td>66.0%</td><td>87.8%</td></tr><tr><td>Heur.</td><td>95.9%</td><td>99.0%</td><td>Heur.</td><td>77.5%</td><td>92.3%</td></tr><tr><td>Ratner (MF)</td><td>96.5%</td><td>99.2%</td><td>Ratner (MF)</td><td>79.8%</td><td>94.4%</td></tr><tr><td>Ratner (LSTM)</td><td>96.7%</td><td>99.1%</td><td>Ratner (LSTM)</td><td>81.5%</td><td>94.0%</td></tr><tr><td>Proposed</td><td>96.7%</td><td>99.3%</td><td>Proposed</td><td>82.1%</td><td>94.8%</td></tr><tr><td colspan="2">5.2 MNIST DATASET</td><td/><td/><td/><td/></tr></table>

Peer review (iclr2019):
 This paper proposes an extension of the influence function study of Koh and Liang (2017) to data augmentation. Influence of augmentation, carried out via a parameterized and differentiable model, on validation loss is approximated and the augmentation model is learned under this approximation. Overall I think it is a valuable and publishable contribution. I do find the paper to be unclear and perhaps could be improved in a few ways. My main comments are: * The biggest question I have is it seems from Eq. 15 that authors are proposing an augmentation approach where the augmented samples replace the original samples and not co-exist with them in the training set. I am not sure why Eq. 15 has to be set up like that, please elaborate. * In Section 3.4 it is stated that only top fully connected layer of F is considered to compute influence function for augmentation. Does this also mean that when F is updated on augmented data only the top layer is updated? Please clarify. * The paper is a bit difficult to follow due to lack of clarity and few errors: - Section 2.1, Adversarial methods, “In these methods, a simple composition…adversarial examples” sentence is unclear - Page 2 footnote “however, they are referred to as unsupervised due to learning is not involved” sentence is unclear - Section 3.3 .tilde{z} in first line should be .tilde{z_i} - Eq. 15 LHS should include .tilde{z_i} - Section 3.4 “adopts” -> “adopt” - Section 3.4 “HVP” used without defining * Empirical evidence, while not extensive, is satisfactory.