AUTOSAMPLING: SEARCH FOR EFFECTIVE DATA SAMPLING SCHEDULES

1 INTRODUCTION . Data sampling policies can greatly influence the performance of model training in computer vision tasks , and therefore finding robust sampling policies can be important . Handcrafted rules , e.g . data resampling , reweighting , and importance sampling , promote better model performance by adjusting the training data frequency and order ( Estabrooks et al. , 2004 ; Weiss et al. , 2007 ; Bengio et al. , 2009 ; Johnson & Guestrin , 2018 ; Katharopoulos & Fleuret , 2018 ; Shrivastava et al. , 2016 ; Jesson et al. , 2017 ) . Handcrafted rules heavily rely on the assumption over the dataset and can not adapt well to datasets with their own characteristics . To handle this issue , learning-based methods ( Li et al. , 2019 ; Jiang et al. , 2017 ; Fan et al. , 2017 ) were designed to automatically reweight or select training data utilizing meta-learning techniques or a policy network . However existing learning-based sampling methods still rely on human priors as proxies to optimize sampling policies , which may fail in practice . Such priors often include assumptions on policy network design for data selection ( Fan et al. , 2017 ) , or dataset conditions like noisiness ( Li et al. , 2019 ; Loshchilov & Hutter , 2015 ) or imbalance ( Wang et al. , 2019 ) . These approaches take images features , losses , importance or their representations as inputs and use the policy network or other learning approaches with small amount of parameters for estimating the sampling probability . However , for example , images with similar visual features can be redundant in training , but their losses or features fed into the policy network are more likely to be close , causing the same probability to be sampled for redundant samples if we rely on aforementioned priors . Therefore , we propose to directly optimize the sampling schedule itself so that no prior knowledge is required for the dataset . Specifically , the sampling schedule refers to order by which data are selected for the entire training course . In this way , we only rely on data themselves to determine the optimal sampling schedule without any prior . Directly optimizing a sampling schedule is challenging due to its inherent high dimension . For example , for the ImageNet classification dataset ( Deng et al. , 2009 ) with around one million samples , the dimension of parameters would be in the same order . While popular approaches such as deep reinforcement learning ( Cubuk et al. , 2018 ; Zhang et al. , 2020 ) , Bayesian optimization ( Snoek et al. , 2015 ) , population-based training ( Jaderberg et al. , 2017 ) or simple random search ( Bergstra & Bengio , 2012 ) have already been utilized to tune low-dimensional hyper-parameters like augmentation schedules , their applications in directly finding good sampling schedules remain unexploited . For instance , the dimension of a data augmentation policy is generally only in dozens , and it needs thousands of training runs ( Cubuk et al. , 2018 ) to sample enough rewards to find an optimal augmentation policy because high-quality rewards require many epochs of training to obtain . As such , optimizing a sampling schedule may require orders of magnitude more rewards than data augmentation to gather and hence training runs , which result in prohibitively slow convergence . To overcome the aforementioned challenge , we propose a data sampling policy search framework , named AutoSampling , to sufficiently learn an optimal sampling schedule in a population-based training fashion ( Jaderberg et al. , 2017 ) . Unlike previous methods , which focus on collecting longterm rewards and updating hyper-parameters or agents offline , our AutoSampling method collects rewards online with a shortened collection cycle but without priors . Specifically , the AutoSampling collects rewards within several training iterations , tens or hundred times shorter than that in existing works ( Ho et al. , 2019 ; Cubuk et al. , 2018 ) . In this manner , we provide the search process with much more frequent feedback to ensure sufficient optimization of the sampling schedule . Each time when a few training iterations pass , we collect the reward from the previous several iterations , accumulate them and later update the sampling distribution using the rewards . Then we perturb the sampling distribution to search in distribution space , and use it to generate new mini-batches for later iterations , which are recorded into the output sampling schedule . As illustrated in Sec . 4.1 , shortened collection cycles with less interference also can better reflect the training value of each data . Our contributions are as follows : • To our best knowledge , we are the first to propose to directly learn a robust sampling schedule from the data themselves without any human prior or condition on the dataset . • We propose the AutoSampling method to handle the optimization difficulty due to the high dimension of sampling schedules , and efficiently learn a robust sampling schedule through shortened reward collection cycle and online update of the sampling schedule . Comprehensive experiments on CIFAR-10/100 and ImageNet datasets ( Krizhevsky , 2009 ; Deng et al. , 2009 ) with different networks show that the Autosampling can increase the top-1 accuracy by up to 2.85 % on CIFAR-10 , 2.19 % on CIFAR-100 , and 2.83 % on ImageNet . 2 BACKGROUND . 2.1 RELATED WORK . Data sampling is of great significance to deep learning , and has been extensively studied . Approaches with human-designed rules take pre-defined heuristic rules to modify the frequency and order by which training data is presented . In particular , one intuitive method is to resample or reweight data according to their frequencies , difficulties or importance in training ( Estabrooks et al. , 2004 ; Weiss et al. , 2007 ; Drummond et al. , 2003 ; Bengio et al. , 2009 ; Lin et al. , 2017 ; Shrivastava et al. , 2016 ; Loshchilov & Hutter , 2015 ; Wang et al. , 2019 ; Johnson & Guestrin , 2018 ; Katharopoulos & Fleuret , 2018 ; Byrd & Lipton , 2018 ; Jesson et al. , 2017 ) . These methods have been widely used in imbalanced training or hard mining problems . However , they are often restricted to certain tasks and datasets based on which they are proposed , and their ability to generalize to a broader range of tasks with different data distribution may be limited . In another word , these methods often implicitly assume certain conditions on the dataset , such as cleanness or imbalance . In addition , learning-based methods have been proposed for finding suitable sampling schemes automatically . Methods using meta-learning or reinforcement learning are also utilized to automatically select or reweight data during training ( Li et al. , 2019 ; Jiang et al. , 2017 ; Ren et al. , 2018 ; Fan et al. , 2017 ) , but they are only tested on small-scale or noisy datasets . Whether or not they can generalize over tasks of other datasets still remain untested . In this work , we directly study the data sampling without any prior , and we also investigate its wide generalization ability across different datasets such as CIFAR-10 , CIFAR-100 and ImageNet using many typical networks . As for hyper-parameter tuning , popular approaches such as deep reinforcement learning ( Cubuk et al. , 2018 ; Zhang et al. , 2020 ) , Bayesian optimization ( Snoek et al. , 2015 ) or simply random search ( Bergstra & Bengio , 2012 ) have already been utilized to tune low-dimensional hyper-parameters and proven to be effective . Nevertheless , they have not been adopted to find good sampling schedule due to its inherent high dimensiona . Some recent works tackle the challenge of optimizing highdimensional hyper-parameter . MacKay et al . ( 2019 ) uses structured best-response functions and Jonathan Lorraine ( 2019 ) achieve this goal through the combinations of the implicit function theorem and efficient inverse Hessian approximations . However , they have not been tested on the task of optimizing sampling schedules , which is the major focus of our work in this paper . 2.2 POPULATION BASED TRAINING . Hyper-parameter tuning task can be framed as a bi-level optimization problem with the following objective function , min h∈H L ( θ∗ , h ) subject to θ∗ = argmax θ∈Θ eval ( θ , h ) ( 1 ) where θ represents the model weight and h = ( h1 , h2 , · · · , hT ) is the hyper-parameter schedule for T training intervals . Population based training ( PBT ) ( Jaderberg et al. , 2017 ) solves the bilevel optimization problem by training a population P of child models in parallel with different hyper-parameter schedules initialized : P = { ( θi , hi , t ) } Np i=1 ( 2 ) where θi , hi respectively represents the child model weight , the corresponding hyper-parameter schedule for the training interval t on worker i , and Np is the number of workers . PBT proceeds in intervals , which usually consists of several epochs of training . During the interval , the population of models are trained in parallel to finish the lower-level optimization of weights θi . Between intervals , an exploit-and-explore procedure is adopted to conduct the upper-level optimization of the hyper-parameter schedule . In particular for interval t , to exploit we evaluate child models on a held-out validation dataset : h∗t , θ ∗ t = argmax pi= ( θi , hi , t ) ∈P eval ( θi , hi ) θ∗ → θi , i = 1 , · · · , Np ( 3 ) We record the best performing hyper-parameter setting h∗t and broadcast the top-performing model θ∗t to all workers . To explore , we initialize new hyper-parameter schedules for interval t+ 1 with different random seeds on all workers , which can be viewed as a search in the hyper-parameter space . The next exploit-and-explore cycle will then be continued . In the end , the top-performing hyper-parameter schedule h∗ = ( h∗1 , h ∗ 2 , · · · , h∗T ) can be obtained . PBT is applied to tune low-dimenisal hyper-parameters such as data augmentation schedules ( Ho et al. , 2019 ; Jaderberg et al. , 2017 ) . However , it can not be directly used for finding sampling strategies due to the high dimension . Unlike PBT , our AutoSampling adopts a multi-exploitation-andexploration structure , leading to much shorter reward collection cycles that contribute to much more and effective rewards for sufficient optimization within a practical computational budget . 3 AUTOSAMPLING WITH SEARCHING . The overview of our AutoSampling is illustrated in Fig.1 . AutoSampling alternately runs multiexploitation step and exploration step . In the exploration step , we 1 ) update the sampling distribution Algorithm 1 : The Multi-Exploitation Step Input : Training dataset D , population P = { ( θi , hi , t ) } Np i=1 , number of workers Np , number of exploitation intervals T , exploitation interval length Ns Initialize H∗ ← ( ) for t = 1 to T do for j = 1 to Ns do for ( θi , ht , i , t ) ∈ P do θi ←5L ( θi , ht , i ) B update the weight of child model i end for h∗t , θ ∗ t = argmaxP eval ( θi , hi ) H∗ ← H∗ + h∗t B update the sampling for child model i for i = 1 to Np do θi ← θ∗t B clone the optimal weight end for end for end for Return H∗ , P using the rewards collected from the multi-exploitation step ( the sampling distribution is uniform distribution initially ) ; 2 ) perturb the updated sampling distribution for child models so that different child models have different sampling distributions ; 3 ) use the corresponding perturbed sampling distribution for each child model to sample mini-batches of training data . In the multi-exploitation step , we 1 ) train multiple child models using the mini-batches sampled from the exploration step ; 2 ) collect short-term rewards from the child models . AutoSampling finishes with a recorded topperforming sampling schedule , which can be transferred to other models .

Summary: The authors mainly concentrate on data sampling. To address the issue of optimizing high-dimensional sampling hyper-parameter in data sampling and release the requirement of prior knowledge from current methods, the authors introduce a searching-based method named AutoSampling. This method is comprised of exploration step and exploitation step which are conducted alternatively. The exploitation step train multi child models with current sampling strategy and save the best model for next iteration. while the exploration step estimates the sampling distribution according to the sampled data in exploitation step and rectifies it to sample all data possibly. The authors have conducted sufficient experiments to verify the superior of their method, especially for the effectiveness and generalizability.