Package: mixupy
mixupymixupy is a python package for data-augmentation inspired bymixup: Beyond Empinical Risk MinimizationIf you like mixupy, give it a star, or fork it and contribute!UsageCreate additional training data for the iris dataset:importnumpyasnpimportpandasaspdfrommixupyimportmixup# Use 'iris' dataset from seaborn packageimportseabornassnsiris=sns.load_dataset('iris')# One-hot encode species columniris_df=pd.get_dummies(iris,columns=['species'],prefix='',prefix_sep='')iris_df# Strictly speaking this is 'input mixup' (see Details section below)set.seed(42)iris_mix=mixup(iris_df)iris_mix.describe()iris_df.describe()# Further infohelp(mixup)InstallationpipinstallmixupyRequires python 3.7 or higher plus pandas and numpypipinstallnumpypandasDetailsThe mixup function enlarges training sets using linear interpolations
of features and associated labels as described inhttps://arxiv.org/abs/1710.09412.Virtual feature-target pairs are produced from randomly drawn
feature-target pairs in the training data.
The method is straight-forward and data-agnostic.  It should
result in a reduction of generalisation error.mixup constructs additional training examples:x' = λ * x_i + (1 - λ) * x_j, where x_i, x_j are raw input vectorsy' = λ * y_i + (1 - λ) * y_j, where y_i, y_j are one-hot label encodings(x_i, y_i) and (x_j ,y_j) are two examples drawn at random from the
training data, and λ ∈ [0, 1] with λ ∼ Beta(α, α) for α ∈ (0, ∞).
The mixup hyper-parameter α controls the strength of interpolation between
feature-target pairs.mixup() parametersParameterDescriptionTypeNotesdataOriginal datapandas data frameRequired parameteralphaHyperparameter specifying strength of interpolationnumericDefaults to 4concatConcatenate mixup data with original databooleanDefaults to Falsebatch_sizeHow many mixup values to produceintegerDefaults to number of 'data' examplesThe 'data' parameter must be a numeric (integers and/or floats) pandas
data frame.  Non-finite values are not permitted.  Categorical variables
should be one-hot encoded.Alpha values must be greater than or equal to zero.  Alpha equal to zero
specifies no interpolation.The mixup function returns a pandas data frame containing interpolated
values.  Optionally, the original values can be concatenated with the
new values using theconcat = Trueoption.Mixup with deep learning versus other learning methodsIt is worthwhile distinguishing between mixup usage with
deep learning and other learning methods.  Mixup with deep learning
can improve generalisation when a new mixed dataset is generated
every epoch or even better for every minibatch.  This level
of granularity may not be possible with other learning
methods.  For example, simple linear modeling may not
benefit much from training on a single (potentially greatly
expanded) pre-mixed dataset.  This single pre-mixed dataset
approach is sometimes referred to as 'input mixup'.In certain situations, under-fitting can occur when conflicts
between synthetic labels of the mixup examples and
labels of the original training data are present.  Some learning
methods may be more prone to this under-fitting than others.Data augmentation as regularisationData augmentation is occasionally referred to as a regularisation
technique.
Regularisation decreases a model's variance by adding prior knowledge
(sometimes using shrinkage).
Increasing training data (using augmentation) also decreases a model's
variance.
Data augmentation is also a form of adding prior knowledge to a model.CitingIf you use mixup in a scientific publication, then consider citing the original paper:mixup: Beyond Empirical Risk MinimizationBy Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, David Lopez-Pazhttps://arxiv.org/abs/1710.09412I have no affiliation with MIT, FAIR or any of the authors.RoadmapImprove docsAdd before and after mixup plots for iris dataAdd more detailed examplesDifferent data types e.g. image, temporal etcDifferent parametersAdd my time series mixup variantApplies mixup technique to two time series separated by 'time_diff' periodImplemented and tested in thisencoder decoder Jupyter notebookAdd label preserving optionAdd support for mixing within the same classUsually doesn't perform as well as mixing within all classesMay still have some utility e.g. unbalanced data setsAlternativesOther implementations:pytorch from hongyi-zhangpytorch from facebookresearchkeras from yu4umxnet from unskyAn R package inspired by 'mixup: Beyond Empirical Risk Minimization'See AlsoDiscussion:inference.vcOpenreviewClosely related research:Manifold Mixup: Better Representations by Interpolating Hidden StatesMixUp as Locally Linear Out-Of-Manifold RegularizationLoosely related research:Label smoothingDropoutContributingPull requests are welcome.  For major changes, please open an issue first to discuss what you would like to change.LicenseGPL-3