Package: tfaug
Table of Contentstfaug packageFeaturesDependanciesFor test scriptSupported AugmentationsInstallSamplesClassification ProblemConvert Images and Labels to Tfrecord Format by TfrecordConverter()Create Dataset by DatasetCreator()Define and Learn Model Using Defined DatasetsSegmentation ProblemConvert Images and Labels to Tfrecord Format by TfrecordConverter()Create Dataset by DatasetCreator()Define and Learn Model Using Defined DatasetsAdjust sampling ratios from multiple tfrecord filesUse AugmentImg Directly1. Initialize2. use in tf.data.map() after batch()tfaug packageTensorflow >= 2 recommends to be feeded data by tf.data.Dataset.
This package supports creation of tf.data.Dataset (generator) and augmentation for image.This package includes below 3 classes:DatasetCreator - creator of tf.data.Dataset from tfrecords or image pathsTfrecordConverter - pack images and labels to tfrecord format (recommended format for better peformance)AugmentImg - image augmentation class. This is used inside DatasetCreator implicitly or you can use it directly.FeaturesAugment input image and label image with same transformations at the same time.Reduce cpu load by generating all transformation matrix at first. (useinput_shapeparameter atDatasetCreator()orAugmentImg())It could adjust sampling ratios from multiple tfrecord files. (useratio_samplesparameter atDatasetCreator().dataset_from_tfrecords) This is effective for class imbalance problems.Augment on batch. It is more efficient than augment each image.Use only tensorflow operators and builtin functions while augmentation. Because any other operations or functions (e.g. numpy functions) may be bottleneck of learning.mentined here.DependanciesPython >= 3.5tensorflow >= 2.0tensorflow-addonsFor test scriptpillownumpymatplotlibSupported Augmentationsstandardizeresizerandom_rotationrandom_flip_left_rightrandom_flip_up_downrandom_shiftrandom_zoomrandom_shearrandom_brightnessrandom_saturationrandom_huerandom_contrastrandom_croprandom_noiserandom_blurInstallpython -m pip install git+https://github.com/piyop/tfaugSamplesSimple Classification and Segmentation Usage is shown below.
Whole ruunable codes is in sample_tfaug.pyClassification ProblemDownload, convert to tfrecord and learn MNIST dataset.
Below examples are part oflearn_mnist()in sample_tfaug.pyImport tfaug and define directory to be store data.fromtfaugimportTfrecordConverter,DatasetCreator,AugmentImgDATADIR='testdata/tfaug/'Load MNIST dataset using tensorflow.os.makedirs(DATADIR+'mnist',exist_ok=True)# load mnist dataset(x_train,y_train),(x_test,y_test)=tf.keras.datasets.mnist.load_data()Convert Images and Labels to Tfrecord Format by TfrecordConverter()Convert training and validation(test) images and labels into Tfrecord format by TfrecordConverter.
Therefore, Tensorflow could be load data from Tfrecord format with least overhead and parallel reading.# save as tfrecordTfrecordConverter().tfrecord_from_ary_label(x_train,y_train,DATADIR+'mnist/train.tfrecord')TfrecordConverter().tfrecord_from_ary_label(x_test,y_test,DATADIR+'mnist/test.tfrecord')Create Dataset by DatasetCreator()Create and apply augmentation to training and validation Tfrecords by DatasetCreator.
For the classification problem, uselabel_type = 'class'for DatasetCreator constractor.
Set image augmentation params to DatasetCreator constractor.batch_size,shuffle_buffer=25,25# create training and validation dataset using tfaug:ds_train,train_cnt=(DatasetCreator(shuffle_buffer=shuffle_buffer,batch_size=batch_size,label_type='class',repeat=True,random_zoom=[0.1,0.1],random_rotation=20,random_shear=[10,10],training=True).dataset_from_tfrecords([DATADIR+'mnist/train.tfrecord']))ds_valid,valid_cnt=(DatasetCreator(shuffle_buffer=shuffle_buffer,batch_size=batch_size,label_type='class',repeat=True,training=False).dataset_from_tfrecords([DATADIR+'mnist/test.tfrecord']))Add constant reguralization to training and validation datasets.# constant reguralizationds_train=ds_train.map(lambdax,y:(x/255,y))ds_train=ds_valid.map(lambdax,y:(x/255,y))Define and Learn Model Using Defined DatasetsDefine Modelmodel=tf.keras.models.Sequential([tf.keras.layers.Flatten(input_shape=(28,28)),tf.keras.layers.Dense(128,activation='relu'),tf.keras.layers.Dropout(0.2),tf.keras.layers.Dense(10)])model.compile(optimizer=tf.keras.optimizers.Adam(0.002),loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),metrics=['sparse_categorical_accuracy'])Learn Model bymodel.fit(). This accepts training and validationtf.data.Datasetwhich is created by DatasetCreator.model.fit()needs number of training and validation iterations per epoch.# learn modelmodel.fit(ds_train,epochs=10,validation_data=ds_valid,steps_per_epoch=train_cnt//batch_size,validation_steps=valid_cnt//batch_size)Evaluation# evaluation resultmodel.evaluate(ds_valid,steps=x_test.shape[0]//batch_size,verbose=2)Segmentation ProblemDownload ADE20k dataset and convert to the tfrecord
Below examples are part oflearn_mnist()in sample_tfaug.pyFirst, we set input image size and batch size for modelinput_size=[256,256]# cropped input image sizebatch_size=5Download and convert ADE20k dataset to tfrecord by defined function download_and_convertADE20k() in sample_tfaug.py# donwloaddownload_and_convert_ADE20k(input_size)Convert Images and Labels to Tfrecord Format by TfrecordConverter()In download_and_convertADE20k(), split original images to patch image byTfrecordConverter.get_patch()Though ADE20k images have not same image size, tensorflow model input should be exactly same size.converter=TfrecordConverter()~~~~~~~~~~~~~~somecodes~~~~~~~~~~~~~~~~~~~~~~img_patches=converter.get_patch(im,input_size,overlap_buffer,x_borders,y_borders,dtype=np.uint8)lbl_pathces=converter.get_patch(lb,input_size,overlap_buffer,x_borders,y_borders,dtype=np.uint8)Save images and labels as separated tfrecord format.image_per_shards=1000~~~~~~~~~~~~~~somecodes~~~~~~~~~~~~~~~~~~~~~~converter.tfrecord_from_path_label(imgs[sti:sti+image_per_shards],path_labels,path_tfrecord)Create Dataset by DatasetCreator()After generate tfrecord files byTfrecordConverter.tfrecord_from_path_label, create training and validation dataset from these tfrecords by DatasetCreator.
For segmentation problem, uselabel_type = 'segmentation'to the constractor of the DatasetCreator.If you useinput_shapeparam inDatasetCreator()like below,AugmentImge()generate all transformation matrix wheninit() is called. It reduces CPU load while learning.# define training and validation dataset using tfaug:tfrecords_train=glob(DATADIR+'ADE20k/ADEChallengeData2016/tfrecord/training_*.tfrecords')ds_train,train_cnt=(DatasetCreator(shuffle_buffer=batch_size,batch_size=batch_size,label_type='segmentation',repeat=True,standardize=True,random_zoom=[0.1,0.1],random_rotation=10,random_shear=[10,10],random_crop=input_size,dtype=tf.float16,input_shape=[batch_size]+input_size+[3],#batch, y, x, channeltraining=True).dataset_from_tfrecords(tfrecords_train))tfrecords_valid=glob(DATADIR+'ADE20k/ADEChallengeData2016/tfrecord/validation_*.tfrecords')ds_valid,valid_cnt=(DatasetCreator(shuffle_buffer=batch_size,batch_size=batch_size,label_type='segmentation',repeat=True,standardize=True,random_crop=input_size,dtype=tf.float16,training=False).dataset_from_tfrecords(tfrecords_valid))Define and Learn Model Using Defined DatasetsLast step is define and fit and evaluate Model.# define modelmodel=def_unet(tuple(input_size+[3]),151)# 150class + padding areamodel.compile(optimizer=tf.keras.optimizers.Adam(0.002),loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),metrics=['sparse_categorical_accuracy'])model.fit(ds_train,epochs=10,validation_data=ds_valid,steps_per_epoch=train_cnt//batch_size,validation_steps=valid_cnt//batch_size)model.evaluate(ds_valid,steps=valid_cnt//batch_size,verbose=2)Adjust sampling ratios from multiple tfrecord filesIf number of images in each class are significantly imvalanced, you may want adjust sampling ratios from each class.DatasetCreator.dataset_from_tfrecordscould accepts sampling ratios.In that case, you must use 2 dimensional nested list representing tfrecord files topath_recordsinDatasetCreator.dataset_from_tfrecords()and assignsampling_ratiosparameter for every 1 dimensional lists in 2 dimensionalpath_records.
A simple example was written in test_tfaug.py like below:dc=DatasetCreator(5,10,label_type='class',repeat=False,**DATAGEN_CONF,training=True)ds,cnt=dc.dataset_from_tfrecords([[path_tfrecord_0,path_tfrecord_0],[path_tfrecord_1,path_tfrecord_1]],ratio_samples=np.array([1,10],dtype=np.float32))Use AugmentImg DirectlyAbove examples ware create tf.data.Dataset by DatasetCreator. If you need to control your dataflow in other way, you could use AugmentImage Directly1. InitializefromtfaugimportAugmentImg#set your augment parameters below:arg_fun=AugmentImg(standardize=False,random_rotation=5,random_flip_left_right=True,random_flip_up_down=True,random_shift=(.1,.1),random_zoom=(.1,.1),random_shear=(.1,.1),random_brightness=.2,random_saturation=None,random_hue=.2,random_contrast=(.2,.5),random_crop=256,interpolation='nearest'clslabel=True,training=True)2. use in tf.data.map() after batch()ds=tf.data.Dataset.zip((tf.data.Dataset.from_tensor_slices(image),tf.data.Dataset.from_tensor_slices(label)))\.shuffle(BATCH_SIZE*10).batch(BATCH_SIZE).map(arg_fun)model.fit(ds)