Package: keras-aug
KerasAugDescriptionKerasAug is a library that includes pure TF/Keras preprocessing and augmentation layers, providing support for various data types such as images, labels, bounding boxes, segmentation masks, and more.Noteleft: the visualization of the layers in KerasAug; right: the visualization of the YOLOV8 pipeline using KerasAugKerasAug aims to provide fast, robust and user-friendly preprocessing and augmentation layers, facilitating seamless integration with TensorFlow, Keras and KerasCV.KerasAug is:🚀 faster thanKerasCVwhich is an official Keras library🧰 supporting various data types, includingimages, labels, bounding boxes, segmentation masks, and more.❤️ dependent only on TensorFlow🌟 seamlessly integrating with thetf.dataandtf.keras.ModelAPIs🔥 compatible with GPU and mixed precision (mixed_float16andmixed_bfloat16)Check out the demo website powered by Streamlit:Apply a transformation to the default or uploaded imageAdjust the arguments of the specified layerWhy KerasAug?KerasAug is generally faster than KerasCVRandomCropAndResize in KerasAug exhibits a remarkable speed-up of+1150%compared to KerasAug. Seekeras-aug/benchmarksfor more details.The APIs of KerasAug are highly stable compared to KerasCVKerasCV struggles to reproduce the YOLOV8 training pipeline, whereas KerasAug executes it flawlessly. SeeQuickstartfor more details.KerasAug comes with built-in support for mixed precision trainingAll layers in KerasAug can run withtf.keras.mixed_precision.set_global_policy('mixed_float16')KerasAug offers the functionality of sanitizing bounding boxes, ensuring the validityThe current layers in KerasAug support the sanitizing process by incorporating thebounding_box_min_area_ratioandbounding_box_max_aspect_ratioarguments.NoteThe degenerate bounding boxes (those located at the bottom of the image) are removed.Installationpipinstallkeras-augtensorflow--upgradeQuickstartRock, Paper and Scissors Image Classificationimportkeras_augimportkeras_cvimporttensorflowastfimporttensorflow_datasetsastfdsfromtensorflowimportkeras# Create a preprocessing pipeline using KerasAugBATCH_SIZE=16NUM_CLASSES=3augmenter=keras.Sequential([keras_aug.layers.RandomFlip(),keras_aug.layers.RandAugment(value_range=(0,255),augmentations_per_image=3,magnitude=15,# [0, 30]magnitude_stddev=0.15,),keras_aug.layers.CutMix(),])defpreprocess_data(images,labels,augment=False):labels=tf.one_hot(labels,NUM_CLASSES)inputs={"images":images,"labels":labels}outputs=augmenter(inputs)ifaugmentelseinputsreturnoutputs["images"],outputs["labels"]train_dataset,test_dataset=tfds.load("rock_paper_scissors",as_supervised=True,split=["train","test"])train_dataset=(train_dataset.batch(BATCH_SIZE).map(lambdax,y:preprocess_data(x,y,augment=True),num_parallel_calls=tf.data.AUTOTUNE,).prefetch(tf.data.AUTOTUNE))test_dataset=(test_dataset.batch(BATCH_SIZE).map(preprocess_data,num_parallel_calls=tf.data.AUTOTUNE).prefetch(tf.data.AUTOTUNE))# Create a model using a pretrained backbonebackbone=keras_cv.models.EfficientNetV2Backbone.from_preset("efficientnetv2_b0_imagenet")model=keras_cv.models.ImageClassifier(backbone=backbone,num_classes=NUM_CLASSES,activation="softmax",)model.compile(loss="categorical_crossentropy",optimizer=keras.optimizers.Adam(learning_rate=1e-5),metrics=["accuracy"],)# Train your modelmodel.fit(train_dataset,validation_data=test_dataset,epochs=8,)# KerasCV Quickstart...
Epoch8/8158/158[==============================]-39s242ms/step-loss:0.7930-accuracy:0.7171-val_loss:0.2488-val_accuracy:0.9946# KerasAug Quickstart...
Epoch8/8158/158[==============================]-34s215ms/step-loss:0.7680-accuracy:0.7567-val_loss:0.2639-val_accuracy:1.0000KerasAug runs faster (215ms/step vs. 242ms/step) than KerasCV and achieves better performance.YOLOV8 Training Pipeline Demoimportkeras_augimportkeras_cvimporttensorflowastfimporttensorflow_datasetsastfdsfromtensorflowimportkerasBATCH_SIZE=16OUTPUT_PATH="output.png"IMAGE_HEIGHT=640IMAGE_WIDTH=640FILL_VALUE=114defvisualize_dataset(inputs,value_range,rows,cols,bounding_box_format,path):inputs=next(iter(inputs.take(1)))images,bounding_boxes=inputs["images"],inputs["bounding_boxes"]keras_cv.visualization.plot_bounding_box_gallery(images,value_range=value_range,rows=rows,cols=cols,y_true=bounding_boxes,scale=5,font_scale=0.7,bounding_box_format=bounding_box_format,path=path,dpi=150,)defunpackage_raw_tfds_inputs(inputs,bounding_box_format):image=inputs["image"]boxes=keras_aug.datapoints.bounding_box.convert_format(inputs["objects"]["bbox"],images=image,source="rel_yxyx",target=bounding_box_format,)bounding_boxes={"classes":tf.cast(inputs["objects"]["label"],dtype=tf.float32),"boxes":tf.cast(boxes,dtype=tf.float32),}return{"images":tf.cast(image,tf.float32),"bounding_boxes":bounding_boxes,}defload_pascal_voc(split,dataset,bounding_box_format):ds=tfds.load(dataset,split=split,with_info=False,shuffle_files=False)ds=ds.map(lambdax:unpackage_raw_tfds_inputs(x,bounding_box_format=bounding_box_format),num_parallel_calls=tf.data.AUTOTUNE,)returndsaugmenter=keras.Sequential(layers=[keras_aug.layers.Resize(IMAGE_HEIGHT,IMAGE_WIDTH,pad_to_aspect_ratio=True,padding_value=FILL_VALUE,bounding_box_format="xywh",),keras_aug.layers.Mosaic(IMAGE_HEIGHT*2,IMAGE_WIDTH*2,fill_value=FILL_VALUE,bounding_box_format="xywh",),keras_aug.layers.RandomAffine(translation_height_factor=0.1,translation_width_factor=0.1,zoom_height_factor=0.5,same_zoom_factor=True,fill_value=FILL_VALUE,bounding_box_format="xywh",bounding_box_min_area_ratio=0.1,bounding_box_max_aspect_ratio=100.0,),keras_aug.layers.Resize(IMAGE_HEIGHT,IMAGE_WIDTH,bounding_box_format="xywh"),# TODO: Blur, MedianBlurkeras_aug.layers.RandomApply(keras_aug.layers.Grayscale(),rate=0.01),keras_aug.layers.RandomApply(keras_aug.layers.RandomCLAHE(value_range=(0,255)),rate=0.01),keras_aug.layers.RandomHSV(value_range=(0,255),hue_factor=0.015,saturation_factor=0.7,value_factor=0.4,),keras_aug.layers.RandomFlip(bounding_box_format="xywh"),])train_ds=load_pascal_voc(split="train",dataset="voc/2007",bounding_box_format="xywh")train_ds=train_ds.ragged_batch(BATCH_SIZE,drop_remainder=True)train_ds=train_ds.map(augmenter,num_parallel_calls=tf.data.AUTOTUNE)visualize_dataset(train_ds,bounding_box_format="xywh",value_range=(0,255),rows=2,cols=2,path=OUTPUT_PATH,)BenchmarkKerasAug is generally faster than KerasCV.TypeLayerKerasAugKerasCVGeometryRandomHFlip21231956fairRandomVFlip18711767fairRandomRotate17031723fairRandomAffine25782355fairRandomCropAndResize2664213+1150%Resize (224, 224)2480222+1017%IntensityRandomBrightness30522768fairRandomContrast*30992976fairRandomBrightnessContrast*2881609+373%RandomColorJitter*2013597+237%RandomGaussianBlur2345203+1055%Invert2691XGrayscale29173116fairEqualize196139+41%AutoContrast30953025fairPosterize30332144fairSolarize31332972fairSharpness29822833fairRegularizationRandomCutout29942795fairRandomGridMask918196+368%MixCutMix29672957fairMixUp18971861fairAutoAugMix79X (Error)RandAugment301246+22%NoteFPS (frames per second)Please refer tobenchmarks/README.mdfor more details.Citing KerasAug@misc{wood2022kerascv,title={KerasCV},author={Wood, Luke and Tan, Zhenyu and Stenbit, Ian and Bischof, Jonathan and Zhu, Scott and Chollet, Fran\c{c}ois and others},year={2022},howpublished={\url{https://github.com/keras-team/keras-cv}},}@misc{chiu2023kerasaug,title={KerasAug},author={Hongyu, Chiu},year={2023},howpublished={\url{https://github.com/james77777778/keras-aug}},}