Package: keras-cv
KerasCVKerasCV is a library of modular computer vision components that work natively
with TensorFlow, JAX, or PyTorch. Built on Keras 3, these models, layers,
metrics, callbacks, etc., can be trained and serialized in any framework and
re-used in another without costly migrations. See "Configuring your backend"
below for more details on multi-framework KerasCV.KerasCV can be understood as a horizontal extension of the Keras API: the
components are new first-party Keras objects that are too specialized to be
added to core Keras. They receive the same level of polish and backwards
compatibility guarantees as the core Keras API, and they are maintained by the
Keras team.Our APIs assist in common computer vision tasks such as data augmentation,
classification, object detection, segmentation, image generation, and more.
Applied computer vision engineers can leverage KerasCV to quickly assemble
production-grade, state-of-the-art training and inference pipelines for all of
these common tasks.Quick LinksList of available models and presetsDeveloper GuidesContributing GuideCall for ContributionsAPI Design GuidelinesInstallationKerasCV supports both Keras 2 and Keras 3. We recommend Keras 3 for all new
users, as it enables using KerasCV models and layers with JAX, TensorFlow and
PyTorch.Keras 2 InstallationTo install the latest KerasCV release with Keras 2, simply run:pip install --upgrade keras-cv tensorflowKeras 3 InstallationThere are currently two ways to install Keras 3 with KerasCV. To install the
latest changes for KerasCV and Keras, you can use our nightly package.pip install --upgrade keras-cv-nightly tf-nightlyTo install the stable versions of KerasCV and Keras 3, you should install Keras
3afterinstalling KerasCV. This is a temporary step while TensorFlow is
pinned to Keras 2, and will no longer be necessary after TensorFlow 2.16.pip install --upgrade keras-cv tensorflow
pip install --upgrade keras[!IMPORTANT]
Keras 3 will not function with TensorFlow 2.14 or earlier.Configuring your backendIf you have Keras 3 installed in your environment (see installation above),
you can use KerasCV with any of JAX, TensorFlow and PyTorch. To do so, set theKERAS_BACKENDenvironment variable. For example:
so by setting theKERAS_BACKENDenvironment variable. For example:exportKERAS_BACKEND=jaxOr in Colab, with:importosos.environ["KERAS_BACKEND"]="jax"importkeras_cv[!IMPORTANT]
Make sure to set theKERAS_BACKENDbefore import any Keras libraries, it
will be used to set up Keras when it is first imported.Once that configuration step is done, you can just import KerasCV and start
using it on top of your backend of choice:importkeras_cvimportkerasfilepath=keras.utils.get_file(origin="https://i.imgur.com/gCNcJJI.jpg")image=np.array(keras.utils.load_img(filepath))image_resized=keras.ops.image.resize(image,(640,640))[None,...]model=keras_cv.models.YOLOV8Detector.from_preset("yolo_v8_m_pascalvoc",bounding_box_format="xywh",)predictions=model.predict(image_resized)Quickstartimporttensorflowastfimportkeras_cvimporttensorflow_datasetsastfdsimportkeras# Create a preprocessing pipeline with augmentationsBATCH_SIZE=16NUM_CLASSES=3augmenter=keras_cv.layers.Augmenter([keras_cv.layers.RandomFlip(),keras_cv.layers.RandAugment(value_range=(0,255)),keras_cv.layers.CutMix(),],)defpreprocess_data(images,labels,augment=False):labels=tf.one_hot(labels,NUM_CLASSES)inputs={"images":images,"labels":labels}outputs=inputsifaugment:outputs=augmenter(outputs)returnoutputs['images'],outputs['labels']train_dataset,test_dataset=tfds.load('rock_paper_scissors',as_supervised=True,split=['train','test'],)train_dataset=train_dataset.batch(BATCH_SIZE).map(lambdax,y:preprocess_data(x,y,augment=True),num_parallel_calls=tf.data.AUTOTUNE).prefetch(tf.data.AUTOTUNE)test_dataset=test_dataset.batch(BATCH_SIZE).map(preprocess_data,num_parallel_calls=tf.data.AUTOTUNE).prefetch(tf.data.AUTOTUNE)# Create a model using a pretrained backbonebackbone=keras_cv.models.EfficientNetV2Backbone.from_preset("efficientnetv2_b0_imagenet")model=keras_cv.models.ImageClassifier(backbone=backbone,num_classes=NUM_CLASSES,activation="softmax",)model.compile(loss='categorical_crossentropy',optimizer=keras.optimizers.Adam(learning_rate=1e-5),metrics=['accuracy'])# Train your modelmodel.fit(train_dataset,validation_data=test_dataset,epochs=8,)ContributorsIf you'd like to contribute, please see ourcontributing guide.To find an issue to tackle, please check ourcall for contributions.We would like to leverage/outsource the Keras community not only for bug reporting,
but also for active development for feature delivery. To achieve this, here is the predefined
process for how to contribute to this repository:Contributors are always welcome to help us fix an issue, add tests, better documentation.If contributors would like to create a backbone, we usually require a pre-trained weight set
with the model for one dataset as the first PR, and a training script as a follow-up. The training script will preferably help us reproduce the results claimed from paper. The backbone should be generic but the training script can contain paper specific parameters such as learning rate schedules and weight decays. The training script will be used to produce leaderboard results.
Exceptions apply to large transformer-based models which are difficult to train. If this is the case,
contributors should let us know so the team can help in training the model or providing GCP resources.If contributors would like to create a meta arch, please try to be aligned with our roadmap and create a PR for design review to make sure the meta arch is modular.If contributors would like to create a new input formatting which is not in our roadmap for the next 6 months, e.g., keypoint, please create an issue and ask for a sponsor.If contributors would like to support a new task which is not in our roadmap for the next 6 months, e.g., 3D reconstruction, please create an issue and ask for a sponsor.Thank you to all of our wonderful contributors!Pretrained WeightsMany models in KerasCV come with pre-trained weights.
With the exception of StableDiffusion and the standard Vision Transformer, all of these weights are trained using Keras and
KerasCV components and training scripts in this repository.
While some models are not trained with the same parameters or preprocessing pipeline
as defined in their original publications, the KerasCV team ensures strong numerical performance.
Performance metrics for the provided pre-trained weights can be found
in the training history for each documented task.
An example of this can be found in the ImageNet classification traininghistory for backbone models.
All results are reproducible using the training scripts in this repository.Historically, many models have been trained on image datasets rescaled via manually
crafted normalization schemes.
The most common variant of manually crafted normalization scheme is subtraction of the
imagenet mean pixel followed by standard deviation normalization based on the imagenet
pixel standard deviation.
This scheme is an artifact of the days of manual feature engineering, but is no longer
required to score state of the art scores using modern deep learning architectures.
Due to this, KerasCV is standardized to operate on images that have been rescaled using
a simple1/255rescaling layer.
This can be seen in all KerasCV training pipelines and code examples.Custom OpsNote that in some of the 3D Object Detection layers, custom TF ops are used. The
binaries for these ops are not shipped in our PyPi package in order to keep our
wheels pure-Python.If you'd like to use these custom ops, you can install from source using the
instructions below.Installing KerasCV with Custom Ops from SourceInstalling custom ops from source requires theBazelbuild
system (version >= 5.4.0). Steps to install Bazel can befound here.git clone https://github.com/keras-team/keras-cv.git
cd keras-cv

python3 build_deps/configure.py

bazel build build_pip_pkg
export BUILD_WITH_CUSTOM_OPS=true
bazel-bin/build_pip_pkg wheels

pip install wheels/keras_cv-*.whlNote that GitHub actions exist to release KerasCV with custom ops, but are
currently disabled. You can use theseactionsin your own fork to create wheels for Linux (manylinux2014), MacOS (both x86 and ARM),
and Windows.DisclaimerKerasCV provides access to pre-trained models via thekeras_cv.modelsAPI.
These pre-trained models are provided on an "as is" basis, without warranties
or conditions of any kind.
The following underlying models are provided by third parties, and are subject to separate
licenses:
StableDiffusion, Vision TransformerCiting KerasCVIf KerasCV helps your research, we appreciate your citations.
Here is the BibTeX entry:@misc{wood2022kerascv,title={KerasCV},author={Wood, Luke and Tan, Zhenyu and Stenbit, Ian and Bischof, Jonathan and Zhu, Scott and Chollet, Fran\c{c}ois and Sreepathihalli, Divyashree and Sampath, Ramesh and others},year={2022},howpublished={\url{https://github.com/keras-team/keras-cv}},}