Package: sap-computer-vision-package
SAP Computer Vision PackageThis package helps with the implementation of Computer Vision use-cases on top of AI Core.
It extendsdetectron2, a state-of-the-art library for object detection and image segmentation. Our package adds image classification and feature extraction (eg., for image retrieval) capabilities. For a fast development of Computer Vision solutions, the package offers training and evaluation methods and other helpful components, such as a large set of augmentation functions. The package can also be used stand-alone without AI Core, and AI Core integration can be added later to the project.The functionalities of the package can be used on AI Core without any programming. For this purpose the package works with theai-core-sdkto provide a command line interface to create AI Core templates for training and serving. From our experience it reduces the time for implementing a Computer Vision use-case on AI Core from several days to several hours.Supported use-casesObject DetectionImage ClassificationImage Feature ExtractionModel Training and Deployment on SAP AI CoreInstallationPrerequisitesBefore installation, make sure that PyTorch and detectron2 are installed. Details on how to install PyTroch can be foundhere. After the installation of PyTorch the matching version ofdetectron2has to be installed. Please check thedetectron2 installation guideto select the proper version. The package is tested withdetectron2=0.6.Mac OSOn MacOS follwing commands can be used to install both:pip install torch==1.10 torchvision
pip install https://github.com/facebookresearch/detectron2/archive/refs/tags/v0.6.zipLinuxFor linux pre-builds ofdetectron2are available:pip install torch==1.10 torchvision
pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cpu/torch1.10/index.htmlMake sure to select the url matching yourtorchversion andcudawhen GPU support is needed. Details can be found in thedetectron2 installation guide.Installation from SourceWhen building from source normallysetup_without_centaur.pyis the suitable setup file. It skips the process of building the model serving binary locally, which only works on linux systems. The binary is only needed in the docker images to serve models.To include local code changes to the installation run:python setup_with_centaur.py developThis is similar topip install -e ., except thatsetup_without_centaur.pyis used instead ofsetup.py.Installation usingpipTo install this package frompypirun:pip install sap-computer-vision-packageGetting StartedUsing the Python Library PartIf you are interested to use our package as a simple extension todetectron2.Using the Package on AI CoreTheai-core-sdkpackage provides an interface to discover and access content packages like thesap-computer-vision-package.InstallSAP AI Core SDKpip install "ai-core-sdk[aicore-content]"Before testing the pipelines on AI Core, make sure that the items in the following checklist are fulfilled.AI Core ChecklistCompleteAI Core OnboardingAccess to the Git repository, Docker repository and S3 bucket onboarded to AI CoreInstallprerequisitesincludingthe correctmetaflowversionInstallsap_computer_vision_packagelocallyConfigureaws credentials andmetaflowConfigure AWS credentials andmetaflowWhen templates are created metaflow pushes tarballs to the S3 bucket. Those tarballs are loaded during pipeline execution. For this to work metaflow needs writing permissions to the S3 bucket onboarded to AI Core andmetaflowhas to configured to use this bucket as its datastore.Details on how to configure an aws profile can be foundhere. In order to enablemetaflowto copy the tarballs into the bucket, theawsclimust not ask for a password when starting a copy process. To achieve this either give thedefaultprofile the permissions to access the bucket or runexport AWS_PROFILE=<profile-with-bucket-access>before creating templates.Full documentation on how to configuremetaflowcan be found in themetaflow documentation. We only need to configure the S3 as the storage backend anddo not needthe configuration for AWS Batch. A mininmal configuration file (~/.metaflowconfig/config.json) looks like this:{
    "METAFLOW_DATASTORE_SYSROOT_S3": "<path-in-bucket>",
    "METAFLOW_DATATOOLS_SYSROOT_S3": "<path-in-bucket>/data",
    "METAFLOW_DEFAULT_DATASTORE": "s3"
}Basic UsageTo show all available templates runaicore-content list sap-cv.The commandaicore-content show sap_cv <pipeline_name>shows detailed information about a specific pipeline and its parameters.The training pipelines are templates for AI Core execution. To run it under your tenant you need the template and the matching docker image:Build Docker image:Python:workflow.create_image(workflow_config)CLI:aicore-content create-image -p sap-cv -w object-detection-train <workflow_config_file>and push it usingdocker push <tag/docker-image-target>Create Templates:Python:workflow.create_template(workflow_config, out_file)CLI:aicore-content create-template -p sap-cv -w object-detection-train <workflow_config> -o <out_file>The template contains several tenant specific entries likeimagePullSecretsetc. Please adjust them by hand or use a pipeline config YAML (see section below).The template has to be pushed into the onboarded git repo (consult AI Core documentation to set it up) and the container to the onboarded docker repository.Templates are built usingmetaflowusing a plugin to create Argo templates. Make sure that a propermetaflowversion (for the argo plugin, use thesap-ai-core-metaflowversion) is installed and that the storage is configured correctly (check section"Configure AWS credentials andmetaflow").Workflow Config .yamlTenant specific values for the template can either be provided through the CLI through additional options. For more information executeaicore-content create-template sap-cv <workflow_name> --help. To simplify the command and make the creation of the template trackable in git it is possible to use a .yaml containing the values.Example:.contentPackage: sap-cv
.workflow: object-detection-train
name: "your-pipeline-name"       #needs to be unique
labels:
  scenarios.ai.sap.com/id: "my-scenario-id"
  ai.sap.com/version: "0.0.1"
annotations:
  scenarios.ai.sap.com/name: "my-scenario-name"
  executables.ai.sap.com/description: "Description of executable"
  executables.ai.sap.com/name: "my-executable-name"
  artifacts.ai.sap.com/datain.kind: "dataset"
image: "docker-registry/docker-repository:tag"
imagePullSecret: "my-docker-registry-secret"
objectStoreSecret: "default-object-store-secret"To use the workflow config during the creation process pass the path to the WORKFLOW_CONFIG tocreate-imageandcreate-templatesubcommands ofaicore-contentCLI, e.g.aicore-content create-template <workflow_config> <out_file>Common IssuesImpossible to have multiple templates for the same pipeline in a tenant.The name forexecutablespecified in the template has to be unique. To overwrite the default name of a pipeline use the--nameoption when creating the template:sap_cv create-template <pipeline_name> -o/--output-file=<choose_name>.json --name=<executable-name>.Template creations gets stucked without an error.When the template creation process gets stuck in this step:$ aicore-content create-template -p sap-cv -w batch_processing <workflow_config> -o test.json
Metaflow 2.4.4 executing BatchProcessing for user:I545048
Validating your flow...
    The graph looks good!
Running pylint...
    Pylint is happy!
Deploying batchprocessing to Argo Workflow Templates...it is most that the permissions to write to the bucket are missing. Make sure to select the correct AWS profile by runningexport AWS_PROFILE=<profile-with-bucket-access>. More details can be found in the section"Configure AWS credentials andmetaflow".Giving Feedback and Reporting BugsIf you are an SAP customer you can give feedback or report bugs by creating an incident via theSAP ONE Support Launchpadusing the component ID "CA-ML-CV".If you are not an SAP customer yet, you can give feedback or report bugs by registering withSAP Communityand asking aquestionusing the tag "SAP AI Core" in the field "SAP Managed Tags".LicenseThis package is distributed under the SAP Developers License, see LICENSE file in the package. The package uses several third party open source components. Please see file DISCLAIMER for more details on their licensing.DisclaimerThis package is distributed under the SAP Developers License. This license information can be found in the LICENSE file in the package.When you build the template Docker images then a third party base image and several additional open source components are loaded.
Please refer to the license of the PyTorch base image at the bottom of their homepage (https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch).
As with all Docker images, this likely also contains other software which may be under other licenses (such as Bash, etc from the base distribution,
along with any direct or indirect dependencies of the primary software being contained).
The list of the additional open source components can be found in the requirements.txt file in the package.
Please refer to the individual documentation of these components for more details on the licenses.
When using pretrained models the weights are loaded either from detectron2 (https://detectron2.readthedocs.io/en/latest/)
or timms (https://rwightman.github.io/pytorch-image-models/), where you can find more information on their licenses.Note that you are responsible for checking and accepting the license terms for all above mentioned third party components as part of the build and training process.