Package: visionai-data-format
visionai-data-formatVisionAIformat isDataversestandardized annotation format to label objects and sequences in the context of Autonomous Driving System(ADS).VisionAIprovides consistent and effective driving environment description and categorization in the real-world case.This tool provides validator ofVisionAIformat schema. Currently, the library supports:Validate createdVisionAIdata formatValidateVisionAIdata attributes with givenOntologyinformation.For a more in-depth understanding of the VisionAI format, please visit this URL:https://linkervision.gitbook.io/dataverse/visionai-format/visionai-data-format.Package (PyPi)|Source codeGetting started(WIP)Install the packagepip3 install visionai-data-formatPrerequisites: You must havePython 3.7and above to use this package.ExampleThe following sections provide examples for the following:Validate VisionAI schemaValidate VisionAI data with given OntologyValidate VisionAI schemaExampleTo validateVisionAIdata structure, could follow the example below:fromvisionai_data_format.schemas.visionai_schemaimportVisionAIModel# your custom visionai datacustom_visionai_data={"visionai":{"frame_intervals":[{"frame_start":0,"frame_end":0}],"frames":{"000000000000":{"objects":{"893ac389-7782-4bc3-8f61-09a8e48c819f":{"object_data":{"bbox":[{"name":"bbox_shape","stream":"camera1","val":[761.565,225.46,98.33000000000004,164.92000000000002]}],"cuboid":[{"name":"cuboid_shape","stream":"lidar1","val":[8.727633224700037,-1.8557590122690717,-0.6544039394148177,0.0,0.0,-1.5807963267948966,1.2,0.48,1.89]}]}}},"frame_properties":{"streams":{"camera1":{"uri":"https://helenmlopsstorageqatest.blob.core.windows.net/vainewformat/kitti/kitti_small/data/000000000000/data/camera1/000000000000.png"},"lidar1":{"uri":"https://helenmlopsstorageqatest.blob.core.windows.net/vainewformat/kitti/kitti_small/data/000000000000/data/lidar1/000000000000.pcd"}}}}},"objects":{"893ac389-7782-4bc3-8f61-09a8e48c819f":{"frame_intervals":[{"frame_start":0,"frame_end":0}],"name":"pedestrian","object_data_pointers":{"bbox_shape":{"frame_intervals":[{"frame_start":0,"frame_end":0}],"type":"bbox"},"cuboid_shape":{"frame_intervals":[{"frame_start":0,"frame_end":0}],"type":"cuboid"}},"type":"pedestrian"}},"coordinate_systems":{"lidar1":{"type":"sensor_cs","parent":"","children":["camera1"]},"camera1":{"type":"sensor_cs","parent":"lidar1","children":[],"pose_wrt_parent":{"matrix4x4":[-0.00159609942076306,-0.005270645688933059,0.999984790046273,0.3321936949138632,-0.9999162467477257,0.012848695454066989,-0.0015282672486530082,-0.022106263278130818,-0.012840436309973332,-0.9999035522454274,-0.0052907123281999745,-0.06171977032225582,0.0,0.0,0.0,1.0]}}},"streams":{"camera1":{"type":"camera","uri":"https://helenmlopsstorageqatest.blob.core.windows.net/vainewformat/kitti/kitti_small/data/000000000000/data/camera1/000000000000.png","description":"Frontal camera","stream_properties":{"intrinsics_pinhole":{"camera_matrix_3x4":[-1.1285209781809271,-706.9900823216068,-181.46849639413674,0.2499212908887926,-3.726606344908137,9.084661126711246,-1.8645282480709864,-0.31027342289053916,707.0385458128643,-1.0805602883730354,603.7910589125847,45.42556655376811],"height_px":370,"width_px":1224}}},"lidar1":{"type":"lidar","uri":"https://helenmlopsstorageqatest.blob.core.windows.net/vainewformat/kitti/kitti_small/data/000000000000/data/lidar1/000000000000.pcd","description":"Central lidar"}},"metadata":{"schema_version":"1.0.0"}}}# validate custom data# If the data structure doesn't meets the VisionAI requirements, it would raise BaseModel error message# otherwise, it will returns dictionary of validated visionai datavalidated_visionai=VisionAIModel(**custom_visionai_data).dict()ExplanationTo begin, we define our customVisionAIdata. Subsequently, we employ theVisionAI(**custom_visionai_data).dict()to ensure the conformity of our custom data with theVisionAIschema. If there are any missing required fields or if the value types deviate from the defined data types, an error will be raised (prompting a list ofVisionAIExceptionexceptions). On the other hand, if the data passes validation, the function will yield a dictionary containing the validatedVisionAIdata.Validate VisionAI data with given OntologyOntology SchemaBefore uploading a dataset to theDataverseplatform, it's advisable to validate VisionAI annotations using theOntologyschema. TheOntologyschema serves as a predefined structure for ProjectOntologydata inDataverse."contexts:
fill this section only if the project ontology is of theclassificationtype.objects: fill this section for project ontologies other thanclassification, such asbounding_boxorsemantic_segmentation.streams:
this section is mandatory as it contains project sensor-related information.tags:
complete this section forsemantic_segmentationproject ontologies.ExampleHere is an example of theOntologySchema and how to validateVisionAIdata using it:fromvisionai_data_format.schemas.ontologyimportOntologycustom_ontology={"objects":{"pedestrian":{"attributes":{"bbox_shape":{"type":"bbox","value":None},"cuboid_shape":{"type":"cuboid","value":None},"activity":{"type":"text","value":[]}}},"truck":{"attributes":{"bbox_shape":{"type":"bbox","value":None},"cuboid_shape":{"type":"cuboid","value":None},"color":{"type":"text","value":[]},"new":{"type":"boolean","value":[]},"year":{"type":"num","value":[]},"status":{"type":"vec","value":["stop","run","small","large"]}}},"car":{"attributes":{"bbox_shape":{"type":"bbox","value":None},"cuboid_shape":{"type":"cuboid","value":None},"color":{"type":"text","value":[]},"new":{"type":"boolean","value":[]},"year":{"type":"num","value":[]},"status":{"type":"vec","value":["stop","run","small","large"]}}},"cyclist":{"attributes":{"bbox_shape":{"type":"bbox","value":None},"cuboid_shape":{"type":"cuboid","value":None}}},"dontcare":{"attributes":{"bbox_shape":{"type":"bbox","value":None},"cuboid_shape":{"type":"cuboid","value":None}}},"misc":{"attributes":{"bbox_shape":{"type":"bbox","value":None},"cuboid_shape":{"type":"cuboid","value":None},"color":{"type":"text","value":[]},"info":{"type":"vec","value":["toyota","new"]}}},"van":{"attributes":{"bbox_shape":{"type":"bbox","value":None},"cuboid_shape":{"type":"cuboid","value":None}}},"tram":{"attributes":{"bbox_shape":{"type":"bbox","value":None},"cuboid_shape":{"type":"cuboid","value":None}}},"person_sitting":{"attributes":{"bbox_shape":{"type":"bbox","value":None},"cuboid_shape":{"type":"cuboid","value":None}}}},"contexts":{"*tagging":{"attributes":{"profession":{"type":"text","value":[]},"roadname":{"type":"text","value":[]},"name":{"type":"text","value":[]},"unknown_object":{"type":"vec","value":["sky","leaves","wheel_vehicle","fire","water"]},"static_status":{"type":"boolean","value":["true","false"]},"year":{"type":"num","value":[]},"weather":{"type":"text","value":[]}}}},"streams":{"camera1":{"type":"camera"},"lidar1":{"type":"lidar"}},"tags":None}# Validate your custom ontologyvalidated_ontology=Ontology(**custom_ontology).dict()# Validate VisionAI data with our ontology, custom_visionai_data is the custom data from upper exampleerrors=VisionAIModel(**custom_visionai_data).validate_with_ontology(ontology=validated_ontology)# Shows the errors# If there is any error occurred, it will returns list of `VisionAIException` exceptions# Otherwise, it will return empty list# example of errors :# >[visionai_data_format.exceptions.visionai.VisionAIException("frame stream sensors {'lidar2'} doesn't match with visionai streams sensor {'camera1', 'lidar1'}.")]print(errors)ExplanationBegin by creating a newOntologythat includes the project ontology. Subsequently, use thevalidate_with_ontology(ontology=validated_ontology)function to check if the currentVisionAIdata aligns with the information in theOntology. The function will return a list ofVisionAIExceptionif any issues are detected; otherwise, it returns an empty list.ToolsConvertBDD+format from/toVisionAIformatConvertBDD+format data toVisionAIformat(Only support box2D and camera sensor data only for now)python3 visionai_data_format/convert_dataset.py -input_format bddp -output_format vision_ai -image_annotation_type 2d_bounding_box -input_annotation_path ./bdd_test.json -source_data_root ./data_root -output_dest_folder ~/visionai_output_dir -uri_root http://storage_test -n_frame 5 -sequence_idx_start 0 -camera_sensor_name camera1 -annotation_name groundtruth -img_extension .jpg --copy_sensor_dataArguments :-input_format: input format (use bddp for BDD+)-output_format: output format (vision_ai)-image_annotation_type: label annotation type for image (2d_bounding_boxfor box2D)-input_annotation_path: source annotation path (BDD+ format json file)-source_data_root: source data root for sensor data and calibration data (will find and copy image from this root)-output_dest_folder: output root folder (VisionAI local root folder)-uri_root: uri root for target upload VAI storage i.e:https://azuresorate/vai_dataset-n_frame: number of frame to be converted (-1 means all), by default -1-sequence_idx_start: sequence start id, by default 0-camera_sensor_name: camera sensor name (default: "", specified it if need to convert camera data)-lidar_sensor_name: lidar sensor name (default: "", specified it if need to convert lidar data)-annotation_name: annotation folder name (default: "groundtruth")-img_extension: image file extention (default: ".jpg")--copy_sensor_data:enable to copy image/lidar dataConvertVisionAIformat data toBDD+format(Only support box2D for now)
The script below could help convertVisionAIannotation data toBDD+json filepython3 visionai_data_format/vai_to_bdd.py -vai_src_folder /path_for_visionai_root_folder -bdd_dest_file /dest_path/bdd.json -company_code 99 -storage_name storge1 -container_name dataset1 -annotation_name groundtruthArguments :-vai_src_folder: VAI root folder contains VAI format json file-bdd_dest_file: BDD+ format file save destination-company_code: company code-storage_name: storage name-container_name: container name (dataset name)-annotation_name: annotation folder name (default: "groundtruth")ConvertKittiformat data toVisionAIformat(Only support KITTI with one camera and one lidar sensor)Important:image type is not restricted, could be ".jpg" or ".png", but we will convert it into ".jpg" inVisionAIformatonly support forP2projection matrix calibration informationCurrently,only supportKITTIdataset with structure folder :.kitti_folder
├──calib
│├──000000.txt
│├──000001.txt
│├──000002.txt
│├──000003.txt
│└──000004.txt
├──data
│├──000000.png
│├──000001.png
│├──000002.png
│├──000003.png
│└──000004.png
├──labels
│├──000000.txt
│├──000001.txt
│├──000002.txt
│├──000003.txt
│└──000004.txt
└──pcd├──000000.pcd├──000001.pcd├──000002.pcd├──000003.pcd└──000004.pcdCommand :python3 visionai_data_format/convert_dataset.py -input_format kitti -output_format vision_ai -image_annotation_type 2d_bounding_box -source_data_root ./data_root -output_dest_folder ~/visionai_output_dir -uri_root http://storage_test -n_frame 5 -sequence_idx_start 0 -camera_sensor_name camera1 -lidar_sensor_name lidar1 -annotation_name groundtruth -img_extension .jpg --copy_sensor_dataArguments :-input_format: input format (use kitti for KITTI)-output_format: output format (vision_ai)-image_annotation_type: label annotation type for image (2d_bounding_box for box2D)-source_data_root: source data root for sensor data and calibration data (will find and copy image from this root)-output_dest_folder: output root folder (VisionAI local root folder)-uri_root: uri root for target upload VAI storage i.e:https://azuresorate/vai_dataset-n_frame: number of frame to be converted (-1 means all), by default -1-sequence_idx_start: sequence start id, by default 0-camera_sensor_name: camera sensor name (default: "", specified it if need to convert camera data)-lidar_sensor_name: lidar sensor name (default: "", specified it if need to convert lidar data)-annotation_name: annotation folder name (default: "groundtruth")-img_extension: image file extention (default: ".jpg")--copy_sensor_data: enable to copy image/lidar dataConvertCOCOformat data toVisionAIformatArguments :-input_format: input format (use coco for COCO format)-output_format: output format (vision_ai)-image_annotation_type: label annotation type for image (2d_bounding_box for box2D)-source_data_root: image data folder-output_dest_folder: output root folder (VisionAI local root folder)-uri_root: uri root for target upload VAI storage i.e:https://azuresorate/vai_dataset-n_frame: number of frame to be converted (-1 means all), by default -1-sequence_idx_start: sequence start id, by default 0-camera_sensor_name: camera sensor name (default: "", specified it if need to convert camera data)-annotation_name: annotation folder name (default: "groundtruth")-img_extension: image file extention (default: ".jpg")--copy_sensor_data: enable to copy image/lidar dataTroubleshooting(WIP)Next steps(WIP)Contributing(WIP)Links to language repos(WIP)Python Readme