Package: datasetinsights
Dataset InsightsUnity Dataset Insights is a python package for downloading, parsing and analyzing synthetic datasets generated using the UnityPerception package.InstallationDatasetinsights is published to PyPI. You can simply runpip install datasetinsightscommand under a supported python environments:Getting StartedDataset StatisticsWe provide a samplenotebookto help you load synthetic datasets generated usingPerception packageand visualize dataset statistics. We plan to support other sample Unity projects in the future.Load DatasetsTheUnity Perceptionpackage provides datasets under thisschema. The datasetinsighs package also provide convenient python modules to parse datasets.For example, you can loadAnnotationDefinitionsinto a python dictionary by providing the corresponding annotation definition ID:fromdatasetinsights.datasets.unity_perceptionimportAnnotationDefinitionsannotation_def=AnnotationDefinitions(data_root=dest,version="my_schema_version")definition_dict=annotation_def.get_definition(def_id="my_definition_id")Similarly, forMetricDefinitions:fromdatasetinsights.datasets.unity_perceptionimportMetricDefinitionsmetric_def=MetricDefinitions(data_root=dest,version="my_schema_version")definition_dict=metric_def.get_definition(def_id="my_definition_id")TheCapturestable provide the collection of simulation captures and annotations. You can load these records directly as a PandasDataFrame:fromdatasetinsights.datasets.unity_perceptionimportCapturescaptures=Captures(data_root=dest,version="my_schema_version")captures_df=captures.filter(def_id="my_definition_id")TheMetricstable can store simulation metrics for a capture or annotation. You can also load these records as a PandasDataFrame:fromdatasetinsights.datasets.unity_perceptionimportMetricsmetrics=Metrics(data_root=dest,version="my_schema_version")metrics_df=metrics.filter_metrics(def_id="my_definition_id")Download DatasetsYou can download the datasets using thedownloadcommand:datasetinsightsdownload--source-uri=<xxx>--output=$HOME/dataThe download command supports HTTP(s), and GCS.Alternatively, you can download dataset directly from pythoninterface.GCSDatasetDownloadercan download a dataset from GCS locations.fromdatasetinsights.io.downloaderimportGCSDatasetDownloadersource_uri=gs://url/to/file.zip# or gs://url/to/folderdest="~/data"downloader=GCSDatasetDownloader()downloader.download(source_uri=source_uri,output=dest)HTTPDatasetDownloadercan a dataset from any HTTP(S) url.fromdatasetinsights.io.downloaderimportHTTPDatasetDownloadersource_uri=http://url.to.file.zipdest="~/data"downloader=HTTPDatasetDownloader()downloader.download(source_uri=source_uri,output=dest)Convert DatasetsIf you are interested in converting the synthetic dataset to COCO format for
annotations that COCO supports, you can run theconvertcommand:datasetinsightsconvert-i<input-directory>-o<output-directory>-fCOCO-Instancesordatasetinsightsconvert-i<input-directory>-o<output-directory>-fCOCO-KeypointsYou will need to provide 2D bounding box definition ID in the synthetic dataset. We currently only support 2D bounding box and human keypoint annotations for COCO format.DockerYou can use the pre-build docker imageunitytechnologies/datasetinsightsto interact with datasets.DocumentationYou can find the API documentation onreadthedocs.ContributingPlease let us know if you encounter a bug by filing an issue. To learn more about making a contribution to Dataset Insights, please see our Contributionpage.LicenseDataset Insights is licensed under the Apache License, Version 2.0. SeeLICENSEfor the full license text.CitationIf you find this package useful, consider citing it using:@misc{datasetinsights2020,
    title={Unity {D}ataset {I}nsights Package},
    author={{Unity Technologies}},
    howpublished={\url{https://github.com/Unity-Technologies/datasetinsights}},
    year={2020}
}