Package: pysolotools-fiftyone
Fiftyone viewer for SOLO DatasetsScreenshot of the example solo dataset displayed in Voxel51Introductionpysolotools-fiftyone is a python package for viewing and interacting with solo datasets using theVoxel51viewer.This package allows the user to create a new Voxel51 viewer in either a web browser or jupyter notebook.SOLO datasets are generated by Unity'sPerception package. Please seeherefor more information about the SOLO schema.It supports the displaying of the following annotations:Bounding Boxes3D Bounding BoxesKeypointsInstance SegmentationSemantic SegmentationDepthOcclusionNormalsPixel PositionPre-RequisitesInstallAnacondaorMiniconda(recommended).Installpre-commit.InstallOpenEXR.Make surepipis installedInstallationpipinstallpysolotools-fiftyoneExecutionCLIThe Solo Fiftyone viewer is executed on the command line with the following command line:pysolotools-fiftyoneexamples/groceriesThis will launch a new fiftyone viewer in your web browser.NotebooksSOLO FiftyOne can be executed inJupyter Notebooks.To use in a notebook:frompysolotools_fiftyone.solo_fiftyoneimportrun_in_notebookpath="./groceries"run_in_notebook(path)Anexample notebookhas been included in the repository for your convenience.The fiftyone will create an entry for each frame. Each entry will consist of a group with at least an RGB capture of the
Unity game screen. If the dataset was created with Normals or Pixel Position labelers, the image group will also contain
a PNG image for each one of them. These 3 types create the background of the frame, and thelabelsare drawn on to of
them. The background image can be changed with thegroupdropdown located at the upper right of the screen.Group dropdownLabels are drawn on top of the background group image. These can be toggled on and off via theLabelsselector left
of the image.Label selectorAnnotation TypesBounding Boxes2D bounding boxes and label type for each visible labeled object in the image. Detailed information includes the amount
of visible pixels of the object.Bounding boxes in Voxel51Semantic SegmentationSemantic segmentation labels each pixel of an image with a label type.Semantic segmentation in Voxel51Instance SegmentationEach pixel of the image is labeled with the ID of the object that it represents.Instance segmentation in Voxel51KeypointsDisplay keypoint (generally, but not limited to, a human skeleton) information.Keypoints in Voxel513D Bounding Boxes3D volume of object in the scene.3D Bounding Boxes in Voxel51DepthPixel distance from the camera in the scene displayed inPlotly's virdis scale. The color scale goes from purple to
yellow, with purple representing objects closer to the camera.Depth image in Voxel51, purple pixels are closer to the camera, yellow pixels are farthest awayOcclusionObject occlusion, or the amount of the object you can see, displayed inPlotly's virdis scale. The scale goes from yellow (fully visible) to purple (fully occluded).
Hovering over a pixel will give you the exact occlusion value.Occlusion in Voxel51, yellow is fully visible, purple is fully occludedNormalsThe normals image displays an object's surface normal in a 3 channel image. The red channel represents the normal's x value,
the green channel represents the y channel, and the blue channel represents the normal's z value.Normals in Voxel51Pixel PositionThe pixel position image displays a pixels cartesian coordinate in the camera's space. The red channel represents the pixel position's x value,
the green channel represents the y channel, and the blue channel represents the z value. The image displayed is a 8bit per channel representation
of a 32bit per channel file format, so some accuracy is lost in the display.Pixel Position in Voxel51Sample SOLO DatasetsInside theexamplesdirectory there are two sample datasetsgroceries: A dataset of grocery items that have all supported labels except keypointskeypoints: A simple dataset that has keypoint annotationsCommunity and FeedbackThe Unity Computer Vision tools are open-source, and we encourage and welcome contributions.
If you wish to contribute, be sure to review ourcontribution guidelinesandcode of conduct.SupportFor feature requests, bugs, or other issues, please file aGitHub issueusing the provided templates we will investigate as soon as possible.LicenseApache License 2.0Know IssuesWhen running with a web browser, it will open with a subset of your frames visible.
The data is still being imported in the background, and refreshing the browser will
update the viewer will the current set of loaded scenes.To properly see some heatmap overlays (for instance depth and occlusion) make sure that you have Voxel51 settings (gear icon at the top of the screen) fieldsColor byset tolabelandLabel opacityto highest (all of the way to the right)