Package: geosynth
GeoSynth: A Photorealistic Synthetic Indoor Dataset for Scene UnderstandingDeep learning has revolutionized many scene perception tasks
over the past decade. Some of these improvements can be attributed
to the development of large labeled datasets. The creation of such
datasets can be an expensive, time-consuming, and is an imperfect
process. To address these issues, we introduce GeoSynth, a diverse
photorealistic synthetic dataset for indoor scene perception tasks.
Each GeoSynth exemplar contains rich labels, including segmentation,
geometry, camera parameters, surface material, lighting, and
more. We demonstrate that supplementing real training data with
GeoSynth can significantly improve network performance on
perception tasks, like semantic segmentation.Watch ourpresentation from IEEE VR 2023.GeoSynth is used internally atGeomagical Labsto help power theIKEA Kreativhome design experience.InstallationGeoSynth requires Python>=3.8and can be installed via:pipinstallgeosynthThis installs:Thegeosynthpython library, providing a pythonic interface for
reading and processing GeoSynth data.Thegeosynthcommand-line tool, which offers a convenient way of
downloading the geosynth dataset.Some optional visualization tools will requirematplotlib.Dataset DownloadAttention!Currently only thedemovariant is available, the full dataset and additional datatypes will be released in the near future.To download just a few scenes of the dataset, download thedemovariant.
Thedemovariant is the default--variantoption:geosynthdownloadnon-hdr--variant=demoIf you also wish to include HDR data, specifyall, instead.
The HDR data more than doubles the size of the download, so only download it if you need it.
It is recommended to only specify the data types you need.By default, the contents will be downloaded to~/data/geosynth/.
To specify an alternative download location, specify the--dstargument.See all download options by runninggeosynth download --help:$geosynthdownload--help

Usage:geosynthdownload[OPTIONS]DTYPES...DownloadtheGeoSynthdata.

╭─Arguments──────────────────────────────────────────────────────────────────────────╮
│*dtypesDTYPES...Assetstodownload.Eitherspecify"non-hdr","all",or│
│asubsetof:[cube_environment_map,depth,extrinsics,│
│gravity,hdr_cube_environment_map,hdr_reflectance,│
│hdr_residual,hdr_rgb,hdr_shading,│
│hdr_sphere_environment_map,instance_segmentation,│
│intrinsics,layout_lines_full,layout_lines_occluded,│
│layout_lines_visible,lighting,normals,reflectance,│
│residual,rgb,semantic_segmentation,shading,│
│sphere_environment_map]).│
│[required]│
╰──────────────────────────────────────────────────────────────────────────────────────╯
╭─Options────────────────────────────────────────────────────────────────────────────╮
│--dstPATHGeoSynthdownloaddirectory.│
│[default:~/data/geosynth]│
│--variant[demo|full]Variantofdatasettodownload.│
│[default:demo]│
│--force--no-forceForceare-download,despitelocally│
│cachedfiles.│
│[default:no-force]│
│--cleanup--no-cleanupDeletezipfilesafterunzipping.│
│[default:cleanup]│
│--helpShowthismessageandexit.│
╰──────────────────────────────────────────────────────────────────────────────────────╯UsageOnce the dataset has been downloaded, data can be accessed in python:fromgeosynthimportGeoSynthgeosynth=GeoSynth("PATH_TO_DATA")# or leave empty for default "~/data/geosynth/".print(f"GeoSynth has{len(geosynth)}scenes.")scene=geosynth[100]# Data can be accessed via indexing like a list.# or iterated over in a for loop:forsceneindataset:# Each Scene object contains attributres for each datatype.# Contents can be read from disk via the ``read`` method.rgb=scene.rgb.read()# (H, W, 3) np.ndarraydepth=scene.depth.read()# (H, W) np.ndarrayintrinsics=scene.intrinsics.read()# (3, 3) camera intrinsicsinstances=scene.instance_segmentation.read()# dictionary of instance masks.# many datatypes have a ``visualize`` methoddepth_viz=scene.depth.visualize(depth)# Returns a (H,W,3) turbo-colorized image.instances_viz=scene.instance_segmentation.visualize(instances,rgb=rgb)LicenseThe GeoSynthcodeis released under theApache-2.0 License.The GeoSynthdataprovided atstorage.googleapis.com/geomagical-geosynth-publicis available under theAttribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) License.