🤗 Simulate is a proof of concept to investigate whether this situation could be improved.
The simulate library is an exploration on how one could use python to easily build & share complex and diverse simulation environments for embodied learning or synthetic data research.
The basic idea is to decouple the creation of the simulation environment (“building”) from the simulation engine used to run it (Unity, Blender, custom engine, etc) by relying on an engine-agnostic sharing format (the open standard glTF format in this case).
The created environments are stored in a language/framework agnostic format and can be loaded and run on a diversity of engines with concise integrations handling more or less of the glTF extensions we use (we provide PoC plugins for Unity, Godot and Blender in the alpha release).
Interfacing with the git-versioning and hosting on the Hugging Face hub allow to download/upload share/reuse assets (objects) as well as full scenes (environments).
The python API was inspired by the awesome kubric library created by Klaus Greff and Andrea Tagliasacchi and the Google team (https://github.com/google-research/kubric) while the Unity engine was inspired in part by the impressive work of the PRIOR team at AllenAI (https://prior.allenai.org/).