Package: bounding-ai-tool
Bounding AI ToolA procedural Blender pipeline for photorealistic rendering.Documentation|Tutorials|Examples|ArXiv paper|Workshop paperFeaturesLoading:*.obj,*.ply,*.blend, BOP, ShapeNet, Haven, 3D-FRONT, etc.Objects: Set or sample object poses, apply physics and collision checking.Materials: Set or sample physically-based materials and texturesLighting: Set or sample lights, automatic lighting of 3D-FRONT scenes.Cameras: Set, sample or load camera poses from file.Rendering: RGB, stereo, depth, normal and segmentation images/sequences.Writing: .hdf5 containers, COCO & BOP annotations.InstallationVia pipThe simplest way to install bounding_ai_tool is via pip:pipinstallbounding_ai_toolGit cloneIf you need to make changes to bounding_ai_tool or you want to make use of the most recent version on the main-branch, clone the repository:gitclonehttps://github.com/DLR-RM/BlenderProcTo still make use of the bounding_ai_tool command and therefore use bounding_ai_tool anywhere on your system, make a local pip installation:cdbounding_ai_tool
pipinstall-e.UsageBounding AI Tool has to be run inside the blender python environment, as only there we can access the blender API.
Therefore, instead of running your script with the usual python interpreter, the command line interface of Bounding AI Tool has to be used.bounding_ai_toolrun<your_python_script>In general, one run of your script first loads or constructs a 3D scene, then sets some camera poses inside this scene and renders different types of images (RGB, distance, semantic segmentation, etc.) for each of those camera poses.
Usually, you will run your script multiple times, each time producing a new scene and rendering e.g. 5-20 images from it.
With a little more experience, it is also possible to change scenes during a single script call, readherehow this is done.QuickstartYou can test your Bounding AI Tool pip installation by runningbounding_ai_toolquickstartThis is an alias tobounding_ai_tool run quickstart.pywherequickstart.pyis:importbounding_ai_toolasbprocimportnumpyasnpbproc.init()# Create a simple object:obj=bproc.object.create_primitive("MONKEY")# Create a point light next to itlight=bproc.types.Light()light.set_location([2,-2,0])light.set_energy(300)# Set the camera to be in front of the objectcam_pose=bproc.math.build_transformation_mat([0,-5,0],[np.pi/2,0,0])bproc.camera.add_camera_pose(cam_pose)# Render the scenedata=bproc.renderer.render()# Write the rendering into an hdf5 filebproc.writer.write_hdf5("output/",data)Bounding AI Tool creates the specified scene and renders the image intooutput/0.hdf5.
To visualize that image, simply call:bounding_ai_toolvishdf5output/0.hdf5Thats it! You rendered your first image with Bounding AI Tool!Debugging in the Blender GUITo understand what is actually going on, Bounding AI Tool has the great feature of visualizing everything inside the blender UI.
To do so, simply call your script with thedebuginstead ofrunsubcommand:bounding_ai_tooldebugquickstart.pyNow the Blender UI opens up, the scripting tab is selected and the correct script is loaded.
To start the Bounding AI Tool pipeline, one now just has to pressRun bounding_ai_tool(see red circle in image).
As in the normal mode, print statements are still printed to the terminal.The pipeline can be run multiple times, as in the beginning of each run the scene is cleared.Breakpoint-Debugging in IDEsAs bounding_ai_tool runs in blenders separate python environment, debugging your bounding_ai_tool script cannot be done in the same way as with any other python script.
Therefore, remote debugging is necessary, which is explained for vscode and PyCharm in the following:Debugging with vscodeFirst, install thedebugpypackage in blenders python environment.bounding_ai_tool pip install debugpyNow add the following configuration to your vscodelaunch.json.{"name":"Attach","type":"python","request":"attach","connect":{"host":"localhost","port":5678}}Finally, add the following lines to the top (after the imports) of your bounding_ai_tool script which you want to debug.importdebugpydebugpy.listen(5678)debugpy.wait_for_client()Now run your bounding_ai_tool script as usual via the CLI and then start the added "Attach" configuration in vscode.
You are now able to add breakpoints and go through the execution step by step.Debugging with PyCharm ProfessionalIn Pycharm, go toEdit configurations...and create anew configurationbased onPython Debug Server.
The configuration will show you, specifically for your version, which pip package to install and which code to add into the script.
The following assumes Pycharm 2021.3:First, install thepydevd-pycharmpackage in blenders python environment.bounding_ai_tool pip install pydevd-pycharm~=212.5457.59Now, add the following code to the top (after the imports) of your bounding_ai_tool script which you want to debug.importpydevd_pycharmpydevd_pycharm.settrace('localhost',port=12345,stdoutToServer=True,stderrToServer=True)Then, first run yourPython Debug Serverconfiguration in PyCharm and then run your bounding_ai_tool script as usual via the CLI.
PyCharm should then go in debug mode, blocking the next code line.
You are now able to add breakpoints and go through the execution step by step.What to do next?As you now ran your first bounding_ai_tool script, your ready to learn the basics:TutorialsRead through the tutorials, to get to know with the basic principles of how Bounding AI Tool is used:Loading and manipulating objectsConfiguring the cameraRendering the sceneWriting the results to fileHow key frames workPositioning objects via the physics simulatorExamplesWe provide a lot ofexampleswhich explain all features in detail and should help you understand how Bounding AI Tool works. Exploring our examples is the best way to learn about what you can do with Bounding AI Tool. We also provide support for some datasets.Basic scene: Basic example, this is the ideal place to start for beginnersCamera sampling: Sampling of different camera positions inside of a shape with constraints for the rotation.Object manipulation: Changing various parameters of objects.Material manipulation: Material selecting and manipulation.Physics positioning: Enabling simple simulated physical interactions between objects in the scene.Semantic segmentation: Generating semantic segmentation labels for a given scene.BOP Challenge: Generate the pose-annotated data used at the BOP Challenge 2020COCO annotations: Write COCO annotations to a .json file for selected objects in the scene.and much more, see ourexamplesfor more details.ContributionsFound a bug? help us by reporting it. Want a new feature in the next Bounding AI Tool release? Create an issue. Made something useful or fixed a bug? Start a PR. Check thecontributions guidelines.This repository was forked from BlenderProc, another GPL-licensed project. We intend to build upon the great work of BlenderProc with additional features.