Repository: bulletphysics/bullet3

Issue:
Support GPU ray cast
GPU accelerated ray casting should be added. 

It can be used for picking, line-of-sight queries and graphics related queries (shadow generation, ray tracer etc). There is a basic demo as a starting point. It supports GPU ray cast against spheres. It needs to be extended to support all collision shapes.

When running out of GPU memory, don't crash
Currently, the GPU OpenCL rigid body pipeline ignores out-of-GPU-memory or other OpenCL errors. We need to deal with it properly.



Discussion:
Latest trunk has GPU raycast for sphere and convex shapes. It can be used for picking and for basic raytracing. The App_BasicGpuDemo uses it for picking, and the App_Bullet3\* demo includes a basic raytraced test.

I'm starting to make some effort on accelerating the GPU raycaster in this branch
(work in progress, not ready for review):
https://github.com/rtrius/bullet3/tree/plbvh_raycast

Presently, the branch implements a Linear BVH that runs on the GPU.
Only the PairBench demo, without large AABBs, is currently supported.

Using the LBVH for the broadphase is ~1-2 ms slower than the 1-axis GPU SAP broadphase. However, the BVH construction algorithm is of extremely low quality and it has yet to be optimized, so there should be a large amount of room for improvement.

That is very exciting! So you are building the tree on the GPU, and also the traversal?
Please keep me posted if you have something to merge, then j can review and pull/merge!

The plbvh_raycast branch is now ready for evaluation.

The latest commit includes slides in the /docs folder that explains the LBVH construction. Additionally, both construction and traversal should be functional and reasonably optimized.

The raycaster is also acclerated using the BVH. With the acceleration, most of the time in the raycasting demo is spent in ray generation and CPU-GPU data transfer. However, ray-triangle mesh and ray-compound tests are still not implemented.

The current method for raycasting is:
    -Update BVH using AABBs from any broadphase
    -BVH traversal generates {ray_index, rigid_index} pairs
    -Sort {ray_index, rigid_index} pairs by ray_index
    -Launch ray-rigid narrowphase/intersection kernel, 1 thread per ray

In my tests, the Linear BVH performs better than the 1-axis SAP for dense scenes, but worse for sparse scenes. Both broadphases are worse than the grid, due to the uniform size of objects. With 27000 AABBs, the break even point where the PLBVH starts to outperform the 1-axis SAP in the PairBench is about scale 76(~5700 overlapping pairs).

```
Some timings for the PairBench with 30x30x30 == 27000 objects, Radeon 5850HD:

At scale: 20 (~226000 overlapping pairs)
    GPU Grid: 9.9 ms
    GPU 1-SAP LDS: 13 ms
    PLBVH: 10.1 ms (3 ms construction, 7 ms traversal)

At scale: 300 (~100 overlapping pairs)
    GPU Grid: 1.9 ms
    GPU 1-SAP LDS: 2.5 ms
    PLBVH: 4 ms (2.8 ms construction, 1.2 ms traversal)
```

Please let me know if there are any issues or places for improvement.

Thanks a lot, that sounds good! 
You provided the timings for finding the overlapping pairs, right?
Do you have any timings for the actual ray test? There is a noticeable slow-down when doing a ray test for many objects, when using the 1-SAP and Grid, because ray test is brute force.
How are the timings of the ray test using the BVH compare?

Here are some timings for the RaycastShadowDemo with the simulation paused,
and the same GPU as above. (Edit: texture size is 512 x 512 == 262 144 rays.)
- ms/frame includes ray generation, CPU-GPU transfer, and writing the result to OpenGL.
- 'primary rays' and 'shadow rays' is the time spent executing BVH/raycast kernels.

## Raycast Scene 1

Timings for Scene 1 (5 x 50 x 5 = 1250 dynamic rigids):
- Brute force: 716ms/frame (294ms primary rays, 313ms shadow rays)
- BVH: 120 ms/frame (25ms primary rays, 26ms shadow rays)

Timings with same camera position as Scene 1, but fewer rigid bodies
(2 x 50 x 2 = 200 dynamic rigids):
- Brute force: 211 ms/frame (48ms primary rays, 50ms shadow rays )
- BVH: 106 ms/frame (14ms primary rays, 13ms shadow rays)
  ![raycastscene1](https://f.cloud.github.com/assets/1758207/2429301/e2e3b976-ac77-11e3-9103-5db090547493.png)

## Raycast Scene 2

Timings for Scene2 (20 x 20 x 20 = 8000 dynamic rigids)
- Brute force: 4100ms/frame (2100ms primary rays, 1900ms shadow rays)
- BVH: 236 ms/frame (85ms primary rays, 34ms shadow rays)
  ![raycastscene2](https://f.cloud.github.com/assets/1758207/2429302/ea24545c-ac77-11e3-8851-63e6830cee67.png)

## Actual profiling output for 'Raycast Scene 2'

(The time for 'Root' over multiple frames varies from 221ms to 244ms):
Profiling: Root (total running time: 235.173 ms) ---
0 -- glFinish (0.00 %) :: 0.000 ms / frame (1 calls)
1 -- window->endRendering (0.22 %) :: 0.521 ms / frame (1 calls)
2 -- gui->draw (1.37 %) :: 3.215 ms / frame (1 calls)
3 -- renderScene (98.30 %) :: 231.175 ms / frame (1 calls)
4 -- texture (0.00 %) :: 0.000 ms / frame (0 calls)
Unaccounted: (0.111 %) :: 0.262 ms
...----------------------------------
...Profiling: renderScene (total running time: 231.175 ms) ---
...0 -- raytrace (99.98 %) :: 231.140 ms / frame (1 calls)
...Unaccounted: (0.015 %) :: 0.035 ms
......----------------------------------
......Profiling: raytrace (total running time: 231.140 ms) ---
......0 -- drawTexturedRect (0.29 %) :: 0.672 ms / frame (1 calls)
......1 -- glGetError (0.00 %) :: 0.001 ms / frame (2 calls)
......2 -- glTexImage2D (0.32 %) :: 0.739 ms / frame (1 calls)
......3 -- get error (0.00 %) :: 0.006 ms / frame (1 calls)
......4 -- write texels (7.16 %) :: 16.544 ms / frame (1 calls)
......5 -- cast shadow rays (19.94 %) :: 46.086 ms / frame (1 calls)
......6 -- init shadow rays (6.98 %) :: 16.132 ms / frame (1 calls)
......7 -- shadowHits.resize (5.88 %) :: 13.596 ms / frame (1 calls)
......8 -- shadowRays.resize (3.99 %) :: 9.223 ms / frame (1 calls)
......9 -- cast primary rays (39.27 %) :: 90.778 ms / frame (1 calls)
......10 -- init hits (1.56 %) :: 3.612 ms / frame (1 calls)
......11 -- hits.resize (3.72 %) :: 8.602 ms / frame (1 calls)
......12 -- Generate primary rays (7.90 %) :: 18.258 ms / frame (1 calls)
......13 -- readbackAllBodiesToCpu (0.47 %) :: 1.079 ms / frame (1 calls)
......14 -- update camera (0.00 %) :: 0.005 ms / frame (1 calls)
......Unaccounted: (2.512 %) :: 5.807 ms
.........----------------------------------
.........Profiling: cast shadow rays (total running time: 46.086 ms) ---
.........0 -- castRaysGPU (99.99 %) :: 46.082 ms / frame (1 calls)
.........Unaccounted: (0.009 %) :: 0.004 ms
............----------------------------------
............Profiling: castRaysGPU (total running time: 46.082 ms) ---
............0 -- raycast copyToHost (11.08 %) :: 5.108 ms / frame (1 calls)
............1 -- ray-rigid intersection (6.17 %) :: 2.845 ms / frame (1 calls)
............2 -- detect ray-rigid pair index ranges (9.96 %) :: 4.588 ms / frame (1 calls)
............3 -- sort ray-rigid pairs (0.26 %) :: 0.121 ms / frame (1 calls)
............4 -- PLBVH testRaysAgainstBvhAabbs() (46.53 %) :: 21.443 ms / frame (1 calls)
............5 -- b3ParallelLinearBvh::build() (6.45 %) :: 2.972 ms / frame (1 calls)
............6 -- raycast copyFromHost (19.11 %) :: 8.807 ms / frame (1 calls)
............Unaccounted: (0.430 %) :: 0.198 ms
...............----------------------------------
...............Profiling: detect ray-rigid pair index ranges (total running time: 4.588 ms) ---
...............0 -- reset ray-rigid pair index ranges (68.55 %) :: 3.145 ms / frame (1 calls)
...............Unaccounted: (31.452 %) :: 1.443 ms
...............----------------------------------
...............Profiling: PLBVH testRaysAgainstBvhAabbs() (total running time: 21.443 ms) ---
...............0 -- PLBVH ray test large AABB (19.36 %) :: 4.151 ms / frame (1 calls)
...............1 -- PLBVH ray test small AABB (78.53 %) :: 16.839 ms / frame (1 calls)
...............Unaccounted: (2.113 %) :: 0.453 ms
...............----------------------------------
...............Profiling: b3ParallelLinearBvh::build() (total running time: 2.972 ms) ---
...............0 -- m_findLeafIndexRangesKernel (4.41 %) :: 0.131 ms / frame (1 calls)
...............1 -- b3GpuParallelLinearBvh::constructBinaryRadixTree() (37.18 %) :: 1.105 ms / frame (1 calls)
...............2 -- Sort leaves by morton codes (23.82 %) :: 0.708 ms / frame (1 calls)
...............3 -- Assign morton codes (4.07 %) :: 0.121 ms / frame (1 calls)
...............4 -- Find AABB of merged nodes (9.35 %) :: 0.278 ms / frame (1 calls)
...............5 -- Separate large and small AABBs (20.83 %) :: 0.619 ms / frame (1 calls)
...............Unaccounted: (0.336 %) :: 0.010 ms
..................----------------------------------
..................Profiling: b3GpuParallelLinearBvh::constructBinaryRadixTree() (total running time: 1.105 ms) ---
..................0 -- m_buildBinaryRadixTreeAabbsRecursiveKernel (46.15 %) :: 0.510 ms / frame (1 calls)
..................1 -- m_findDistanceFromRootKernel (12.94 %) :: 0.143 ms / frame (1 calls)
..................2 -- m_buildBinaryRadixTreeInternalNodesKernel (19.00 %) :: 0.210 ms / frame (1 calls)
..................3 -- m_buildBinaryRadixTreeLeafNodesKernel (10.68 %) :: 0.118 ms / frame (1 calls)
..................4 -- m_computeAdjacentPairCommonPrefixKernel (11.04 %) :: 0.122 ms / frame (1 calls)
..................Unaccounted: (0.181 %) :: 0.002 ms
.....................----------------------------------
.....................Profiling: m_buildBinaryRadixTreeAabbsRecursiveKernel (total running time: 0.510 ms) ---
.....................0 -- copy maxDistanceFromRoot to CPU (30.39 %) :: 0.155 ms / frame (1 calls)
.....................Unaccounted: (69.608 %) :: 0.355 ms
.........----------------------------------
.........Profiling: cast primary rays (total running time: 90.778 ms) ---
.........0 -- castRaysGPU (100.00 %) :: 90.775 ms / frame (1 calls)
.........Unaccounted: (0.003 %) :: 0.003 ms
............----------------------------------
............Profiling: castRaysGPU (total running time: 90.775 ms) ---
............0 -- raycast copyToHost (5.46 %) :: 4.957 ms / frame (1 calls)
............1 -- ray-rigid intersection (7.01 %) :: 6.367 ms / frame (1 calls)
............2 -- detect ray-rigid pair index ranges (17.13 %) :: 15.552 ms / frame (1 calls)
............3 -- sort ray-rigid pairs (0.13 %) :: 0.122 ms / frame (1 calls)
............4 -- PLBVH testRaysAgainstBvhAabbs() (57.84 %) :: 52.507 ms / frame (1 calls)
............5 -- b3ParallelLinearBvh::build() (3.37 %) :: 3.059 ms / frame (1 calls)
............6 -- raycast copyFromHost (8.83 %) :: 8.014 ms / frame (1 calls)
............Unaccounted: (0.217 %) :: 0.197 ms
...............----------------------------------
...............Profiling: detect ray-rigid pair index ranges (total running time: 15.552 ms) ---
...............0 -- reset ray-rigid pair index ranges (49.18 %) :: 7.649 ms / frame (1 calls)
...............Unaccounted: (50.817 %) :: 7.903 ms
...............----------------------------------
...............Profiling: PLBVH testRaysAgainstBvhAabbs() (total running time: 52.507 ms) ---
...............0 -- PLBVH ray test large AABB (9.62 %) :: 5.053 ms / frame (1 calls)
...............1 -- PLBVH ray test small AABB (89.46 %) :: 46.975 ms / frame (1 calls)
...............Unaccounted: (0.912 %) :: 0.479 ms
...............----------------------------------
...............Profiling: b3ParallelLinearBvh::build() (total running time: 3.059 ms) ---
...............0 -- m_findLeafIndexRangesKernel (4.09 %) :: 0.125 ms / frame (1 calls)
...............1 -- b3GpuParallelLinearBvh::constructBinaryRadixTree() (36.81 %) :: 1.126 ms / frame (1 calls)
...............2 -- Sort leaves by morton codes (24.75 %) :: 0.757 ms / frame (1 calls)
...............3 -- Assign morton codes (3.89 %) :: 0.119 ms / frame (1 calls)
...............4 -- Find AABB of merged nodes (9.02 %) :: 0.276 ms / frame (1 calls)
...............5 -- Separate large and small AABBs (21.22 %) :: 0.649 ms / frame (1 calls)
...............Unaccounted: (0.229 %) :: 0.007 ms
..................----------------------------------
..................Profiling: b3GpuParallelLinearBvh::constructBinaryRadixTree() (total running time: 1.126 ms) ---
..................0 -- m_buildBinaryRadixTreeAabbsRecursiveKernel (46.80 %) :: 0.527 ms / frame (1 calls)
..................1 -- m_findDistanceFromRootKernel (12.70 %) :: 0.143 ms / frame (1 calls)
..................2 -- m_buildBinaryRadixTreeInternalNodesKernel (18.56 %) :: 0.209 ms / frame (1 calls)
..................3 -- m_buildBinaryRadixTreeLeafNodesKernel (10.66 %) :: 0.120 ms / frame (1 calls)
..................4 -- m_computeAdjacentPairCommonPrefixKernel (10.83 %) :: 0.122 ms / frame (1 calls)
..................Unaccounted: (0.444 %) :: 0.005 ms
.....................----------------------------------
.....................Profiling: m_buildBinaryRadixTreeAabbsRecursiveKernel (total running time: 0.527 ms) ---
.....................0 -- copy maxDistanceFromRoot to CPU (31.88 %) :: 0.168 ms / frame (1 calls)
.....................Unaccounted: (68.121 %) :: 0.359 ms

Most crashes due to memory-overflow should be solved in latest trunk.


Patch:
diff --git a/examples/pybullet/gym/pybullet_envs/bullet/cartpole_bullet.py b/examples/pybullet/gym/pybullet_envs/bullet/cartpole_bullet.py
index e28a9e962d..277e52ba2c 100644
--- a/examples/pybullet/gym/pybullet_envs/bullet/cartpole_bullet.py
+++ b/examples/pybullet/gym/pybullet_envs/bullet/cartpole_bullet.py
@@ -64,10 +64,10 @@ def _step(self, action):
 #    time.sleep(self.timeStep)
     self.state = p.getJointState(self.cartpole, 1)[0:2] + p.getJointState(self.cartpole, 0)[0:2]
     theta, theta_dot, x, x_dot = self.state
-    
+
     dv = 0.1
     deltav = [-10.*dv,-5.*dv, -2.*dv, -0.1*dv, 0, 0.1*dv, 2.*dv,5.*dv, 10.*dv][action]
-    
+
     p.setJointMotorControl2(self.cartpole, 0, p.VELOCITY_CONTROL, targetVelocity=(deltav + self.state[3]))
 
     done =  x < -self.x_threshold \
@@ -99,3 +99,8 @@ def _reset(self):
 
   def _render(self, mode='human', close=False):
       return
+
+  render = _render
+  reset = _reset
+  seed = _seed
+  step = _step
diff --git a/examples/pybullet/gym/pybullet_envs/bullet/kukaCamGymEnv.py b/examples/pybullet/gym/pybullet_envs/bullet/kukaCamGymEnv.py
index 4e7d7e70ec..8ed171be05 100644
--- a/examples/pybullet/gym/pybullet_envs/bullet/kukaCamGymEnv.py
+++ b/examples/pybullet/gym/pybullet_envs/bullet/kukaCamGymEnv.py
@@ -56,8 +56,8 @@ def __init__(self,
     observationDim = len(self.getExtendedObservation())
     #print("observationDim")
     #print(observationDim)
-    
-    observation_high = np.array([np.finfo(np.float32).max] * observationDim)    
+
+    observation_high = np.array([np.finfo(np.float32).max] * observationDim)
     if (self._isDiscrete):
       self.action_space = spaces.Discrete(7)
     else:
@@ -74,15 +74,15 @@ def _reset(self):
     p.setPhysicsEngineParameter(numSolverIterations=150)
     p.setTimeStep(self._timeStep)
     p.loadURDF(os.path.join(self._urdfRoot,"plane.urdf"),[0,0,-1])
-    
+
     p.loadURDF(os.path.join(self._urdfRoot,"table/table.urdf"), 0.5000000,0.00000,-.820000,0.000000,0.000000,0.0,1.0)
-    
+
     xpos = 0.5 +0.2*random.random()
     ypos = 0 +0.25*random.random()
     ang = 3.1415925438*random.random()
     orn = p.getQuaternionFromEuler([0,0,ang])
     self.blockUid =p.loadURDF(os.path.join(self._urdfRoot,"block.urdf"), xpos,ypos,-0.1,orn[0],orn[1],orn[2],orn[3])
-            
+
     p.setGravity(0,0,-10)
     self._kuka = kuka.Kuka(urdfRootPath=self._urdfRoot, timeStep=self._timeStep)
     self._envStepCounter = 0
@@ -98,7 +98,7 @@ def _seed(self, seed=None):
     return [seed]
 
   def getExtendedObservation(self):
-     
+
      #camEyePos = [0.03,0.236,0.54]
      #distance = 1.06
      #pitch=-56
@@ -118,13 +118,13 @@ def getExtendedObservation(self):
      viewMat = [-0.5120397806167603, 0.7171027660369873, -0.47284144163131714, 0.0, -0.8589617609977722, -0.42747554183006287, 0.28186774253845215, 0.0, 0.0, 0.5504802465438843, 0.8348482847213745, 0.0, 0.1925382763147354, -0.24935829639434814, -0.4401884973049164, 1.0]
      #projMatrix = camInfo[3]#[0.7499999403953552, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, -1.0000200271606445, -1.0, 0.0, 0.0, -0.02000020071864128, 0.0]
      projMatrix = [0.75, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, -1.0000200271606445, -1.0, 0.0, 0.0, -0.02000020071864128, 0.0]
-     
+
      img_arr = p.getCameraImage(width=self._width,height=self._height,viewMatrix=viewMat,projectionMatrix=projMatrix)
      rgb=img_arr[2]
      np_img_arr = np.reshape(rgb, (self._height, self._width, 4))
      self._observation = np_img_arr
      return self._observation
-  
+
   def _step(self, action):
     if (self._isDiscrete):
       dv = 0.01
@@ -142,7 +142,7 @@ def _step(self, action):
       realAction = [dx,dy,-0.002,da,f]
 
     return self.step2( realAction)
-     
+
   def step2(self, action):
     for i in range(self._actionRepeat):
       self._kuka.applyAction(action)
@@ -158,11 +158,11 @@ def step2(self, action):
 
     #print("self._envStepCounter")
     #print(self._envStepCounter)
-    
+
     done = self._termination()
     reward = self._reward()
     #print("len=%r" % len(self._observation))
-    
+
     return np.array(self._observation), reward, done, {}
 
   def _render(self, mode='human', close=False):
@@ -190,18 +190,18 @@ def _termination(self):
     #print (self._kuka.endEffectorPos[2])
     state = p.getLinkState(self._kuka.kukaUid,self._kuka.kukaEndEffectorIndex)
     actualEndEffectorPos = state[0]
-      
+
     #print("self._envStepCounter")
     #print(self._envStepCounter)
     if (self.terminated or self._envStepCounter>maxSteps):
       self._observation = self.getExtendedObservation()
       return True
-    maxDist = 0.005 
+    maxDist = 0.005
     closestPoints = p.getClosestPoints(self._kuka.trayUid, self._kuka.kukaUid,maxDist)
-     
+
     if (len(closestPoints)):#(actualEndEffectorPos[2] <= -0.43):
       self.terminated = 1
-      
+
       #print("closing gripper, attempting grasp")
       #start grasp and terminate
       fingerAngle = 0.3
@@ -212,7 +212,7 @@ def _termination(self):
         fingerAngle = fingerAngle-(0.3/100.)
         if (fingerAngle<0):
           fingerAngle=0
-    
+
       for i in range (1000):
         graspAction = [0,0,0.001,0,fingerAngle]
         self._kuka.applyAction(graspAction)
@@ -227,18 +227,18 @@ def _termination(self):
         if (actualEndEffectorPos[2]>0.5):
           break
 
-    
+
       self._observation = self.getExtendedObservation()
       return True
     return False
-    
+
   def _reward(self):
-    
+
     #rewards is height of target object
     blockPos,blockOrn=p.getBasePositionAndOrientation(self.blockUid)
-    closestPoints = p.getClosestPoints(self.blockUid,self._kuka.kukaUid,1000, -1, self._kuka.kukaEndEffectorIndex) 
+    closestPoints = p.getClosestPoints(self.blockUid,self._kuka.kukaUid,1000, -1, self._kuka.kukaEndEffectorIndex)
 
-    reward = -1000    
+    reward = -1000
     numPt = len(closestPoints)
     #print(numPt)
     if (numPt>0):
@@ -254,3 +254,7 @@ def _reward(self):
     #print(reward)
     return reward
 
+  render = _render
+  reset = _reset
+  seed = _seed
+  step = _step
diff --git a/examples/pybullet/gym/pybullet_envs/bullet/kukaGymEnv.py b/examples/pybullet/gym/pybullet_envs/bullet/kukaGymEnv.py
index b9ef16017b..6fa57e76ae 100644
--- a/examples/pybullet/gym/pybullet_envs/bullet/kukaGymEnv.py
+++ b/examples/pybullet/gym/pybullet_envs/bullet/kukaGymEnv.py
@@ -44,7 +44,7 @@ def __init__(self,
     self._maxSteps = maxSteps
     self.terminated = 0
     self._cam_dist = 1.3
-    self._cam_yaw = 180 
+    self._cam_yaw = 180
     self._cam_pitch = -40
 
     self._p = p
@@ -61,8 +61,8 @@ def __init__(self,
     observationDim = len(self.getExtendedObservation())
     #print("observationDim")
     #print(observationDim)
-    
-    observation_high = np.array([largeValObservation] * observationDim)    
+
+    observation_high = np.array([largeValObservation] * observationDim)
     if (self._isDiscrete):
       self.action_space = spaces.Discrete(7)
     else:
@@ -80,15 +80,15 @@ def _reset(self):
     p.setPhysicsEngineParameter(numSolverIterations=150)
     p.setTimeStep(self._timeStep)
     p.loadURDF(os.path.join(self._urdfRoot,"plane.urdf"),[0,0,-1])
-    
+
     p.loadURDF(os.path.join(self._urdfRoot,"table/table.urdf"), 0.5000000,0.00000,-.820000,0.000000,0.000000,0.0,1.0)
-    
+
     xpos = 0.55 +0.12*random.random()
     ypos = 0 +0.2*random.random()
     ang = 3.14*0.5+3.1415925438*random.random()
     orn = p.getQuaternionFromEuler([0,0,ang])
     self.blockUid =p.loadURDF(os.path.join(self._urdfRoot,"block.urdf"), xpos,ypos,-0.15,orn[0],orn[1],orn[2],orn[3])
-            
+
     p.setGravity(0,0,-10)
     self._kuka = kuka.Kuka(urdfRootPath=self._urdfRoot, timeStep=self._timeStep)
     self._envStepCounter = 0
@@ -115,7 +115,7 @@ def getExtendedObservation(self):
      dir0 = [gripperMat[0],gripperMat[3],gripperMat[6]]
      dir1 = [gripperMat[1],gripperMat[4],gripperMat[7]]
      dir2 = [gripperMat[2],gripperMat[5],gripperMat[8]]
-             
+
      gripperEul =  p.getEulerFromQuaternion(gripperOrn)
      #print("gripperEul")
      #print(gripperEul)
@@ -126,17 +126,17 @@ def getExtendedObservation(self):
      #print(projectedBlockPos2D)
      #print("blockEulerInGripper")
      #print(blockEulerInGripper)
-     
+
      #we return the relative x,y position and euler angle of block in gripper space
      blockInGripperPosXYEulZ =[blockPosInGripper[0],blockPosInGripper[1],blockEulerInGripper[2]]
-     
+
      #p.addUserDebugLine(gripperPos,[gripperPos[0]+dir0[0],gripperPos[1]+dir0[1],gripperPos[2]+dir0[2]],[1,0,0],lifeTime=1)
      #p.addUserDebugLine(gripperPos,[gripperPos[0]+dir1[0],gripperPos[1]+dir1[1],gripperPos[2]+dir1[2]],[0,1,0],lifeTime=1)
      #p.addUserDebugLine(gripperPos,[gripperPos[0]+dir2[0],gripperPos[1]+dir2[1],gripperPos[2]+dir2[2]],[0,0,1],lifeTime=1)
-     
+
      self._observation.extend(list(blockInGripperPosXYEulZ))
      return self._observation
-  
+
   def _step(self, action):
     if (self._isDiscrete):
       dv = 0.005
@@ -154,7 +154,7 @@ def _step(self, action):
       f = 0.3
       realAction = [dx,dy,-0.002,da,f]
     return self.step2( realAction)
-     
+
   def step2(self, action):
     for i in range(self._actionRepeat):
       self._kuka.applyAction(action)
@@ -168,7 +168,7 @@ def step2(self, action):
 
     #print("self._envStepCounter")
     #print(self._envStepCounter)
-    
+
     done = self._termination()
     npaction = np.array([action[3]]) #only penalize rotation until learning works well [action[0],action[1],action[3]])
     actionCost = np.linalg.norm(npaction)*10.
@@ -177,9 +177,9 @@ def step2(self, action):
     reward = self._reward()-actionCost
     #print("reward")
     #print(reward)
- 
+
     #print("len=%r" % len(self._observation))
-    
+
     return np.array(self._observation), reward, done, {}
 
   def _render(self, mode="rgb_array", close=False):
@@ -208,18 +208,18 @@ def _termination(self):
     #print (self._kuka.endEffectorPos[2])
     state = p.getLinkState(self._kuka.kukaUid,self._kuka.kukaEndEffectorIndex)
     actualEndEffectorPos = state[0]
-      
+
     #print("self._envStepCounter")
     #print(self._envStepCounter)
     if (self.terminated or self._envStepCounter>self._maxSteps):
       self._observation = self.getExtendedObservation()
       return True
-    maxDist = 0.005 
+    maxDist = 0.005
     closestPoints = p.getClosestPoints(self._kuka.trayUid, self._kuka.kukaUid,maxDist)
-     
+
     if (len(closestPoints)):#(actualEndEffectorPos[2] <= -0.43):
       self.terminated = 1
-      
+
       #print("terminating, closing gripper, attempting grasp")
       #start grasp and terminate
       fingerAngle = 0.3
@@ -230,7 +230,7 @@ def _termination(self):
         fingerAngle = fingerAngle-(0.3/100.)
         if (fingerAngle<0):
           fingerAngle=0
-    
+
       for i in range (1000):
         graspAction = [0,0,0.001,0,fingerAngle]
         self._kuka.applyAction(graspAction)
@@ -245,19 +245,19 @@ def _termination(self):
         if (actualEndEffectorPos[2]>0.5):
           break
 
-    
+
       self._observation = self.getExtendedObservation()
       return True
     return False
-    
+
   def _reward(self):
-    
+
     #rewards is height of target object
     blockPos,blockOrn=p.getBasePositionAndOrientation(self.blockUid)
-    closestPoints = p.getClosestPoints(self.blockUid,self._kuka.kukaUid,1000, -1, self._kuka.kukaEndEffectorIndex) 
+    closestPoints = p.getClosestPoints(self.blockUid,self._kuka.kukaUid,1000, -1, self._kuka.kukaEndEffectorIndex)
 
     reward = -1000
-    
+
     numPt = len(closestPoints)
     #print(numPt)
     if (numPt>0):
@@ -276,10 +276,7 @@ def _reward(self):
     #print(reward)
     return reward
 
-    def reset(self):
-        """Resets the state of the environment and returns an initial observation.
-
-        Returns: observation (object): the initial observation of the
-            space.
-        """
-        return self._reset()
\ No newline at end of file
+  render = _render
+  reset = _reset
+  seed = _seed
+  step = _step
diff --git a/examples/pybullet/gym/pybullet_envs/bullet/minitaur_duck_gym_env.py b/examples/pybullet/gym/pybullet_envs/bullet/minitaur_duck_gym_env.py
index 44b41dd7dd..b5c3375b47 100644
--- a/examples/pybullet/gym/pybullet_envs/bullet/minitaur_duck_gym_env.py
+++ b/examples/pybullet/gym/pybullet_envs/bullet/minitaur_duck_gym_env.py
@@ -385,3 +385,8 @@ def _noisy_observation(self):
           scale=self._observation_noise_stdev, size=observation.shape) *
                       self.minitaur.GetObservationUpperBound())
     return observation
+
+  render = _render
+  reset = _reset
+  seed = _seed
+  step = _step
diff --git a/examples/pybullet/gym/pybullet_envs/bullet/minitaur_gym_env.py b/examples/pybullet/gym/pybullet_envs/bullet/minitaur_gym_env.py
index 38ec631a5f..1581280677 100644
--- a/examples/pybullet/gym/pybullet_envs/bullet/minitaur_gym_env.py
+++ b/examples/pybullet/gym/pybullet_envs/bullet/minitaur_gym_env.py
@@ -376,3 +376,8 @@ def _noisy_observation(self):
           scale=self._observation_noise_stdev, size=observation.shape) *
                       self.minitaur.GetObservationUpperBound())
     return observation
+
+  render = _render
+  reset = _reset
+  seed = _seed
+  step = _step
diff --git a/examples/pybullet/gym/pybullet_envs/bullet/racecarGymEnv.py b/examples/pybullet/gym/pybullet_envs/bullet/racecarGymEnv.py
index 7537d54c12..bdde5d963d 100644
--- a/examples/pybullet/gym/pybullet_envs/bullet/racecarGymEnv.py
+++ b/examples/pybullet/gym/pybullet_envs/bullet/racecarGymEnv.py
@@ -50,8 +50,8 @@ def __init__(self,
     #self.reset()
     observationDim = 2 #len(self.getExtendedObservation())
     #print("observationDim")
-    #print(observationDim) 
-    # observation_high = np.array([np.finfo(np.float32).max] * observationDim)    
+    #print(observationDim)
+    # observation_high = np.array([np.finfo(np.float32).max] * observationDim)
     observation_high = np.ones(observationDim) * 1000 #np.inf
     if (isDiscrete):
       self.action_space = spaces.Discrete(9)
@@ -59,7 +59,7 @@ def __init__(self,
        action_dim = 2
        self._action_bound = 1
        action_high = np.array([self._action_bound] * action_dim)
-       self.action_space = spaces.Box(-action_high, action_high) 
+       self.action_space = spaces.Box(-action_high, action_high)
     self.observation_space = spaces.Box(-observation_high, observation_high)
     self.viewer = None
 
@@ -74,14 +74,14 @@ def _reset(self):
     #	pos,orn = self._p.getBasePositionAndOrientation(i)
     #	newpos = [pos[0],pos[1],pos[2]-0.1]
     #	self._p.resetBasePositionAndOrientation(i,newpos,orn)
-    
+
     dist = 5 +2.*random.random()
     ang = 2.*3.1415925438*random.random()
-    
+
     ballx = dist * math.sin(ang)
     bally = dist * math.cos(ang)
     ballz = 1
-    
+
     self._ballUniqueId = self._p.loadURDF(os.path.join(self._urdfRoot,"sphere2.urdf"),[ballx,bally,ballz])
     self._p.setGravity(0,0,-10)
     self._racecar = racecar.Racecar(self._p,urdfRootPath=self._urdfRoot, timeStep=self._timeStep)
@@ -101,18 +101,18 @@ def _seed(self, seed=None):
   def getExtendedObservation(self):
      self._observation = [] #self._racecar.getObservation()
      carpos,carorn = self._p.getBasePositionAndOrientation(self._racecar.racecarUniqueId)
-     ballpos,ballorn = self._p.getBasePositionAndOrientation(self._ballUniqueId)      
+     ballpos,ballorn = self._p.getBasePositionAndOrientation(self._ballUniqueId)
      invCarPos,invCarOrn = self._p.invertTransform(carpos,carorn)
      ballPosInCar,ballOrnInCar = self._p.multiplyTransforms(invCarPos,invCarOrn,ballpos,ballorn)
-    
+
      self._observation.extend([ballPosInCar[0],ballPosInCar[1]])
      return self._observation
-     
+
   def _step(self, action):
     if (self._renders):
       basePos,orn = self._p.getBasePositionAndOrientation(self._racecar.racecarUniqueId)
       #self._p.resetDebugVisualizerCamera(1, 30, -40, basePos)
-    
+
     if (self._isDiscrete):
 	    fwd = [-1,-1,-1,0,0,0,1,1,1]
 	    steerings = [-0.6,0,0.6,-0.6,0,0.6,-0.6,0,0.6]
@@ -128,14 +128,14 @@ def _step(self, action):
       if self._renders:
         time.sleep(self._timeStep)
       self._observation = self.getExtendedObservation()
-      
+
       if self._termination():
         break
       self._envStepCounter += 1
     reward = self._reward()
     done = self._termination()
     #print("len=%r" % len(self._observation))
-    
+
     return np.array(self._observation), reward, done, {}
 
   def _render(self, mode='human', close=False):
@@ -159,13 +159,13 @@ def _render(self, mode='human', close=False):
     rgb_array = rgb_array[:, :, :3]
     return rgb_array
 
-    
+
   def _termination(self):
     return self._envStepCounter>1000
-    
+
   def _reward(self):
-    closestPoints = self._p.getClosestPoints(self._racecar.racecarUniqueId,self._ballUniqueId,10000) 
-    
+    closestPoints = self._p.getClosestPoints(self._racecar.racecarUniqueId,self._ballUniqueId,10000)
+
     numPt = len(closestPoints)
     reward=-1000
     #print(numPt)
@@ -174,3 +174,8 @@ def _reward(self):
       reward = -closestPoints[0][8]
       #print(reward)
     return reward
+
+  render = _render
+  reset = _reset
+  seed = _seed
+  step = _step
diff --git a/examples/pybullet/gym/pybullet_envs/bullet/racecarZEDGymEnv.py b/examples/pybullet/gym/pybullet_envs/bullet/racecarZEDGymEnv.py
index d9546de8c6..277453e14b 100644
--- a/examples/pybullet/gym/pybullet_envs/bullet/racecarZEDGymEnv.py
+++ b/examples/pybullet/gym/pybullet_envs/bullet/racecarZEDGymEnv.py
@@ -40,7 +40,7 @@ def __init__(self,
     self._renders = renders
     self._width = 100
     self._height = 10
-    
+
     self._isDiscrete = isDiscrete
     if self._renders:
       self._p = bullet_client.BulletClient(
@@ -53,8 +53,8 @@ def __init__(self,
     observationDim = len(self.getExtendedObservation())
     #print("observationDim")
     #print(observationDim)
-    
-    observation_high = np.array([np.finfo(np.float32).max] * observationDim)    
+
+    observation_high = np.array([np.finfo(np.float32).max] * observationDim)
     if (isDiscrete):
       self.action_space = spaces.Discrete(9)
     else:
@@ -77,14 +77,14 @@ def _reset(self):
       pos,orn = self._p.getBasePositionAndOrientation(i)
       newpos = [pos[0],pos[1],pos[2]+0.1]
       self._p.resetBasePositionAndOrientation(i,newpos,orn)
-    
+
     dist = 5 +2.*random.random()
     ang = 2.*3.1415925438*random.random()
-    
+
     ballx = dist * math.sin(ang)
     bally = dist * math.cos(ang)
     ballz = 1
-    
+
     self._ballUniqueId = self._p.loadURDF(os.path.join(self._urdfRoot,"sphere2red.urdf"),[ballx,bally,ballz])
     self._p.setGravity(0,0,-10)
     self._racecar = racecar.Racecar(self._p,urdfRootPath=self._urdfRoot, timeStep=self._timeStep)
@@ -104,13 +104,13 @@ def _seed(self, seed=None):
   def getExtendedObservation(self):
      carpos,carorn = self._p.getBasePositionAndOrientation(self._racecar.racecarUniqueId)
      carmat = self._p.getMatrixFromQuaternion(carorn)
-     ballpos,ballorn = self._p.getBasePositionAndOrientation(self._ballUniqueId)      
+     ballpos,ballorn = self._p.getBasePositionAndOrientation(self._ballUniqueId)
      invCarPos,invCarOrn = self._p.invertTransform(carpos,carorn)
      ballPosInCar,ballOrnInCar = self._p.multiplyTransforms(invCarPos,invCarOrn,ballpos,ballorn)
      dist0 = 0.3
      dist1 = 1.
      eyePos = [carpos[0]+dist0*carmat[0],carpos[1]+dist0*carmat[3],carpos[2]+dist0*carmat[6]+0.3]
-     targetPos = [carpos[0]+dist1*carmat[0],carpos[1]+dist1*carmat[3],carpos[2]+dist1*carmat[6]+0.3]      
+     targetPos = [carpos[0]+dist1*carmat[0],carpos[1]+dist1*carmat[3],carpos[2]+dist1*carmat[6]+0.3]
      up = [carmat[2],carmat[5],carmat[8]]
      viewMat = self._p.computeViewMatrix(eyePos,targetPos,up)
      #viewMat = self._p.computeViewMatrixFromYawPitchRoll(carpos,1,0,0,0,2)
@@ -122,12 +122,12 @@ def getExtendedObservation(self):
      np_img_arr = np.reshape(rgb, (self._height, self._width, 4))
      self._observation = np_img_arr
      return self._observation
-     
+
   def _step(self, action):
     if (self._renders):
       basePos,orn = self._p.getBasePositionAndOrientation(self._racecar.racecarUniqueId)
       #self._p.resetDebugVisualizerCamera(1, 30, -40, basePos)
-    
+
     if (self._isDiscrete):
             fwd = [-1,-1,-1,0,0,0,1,1,1]
             steerings = [-0.6,0,0.6,-0.6,0,0.6,-0.6,0,0.6]
@@ -143,14 +143,14 @@ def _step(self, action):
       if self._renders:
         time.sleep(self._timeStep)
       self._observation = self.getExtendedObservation()
-      
+
       if self._termination():
         break
       self._envStepCounter += 1
     reward = self._reward()
     done = self._termination()
     #print("len=%r" % len(self._observation))
-    
+
     return np.array(self._observation), reward, done, {}
 
   def _render(self, mode='human', close=False):
@@ -177,10 +177,10 @@ def _render(self, mode='human', close=False):
 
   def _termination(self):
     return self._envStepCounter>1000
-    
+
   def _reward(self):
-    closestPoints = self._p.getClosestPoints(self._racecar.racecarUniqueId,self._ballUniqueId,10000) 
-    
+    closestPoints = self._p.getClosestPoints(self._racecar.racecarUniqueId,self._ballUniqueId,10000)
+
     numPt = len(closestPoints)
     reward=-1000
     #print(numPt)
@@ -189,3 +189,8 @@ def _reward(self):
       reward = -closestPoints[0][8]
       #print(reward)
     return reward
+
+  render = _render
+  reset = _reset
+  seed = _seed
+  step = _step
diff --git a/examples/pybullet/gym/pybullet_envs/bullet/simpleHumanoidGymEnv.py b/examples/pybullet/gym/pybullet_envs/bullet/simpleHumanoidGymEnv.py
index c632e7f318..53b19b7a8a 100644
--- a/examples/pybullet/gym/pybullet_envs/bullet/simpleHumanoidGymEnv.py
+++ b/examples/pybullet/gym/pybullet_envs/bullet/simpleHumanoidGymEnv.py
@@ -46,8 +46,8 @@ def __init__(self,
     observationDim = len(self.getExtendedObservation())
     #print("observationDim")
     #print(observationDim)
-    
-    observation_high = np.array([np.finfo(np.float32).max] * observationDim)    
+
+    observation_high = np.array([np.finfo(np.float32).max] * observationDim)
     self.action_space = spaces.Discrete(9)
     self.observation_space = spaces.Box(-observation_high, observation_high)
     self.viewer = None
@@ -57,14 +57,14 @@ def _reset(self):
     #p.setPhysicsEngineParameter(numSolverIterations=300)
     p.setTimeStep(self._timeStep)
     p.loadURDF(os.path.join(self._urdfRoot,"plane.urdf"))
-    
+
     dist = 5 +2.*random.random()
     ang = 2.*3.1415925438*random.random()
-    
+
     ballx = dist * math.sin(ang)
     bally = dist * math.cos(ang)
     ballz = 1
-        
+
     p.setGravity(0,0,-10)
     self._humanoid = simpleHumanoid.SimpleHumanoid(urdfRootPath=self._urdfRoot, timeStep=self._timeStep)
     self._envStepCounter = 0
@@ -82,7 +82,7 @@ def _seed(self, seed=None):
   def getExtendedObservation(self):
      self._observation = self._humanoid.getObservation()
      return self._observation
-     
+
   def _step(self, action):
     self._humanoid.applyAction(action)
     for i in range(self._actionRepeat):
@@ -96,7 +96,7 @@ def _step(self, action):
     reward = self._reward()
     done = self._termination()
     #print("len=%r" % len(self._observation))
-    
+
     return np.array(self._observation), reward, done, {}
 
   def _render(self, mode='human', close=False):
@@ -104,8 +104,13 @@ def _render(self, mode='human', close=False):
 
   def _termination(self):
     return self._envStepCounter>1000
-    
+
   def _reward(self):
     reward=self._humanoid.distance
     print(reward)
     return reward
+
+  render = _render
+  reset = _reset
+  seed = _seed
+  step = _step
diff --git a/examples/pybullet/gym/pybullet_envs/env_bases.py b/examples/pybullet/gym/pybullet_envs/env_bases.py
index 5d49c22c3b..ceafb64048 100644
--- a/examples/pybullet/gym/pybullet_envs/env_bases.py
+++ b/examples/pybullet/gym/pybullet_envs/env_bases.py
@@ -43,7 +43,7 @@ def _reset(self):
 			conInfo = p.getConnectionInfo()
 			if (conInfo['isConnected']):
 				self.ownsPhysicsClient = False
-				
+
 				self.physicsClientId = 0
 			else:
 				self.ownsPhysicsClient = True
@@ -75,12 +75,12 @@ def _render(self, mode, close):
 			self.isRender = True
 		if mode != "rgb_array":
 			return np.array([])
-		
+
 		base_pos=[0,0,0]
 		if (hasattr(self,'robot')):
 			if (hasattr(self.robot,'body_xyz')):
 				base_pos = self.robot.body_xyz
-		
+
 		view_matrix = p.computeViewMatrixFromYawPitchRoll(
 			cameraTargetPosition=base_pos,
 			distance=self._cam_dist,
@@ -100,6 +100,7 @@ def _render(self, mode, close):
 		rgb_array = rgb_array[:, :, :3]
 		return rgb_array
 
+
 	def _close(self):
 		if (self.ownsPhysicsClient):
 			if (self.physicsClientId>=0):
@@ -109,6 +110,17 @@ def _close(self):
 	def HUD(self, state, a, done):
 		pass
 
+	# backwards compatibility for gym >= v0.9.x
+	# for extension of this class.
+	def step(self, *args, **kwargs):
+		return self._step(*args, **kwargs)
+
+	close = _close
+	render = _render
+	reset = _reset
+	seed = _seed
+
+
 class Camera:
 	def __init__(self):
 		pass