Repository: stepjam/RLBench
Issue: Vision Sensor causes slow steps, and different speeds in general
Root cause: Configuration Error

When I create a Vision sensor in the environment, even if I don't invoke capture_rgb(), the task steps become dramatically slower. Using cProfile shows some of the calls taking .17s and the others taking 2-4 seconds, and this is occurring in the task.step() function (and further down the stack, in sim.py extstep.) In comparison, without, it takes about ~0.008 for some and ~0.155 for others. Could you explain the discrepancy in the speeds for the vision, as well as the discrepancy in the speeds between the calls? 

I tried to work around this, and in the scenario that I load in the Vision sensor in the task (in the ttm) itself, I see the same calls to the step (not the capture_rgb()) taking .02s & .6s. Is there rendering being done in the step? This is still too slow for our purposes. What is the proper way to add a vision sensor to speed up as fast as possible?

Debugging: Looks like the calls that take longer are due to calling extstep 20 times versus once.
I'm also seeing:
```
 pyrep.py:209: UserWarning: Could no
t change simulation timestep. You may need to change it to "custom dt" using simulation
 settings dialog. 
```
which may be related.

I just pushed something that will help with:

> When I create a Vision sensor in the environment, even if I don't invoke capture_rgb(), the task steps become dramatically slower.

This should no longer be the case. 

To get the fastest step time possible

- Only enable the cameras that you need using the `ObservationConfig`. 
- Using the OPENGL renderer over the OPENGL3 rendrer will be much faster. The former uses shadows which does increase rendering time. 
- If you must have shadows, then decreasing the number of lights will decrease rendering time. 

Thanks. Any ideas for the different timings for steps? 

Not sure what you mean; but on my desktop I get 0.07s per step with **all** observations enabled (rgb, depth, mask) for all 4 cameras (OPENGL3 renderer), using the joint velocity action mode.

pyrep.py:209: UserWarning: Could no
t change simulation timestep. You may need to change it to "custom dt" using simulation
 settings dialog. 

Maybe related to this? Basically it’s calling simextstep ~20 times on some of the steps.. 

Hi,
It's unrelated to that.
When you send a open/close command, this steps the simulator several times. So could be that.

Sent from my phone (please excuse any typos!)

________________________________
From: Bryan Chen <notifications@github.com>
Sent: Wednesday, April 1, 2020 9:43:00 PM
To: stepjam/RLBench <RLBench@noreply.github.com>
Cc: Stephen James <sjames_93@hotmail.co.uk>; Comment <comment@noreply.github.com>
Subject: Re: [stepjam/RLBench] Vision Sensor causes slow steps, and different speeds in general (#53)


pyrep.py:209: UserWarning: Could no
t change simulation timestep. You may need to change it to "custom dt" using simulation
settings dialog.

Maybe related to this? Basically it’s calling simextstep ~20 times on some of the steps..

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<https://github.com/stepjam/RLBench/issues/53#issuecomment-607479717>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABUSO6RMMJBQA6USWO2PIH3RKORNJANCNFSM4LYJBMZA>.


Ah, that was it! Thanks! (Assuming you meant the gripper open/close, after I fixed the action it stopped.)

I think perhaps the velocity should be changed when headless? I think that would make it faster for sims. 
`                done = self._robot.gripper.actuate(ee_action, velocity=0.04 => 1)
`