Repository: stepjam/RLBench
Issue: Environment hangs
Root cause: Environment Incompatibility

Thank you for taking the time to create and maintain RLBench library!

I noticed that the environment hangs for me. I first tried the [RL example from README](https://github.com/stepjam/RLBench#reinforcement-learning) in the python prompt. (pasting here for reference):

```
from rlbench.environment import Environment
from rlbench.action_modes import ActionMode, ArmActionMode
from rlbench.tasks import ReachTarget
import numpy as np

action_mode = ActionMode(ArmActionMode.ABS_JOINT_VELOCITY)
env = Environment(action_mode)
env.launch()

task = env.get_task(ReachTarget)
descriptions, obs = task.reset()
obs, reward, terminate = task.step(np.random.normal(size=env.action_size))
```
The prompt hangs on `descriptions, obs = task.reset()`. 

Then I tried running the [full single task RL example](https://github.com/stepjam/RLBench/blob/master/examples/single_task_rl.py) but that also hangs for me.

Output:
```
(rlbench) ➜  examples git:(master) python3 single_task_rl.py

Reset Episode
['reach the red target', 'touch the red ball with the panda gripper', 'reach the red sphere']
[ 0.17079188 -0.0264201  -0.10311306  0.03359457  0.03345952 -0.12136246
  0.06670413  1.        ]
[ 0.14312055  0.02824538 -0.02325131  0.07150904 -0.10360528 -0.00772027
 -0.044692    1.        ]
[-0.10803643 -0.00968025 -0.02897226  0.02368353  0.16428308 -0.02507495
  0.14132662  1.        ]
[ 0.10310517  0.03861239 -0.10995498  0.02547699 -0.2368444  -0.05069968
 -0.13309511  1.        ]
[-0.12409099 -0.05137944 -0.03368804 -0.07796257 -0.07599648  0.15202908
  0.07392018  1.        ]
[-0.04220587  0.08722054 -0.12057854 -0.06980598 -0.00843292  0.04940882
  0.02971151  1.        ]
[ 0.13746512  0.0481053  -0.04097826  0.07410873 -0.00468852  0.0017098
 -0.04300822  1.        ]
[-0.01620883 -0.01427259 -0.07329133  0.11900221  0.14108984  0.08420374
  0.09554812  1.        ]
[ 0.15239775 -0.0193449   0.08431747 -0.09703287  0.15586955 -0.11501804
 -0.03423841  1.        ]
[-0.02746633 -0.11027182  0.0221979  -0.05900184  0.02954036 -0.00927814
  0.05863471  1.        ]
[-0.0525069  -0.08002597 -0.01114554  0.01513193 -0.0540802   0.08797246
  0.07335001  1.        ]
[ 0.15051924 -0.05339467 -0.04653889 -0.19188749  0.02017341  0.06923941
  0.07673441  1.        ]
[-0.11251707 -0.03376517 -0.10000211 -0.08255756 -0.08937531 -0.05511788
 -0.24687996  1.        ]
[-0.0951936  -0.00999391 -0.05834677 -0.13333046  0.09394424  0.02070207
 -0.06323385  1.        ]
[-0.00927133 -0.10129165  0.06955679 -0.19159371 -0.00978443 -0.0849275
 -0.00752591  1.        ]
```

I went through some existing issues and incorporated the suggestions like [this](https://github.com/stepjam/RLBench/issues/89#issuecomment-691140680) but have not figured out why is the environment getting stuck

Hi. Hmm OK, I'll need a little more info.
Since you linked issue #89, I guess you are trying to run this on a server without a display attached?

Yes that is correct :)

Let's check the output from PyRep.
Could you add the following flag (`verbosity=Verbosity.LOAD_INFOS` ; you'll need to import: `from pyrep.const import Verbosity`) to this line:
https://github.com/stepjam/RLBench/blob/master/rlbench/environment.py#L135

You'll need to reinstall RLBench after adding this line.

Oops sorry @stepjam  but I missed your comment. I will get back to you with info in a couple of hours :)

Now I am getting a new error:
```
(rlbench) ➜  examples git:(master) ✗ Xvfb :99 -screen 0 1024x768x24 +extension GLX +render -noreset &

[1] 686809
(rlbench) ➜  examples git:(master) ✗ export DISPLAY=:99

(rlbench) ➜  examples git:(master) ✗ python3 single_task_rl.py
Traceback (most recent call last):
  File "single_task_rl.py", line 25, in <module>
    env.launch()
  File "/private/home/sodhani/.conda/envs/rlbench/lib/python3.8/site-packages/rlbench/environment.py", line 136, in launch
    self._pyrep.launch(join(DIR_PATH, TTT_FILE), headless=self._headlessi, verbosity=Verbosity.LOAD_INFOS)
AttributeError: 'Environment' object has no attribute '_headlessi'
```
Added the lines as follows:

```
 def launch(self):
132         if self._pyrep is not None:
133             raise RuntimeError('Already called launch!')
134         self._pyrep = PyRep()
135         from pyrep.const import Verbosity
136         self._pyrep.launch(join(DIR_PATH, TTT_FILE), headless=self._headlessi, verbosity=Verbosity.LOAD_INFOS)
137
138         arm_class, gripper_class, _ = SUPPORTED_ROBOTS[
139             self._robot_configuration]
```

I think you have a typo there. Should be `self._headless` and not `self._headlessi`. 

oops you are right - :) Thanks for catching it

Added the output:

<details><summary>Output</summary>
<p>


(rlbench) ➜  examples git:(master) ✗ python3 single_task_rl.py
[CoppeliaSim:loadinfo]   CoppeliaSim V4.1.0., (rev. 1), flavor: 0
[CoppeliaSim:loadinfo]   using the default Lua library.
[CoppeliaSim:error]   could not find or correctly load the video compression library.
        Try following:

        >sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev
        if above fails, try first:
        >sudo apt-get -f install

[CoppeliaSim:loadinfo]   if CoppeliaSim crashes now, try to install libgl1-mesa-dev on your system:
        >sudo apt install libgl1-mesa-dev
[CoppeliaSim:loadinfo]   OpenGL Mesa/X.org, Renderer: llvmpipe (LLVM 11.0.0, 256 bits), Version: 3.1 Mesa 20.2.6
[CoppeliaSim:loadinfo]   simulator launched.
[CoppeliaSim:loadinfo]   plugin 'DynamicsBullet-2-78': loading...
[CoppeliaSim:loadinfo]   plugin 'DynamicsBullet-2-78': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'DynamicsBullet-2-83': loading...
[CoppeliaSim:loadinfo]   plugin 'DynamicsBullet-2-83': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'DynamicsNewton': loading...
[CoppeliaSim:loadinfo]   plugin 'DynamicsNewton': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'DynamicsOde': loading...
[CoppeliaSim:loadinfo]   plugin 'DynamicsOde': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'DynamicsVortex': loading...
[CoppeliaSim:loadinfo]   plugin 'DynamicsVortex': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'Geometric': loading...
[CoppeliaSim:loadinfo]   plugin 'Geometric': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'Assimp': loading...
[CoppeliaSim:loadinfo]   plugin 'Assimp': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'BlueZero': loading...
[CoppeliaSim:loadinfo]   plugin 'BlueZero': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'Bwf': loading...
[CoppeliaSim:loadinfo]   plugin 'Bwf': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'CodeEditor': loading...
[CoppeliaSim:loadinfo]   plugin 'CodeEditor': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'ConvexDecompose': loading...
[CoppeliaSim:loadinfo]   plugin 'ConvexDecompose': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'CustomUI': loading...
[CoppeliaSim:loadinfo]   plugin 'CustomUI': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'GLTF': loading...
[CoppeliaSim:loadinfo]   plugin 'GLTF': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'IK': loading...
[CoppeliaSim:loadinfo]   plugin 'IK': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'LuaRemoteApiClient': loading...
[CoppeliaSim:loadinfo]   plugin 'LuaRemoteApiClient': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'OMPL': loading...
[CoppeliaSim:loadinfo]   plugin 'OMPL': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'OpenGL3Renderer': loading...
[CoppeliaSim:loadinfo]   plugin 'OpenGL3Renderer': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'OpenMesh': loading...
[CoppeliaSim:loadinfo]   plugin 'OpenMesh': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'Qhull': loading...
[CoppeliaSim:loadinfo]   plugin 'Qhull': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'ROS2Interface': loading...
[CoppeliaSim:error]   plugin 'ROS2Interface': load failed (could not load). The plugin probably couldn't load dependency libraries. For additional infos, modify the script 'libLoadErrorCheck.sh', run it and inspect the output.
[CoppeliaSim:loadinfo]   plugin 'ROSInterface': loading...
[CoppeliaSim:error]   plugin 'ROSInterface': load failed (could not load). The plugin probably couldn't load dependency libraries. For additional infos, modify the script 'libLoadErrorCheck.sh', run it and inspect the output.
[CoppeliaSim:loadinfo]   plugin 'RRS1': loading...
[CoppeliaSim:loadinfo]   plugin 'RRS1': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'ReflexxesTypeII': loading...
[CoppeliaSim:loadinfo]   plugin 'ReflexxesTypeII': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'RemoteApi': loading...
[simExtRemoteApi:loadinfo]   starting a remote API server on port 19997
[CoppeliaSim:loadinfo]   plugin 'RemoteApi': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'URLDrop': loading...
[CoppeliaSim:loadinfo]   plugin 'URLDrop': load succeeded.
[CoppeliaSim:loadinfo]   plugin 'Vision': loading...
[CoppeliaSim:loadinfo]   plugin 'Vision': load succeeded.
[CoppeliaSim:loadinfo]   using the 'Geometric' plugin.
[CoppeliaSim:loadinfo]   using the 'IK' plugin.
Reset Episode
['reach the red target', 'touch the red ball with the panda gripper', 'reach the red sphere']
[ 0.04948497 -0.04956619  0.06719672  0.04233291  0.15721883 -0.07056609
  0.06251304  1.        ]
[CoppeliaSim:loadinfo]   checking for an updated CoppeliaSim version...
[ 0.00570323 -0.13188903 -0.03201584 -0.05753577 -0.03109085 -0.09485188
 -0.01519833  1.        ]
[ 0.0100459  -0.10105997  0.20868957  0.09181551 -0.01496104 -0.01077643
 -0.10662628  1.        ]
[-0.06575316  0.06549395 -0.12467912  0.01852624 -0.06711564 -0.23925829
  0.05644647  1.        ]
[ 0.04371544 -0.01984365 -0.13203219  0.04824168  0.08031941 -0.07178624
  0.08128195  1.        ]
[CoppeliaSim:loadinfo]   This CoppeliaSim version is up-to-date.
[ 0.08617661 -0.09200831 -0.0964203   0.11651521 -0.07293697 -0.03211918
  0.10363882  1.        ]
[-0.07891045  0.00148423 -0.00681395  0.01753058 -0.08609647 -0.01686021
 -0.04780833  1.        ]
[0.16229348 0.05730909 0.02287597 0.16025766 0.19937704 0.06729786
 0.01968983 1.        ]
[ 0.05366387 -0.00306434  0.02370401 -0.07650011 -0.03929191  0.25164677
  0.03332103  1.        ]
[ 0.00497177  0.11833687  0.16885223  0.03729433 -0.06097733 -0.03427348
 -0.18000283  1.        ]
[ 0.18381203  0.09614005 -0.26790433 -0.01227695  0.05204521  0.0208433
  0.05394199  1.        ]
[-0.06133404  0.19781443  0.11327131  0.13771381 -0.18735617 -0.03288778
  0.00828835  1.        ]
[ 0.01627417 -0.08510317 -0.12907477 -0.03531879  0.01156986  0.07117835
 -0.13248277  1.        ]

</p>
</details>

Should I install the libraries it suggests?

You can try it, but I don't think this is related to your problem. And I don't think that your code hangs on `task.reset()`. It seems to run fine for some steps and then stops at some point. Seem to be related to `task.step()`. Unfortunately, I don't know what the problem is. Hopefully, Stephen can help. 

Thanks @robfiras  :) Lets wait for @stepjam :)

Hi. No the libraries should not be related. 
How are you launching the program?

If you are not already, can you try launching like this:

```
Xvfb :99 -screen 0 1024x768x24 +extension GLX +render -noreset &
export DISPLAY=:99
python my_prog.py
```


The program is launched with screen and display variable set.

Thanks

Shagun
https://shagunsodhani.com

On Wed, Feb 3, 2021, 6:30 AM Stephen James <notifications@github.com> wrote:

> Hi. No the libraries should not be related.
> How are you launching the program?
>
> If you are not already, can you try launching like this:
>
> Xvfb :99 -screen 0 1024x768x24 +extension GLX +render -noreset &
> export DISPLAY=:99
> python my_prog.py
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/stepjam/RLBench/issues/104#issuecomment-772440183>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAKCR2KNCF6HOWNREHF5T7LS5EXVNANCNFSM4WNP53RA>
> .
>


Hmm... There's been a few updates to RLBench and PyRep, specifically how joint angles are reset. Can you make sure you are on the most recent versions, and we can go from there. Thanks!

Also, I imagine if you run the [pyrep rl example](https://github.com/stepjam/PyRep/blob/master/examples/example_reinforcement_learning_env.py), then you also get the same issue?

> Hmm... There's been a few updates to RLBench and PyRep, specifically how joint angles are reset. Can you make sure you are on the most recent versions, and we can go from there. Thanks!

I installed everything 13 days back. Happy to reinstall.

> Also, I imagine if you run the [pyrep rl example](https://github.com/stepjam/PyRep/blob/master/examples/example_reinforcement_learning_env.py), then you also get the same issue?

Trying it now.

Reinstall and ran the pyrep example. It is stuck as well

```
(rlbench) ➜  examples git:(master) python example_reinforcement_learning_env.py
Starting episode 0
Starting episode 1
```

OK; so this is a PyRep issue.
Actually, it seems like this has been an issue before: https://github.com/stepjam/PyRep/issues/222.
I'll close this, and reopen the PyRep Issue.