Repository: stepjam/RLBench
Issue: Could not initialize GLX - in headless mode
Root cause: Environment Incompatibility

I am trying to run RLBench on a headless server. I set the `headless` flag as `True` while creating an `Environment`. But whenever I do `task.reset()`, the script crashes with the following error :

```shell
qt.glx: qglx_findConfig: Failed to finding matching FBConfig for QSurfaceFormat(version 3.2, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize 0, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize 0, stencilBufferSize 0, samples -1, swapBehavior QSurfaceFormat::SingleBuffer, swapInterval 1, colorSpace QSurfaceFormat::DefaultColorSpace, profile  QSurfaceFormat::CoreProfile)
qt.glx: qglx_findConfig: Failed to finding matching FBConfig for QSurfaceFormat(version 3.2, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize 0, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize 0, stencilBufferSize 0, samples -1, swapBehavior QSurfaceFormat::SingleBuffer, swapInterval 1, colorSpace QSurfaceFormat::DefaultColorSpace, profile  QSurfaceFormat::CoreProfile)
Could not initialize GLX
Aborted (core dumped)
```
I ran the examples in PyRep repo and that worked perfectly fine.
Can anyone help me with this? Thanks a lot!

I suppose this has something to do with the `VisionSensor` used in `get_observation()` in `scene.py`. I am guessing that I can't work with RGB(D) images if I am using a headless server then?

Did you follow the headless instruction in PyRep? https://github.com/stepjam/PyRep#running-headless

Yes, I did. I think the problem is coming up while calling [`sensor.handle_explicitly()` in `scene.py`](https://github.com/stepjam/RLBench/blob/20988254b773aae433146fff3624d8bcb9ed7330/rlbench/backend/scene.py#L186). 
Is it possible to use `VisionSensor` if using a headless server?

I was looking at your comment [here](https://github.com/stepjam/PyRep/pull/43#issuecomment-565983234), and I think you're calling `sensor.capture_rgb()` without calling `sensor.handle_explicitly()`. Is it okay to comment this out in `scene.py`? 
`sensor.capture_rgb()` wasn't raising any errors.

Theres no error raised with you because `sensor.capture_rgb()` doesn't actually render anything, it returns an already rendered image, which will be blank unless you call `sensor.handle_explicitly()`. There must be an issue with _how_ you are running the program.
Could you tell me exactly the command you are running when launching rlbench?


> Is it possible to use `VisionSensor` if using a headless server?

Yes, just to be clear, RLBench runs fine headlessly.


> Could you tell me exactly the command you are running when launching rlbench?

I was just trying this on the examples in the repo to fix this right now. I am running the program like this : 
```shell
xvfb-run -a python3 single_task_rl.py
```

OK. Perhaps try launching this way instead:
```
Xvfb :99 -screen 0 1024x768x24 +extension GLX +render -noreset &
export DISPLAY=:99
python3 single_task_rl.py
```

Thanks a lot that worked! Can you tell me what's happening in the first line?

I think `xvfb-run -a python3 single_task_rl.py` doesn't work because only the python interpreter gets access to the display, and not the CoppeliaSim process that spawns off from that. Whereas the other solution creates a permanent display (line 1) and then everything  from that point (after calling `export DISPLAY=:99`) uses this new display.

Ohh okay. That's interesting, didn't think of that. Thanks a lot! Amazing work with this :)

You can now do GPU rendering: https://github.com/stepjam/RLBench/blob/master/README.md#running-headless