zhiminy/EvalEng huggingface dataset:
Repository: stepjam/RLBench
Issue: RLBenchEnv related issues
Root cause: Documentation Deficiency

1. Is there a way I can create multiple environment ?
Currently, If I try to make more than 1 environment, I got following error:
```
env = gym.make('reach_target-state-v0')
env2 = gym.make('reach_target-state-v0')
```


```
Attribute Qt::AA_UseDesktopOpenGL must be set before QCoreApplication is created.
WARNING: QApplication was not created in the main() thread.

(python3:49361): GLib-CRITICAL **: 17:55:14.764: g_main_context_push_thread_default: assertion 'acquired_context' failed
```
2. What is the proper way to change action mode when I am using RLBenchEnv?
Right now, I feel like there is no easy way to change action mode, I can also change the code by doing modification to the RLbench code, but I really want to avoid making change on the original code. Currently I use the following code to change action mode, but I wonder if it may cause some error since during the initialization stage, the environment is initialized under the default action mode, so some setting can be different, such as self._robot.arm.set_control_loop_enabled
```
env = gym.make('reach_target-state-v0')
env.env._action_mode = ArmActionMode.EE_POSE_PLAN_EE_FRAME 
```
Thanks! :)

Hi.
1) Each RLBench env needs to be created in a new process. See here: https://github.com/stepjam/PyRep#running-multiple-pyrep-instances

2) You select the action mode when defining the environment. See the examples, e.g. https://github.com/stepjam/RLBench/blob/master/examples/single_task_rl.py#L23

Thanks for getting back to me Stephen:)
Regarding question 2, I have seen the example, but like I have mentioned, I would like to use RLBenchEnv which inherits from gym instead of Environment, because we have been trying to use some existing open source algorithm such as stable baseline as our baseline. 
Shall I just make an **env** by using gym.make and create a new environment class with the desired action mode, and replace the original environment from the **env** with the new environment I created?
thanks:)


Sorry for delay. I assume this has been resolved. Feel free to reopen if needed!

> 

Hi, have you found a way to use gym-like env with specified action-mode? This issue has been marked as closed but I did not see your question been solved.

> Thanks for getting back to me Stephen:)
> Regarding question 2, I have seen the example, but like I have mentioned, I would like to use RLBenchEnv which inherits from gym instead of Environment, because we have been trying to use some existing open source algorithm such as stable baseline as our baseline.
> Shall I just make an **env** by using gym.make and create a new environment class with the desired action mode, and replace the original environment from the **env** with the new environment I created?
> thanks:)

We also encountered the same issue, how did you solve it?


Hi, Stephen, I tried to make two gym environments and saw your provided example, but it's a bit difficult for me to implement, could you show me more details about making two gym environments?
Repository: stepjam/RLBench
Issue: Best practice for overriding observation config.
Root cause: Documentation Deficiency

Hi, I'm working on a custom task that generates RGBD observation and state vector different from the default
vision and low-level state defined in the RLBench.

For example, I want to generate RGB observation with OPENGL3 renderer with a size of 64*64. Thus, I am trying
to implement my own setting of the ObservationConfig() in RLBench.

If I am to acquire such information for each step and reset of the environment, what can be the best practice to define
such custom configuration not altering the default setup?

Thanks in advance!


The ObservationConfig is meant to customise the observations. So if you want the OPENGL3 renderer with res 64x64, you can do:

```python
cam_config= CameraConfig(image_size=(64, 64), render_mode=RenderMode.OPENGL3)
obs_config = ObservationConfig(left_shoulder_camera=cam_config, front_camera=cam_config)
```

And then pass this to the env:

```python
action_mode = ActionMode(ArmActionMode.ABS_JOINT_VELOCITY)
env = Environment(action_mode, obs_config=obs_config)
```

Thank you for the prompt response! I'm impressed by the systematic design every time I use it.
Repository: stepjam/RLBench
Issue: [Task builder] Adding static objects under the task dummy hierarchy
Root cause: Documentation Deficiency

Hi, I'm working on the custom environment where there are respondable (collidiable, but static) walls around the table
of default RLBench table.

AFAIK, RLBench tries to spawn the task dummy inside the workspace with some rotation for each episode.
What could be the best practice if I don't want the walls to be spawned with the random rotation for each episode?

Thanks in advance!

Hi.
You can simply override [is_static_workspace](https://github.com/stepjam/RLBench/blob/master/rlbench/backend/task.py#L159) in your task file, and return True.

Thank you for your help!
Repository: stepjam/RLBench
Issue: Raise error "QObject::~QObject: Timers cannot be stopped from another thread"
Root cause: Resource Mishandling

I look this good framework for study.
When run single_task_rl.py with stackBlocks task it is stopped and raised this error.

![image](https://user-images.githubusercontent.com/37202668/67453245-11a81700-f661-11e9-8263-0a8cef049136.png)

How can I fix this?



Hi,
Sorry for the delay in getting back to you (I've been away).
Yes this is a know problem with PyRep, but this is not something that you need to worry about; it does not crash your program, but always prints out when the simulation is shutdown.
Best,
Stephen
Repository: stepjam/RLBench
Issue: Switching Between Action Modes
Root cause: Unimplemented Feature Gap

Is there a way to easily switch between action modes during a task? I would like to use part of the get_demos trajectory to move the robot towards the object in a scene, and then execute a policy. However, it seems like the get_demos code generates a sequence of waypoints in joint positions, so in order to use it I set the ActionMode to ABS_JOINT_POSITION and execute each point in _path_points,, but the policy I execute after should operate in  EE_POSE_EE_FRAME. Is there a way for me to switch between these action modes inside a task? Or is there a better way for me to execute part of the demo trajectory without setting my action mode to ABS_JOINT_POSITION?

Thanks!

No currently, there is not. But it is easy to add in. I'll put this on my todo list.

This will now be much easier in V1.2 when it releases; action modes are now classes, and you can now easily implement custom ones from scratch.
Repository: stepjam/RLBench
Issue: How to get camera intrinsics?
Root cause: Documentation Deficiency

How do I get the intrinsic matrix of the cameras for RLBench? Thanks!

Hi.
There will be an update to version 1.1.0 which allows this in a nice way. For now, you can simply do:
```python
left_cam_intr = my_task_env._scene._cam_over_shoulder_left.get_intrinsic_matrix()
right_cam_intr = my_task_env._scene._cam_over_shoulder_right.get_intrinsic_matrix()
...
```

You will need to swap to the 'dev' branch for PyRep; https://github.com/stepjam/PyRep/blob/dev/pyrep/objects/vision_sensor.py#L168

Thank you so much, this is exactly what I needed!
Repository: stepjam/RLBench
Issue: RLBench returning incorrectly shaped gripper_matrix and wrist_camera_matrix?
Root cause: External Dependency Breakage

I pulled the current version of RLBench from master, and now whenever I tried to get an observation from the environment I get the following error

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/eric/venvs/new_bench/lib/python3.7/site-packages/gym/envs/registration.py", line 145, in make
    return registry.make(id, **kwargs)
  File "/home/eric/venvs/new_bench/lib/python3.7/site-packages/gym/envs/registration.py", line 90, in make
    env = spec.make(**kwargs)
  File "/home/eric/venvs/new_bench/lib/python3.7/site-packages/gym/envs/registration.py", line 60, in make
    env = cls(**_kwargs)
  File "/home/eric/new_Github/RLBench/rlbench/gym/rlbench_env.py", line 38, in __init__
    _, obs = self.task.reset()
  File "/home/eric/new_Github/RLBench/rlbench/task_environment.py", line 91, in reset
    return desc, self._scene.get_observation()
  File "/home/eric/new_Github/RLBench/rlbench/backend/scene.py", line 261, in get_observation
    if self._obs_config.gripper_matrix else None),
  File "<__array_function__ internals>", line 6, in reshape
  File "/home/eric/venvs/new_bench/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 299, in reshape
    return _wrapfunc(a, 'reshape', newshape, order=order)
  File "/home/eric/venvs/new_bench/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 58, in _wrapfunc
    return bound(*args, **kwds)
ValueError: cannot reshape array of size 16 into shape (3,4)

I went into the scene.py file, and found that tip.get_matrix().shape was returning a [4,4] matrix, not a [3,4] as the one specified in the file. I changed it (And wrist_camera_matrix) to 4,4 instead and now everything appears to work. Is there something I did incorrectly or is this a bug?

Thanks for pointing out. There was a recent update to PyRep. Will fix shortly.
Repository: stepjam/RLBench
Issue: depth_in_meters not working in ObservationConfig?
Root cause: Validation Gap

I'm trying to extract the depth image from the cameras in meters. Using PyRep 4.1, I did the following:

front_cam_config = CameraConfig(depth_in_meters=True)
left_shoulder_cam_config = CameraConfig(depth_in_meters=True)
obs_config = ObservationConfig(left_shoulder_camera=left_shoulder_cam_config,front_camera=front_cam_config)

However, regardless if I set depth_in_meters = True or False, the depth images from the returned observation from step() still come out in a range between [0,1]. This looks like the scaled down depth image values to me and not in meters, so it looks like the flag depth_in_meters isn't being processed? Also, I tried grabbing the depth image directly from the vision sensor by doing

my_front_cam.capture_depth(in_meters=True)

and this gave me a depth image that had extremely large and small depth values (ranging from -1e+21 to 1e+21), which doesn't make sense to me as meter values. I tried grabbing the near and far clipping plane distances, which appear to be around ~0 meters and ~4.5 meters respectively, so either the clipping plane distances reported are incorrect or the depth values I'm getting out don't make sense with that? I also tried to transform the scaled-down depth image to meters using the near and far clipping plane distances to rescale the depth image, but the resulting point cloud looks warped, which makes me think the reported clipping planes may be incorrect?

Not sure how to resolve the issue above, but using capture_pointcloud() gives me what I need so I'm going to just use that!
Repository: stepjam/RLBench
Issue: Learning without vision sensor 
Root cause: Documentation Deficiency

I would like to develop a learning skill in my robot but the sensors of my robot doesn´t include vision sensor. So is it possible use the RLBench without vision sensor? If yes, could you share some example?

Sure. `task.step()` returns an [Observation](https://github.com/stepjam/RLBench/blob/master/rlbench/backend/observation.py) object. You can choose whatever information you like from it to train your algorithm.

If you want to disable the cameras completely to speed up the simulation, here is a minimal example showing how you turn off the cameras:
```python
from rlbench.environment import Environment
from rlbench.action_modes import ActionMode, ArmActionMode
from rlbench.observation_config import ObservationConfig, CameraConfig
from rlbench.tasks import ReachTarget
import numpy as np

action_mode = ActionMode(ArmActionMode.ABS_JOINT_VELOCITY)

# setup the camera configuration -> disable rgb, masks and depth maps
cam_config = CameraConfig()
cam_config.set_all(False)

# setup the observation configuration -> all cameras get hte same configuration (deactivation)
obs_config = ObservationConfig(front_camera=cam_config,
                               wrist_camera=cam_config,
                               left_shoulder_camera=cam_config,
                               right_shoulder_camera=cam_config)

# setup environment -> pass observation configuration
env = Environment(action_mode=action_mode, obs_config=obs_config)
env.launch()

# start task
task = env.get_task(ReachTarget)
descriptions, obs = task.reset()

# run an episode
for step in range(40):

    # let your fancy algorithm choose the action based on the observation
    action = np.random.normal(size=env.action_size)

    # obs will not contain any camera data
    obs, reward, terminate = task.step(action)

env.shutdown()
```
Take a look at [ObservationConfig and CameraConfig](https://github.com/stepjam/RLBench/blob/master/rlbench/observation_config.py) to see what you can enable and disable. 

Thanks for answering @robfiras
Repository: stepjam/RLBench
Issue: Gripper action should not run in loop
Root cause: Architectural Constraint

I think that it is misleading that the gripper action is running in a loop. From the perspective of the agent/learner, this leads to different effective step-sizes of the simulation depending on whether the gripper is used or not. 

My agents utilize the gripper action to have a greater step-size during reaching in any of the tasks; i.e., the are constantly closing and opening the gripper while approaching an object. 

I believe this has been resolved in newer versions :)
Repository: stepjam/RLBench
Issue: Recommended way to change environment parameters
Root cause: Documentation Deficiency

Hello,

thank you for sharing with the community this great project!
Since I am interested in evaluating the robustness of the learned policies against environmental perturbation I was wondering if there is a recommended way to modify the environment characteristics like the robot mass, the objects mass, friction etc.

Thank you in advance,
Luca

Sorry for delay. In the task file (e.g. rlbench/tasks/reach_target.py), you can randomize these at the beginning of each episode. E.g.

```
s = Shape(‘my_object’)
s.set_mass(0.1)
```

Feel free to reopen if you have more questions
Repository: stepjam/RLBench
Issue: What are gripper touch forces in observation?

I'm printing out the gripper touch forces on the observation state and I see it is a vector of 6 floats. Am I to interpret that as the 3-D contact force felt by each of the two fingers (e.g. first 3 values are force for one finger and the last 3 are forces for the other finger)? Or is it a 6-D wrench with 3 force values and 3 torque values?

Hi,
Sorry for the delay in getting back to you (I've been away).

The observation gives you the former: the applied forces along the sensor's x, y and z-axes for each of the grippers.

Best,
Stephen
Repository: stepjam/RLBench
Issue: 'QWidget: Must construct a QApplication before a QWidget'
Root cause: Environment Incompatibility

Hi,

First of all, thank you for opening your beautiful project to everyone. When I want to try out the sample scenarios in the examples folder: 'QWidget: Must construct a QApplication before a QWidget
Aborted (core dumped) 'error and it doesn't work. I'm running 'python single_task_rlbench.py' with this command. Have you had an error like this before? If so, do you know the solution?

Thanks.
Alper

Thank you!
Sorry for delay. I’ll assume that this is no longer an issue. Please feel free to reopen a new issue if it is still a problem.
Repository: stepjam/RLBench
Issue: get_scene_data methods in Environment
Root cause: Environment Incompatibility

Hi Stephen; 

Thanks for this wonderful Benchmark! 

I got a question while trying to retrieve scene data. 

It showed that Environment doesn't have the get_scene_data method
`['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_check_dataset_structure', '_set_arm_control_action', '_string_to_task', 'action_size', 'get_demos', 'get_task', 'launch', 'shutdown']`

However, I can see that defined in `rlbench/environment.py`

I am not sure why this is happening, can you help me with this?

Best,
ZiJia

sorry I have mistaken my RLBench and python3 version.
Repository: stepjam/RLBench
Issue: Panda URDF and world to robot coordinate frame transformation matrix 
Root cause: Documentation Deficiency

Hi,

Thank you for spending the time to develop such an extensive framework. 

I have been running into a few issues as I have wanted to use my own inverse kinematics solver to have more control. However, I am running into a few in this regard and was hoping you could help on either front:

1. Could your provide the original URDFs used to import the robots to CoppeliaSim? I am using the Panda, and while I can find other URDFs online, however they seem to differ slightly (e.g. the other URDFs have a "panda_hand" link which the one in CoppeliaSim does not seem to have).
2. Could you provide a transformation matrix that maps the global coordinate frame and the robots intrinsic coordinate frame?

Thank you in advance,

Stephane

Hi! This is actually more of a CoppeliaSim question than a RLBench one. I’d recommend asking your question here: https://forum.coppeliarobotics.com/

FYI I raised a PR to add the panda urdf
https://github.com/stepjam/RLBench/pull/201
Repository: stepjam/RLBench
Issue: Camera number and position in RLBench
Root cause: Documentation Deficiency

@stepjam Hi, thanks for your awesome work! I'm currently looking into designing a new task. But I found that in all tasks the number and position of cameras are fixed. Where can I modify these attributes?

Thanks.

Controlling the cameras positions is actually fairly straight forward. For example, the following will allow you to access the camera in your code:
```Python
from pyrep.objects.vision_sensor import VisionSensor
cam_over_shoulder_left = VisionSensor('cam_over_shoulder_left')
```
You can see this in [`scene.py`](https://github.com/stepjam/RLBench/blob/f214b6ecc177babce12d3ad13dd92049c824c1d9/rlbench/backend/scene.py#L41-L45).

To adjust the position I use the following:
```Python
import numpy as np

cams = [VisionSensor('cam_over_shoulder_right'),
        VisionSensor('cam_over_shoulder_left'),
        VisionSensor('cam_front') ]
cam_positions = [cam.get_position() for cam in cams]

for camera, position in zip(cams, cam_positions):
    new_position = list(position + np.random.normal(0, 0.05, len(position)))
    camera.set_position(new_position)
```
Specifically, I added something similar to this in [`domain_randomization_scene.py`](https://github.com/stepjam/RLBench/blob/master/rlbench/sim2real/domain_randomization_scene.py)

I'm a little less sure about adding new cameras. One approach could be adding them in the CoppeliaSim model itself and then accessing them the way I mentioned above.

I think that PyRep's [`vision_sensor.py`](https://github.com/stepjam/PyRep/blob/a420d722a8098e895705ebd923f9dd88f3b85736/pyrep/objects/vision_sensor.py#L17-L105) has a create method that can generate a new camera but I have no experience with it. This seems to be the approach taken by the [cinematic recorder](https://github.com/stepjam/RLBench/blob/f214b6ecc177babce12d3ad13dd92049c824c1d9/tools/cinematic_recorder.py#L124-L128).

A final idea, the [Object copy](https://github.com/stepjam/PyRep/blob/a420d722a8098e895705ebd923f9dd88f3b85736/pyrep/objects/object.py#L684-L692) method in PyRep may also be helpful.

Let me know if this helps or if you've found alternate way of doing this!
Repository: stepjam/RLBench
Issue: [Announcement] CoppeliaSim Version Expired
Root cause: External Dependency Breakage

Just an announcement that I am aware Version 4.1 of CoppeliaSim has expired. This should be resolved within the next few days. After the fix, you will need to update PyRep/CoppeliaSim: https://github.com/stepjam/PyRep#install 

In the meantime, see here for a quick fix:
https://github.com/stepjam/PyRep/issues/285#issuecomment-881340474

How are you able to get performance? Our experiments run very slow. It seems that it is the CSim part that is slow: sampling using RLPyt  (for 1000 'samples', up to half an hour headless, with all cameras (@128x128)), not our optimization step so much. I'm not passing much in terms of parameters to the RLBench env.

This is now resolved. Please go to PyRep, and simply rerun `pip install .` (after pulling from master)
**There is no need to redownload CoppeliaSim 4.1**
Repository: stepjam/RLBench
Issue: How are contacts modeled in RLBench?

I was wondering how contacts are modeled in RLBench? In order to grasp an object, does RLBench set a fixed kinematic constraint between the gripper and object to maintain contact, or are collision models between the gripper and object simulated when holding the object? Thanks!

Hi.
So this actually depends on the task. Usually, for rigid objects, ‘fake’ grasping is employed: this mean that if a graspable object* is detected within the gripper after a close command, the object will be attached the the arm (therefore, the object will not ‘slip’). However, for articulated objects (e.g. draws, door handles, etc), this does not happen otherwise this would break the articulated joint. If you want to disable fake grasping, you can set it here: https://github.com/stepjam/RLBench/blob/master/rlbench/task_environment.py#L41.

Please reopen if this does not answer your question. 

*graspable objects are marked in individual task files, e.g. https://github.com/stepjam/RLBench/blob/master/rlbench/tasks/pick_and_lift.py#L19
Repository: stepjam/RLBench
Issue: Simulator warning: At least one static respondable shape built on top of a dynamic tree structure
Root cause: External Dependency Breakage

Hi there,

I reinstalled the latest version of this repo, and when I am trying to collect live demos, Coppelia simulator always gives this warning: "It appears that there is at least one static respondable shape built on top of a dynamic tree structure. This is never recommended since it might lead to some strange and unrealistic behaviour." See attached screenshot below, for context, I am running the `RLBench/examples/single_task_rl_with_demos.py ` file directly off of repo. I used the repo several months ago, and this error is a new one. 

And I believe beacause of this, when I tried to collect demos for other tasks (such as CloseJar, BeatTheBuzz), it will give the following error and can't collect any demos.
`in _get_live_demos`
`   'Could not collect demos. Maybe a problem with the task?')`
`RuntimeError: Could not collect demos. Maybe a problem with the task?`
![Screenshot from 2021-10-06 06-18-58](https://user-images.githubusercontent.com/16480267/136210684-bde3ac40-7a6f-4b9c-af92-3eb99adfe2dc.png)

Any help/idea why this is the case and how to change the task will be much appreciated! Thanks!

This issue is due to my modifications, sorry for the confusion, closing the issue.
Repository: stepjam/RLBench
Issue: Question - Disable visualisation to speed up running simulator?
Root cause: Documentation Deficiency

Presumably I could just dig through the code a bit longer to figure this out, but I was wondering if there's a quick simple trick to disable the visualization of the robot for data collection? 

For the dataset collection, I assume you mean _tools/dataset_generator.py_?
If so, it should already be headless 🙂