Repository: stepjam/RLBench
Issue: Incompatibility between qt version between conda and coppeliasim
Root cause: Environment Incompatibility

Hi,

I was trying to visualize the observation at running time using matplotlib. However, it seems matplotlib is incompatible with the QT library used in RLBench. So I'm wondering have you ever try to visualize images online? Are there any suggestions on the alternatives?

Here is the code to reproduce my problem, I just add matplotlib imshow functions in the imitation learning example.
```
from rlbench.environment import Environment
from rlbench.action_modes import ArmActionMode, ActionMode
from rlbench.observation_config import ObservationConfig
from rlbench.tasks import ReachTarget
import numpy as np

from matplotlib import pyplot as plt


class ImitationLearning(object):

    def predict_action(self, batch):
        return np.random.uniform(size=(len(batch), 7))

    def behaviour_cloning_loss(self, ground_truth_actions, predicted_actions):
        return 1


# To use 'saved' demos, set the path below, and set live_demos=False
live_demos = True
DATASET = '' if live_demos else 'PATH/TO/YOUR/DATASET'

obs_config = ObservationConfig()
obs_config.set_all(True)

action_mode = ActionMode(ArmActionMode.ABS_JOINT_VELOCITY)
env = Environment(
    action_mode, DATASET, obs_config, False)
env.launch()

task = env.get_task(ReachTarget)

il = ImitationLearning()

demos = task.get_demos(2, live_demos=live_demos)  # -> List[List[Observation]]
demos = np.array(demos).flatten()

# An example of using the demos to 'train' using behaviour cloning loss.
for i in range(100):
    print("'training' iteration %d" % i)
    batch = demos[0]  # np.random.choice(demos, replace=False)
    batch_images = [obs.left_shoulder_rgb for obs in demos[0]]
    # just add this new line
    plt.imshow(batch_images[0])
    plt.show()
    predicted_actions = il.predict_action(batch_images)
    ground_truth_actions = [obs.joint_velocities for obs in batch]
    loss = il.behaviour_cloning_loss(ground_truth_actions, predicted_actions)

print('Done')
env.shutdown()
```

The error is:
```
Cannot mix incompatible Qt library (version 0x50905) with this library (version 0x50c05)
[1]    74089 abort (core dumped)  python examples/imitation_learning.py
```

Are you using a conda environment? Seems conda environment use a different qt version than Coppelia.

@lianghongzhuo 
Yes, I use conda. Is this the problem between conda and Coppelia?

Yes, if you install packages that using Qt, conda will install another specific version of qt in your conda environment. If that qt version is different than the qt version in the Coppelia binary folder you will get an error like that.

I see. Thank you for your information. 
But in my case, there will be no such error if I don't use matplotlib. I'm still wondering why the error will be triggered only when using matploblib?

I can't reproduce your issue in a python3.7 conda environment. Maybe matplotlib is not the reason that cause your error.
Could you check your output of `conda list` if there is qt related packages installed?

The output is
```
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
absl-py                   0.10.0                   pypi_0    pypi
atari-py                  0.2.6                    pypi_0    pypi
backcall                  0.2.0                    pypi_0    pypi
box2d-py                  2.3.8                    pypi_0    pypi
ca-certificates           2020.10.14                    0  
certifi                   2020.11.8        py36h06a4308_0  
cffi                      1.14.3                   pypi_0    pypi
cloudpickle               1.6.0                    pypi_0    pypi
cudatoolkit               10.1.243             h6bb024c_0  
cycler                    0.10.0                   pypi_0    pypi
cython                    0.29.21                  pypi_0    pypi
decorator                 4.4.2                    pypi_0    pypi
detectron2                0.3+cu101                pypi_0    pypi
dill                      0.3.2                    pypi_0    pypi
future                    0.18.2                   pypi_0    pypi
fvcore                    0.1.2.post20201122          pypi_0    pypi
glfw                      2.0.0                    pypi_0    pypi
gym                       0.17.3                   pypi_0    pypi
h5py                      2.10.0                   pypi_0    pypi
hickle                    4.0.1                    pypi_0    pypi
html-testrunner           1.2.1                    pypi_0    pypi
imageio                   2.9.0                    pypi_0    pypi
ipython                   7.16.1                   pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
jedi                      0.17.2                   pypi_0    pypi
jinja2                    2.11.2                   pypi_0    pypi
kiwisolver                1.2.0                    pypi_0    pypi
ld_impl_linux-64          2.33.1               h53a641e_7  
libedit                   3.1.20191231         h14c3975_1  
libffi                    3.3                  he6710b0_2  
libgcc-ng                 9.1.0                hdf63c60_0  
libstdcxx-ng              9.1.0                hdf63c60_0  
markupsafe                1.1.1                    pypi_0    pypi
matplotlib                3.3.2                    pypi_0    pypi
mujoco-py                 1.50.1.0                 pypi_0    pypi
ncurses                   6.2                  he6710b0_1  
opencv-python             3.4.2.17                 pypi_0    pypi
openssl                   1.1.1h               h7b6447c_0  
parso                     0.7.1                    pypi_0    pypi
pexpect                   4.8.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    6.2.2                    pypi_0    pypi
pip                       20.2.4           py36h06a4308_0  
prompt-toolkit            3.0.8                    pypi_0    pypi
ptyprocess                0.6.0                    pypi_0    pypi
pycocotools               2.0.2                    pypi_0    pypi
pycparser                 2.20                     pypi_0    pypi
pyglet                    1.5.0                    pypi_0    pypi
pygments                  2.7.1                    pypi_0    pypi
pyparsing                 2.4.7                    pypi_0    pypi
pyquaternion              0.9.9                    pypi_0    pypi
python                    3.6.12               hcff3b4d_2  
python-dateutil           2.8.1                    pypi_0    pypi
pyyaml                    5.3.1                    pypi_0    pypi
readline                  8.0                  h7b6447c_0  
scipy                     1.5.2                    pypi_0    pypi
setuptools                50.3.1           py36h06a4308_1  
sqlite                    3.33.0               h62c20be_0  
tk                        8.6.10               hbc83047_0  
tqdm                      4.50.2                   pypi_0    pypi
traitlets                 4.3.3                    pypi_0    pypi
wcwidth                   0.2.5                    pypi_0    pypi
wheel                     0.35.1             pyhd3eb1b0_0  
xz                        5.2.5                h7b6447c_0  
yacs                      0.1.8                    pypi_0    pypi
zlib                      1.2.11               h7b6447c_3  
```

But happy to know it may not be the problem of matplotlib. I can set a new virtual environment and reintall.

in my experience, opencv-python is the one package that causes the problem.
I fixed it by using opencv headless package

Hi @lianghongzhuo 

Sorry for the late reply, this issue took me a little bit longer to solve.

I tried several things:
1. to remove OpenCV and other libraries possibly using QT.
2. update conda's library so that the qt version (in conda) matches Coppelia's.
3. totally remove conda's qt library (libqt*.so)
But none of the above work.

**After doing more search, I found a related issue: https://github.com/stepjam/PyRep/issues/76 where it's caused by the python's version. Then, I change the python version to 3.8.5 (from 3.6) and the problem is solved.** 

Now the program can be correctly executed even with OpenCV (the normal one, not headless).

Since I think there will be users having this problem again in the future, I'll change the topic to 'incompatibility between qt version between conda and coppeliasim) for easier search (feel free to change it back if inappropriate).


I tried your code on a new python3.6 environment. Still can't reproduce your error. (installed matplotlib pyrep opnecv-python and rlbench)
But an interesting finding is that since python3.6 is also my default python version in ubuntu18 system. the conda environment can also find my package installed in `$HOME/.local/lib/python3.6` folder

This is quite interesting... My system python is 3.7 and I also found such an issue when I try a conda with python 3.7.

I don't know the exact reason why my previous solution doesn't work, but I doubt it may be a mixed-use of conda / pip or some more complicated things. For now, I'll just switch to python 3.8 as a workaround.

I had exactly the same error. I created a fresh conda environment, installed PyTorch, installed PyRep, installed RLBench, and it now works fine - so conda itself isn't the problem. There must have been some incompatibility with the set of packages I use normally; I haven't tested, but trying to install matplotlib with conda also tries to install qt, so that might be the problem.