Repository: vispy/vispy

Issue:
Transparency issue resurfaced for PyQt5 on OSX
A colleague noticed that our PyQt5 application that uses VisPy is showing a transparency issue that we used to see with the old version of vispy's pyqt5 backend but only on OSX (and maybe Windows). The issue is whenever you have something semi-transparent in the canvas on OSX it shows *through* the window allowing you to see the windows behind. Here is a minimal example:

```
from vispy.app import use_app
from vispy.scene import SceneCanvas
from vispy.scene.visuals import Polygon
canvas = SceneCanvas()
v = canvas.central_widget.add_view()
v.bgcolor = 'gray'
v.camera = 'panzoom'
cx, cy = (0.5, 0.5)
halfx, halfy = (0.1, 0.1)
poly_coords = [(cx - halfx, cy - halfy),
               (cx + halfx, cy - halfy),
               (cx + halfx, cy + halfy),
               (cx - halfx, cy + halfy)]
poly = Polygon(poly_coords, color=(1.0, 0.0, 0.0, 0.5), border_color=(1.0, 1.0, 1.0, 0.2),
               border_width=3,  parent=v.scene)
app = use_app('pyqt5')
canvas.show()
app.run()
```

Which produces something like:

![image (1)](https://user-images.githubusercontent.com/1828519/86391292-2f516580-bc5f-11ea-9680-f22a610d528c.png)

Googling around shows that this should be related to the alpha buffer configured here:

https://github.com/vispy/vispy/blob/77f016a0c6121e1b0c024bdf4e03d5bf84aa9879/vispy/app/backends/_qt.py#L233

However, setting this to -1 (default `alpha_size` is 8) or commenting out this line still produce the same results on OSX. I've had a coworker (CC @rayg-ssec) test this to confirm. On my Ubuntu laptop I don't see this issue at all. I'm wondering if any of the napari folks and other contributors have seen something like this (CC @tlambert03 @kmuehlbauer @larsoner @sofroniewn @kne42)?


Discussion:
I havn't seen that in napari, but I am able to reproduce the error using your standalone script on my mac, so it's definitely very repeatable.

I'd need to look in a little deeper to see why we havn't seen in napari as I know we have shown mesh objects with alpha transparency.
@djhoese I can't reproduce this on my kubuntu. I can check on OpenSuse after the weekend, but I can't remember having issues with that on OpenSuse.
@kmuehlbauer I don't think testing on various unix-ish environments is needed. The main culprit is OSX with Windows being another possibility.

> I havn't seen that in napari

@sofroniewn That's why I wanted to check with the napari folks. I figured you have the most examples of these different visualizations. Is it possible that this is a PyQt-specific issue and you usually use PySide2? Or maybe this has only been introduced in a semi-recent version of Qt/PyQt?
@sofroniewn Any update on if you've seen this issue in napari?
I havn't been able to create a napari example yet and no one has reported the issue, but I can confirm that the issue is PyQt5 specific. When I run your vispy example script above with PyQt5 I get the error, when I run it with PySide2 I don't get the error, both on my mac.

I imagine I could make a napari specific example with PyQt5 that would show the same problem.
@sofroniewn Can you double check the pyside versus pyqt5 behavior? I just had one of my coworkers try it on their mac and they still see transparency with pyside2:

```
Platform: Darwin-19.6.0-x86_64-i386-64bit
Python:   3.7.8 | packaged by conda-forge | (default, Jul 31 2020, 02:37:09)  [Clang 10.0.1 ]
NumPy:    1.19.1
Backend:  PySide2
pyqt4:    None
pyqt5:    None
pyside:   None
pyside2:  ('PySide2', '5.13.2', '5.12.5')
```
NOTE: @sofroniewn you should double check the conda build of your pyside2 installation; specifically `qt`. I'm curious if this is related to a specific version of Qt and your installation of Pyside2 was using an older one.
> @sofroniewn Can you double check the pyside versus pyqt5 behavior? I just had one of my coworkers try it on their mac and they still see transparency with pyside2:

I'll try and take a look tonight

Resolving patch:
diff --git a/vispy/app/backends/_qt.py b/vispy/app/backends/_qt.py
--- a/vispy/app/backends/_qt.py
+++ b/vispy/app/backends/_qt.py
@@ -845,6 +845,16 @@ def paintGL(self):
         self._vispy_canvas.set_current()
         self._vispy_canvas.events.draw(region=None)
 
+        # Clear the alpha channel with QOpenGLWidget (Qt >= 5.4), otherwise the
+        # window is translucent behind non-opaque objects.
+        # Reference:  MRtrix3/mrtrix3#266
+        if QT5_NEW_API or QT6_NEW_API:
+            context = self._vispy_canvas.context
+            context.set_color_mask(False, False, False, True)
+            context.clear(color=True, depth=False, stencil=False)
+            context.set_color_mask(True, True, True, True)
+            context.flush()
+
 
 # Select CanvasBackend
 if USE_EGL:


Test patch:
diff --git a/vispy/gloo/tests/test_glir.py b/vispy/gloo/tests/test_glir.py
--- a/vispy/gloo/tests/test_glir.py
+++ b/vispy/gloo/tests/test_glir.py
@@ -25,16 +25,16 @@ def test_queue():
         assert cmds[i] == ('FOO', 'BAR', i)
 
     # Test filter 1
-    cmds1 = [('DATA', 1), ('SIZE', 1), ('FOO', 1), ('SIZE', 1), ('FOO', 1), 
+    cmds1 = [('DATA', 1), ('SIZE', 1), ('FOO', 1), ('SIZE', 1), ('FOO', 1),
              ('DATA', 1), ('DATA', 1)]
     cmds2 = [c[0] for c in q._shared._filter(cmds1, parser)]
     assert cmds2 == ['FOO', 'SIZE', 'FOO', 'DATA', 'DATA']
 
     # Test filter 2
-    cmds1 = [('DATA', 1), ('SIZE', 1), ('FOO', 1), ('SIZE', 2), ('SIZE', 2), 
+    cmds1 = [('DATA', 1), ('SIZE', 1), ('FOO', 1), ('SIZE', 2), ('SIZE', 2),
              ('DATA', 2), ('SIZE', 1), ('FOO', 1), ('DATA', 1), ('DATA', 1)]
     cmds2 = q._shared._filter(cmds1, parser)
-    assert cmds2 == [('FOO', 1), ('SIZE', 2), ('DATA', 2), ('SIZE', 1), 
+    assert cmds2 == [('FOO', 1), ('SIZE', 2), ('DATA', 2), ('SIZE', 1),
                      ('FOO', 1), ('DATA', 1), ('DATA', 1)]
 
     # Define shader
@@ -94,11 +94,14 @@ def test_log_parser():
     assert lines[i].startswith(expected[0])
     assert lines[i].endswith(expected[1])
     assert int(lines[i][len(expected[0]):-len(expected[1])]) is not None
-    i += 1
 
     # The 'CURRENT' command may have been called multiple times
-    while lines[i] == lines[i - 1]:
+    while lines[i].startswith('["CURRENT",'):
         i += 1
+        if lines[i] == json.dumps(['FUNC', 'colorMask', False, False, False, True]):
+            # Qt workaround, see #2040
+            i += 4
+
     assert lines[i] == json.dumps(['FUNC', 'clearColor', 1.0, 1.0, 1.0, 1.0])
     i += 1
     assert lines[i] == json.dumps(['FUNC', 'clear', 17664])