Repository: napari/napari

Issue:
Change pytest-xvfb to explicit xvfb run
# Description
<!-- What does this pull request (PR) do? Why is it necessary? -->
<!-- Tell us about your new feature, improvement, or fix! -->
<!-- If your change includes user interface changes, please add an image, or an animation "An image is worth a thousand words!" -->
<!-- You can use https://www.cockos.com/licecap/ or similar to create animations -->

To remove x11 errors which cause fail negative test results.

## Type of change
<!-- Please delete options that are not relevant. -->
- [ ] Bug-fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] This change requires a documentation update

# References
<!-- What resources, documentation, and guides were used in the creation of this PR? -->
<!-- If this is a bug-fix or otherwise resolves an issue, reference it here with "closes #(issue)" -->

# How has this been tested?
<!-- Please describe the tests that you ran to verify your changes. -->
- [ ] example: the test suite for my feature covers cases x, y, and z
- [ ] example: all tests pass with my change

## Final checklist:
- [ ] My PR is the minimum possible work for the desired functionality
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] If I included new strings, I have used `trans.` to make them localizable.
      For more information see our [translations guide](https://napari.org/docs/dev/guides/translations.html).



Discussion:
Will merge this tomorrow unless I hear otherwise, thanks!!
I would prefer to check if merging #3127 solves current CI failures. 
@Czaki, I just merged #3127 
I have no idea how to deal with this:

```
  [xcb] Unknown request in queue while appending request
  [xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
  [xcb] Aborting, sorry about that.
  python: ../../src/xcb_io.c:151: append_pending_request: Assertion `!xcb_xlib_unknown_req_pending' failed.
  Fatal Python error: Aborted
```
# [Codecov](https://codecov.io/gh/napari/napari/pull/3129?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) Report
> Merging [#3129](https://codecov.io/gh/napari/napari/pull/3129?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (3b1e092) into [master](https://codecov.io/gh/napari/napari/commit/8c267c625a68faf50348ab1371810c7bc806a05b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (8c267c6) will **decrease** coverage by `0.01%`.
> The diff coverage is `14.28%`.

[![Impacted file tree graph](https://codecov.io/gh/napari/napari/pull/3129/graphs/tree.svg?width=650&height=150&src=pr&token=1LfdDeezHt&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)](https://codecov.io/gh/napari/napari/pull/3129?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)

```diff
@@            Coverage Diff             @@
##           master    #3129      +/-   ##
==========================================
- Coverage   81.56%   81.55%   -0.02%     
==========================================
  Files         519      519              
  Lines       43823    43830       +7     
==========================================
+ Hits        35744    35745       +1     
- Misses       8079     8085       +6     
```


| [Impacted Files](https://codecov.io/gh/napari/napari/pull/3129?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | |
|---|---|---|
| [napari/conftest.py](https://codecov.io/gh/napari/napari/pull/3129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bmFwYXJpL2NvbmZ0ZXN0LnB5) | `81.81% <14.28%> (-2.80%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/napari/napari/pull/3129?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/napari/napari/pull/3129?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None). Last update [8c267c6...3b1e092](https://codecov.io/gh/napari/napari/pull/3129?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).

> [xcb] Aborting, sorry about that.

😂
I do not have any idea what happens, but I got segfault when running locally with tox.
Here is stack trace:

```
Thread 1 "python" received signal SIGSEGV, Segmentation fault.
0x00007ffff2c6a63f in _XSend () from /lib/x86_64-linux-gnu/libX11.so.6
(gdb) bt
#0  0x00007ffff2c6a63f in _XSend () from /lib/x86_64-linux-gnu/libX11.so.6
#1  0x00007ffff2c61c4a in XQueryExtension () from /lib/x86_64-linux-gnu/libX11.so.6
#2  0x00007ffff2c55b57 in XInitExtension () from /lib/x86_64-linux-gnu/libX11.so.6
#3  0x00007fff73eda511 in XextAddDisplay () from /lib/x86_64-linux-gnu/libXext.so.6
#4  0x00007fffeb6be97b in ?? () from /lib/x86_64-linux-gnu/libGLX_nvidia.so.0
#5  0x00007fffeb6ae941 in glXDestroyContext () from /lib/x86_64-linux-gnu/libGLX_nvidia.so.0
#6  0x00007fffeb9b3c41 in QGLXContext::~QGLXContext() () from /tmp/.tox/py38-linux-pyqt/lib/python3.8/site-packages/PyQt5/Qt5/plugins/xcbglintegrations/libqxcb-glx-integration.so
#7  0x00007fffeb9b3c69 in QGLXContext::~QGLXContext() () from /tmp/.tox/py38-linux-pyqt/lib/python3.8/site-packages/PyQt5/Qt5/plugins/xcbglintegrations/libqxcb-glx-integration.so
#8  0x00007fffb0f947eb in QOpenGLContext::destroy() () from /tmp/.tox/py38-linux-pyqt/lib/python3.8/site-packages/PyQt5/Qt5/lib/libQt5Gui.so.5
#9  0x00007fffb0f94ac7 in QOpenGLContext::~QOpenGLContext() () from /tmp/.tox/py38-linux-pyqt/lib/python3.8/site-packages/PyQt5/Qt5/lib/libQt5Gui.so.5
#10 0x00007fffb18b17dd in sipQOpenGLContext::~sipQOpenGLContext() () from /tmp/.tox/py38-linux-pyqt/lib/python3.8/site-packages/PyQt5/QtGui.abi3.so
#11 0x00007fffb491f465 in cleanup_qobject(_sipSimpleWrapper*, void*) () from /tmp/.tox/py38-linux-pyqt/lib/python3.8/site-packages/PyQt5/QtCore.abi3.so
#12 0x00007fffb1aff5d9 in sip_api_visit_wrappers (visitor=0x7fffb491f3a0 <cleanup_qobject(_sipSimpleWrapper*, void*)>, closure=0x5556c32edbd0) at siplib.c:13280
#13 0x00007fffb491f0f0 in cleanup_on_exit(_object*, _object*) () from /tmp/.tox/py38-linux-pyqt/lib/python3.8/site-packages/PyQt5/QtCore.abi3.so
#14 0x00007ffff7d1bc2a in cfunction_vectorcall_NOARGS (func=func@entry=0x7fffb4a27860, args=<optimized out>, nargsf=nargsf@entry=0, kwnames=<optimized out>) at Objects/methodobject.c:459
#15 0x00007ffff7cd7013 in PyVectorcall_Call (callable=0x7fffb4a27860, tuple=<optimized out>, kwargs=<optimized out>) at Objects/call.c:199
#16 0x00007ffff7e59527 in atexit_callfuncs (module=<optimized out>) at ./Modules/atexitmodule.c:87
#17 0x00007ffff7df1b2d in call_py_exitfuncs (istate=0x55555555cc10) at Python/pylifecycle.c:2236
#18 Py_FinalizeEx () at Python/pylifecycle.c:1183
#19 Py_FinalizeEx () at Python/pylifecycle.c:1154
#20 0x00007ffff7e16d35 in Py_RunMain () at Modules/main.c:646
#21 0x00007ffff7e1748a in pymain_main (args=0x7fffffffe6a0) at Modules/main.c:674
#22 Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:698
--Type <RET> for more, q to quit, c to continue without paging--
#23 0x00007ffff7a59565 in __libc_start_main (main=0x555555555060 <main>, argc=9, argv=0x7fffffffe808, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7fffffffe7f8) at ../csu/libc-start.c:332
#24 0x000055555555509e in _start ()

```

So it looks like there is a problem with some OpenGL Widget destroy. Did you have any idea which PR lately plays with such objects? 
Do we have a sense if we still want to go with this approach? Do you think it might help #3034 which has quite a lot of test fails that are hard to pin down?

Resolving patch:
diff --git a/napari/_qt/qt_viewer.py b/napari/_qt/qt_viewer.py
--- a/napari/_qt/qt_viewer.py
+++ b/napari/_qt/qt_viewer.py
@@ -326,7 +326,6 @@ def _create_performance_dock_widget(self):
                 QtPerformance(),
                 name=trans._('performance'),
                 area='bottom',
-                shortcut='Ctrl+Shift+P',
             )
         return None
 
diff --git a/napari/conftest.py b/napari/conftest.py
--- a/napari/conftest.py
+++ b/napari/conftest.py
@@ -323,17 +323,6 @@ def pytest_internalerror(excinfo):
         raise excinfo.value
 
 
-def pytest_collection_modifyitems(session, config, items):
-
-    put_at_end = ('test_trace_on_start',)
-    at_end = []
-    for i, item in enumerate(items):
-        if item.name in put_at_end:
-            at_end.append(items.pop(i))
-
-    items.extend([x for _, x in sorted(zip(put_at_end, at_end))])
-
-
 @pytest.fixture(autouse=True)
 def fresh_settings(monkeypatch):
     from napari import settings


Test patch:
diff --git a/.github/workflows/test_comprehensive.yml b/.github/workflows/test_comprehensive.yml
--- a/.github/workflows/test_comprehensive.yml
+++ b/.github/workflows/test_comprehensive.yml
@@ -53,6 +53,9 @@ jobs:
     needs: code
     name: ${{ matrix.platform }} py${{ matrix.python }} ${{ matrix.toxenv }} ${{ matrix.MIN_REQ && 'min_req' }}
     runs-on: ${{ matrix.platform }}
+    defaults:
+      run:
+        shell: bash -l {0}
     strategy:
       fail-fast: false
       matrix:
@@ -139,8 +142,9 @@ jobs:
       # in which case we are declaring one specific tox environment to run.
       # see tox.ini for more
       - name: Test with tox
-        shell: bash -l {0}
-        run: tox
+        uses: GabrielBB/xvfb-action@v1
+        with:
+          run: tox
         env:
           PLATFORM: ${{ matrix.platform }}
           TOXENV: ${{ matrix.toxenv }}
diff --git a/.github/workflows/test_prereleases.yml b/.github/workflows/test_prereleases.yml
--- a/.github/workflows/test_prereleases.yml
+++ b/.github/workflows/test_prereleases.yml
@@ -53,7 +53,9 @@ jobs:
 
       - name: Test with tox
         # run tests using pip install --pre
-        run: tox -v --pre
+        uses: GabrielBB/xvfb-action@v1
+        with:
+          run: tox -v --pre
         env:
           PLATFORM: ${{ matrix.platform }}
           BACKEND: ${{ matrix.backend }}
diff --git a/.github/workflows/test_pull_requests.yml b/.github/workflows/test_pull_requests.yml
--- a/.github/workflows/test_pull_requests.yml
+++ b/.github/workflows/test_pull_requests.yml
@@ -140,7 +140,9 @@ jobs:
       # `tox -e py38-linux-pyqt,py38-linux-pyside`
       # see tox.ini for more
       - name: Test with tox
-        run: tox
+        uses: GabrielBB/xvfb-action@v1
+        with:
+          run: tox
         env:
           PLATFORM: ${{ matrix.platform }}
           BACKEND: ${{ matrix.backend }}
@@ -168,10 +170,13 @@ jobs:
             libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 \
             libxcb-xinerama0 libxcb-xinput0 libxcb-xfixes0
 
-      - name: Test
+      - name: Install this commit
         run: |
-          pip install --upgrade pip
-          pip install pytest-xvfb
           # pip install this git sha directly from github
+          pip install --upgrade pip
           pip install git+git://github.com/${{ github.repository }}.git@${{ github.sha }}#egg=napari[all,testing]
-          pytest --pyargs napari -v --color=yes
+
+      - name: Test
+        uses: GabrielBB/xvfb-action@v1
+        with:
+          run: pytest --pyargs napari --color=yes
diff --git a/napari/_qt/perf/_tests/test_perf.py b/napari/_qt/perf/_tests/test_perf.py
--- a/napari/_qt/perf/_tests/test_perf.py
+++ b/napari/_qt/perf/_tests/test_perf.py
@@ -1,78 +1,51 @@
 import json
 import os
+import subprocess
 import sys
 from pathlib import Path
 
-import numpy as np
-import pytest
-import qtpy
-from qtpy.QtWidgets import QApplication
-
-from napari._qt.perf import qt_event_tracing
-from napari._qt.utils import delete_qapp
-from napari.utils import perf
-
-if os.getenv('CI', '0') != '0' and (
-    sys.version_info >= (3, 9)
-    or (
-        (sys.platform.startswith('linux') or sys.platform.startswith('win'))
-        and qtpy.API_NAME == "PySide2"
-    )
-):
-    # this test is covered by other platforms, and also seems to work locally
-    # on linux
-    pytest.skip(
-        "Perfmon segfaults on linux or windows CI with pyside2, or with python 3.9",
-        allow_module_level=True,
-    )
-
-
-@pytest.fixture(scope="module")
-def qapp():
-    """A modified QApplicationWithTracing just for this test module.
-
-    Note: Because of the difficulty in destroying a QApplication that overrides
-    .notify() like QApplicationWithTracing does, this test must be run last
-    (globally).  So in napari/conftest.py::pytest_collection_modifyitems,
-    we ensure that this test is always last.
-    """
-
-    # before creating QApplicationWithTracing, we need to monkeypatch
-    # the `perf.perf_timer` context manager that gets used in the
-    # qt_event_tracing module
-
-    original_perf_timer = qt_event_tracing.perf.perf_timer
-    _, perf_timer, _, _ = perf._timers._create_timer()
-    qt_event_tracing.perf.perf_timer = perf_timer
-
-    try:
-        if qtpy.API_NAME == 'PySide2' and QApplication.instance():
-            delete_qapp(QApplication.instance())
-        yield qt_event_tracing.QApplicationWithTracing([])
-    finally:
-        qt_event_tracing.perf.perf_timer = original_perf_timer
-
-
-def test_trace_on_start(tmp_path, monkeypatch, make_napari_viewer):
+from napari._tests.utils import skip_local_popups
+
+# NOTE:
+# for some reason, running this test fails in a subprocess with a segfault
+# if you don't show the viewer...
+PERFMON_SCRIPT = """
+import napari
+from qtpy.QtCore import QTimer
+
+v = napari.view_points()
+QTimer.singleShot(100, napari._qt.qt_event_loop.quit_app)
+napari.run()
+"""
+
+CONFIG = {
+    "trace_qt_events": True,
+    "trace_file_on_start": '',
+    "trace_callables": ["chunk_loader"],
+    "callable_lists": {
+        "chunk_loader": [
+            "napari.components.experimental.chunk._loader.ChunkLoader.load_request",
+            "napari.components.experimental.chunk._loader.ChunkLoader._on_done",
+        ]
+    },
+}
+
+
+@skip_local_popups
+def test_trace_on_start(tmp_path: Path):
     """Make sure napari can write a perfmon trace file."""
-
-    timers, _, _, _ = perf._timers._create_timer()
-    monkeypatch.setattr(perf._timers, 'timers', timers)
-
-    # Check perfmon is enabled
     trace_path = tmp_path / "trace.json"
-    timers.start_trace_file(trace_path)
-
-    viewer = make_napari_viewer()
-    data = np.random.random((10, 15))
-    viewer.add_image(data)
-    viewer.close()
+    config_path = tmp_path / "perfmon.json"
+    CONFIG['trace_file_on_start'] = str(trace_path)
+    config_path.write_text(json.dumps(CONFIG))
 
-    timers.stop_trace_file()
+    env = os.environ.copy()
+    env.update({'NAPARI_PERFMON': str(config_path), 'NAPARI_CONFIG': ''})
+    subprocess.run([sys.executable, '-c', PERFMON_SCRIPT], env=env, check=True)
 
     # Make sure file exists and is not empty.
-    assert Path(trace_path).exists(), "Trace file not written"
-    assert Path(trace_path).stat().st_size > 0, "Trace file is empty"
+    assert trace_path.exists(), "Trace file not written"
+    assert trace_path.stat().st_size > 0, "Trace file is empty"
 
     # Assert every event contains every important field.
     with open(trace_path) as infile:
diff --git a/napari/_tests/test_sys_info.py b/napari/_tests/test_sys_info.py
new file mode 100644
--- /dev/null
+++ b/napari/_tests/test_sys_info.py
@@ -0,0 +1,16 @@
+from napari.utils.info import sys_info
+
+
+# vispy use_app tries to start Qt, which can cause segfaults when running
+# sys_info on CI unless we provide a pytest Qt app
+def test_sys_info(qapp):
+    str_info = sys_info()
+    assert isinstance(str_info, str)
+    assert '<br>' not in str_info
+    assert '<b>' not in str_info
+    assert "Plugins" in str_info
+
+    html_info = sys_info(as_html=True)
+    assert isinstance(html_info, str)
+    assert '<br>' in html_info
+    assert '<b>' in html_info
diff --git a/napari/utils/_tests/test_info.py b/napari/utils/_tests/test_info.py
--- a/napari/utils/_tests/test_info.py
+++ b/napari/utils/_tests/test_info.py
@@ -1,26 +1,12 @@
 import subprocess
 from typing import NamedTuple
 
-from .. import info
-from ..info import citation_text, sys_info
-
-
-def test_sys_info():
-    str_info = sys_info()
-    assert isinstance(str_info, str)
-    assert '<br>' not in str_info
-    assert '<b>' not in str_info
-    assert "Plugins" in str_info
-
-    html_info = sys_info(as_html=True)
-    assert isinstance(html_info, str)
-    assert '<br>' in html_info
-    assert '<b>' in html_info
+from napari.utils import info
 
 
 def test_citation_text():
-    assert isinstance(citation_text, str)
-    assert 'doi' in citation_text
+    assert isinstance(info.citation_text, str)
+    assert 'doi' in info.citation_text
 
 
 def test_linux_os_name_file(monkeypatch, tmp_path):