Repository: mne-tools/mne-python

Issue:
Replace Travis with Azure or GitHub Actions
Travis seems to be very slow especially with our macOS tests, to the point where the job routinely times out after 50 minutes with only 46% of all tests run so far. I suggest that we abandon Travis and use Azure or GitHub actions instead (somehow I'd prefer GH actions because this seems to be the go-to solution, but I have never actually tried it).


Discussion:
We can work from SciPy's macOS GitHub action

https://github.com/scipy/scipy/blob/master/.github/workflows/testing.yml

This combined with our code for Travis would make for a nice, hopefully not so painful CI PR if someone wants to learn more about the process

The one problem with the GitHub action IIRC is that it won't kick in until merged with `master`, so for testing whoever implements it will have to enable it for their fork and iterate/monitor progress there
I probably won't have time to work on this in the next weeks (so if you want to take a stab please go ahead). Did I understand you correctly that the CI won't kick in in a normal PR? This can't be true, because browsing the SciPy PRs shows that the macOS job is run...
> Did I understand you correctly that the CI won't kick in in a normal PR? This can't be true, because browsing the SciPy PRs shows that the macOS job is run...

The PR to add the GitHub action itself will not have the action run. Once the changes are in `master`, each PR runs the GitHub actions in master -- or at least this is how it was when I added circleci-redirector-action. This is different from Travis/AppVeyor/Azure/CircleCI, where the config YAML in a given PR is actually used in execution. So it makes iterating on CIs a little bit harder / more annoying, but once things work and are in `master`, it otherwise has fine behavior.

Resolving patch:
diff --git a/mne/conftest.py b/mne/conftest.py
--- a/mne/conftest.py
+++ b/mne/conftest.py
@@ -175,9 +175,9 @@ def __init__(self, exception_handler=None):
 
 
 @pytest.fixture(scope='session')
-def travis_macos():
-    """Determine if running on Travis macOS."""
-    return (os.getenv('TRAVIS', 'false').lower() == 'true' and
+def ci_macos():
+    """Determine if running on MacOS CI."""
+    return (os.getenv('CI', 'false').lower() == 'true' and
             sys.platform == 'darwin')
 
 
@@ -189,10 +189,10 @@ def azure_windows():
 
 
 @pytest.fixture()
-def check_gui_ci(travis_macos, azure_windows):
+def check_gui_ci(ci_macos, azure_windows):
     """Skip tests that are not reliable on CIs."""
-    if azure_windows or travis_macos:
-        pytest.skip('Skipping GUI tests on Travis OSX and Azure Windows')
+    if azure_windows or ci_macos:
+        pytest.skip('Skipping GUI tests on MacOS CIs and Azure Windows')
 
 
 @pytest.fixture(scope='session', params=[testing._pytest_param()])
diff --git a/mne/viz/backends/_pyvista.py b/mne/viz/backends/_pyvista.py
--- a/mne/viz/backends/_pyvista.py
+++ b/mne/viz/backends/_pyvista.py
@@ -190,6 +190,8 @@ def __init__(self, fig=None, size=(600, 600), bgcolor='black',
             warnings.filterwarnings("ignore", category=FutureWarning)
             if MNE_3D_BACKEND_TESTING:
                 self.tube_n_sides = 3
+                # smooth_shading=True fails on MacOS CIs
+                self.figure.smooth_shading = False
             with _disabled_depth_peeling():
                 self.plotter = self.figure.build()
             self.plotter.hide_axes()


Test patch:
diff --git a/mne/beamformer/tests/test_lcmv.py b/mne/beamformer/tests/test_lcmv.py
--- a/mne/beamformer/tests/test_lcmv.py
+++ b/mne/beamformer/tests/test_lcmv.py
@@ -819,7 +819,7 @@ def test_lcmv_reg_proj(proj, weight_norm):
     (0.05, 'unit-noise-gain', False, None, 83, 86),
     (0.05, 'unit-noise-gain', False, 0.8, 83, 86),  # depth same for wn != None
     # no reg
-    (0.00, 'unit-noise-gain', True, None, 63, 99),  # TODO: Still not stable
+    (0.00, 'unit-noise-gain', True, None, 45, 99),  # TODO: Still not stable
 ])
 def test_localization_bias_fixed(bias_params_fixed, reg, weight_norm, use_cov,
                                  depth, lower, upper):
diff --git a/mne/viz/_brain/tests/test_brain.py b/mne/viz/_brain/tests/test_brain.py
--- a/mne/viz/_brain/tests/test_brain.py
+++ b/mne/viz/_brain/tests/test_brain.py
@@ -405,12 +405,10 @@ def test_brain_timeviewer_traces(renderer_interactive, hemi, src, tmpdir):
 
 @testing.requires_testing_data
 @pytest.mark.slowtest
-def test_brain_linkviewer(renderer_interactive, travis_macos):
+def test_brain_linkviewer(renderer_interactive):
     """Test _LinkViewer primitives."""
     if renderer_interactive._get_3d_backend() != 'pyvista':
         pytest.skip('Linkviewer only supported on PyVista')
-    if travis_macos:
-        pytest.skip('Linkviewer tests unstable on Travis macOS')
     brain_data = _create_testing_brain(hemi='split', show_traces=False)
 
     link_viewer = _LinkViewer(