Repository: nilearn/nilearn

Issue:
nilearn.plotting overrides the matplotlib backend, causing troubles on remote SSH development
I am working on a remote settings, accessing a machine (drago/margaret for INRIA context) through a PyCharm Remote Interpreter.
I have been struggling to display nilearn figures in the scientific view of PyCharm, leading to a painful process of saving the image => rsync figures dir => visualize figures once there are on personal laptop.

I did a simple experiment drawing first a simple `plot(list(range(10)))` which is displayed and then a nilearn figure which does not show up (script at the end of the issue).

I found out the reason for this, which is that in `nilearn.plotting.__init__`, the matplotlib backend is set to `agg` if not in `[inline, agg]`, but the backend used initially (which is working for me) is `module://backend_interagg`.

Setting the backend to the initial value after importing `nilearn.datasets` fixes the problem.

I don't know exactly how those matplotlib backends work, if the list of accepted backends in `nilearn.datasets.__init__` could be extended or if we could have a more robust list of valid matplotlib backend. Also, I feel that nilearn shouldn't override matplotlib backend silently? 

<!--Please fill in the following information, to the best of your ability.-->
Nilearn version: 0.7.1

### Expected behavior

Nilearn does not override matplotlib backend silently.

### Actual behavior

Nilearn sets backend to "agg" if not in ["inline", "agg"], causing trouble with remote SSH development.

### Steps and code to reproduce bug

Run the following script through SSH remote interpreter

```python
import matplotlib
initial_backend = matplotlib.get_backend().lower()
print(initial_backend)

import matplotlib.pyplot as plt
import numpy as np

arr = np.zeros((100, 100))
plt.figure()
plt.plot(list(range(10)))
plt.show()
# >> The show is displayed in PyCharm

from nilearn import datasets, plotting
print(matplotlib.get_backend().lower())
# >> Backend has switched to "agg"

sample_brain_map = datasets.fetch_neurovault_motor_task().images[0]
plotting.plot_stat_map(
    sample_brain_map,
    threshold=3,
    title="Before setting back the backend",
)

plt.show()
# >> Does not show up

matplotlib.use(initial_backend)
plotting.plot_stat_map(
    sample_brain_map,
    threshold=3,
    title="After setting back the backend",
)

plt.show()
# >> Shows up

```



Discussion:
Thanks for opening this @RaphaelMeudec 

Linking the relevant part of the code:

https://github.com/nilearn/nilearn/blob/6fa1359d2ca9e8174e10f48a54d747a6382c6144/nilearn/plotting/__init__.py#L32-L38

As well as #2560 where some explanations for this can be found.

I believe there are two things here:

- Changing the backend silently is confusing for users
- Should we change the list of "authorized" backends to cover this use case

For the first point, I think that adding a warning when we change the backend would be useful. At least, users wouldn't have to spend hours understanding why their plots don't show up. 

For the second point, I'm less sure because this was added intentionally to avoid annoying failures at the end of scripts ran remotly as pointed out by @GaelVaroquaux in #2560.

Maybe we could add some way to configure a bypass for this, such that users who know what they're doing could setup nilearn to always use whatever backend they have? WDYT?
I would say if nilearn isn't silent anymore, it will easily guide users to eventually get back to a specific backend.

For the 2nd point, could it be possible to make sure that matplotlib isn't going to crash because of X server (according to @GaelVaroquaux's comment [here](https://github.com/nilearn/nilearn/issues/2560#issuecomment-682333518)), and if it is not crashing, accept the mpl backend even if it is not in [inline, agg]?
> For the 2nd point, could it be possible to make sure that matplotlib isn't going to crash because of X server (according to @GaelVaroquaux's comment here), and if it is not crashing, accept the mpl backend even if it is not in [inline, agg]?

Do you have a technical solution for that?

We could probably expand the list of backends known to be robust.

Is it possible that this check was a workaround an issue that has since been addressed in matplotlib?
matplotlibrc says:
```
## The default backend.  If you omit this parameter, the first working
## backend from the following list is used:
##     MacOSX Qt5Agg Gtk3Agg TkAgg WxAgg Agg
```
in a couple of servers without an X server I tried, with the default matplotlib configuration I end up using Agg.
@jeromedockes I don't think this is a solution for what Gael is pointing out: if you set an incompatible backend in your matplotlibrc file, you can still have errors that show up only when you run plt.show(). But I don't really see why it should be handled by nilearn. Is there a lot of nilearn users that have this problem?
What about forcing an early call to `matplotlib.pyplot.switch_backend(matplotlib.get_backend())`?

- it does not change the backend chosen by the user
- it checks [compatibility between the session and the backend](https://github.com/matplotlib/matplotlib/blob/49609c36a922baaab25f61f5080fd1eef53604a5/lib/matplotlib/pyplot.py#L265)
- it would comply with the idea to check early that plt.show will raise an error (unless it is possible that plt.show raises an error without the backend being compatible)
- no need anymore for nilearn to have its list of accepted backend
I understood "non technical users" in Gaël's comment to mean "users who don't edit the matplotlib configuration".

I think if you edit your matplotlibrc you have to read the corresponding documentation and fixing your matplotlib configuration is outside the scope of nilearn.
if a user knows about and edits their matplotlibrc, probably they also use matplotlib directly, outside of nilearn. if they chose an incompatible backend it will fail then too, so they will have to fix their configuration anyway

IMO nilearn plotting should work with the default matplotlib configuration, or any valid custom configuration

> What about forcing an early call to matplotlib.pyplot.switch_backend(matplotlib.get_backend())?

yes `switch_backend` does more checks but still it would be nice if we could describe more precisely and reproduce the error we are trying to address with this check
@GaelVaroquaux do you have precise examples of matplotlib errors that new users could have and show up on .show() only? We've been trying to generate some with @jeromedockes but did not succeed.

We think we could drop this custom matplotlib check as matplotlib has increased its own checks. But if you're not convinced, we can simply switch the current function to a call to `switch_backend` + add a warning for the user about the mpl backend switch. WDYT? 

Resolving patch:
diff --git a/nilearn/plotting/__init__.py b/nilearn/plotting/__init__.py
--- a/nilearn/plotting/__init__.py
+++ b/nilearn/plotting/__init__.py
@@ -2,9 +2,8 @@
 Plotting code for nilearn
 """
 # Original Authors: Chris Filo Gorgolewski, Gael Varoquaux
-import os
-import sys
 import importlib
+import warnings
 
 
 ###############################################################################
@@ -29,13 +28,17 @@ def _set_mpl_backend():
                                           OPTIONAL_MATPLOTLIB_MIN_VERSION)
         current_backend = matplotlib.get_backend().lower()
 
-        if 'inline' in current_backend or 'nbagg' in current_backend:
-            return
-        # Set the backend to a non-interactive one for unices without X
-        # (see gh-2560)
-        if (sys.platform not in ('darwin', 'win32') and
-                'DISPLAY' not in os.environ):
-            matplotlib.use('Agg')
+        try:
+            # Making sure the current backend is usable by matplotlib
+            matplotlib.use(current_backend)
+        except Exception:
+            # If not, switching to default agg backend
+            matplotlib.use("Agg")
+        new_backend = matplotlib.get_backend().lower()
+
+        if new_backend != current_backend:
+            # Matplotlib backend has been changed, let's warn the user
+            warnings.warn(f"Backend changed to {new_backend}...")
 
 
 _set_mpl_backend()


Test patch:
diff --git a/nilearn/plotting/tests/test_matplotlib_backend.py b/nilearn/plotting/tests/test_matplotlib_backend.py
new file mode 100644
--- /dev/null
+++ b/nilearn/plotting/tests/test_matplotlib_backend.py
@@ -0,0 +1,46 @@
+from unittest.mock import patch
+
+SKIP_REASON = "matplotlib missing; necessary for this test"
+try:
+    import matplotlib  # noqa: F401
+except ImportError:
+    MATPLOTLIB_INSTALLED = False
+else:
+    MATPLOTLIB_INSTALLED = True
+import pytest
+
+from nilearn.plotting import _set_mpl_backend
+
+
+@pytest.mark.skipif(not MATPLOTLIB_INSTALLED, reason=SKIP_REASON)
+@patch("matplotlib.use")
+@patch("matplotlib.get_backend", side_effect=["backend_1", "backend_2"])
+def test_should_raise_warning_if_backend_changes(*_):
+    # The backend values returned by matplotlib.get_backend are different.
+    # Warning should be raised to inform user of the backend switch.
+    with pytest.warns(UserWarning, match="Backend changed to backend_2..."):
+        _set_mpl_backend()
+
+
+@pytest.mark.skipif(not MATPLOTLIB_INSTALLED, reason=SKIP_REASON)
+@patch("matplotlib.use")
+@patch("matplotlib.get_backend",
+       side_effect=["backend_1", "backend_1"])
+def test_should_not_raise_warning_if_backend_is_not_changed(*_):
+    # The backend values returned by matplotlib.get_backend are identical.
+    # Warning should not be raised.
+    with pytest.warns(None):
+        _set_mpl_backend()
+
+
+@pytest.mark.skipif(not MATPLOTLIB_INSTALLED, reason=SKIP_REASON)
+@patch("matplotlib.use",
+       side_effect=[Exception("Failed to switch backend"), True])
+def test_should_switch_to_agg_backend_if_current_backend_fails(use_mock):
+    # First call to `matplotlib.use` raises an exception, hence the default Agg
+    # backend should be triggered
+    _set_mpl_backend()
+
+    assert use_mock.call_count == 2
+    # Check that the most recent call to `matplotlib.use` has arg `Agg`
+    use_mock.assert_called_with("Agg")