Repository: mne-tools/mne-python

Issue:
Progressbar not visible in Spyder
Performing a cluster based test (unfortunatelly one that takes a lot of time) I no longer see the progressbar in Spyder:
```
stat_fun(H1): min=-5.699470 max=3.482243
Running initial clustering
Found 17 clusters
Permuting 9999 times...
HBox(children=(FloatProgress(value=0.0, max=9999.0), HTML(value='')))
```
this is most likely a result of using auto mode of `tqdm` progressbar, which seems to be unable to tell whether it is in a notebook or ipython console. It would be nice to have some option to control this behavior (`progressbar='text'` maybe?) as now everyone who works in ipython/Spyder will likely have no progressbar, which for long running tasks is a bit inconvenient.


Discussion:
This sounds more like an upstream bug with tqdm. Can you look into fixing it there (if it's not already fixed; try using tqdm locally as IIRC we prefer an installed system version when available and new enough)?
did github automatically self-assigned me? weird, I just clicked "submit". Anyway - I can do it.

@larsoner 
Unfortunatelly from what I read on multiple occasions it is not possible to distinguish whether the code is running from notebook or ipython (or Spyder). :(
Perhaps they should support magic `%tqdm text` for example to set within a given session which auto backend will be selected. Might be worth proposing and implementing that at their end. If they reject it, we can add some MNE config value that would triage the class used, but it makes more sense for them to implement it if they're up for it.
@larsoner Good idea, I'll suggest that there. Although the response I imagine might be "just use `tqdm.tqdm` instead of auto or `tqdm.notebook`", but I'll try. :)
> Although the response I imagine might be "just use tqdm.tqdm instead of auto or tqdm.notebook", but I'll try. :)

Yes if they do this we can support it locally. But they will be forcing every project that uses `tqdm` to reimplement this same triaging mechanism. So hopefully they see the value in it :)
@mmagnuski any progress at the `tqdm` end? We're recommending Spyder as the IDE in our release, so if they're not fixing it, we should.

Resolving patch:
diff --git a/mne/utils/config.py b/mne/utils/config.py
--- a/mne/utils/config.py
+++ b/mne/utils/config.py
@@ -117,6 +117,7 @@ def set_memmap_min_size(memmap_min_size):
     'MNE_SKIP_NETWORK_TESTS',
     'MNE_SKIP_TESTING_DATASET_TESTS',
     'MNE_STIM_CHANNEL',
+    'MNE_TQDM',
     'MNE_USE_CUDA',
     'MNE_USE_NUMBA',
     'SUBJECTS_DIR',
diff --git a/mne/utils/progressbar.py b/mne/utils/progressbar.py
--- a/mne/utils/progressbar.py
+++ b/mne/utils/progressbar.py
@@ -14,6 +14,8 @@
 
 import numpy as np
 
+from .check import _check_option
+from .config import get_config
 from ._logging import logger
 
 
@@ -42,8 +44,16 @@ class ProgressBar(object):
     def __init__(self, iterable=None, initial_value=0, mesg=None,
                  max_total_width='auto', max_value=None,
                  **kwargs):  # noqa: D102
-        from ..externals.tqdm import auto
-        tqdm = auto.tqdm
+        # The following mimics this, but with configurable module to use
+        # from ..externals.tqdm import auto
+        from ..externals import tqdm
+        which_tqdm = get_config('MNE_TQDM', 'tqdm.auto')
+        _check_option('MNE_TQDM', which_tqdm[:5], ('tqdm', 'tqdm.'),
+                      extra='beginning')
+        logger.debug(f'Using ProgressBar with {which_tqdm}')
+        if which_tqdm != 'tqdm':
+            tqdm = getattr(tqdm, which_tqdm.split('.', 1)[1])
+        tqdm = tqdm.tqdm
         defaults = dict(
             leave=True, mininterval=0.016, miniters=1, smoothing=0.05,
             bar_format='{percentage:3.0f}%|{bar}| {desc} : {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt:>11}{postfix}]',  # noqa: E501
@@ -142,7 +152,7 @@ def __exit__(self, type_, value, traceback):  # noqa: D105
 
     def __del__(self):
         """Ensure output completes."""
-        if getattr(self, '_tqdm') is not None:
+        if getattr(self, '_tqdm', None) is not None:
             self._tqdm.close()
 
 


Test patch:
diff --git a/mne/utils/tests/test_progressbar.py b/mne/utils/tests/test_progressbar.py
--- a/mne/utils/tests/test_progressbar.py
+++ b/mne/utils/tests/test_progressbar.py
@@ -5,7 +5,8 @@
 import pytest
 
 from mne.parallel import parallel_func
-from mne.utils import ProgressBar, array_split_idx, use_log_level
+from mne.utils import (ProgressBar, array_split_idx, use_log_level,
+                       modified_env, catch_logging)
 
 
 def test_progressbar():
@@ -25,6 +26,18 @@ def iter_func(a):
             pass
     pytest.raises(Exception, iter_func, ProgressBar(20))
 
+    # Make sure different progress bars can be used
+    with catch_logging() as log, modified_env(MNE_TQDM='tqdm'), \
+            use_log_level('debug'), ProgressBar(np.arange(3)) as pbar:
+        for p in pbar:
+            pass
+    log = log.getvalue()
+    assert 'Using ProgressBar with tqdm\n' in log
+    with modified_env(MNE_TQDM='broken'), pytest.raises(ValueError):
+        ProgressBar(np.arange(3))
+    with modified_env(MNE_TQDM='tqdm.broken'), pytest.raises(AttributeError):
+        ProgressBar(np.arange(3))
+
 
 def _identity(x):
     return x