Repository: mne-tools/mne-python

Issue:
brain.screenshot(time_viewer=True) crash
I was running the mne-bids-pipeline (so with mne-23dev cloned from GitHub in backend), but a bug appeared just after the creation of the reports of the first subject. After investigation, it turns out that the bug comes from MNE, from the option time_viewer=True in the screenshot method. By setting it to false, the image plots nicely.


I tried to make a reproducible example with sample data, but this does not reproduce the issue (Else). So I've also included here the (IF True) my [stc file](https://drive.google.com/file/d/1rcaBOMTbLnmyWqvUn9dRy4G3V5PEJehY/view?usp=sharing) and my [subjects file](https://drive.google.com/drive/folders/13TeY0L88dsag6AD3VfYj7GTOFqS_He_k?usp=sharing) which isolate the code on my computer.
```python
from pathlib import Path
import matplotlib
import matplotlib.pyplot as plt
import mne
from mne.datasets import sample
matplotlib.use('Qt5Agg')

if True:
    stc_fname = Path("mne_data/ds003392/derivatives",
                     'mne-bids-pipeline/sub-01/meg/',
                     'sub-01_task-localizer_coherent+dSPM+hemi-lh.stc')

    subjects_dir = Path("mne_data/ds003392/derivatives/freesurfer",
                        'subjects')
    subject = 'sub-01'
    stc = mne.read_source_estimate(stc_fname, subject=subject)
else:
    data_path = sample.data_path()
    fname = data_path + '/MEG/sample/sample_audvis-meg'

    stc = mne.read_source_estimate(fname)
    subject = "sample"
    subjects_dir = Path("C:/Users/charb/mne_data/MNE-sample-data/subjects")
brain = stc.plot(subject=subject, subjects_dir=subjects_dir,
                 initial_time=0.13, views=['lat'], hemi='split')
brain.toggle_interface()
brain._renderer.plotter.reset_camera()
brain._renderer.plotter.subplot(0, 0)
brain._renderer.plotter.reset_camera()
figs, ax = plt.subplots(figsize=(15, 10))

# BUG : Set time_viewer to False to remove the Bug
image = brain.screenshot(time_viewer=True)
ax.imshow(image)
ax.axis('off')
plt.show()

```

#### Expected results
An interactive plot.


#### Actual results
When I run the code, the figure appears one second and then crashes

```
(mne_dev) C:\Users\charb\Desktop\parietal>C:/Users/charb/anaconda3/envs/mne_dev/python.exe c:/Users/charb/Desktop/parietal/test.py
Using pyvista 3d backend.

Using control points [13.01218636 16.52222281 48.36637508]
Traceback (most recent call last):
  File "c:\Users\charb\Desktop\parietal\test.py", line 51, in <module>
    image = brain.screenshot(time_viewer=True)
  File "C:\Users\charb\anaconda3\envs\mne_dev\lib\site-packages\mne\viz\_brain\_brain.py", line 2631, in screenshot
    trace_img = np.reshape(
  File "<__array_function__ internals>", line 5, in reshape
  File "C:\Users\charb\anaconda3\envs\mne_dev\lib\site-packages\numpy\core\fromnumeric.py", line 299, in reshape
    return _wrapfunc(a, 'reshape', newshape, order=order)
  File "C:\Users\charb\anaconda3\envs\mne_dev\lib\site-packages\numpy\core\fromnumeric.py", line 58, in _wrapfunc
    return bound(*args, **kwds)
ValueError: cannot reshape array of size 1082952 into shape (1015,4)

(mne_dev) C:\Users\charb\Desktop\parietal>    

```
#### Additional information
I'm working on Windows.

```
(mne_dev) C:\Users\charb\Desktop\parietal\mne-bids-pipeline>python
Python 3.9.2 | packaged by conda-forge | (default, Feb 21 2021, 04:59:43) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import mne; mne.sys_info()
Platform:      Windows-10-10.0.19041-SP0
Python:        3.9.2 | packaged by conda-forge | (default, Feb 21 2021, 04:59:43) [MSC v.1916 64 bit (AMD64)]
Executable:    C:\Users\charb\anaconda3\envs\mne_dev\python.exe
CPU:           Intel64 Family 6 Model 158 Stepping 10, GenuineIntel: 8 cores
Memory:        7.9 GB

mne:           0.23.dev0
numpy:         1.20.2 {blas=NO_ATLAS_INFO, lapack=lapack}
scipy:         1.6.2
matplotlib:    3.4.1 {backend=Qt5Agg}

sklearn:       0.24.1
numba:         0.53.1
nibabel:       3.2.1
nilearn:       0.7.1
dipy:          1.4.0
cupy:          Not found
pandas:        1.2.4
mayavi:        4.7.2
pyvista:       0.29.1 {pyvistaqt=0.3.0, OpenGL 4.5.0 - Build 26.20.100.6911 via Intel(R) UHD Graphics 630}
vtk:           9.0.1
PyQt5:         5.12.3
>>>
```





Discussion:
Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴🏽‍♂️

I'm guessing this is some sort of off-by-one error as:
```
>>> 1082952 // 4 / 1014
267.0
```
@GuillaumeFavelier can you look?
I added it in the list, I'll fix it ASAP.
Thanks @GuillaumeFavelier!

I don't want to be pushy, but if you could kindly move this up on your priorities list this would be most appreciated :) I'm working with @crsegerie and unfortunately this bug is currently a blocker in his project.

Let me know if there's any way I can help you track this down! Happy to have a call too.

🤗
Yep. It would be most appreciated! And props to @hoechenberger who first isolated the bug
I'm trying to reproduce so I downloaded `sub-01_task-localizer_coherent+dSPM+hemi-lh.stc` and the `sub-01` folder but I obtain the following when I run the script:

```
/tmp/bug.py in <module>
     10     subjects_dir = 'subjects'
     11     subject = 'sub-01'
---> 12     stc = mne.read_source_estimate(stc_fname)
     13 else:
     14     data_path = sample.data_path()

~/source/mne-python/mne/source_estimate.py in read_source_estimate(fname, subject)
    301             fname += '-stc'
    302         elif any(stc_exist) or any(w_exist):
--> 303             raise IOError("Hemisphere missing for %r" % fname_arg)
    304         else:
    305             raise IOError("SourceEstimate File(s) not found for: %r"

OSError: Hemisphere missing for 'sub-01_task-localizer_coherent+dSPM+hemi-lh.stc'
```
Also, as a way to mitigate since it's a blocker, does it work if you change the `size` parameter in `stc.plot` before calling `screenshot()`?
> ```
> OSError: Hemisphere missing for 'sub-01_task-localizer_coherent+dSPM+hemi-lh.stc'
> ```

@crsegerie Can you share the STC for the right hemisphere too? Both are needed here
Yes @GuillaumeFavelier, if I add size=(500, 500) in the stc.plot, the bug disappears !
> the bug disappears !

I like your enthusiasm :) For me, it's still there in a corner case of `stc.plot` :sweat_smile: 
Here is the right hemisphere : https://drive.google.com/file/d/1y7puZ7RPMGV6uB6tN_Pu-MgamuaAFvlb/view?usp=sharing

(In the script only left hemisphere is imported ? Weird, but ok...)
> (In the script only left hemisphere is imported ? Weird, but ok...)

Yeah, MNE tries to automatically find the other hemisphere. It's a bit odd, yes…
Thanks for sharing. I still cannot reproduce locally but I'll try to simulate this scenario anyway and come up with a fix eventually.

About the origin of the issue itself, it's not clear so I can only guess here. Reasons why the trace plot would not have the correct size could be either unexpected DPI configuration or PyQt5 inconsistencies on Windows for example.
@GuillaumeFavelier yes I also suspected some kind of hi-DPI interaction

@crsegerie can you check if you have any of those settings for high-resolution displays activated?

https://windowsreport.com/fix-hidpi-issues-windows-10/
I checked my graphic settings, but did not find any anomalies.
if it is more convenient for you, we can arrange a meeting if you want to explore my debugger on VS Code? 
@GuillaumeFavelier So the default `size` is 800; why would changing that to 500 change anything? What's the rationale? (It seems to work so this is great – but I have no idea **why** that works?)
> PyQt5 inconsistencies on Windows for example.

Probably this, or something similar. My guess is that there is a situation where the sizes reported are inconsistent, or we don't use them correctly, and that whether or not this situation occurs depends on the size itself and/or interactions with the window (resizes, etc.). So trying different `size` options will tell us if some work and others don't, and then we could start looking at various MNE / Qt / matplotlib / vtk variables so see what each layer "thinks" the size should be, and thus figure out who's getting it wrong and why.
Thanks @larsoner!

I just realized that @crsegerie switched from `size=800` (default) to `size=(500, 500)`, i.e., a tuple. Maybe that's also part of the solution…
IIRC an `int` just gets turned into `(size, size)` internally so `size=800` should give identical results to `(800, 800)`
Ok. I've tested (500, 500), 500, (800, 800), and 800 and it's not a matter of tuple not tuple... Only the 500 are working
> Ok. I've tested (500, 500), 500, (800, 800), and 800 and it's not a matter of tuple not tuple... Only the 500 are working

You mean `500` and `(500, 500)` are working, yes?
Only the size matter, not the tuple
OK. 500 working, 600 working, 799 working, 800 not working, 801 not working
Can you see if you can reproduce with a simple example using `sample` data? For example:
```
import os
import os.path as op
import mne
from mne.datasets import sample

data_path = sample.data_path()
sample_dir = op.join(data_path, 'MEG', 'sample')
subjects_dir = op.join(data_path, 'subjects')
fname_stc = op.join(sample_dir, 'sample_audvis-meg')
stc = mne.read_source_estimate(fname_stc, subject='sample')
brain = stc.plot(subjects_dir=subjects_dir, size=500, inital_time=0.1)
img = brain.screenshot(time_viewer=True)
print(img.shape)
```
For me I get:
```
(669, 500, 3)
```
Can you tell me if this works for you at 500 and 800? And if one or both of them succeed, what the `print` statement prints?

Also, what is your screen resolution? And can you tell me if this prints something other than 1.:
```
>>> python -c "from mne.conftest import pixel_ratio; print(pixel_ratio.__wrapped__())"
1.0
```
There is a small typo inital_time > initial_time.

For me the code works for both 500 and 800.

For 500 I obtain : (66**7**, 500, 3)
For 800 I obtain : (1067, 800, 3)

My screen resolution : 1920x1080.

When I run the tests, I get 1.
So the only difference is my 667 and your 669

Resolving patch:
diff --git a/mne/viz/_brain/_brain.py b/mne/viz/_brain/_brain.py
--- a/mne/viz/_brain/_brain.py
+++ b/mne/viz/_brain/_brain.py
@@ -1229,7 +1229,7 @@ def _configure_picking(self):
     def _configure_tool_bar(self):
         self._renderer._tool_bar_load_icons()
         self._renderer._tool_bar_set_theme(self.theme)
-        self._renderer._tool_bar_initialize()
+        self._renderer._tool_bar_initialize(name="Toolbar")
         self._renderer._tool_bar_add_file_button(
             name="screenshot",
             desc="Take a screenshot",
@@ -2611,9 +2611,11 @@ def screenshot(self, mode='rgb', time_viewer=False):
             Image pixel values.
         """
         img = self._renderer.screenshot(mode)
+        logger.debug(f'Got screenshot of size {img.shape}')
         if time_viewer and self.time_viewer and \
                 self.show_traces and \
                 not self.separate_canvas:
+            from matplotlib.image import imread
             canvas = self.mpl_canvas.fig.canvas
             canvas.draw_idle()
             fig = self.mpl_canvas.fig
@@ -2624,15 +2626,26 @@ def screenshot(self, mode='rgb', time_viewer=False):
                 # (e.g., macOS w/Qt 5.14+ and VTK9) then things won't work,
                 # so let's just calculate the DPI we need to get
                 # the correct size output based on the widths being equal
-                dpi = img.shape[1] / fig.get_size_inches()[0]
-                fig.savefig(output, dpi=dpi, format='raw',
+                size_in = fig.get_size_inches()
+                dpi = fig.get_dpi()
+                want_size = tuple(x * dpi for x in size_in)
+                n_pix = want_size[0] * want_size[1]
+                logger.debug(
+                    f'Saving figure of size {size_in} @ {dpi} DPI '
+                    f'({want_size} = {n_pix} pixels)')
+                # Sometimes there can be off-by-one errors here (e.g.,
+                # if in mpl int() rather than int(round()) is used to
+                # compute the number of pixels) so rather than use "raw"
+                # format and try to reshape ourselves, just write to PNG
+                # and read it, which has the dimensions encoded for us.
+                fig.savefig(output, dpi=dpi, format='png',
                             facecolor=self._bg_color, edgecolor='none')
                 output.seek(0)
-                trace_img = np.reshape(
-                    np.frombuffer(output.getvalue(), dtype=np.uint8),
-                    newshape=(-1, img.shape[1], 4))[:, :, :3]
-            img = concatenate_images(
-                [img, trace_img], bgcolor=self._brain_color[:3])
+                trace_img = imread(output, format='png')[:, :, :3]
+                trace_img = np.clip(
+                    np.round(trace_img * 255), 0, 255).astype(np.uint8)
+            bgcolor = np.array(self._brain_color[:3]) / 255
+            img = concatenate_images([img, trace_img], bgcolor=bgcolor)
         return img
 
     @contextlib.contextmanager
diff --git a/mne/viz/backends/_qt.py b/mne/viz/backends/_qt.py
--- a/mne/viz/backends/_qt.py
+++ b/mne/viz/backends/_qt.py
@@ -391,7 +391,7 @@ def _window_get_mplcanvas(self, brain, interactor_fraction, show_traces,
 
     def _window_adjust_mplcanvas_layout(self):
         canvas = self._mplcanvas.canvas
-        dock, dock_layout = _create_dock_widget(
+        self._mpl_dock, dock_layout = _create_dock_widget(
             self._window, "Traces", Qt.BottomDockWidgetArea)
         dock_layout.addWidget(canvas)
 
@@ -414,10 +414,14 @@ def _window_ensure_minimum_sizes(self):
         # plotter.frame:      QFrame with QVBoxLayout with plotter.interactor as centralWidget  # noqa
         # plotter.ren_win:    vtkXOpenGLRenderWindow
         self._interactor.setMinimumSize(*sz)
+        # Lines like this are useful for debugging these issues:
+        # print('*' * 80)
+        # print(0, self._interactor.app_window.size().height(), self._interactor.size().height(), self._mpl_dock.widget().height(), self._mplcanvas.canvas.size().height())  # noqa
         if adjust_mpl:
             mpl_h = int(round((sz[1] * self._interactor_fraction) /
                               (1 - self._interactor_fraction)))
             self._mplcanvas.canvas.setMinimumSize(sz[0], mpl_h)
+            self._mpl_dock.widget().setMinimumSize(sz[0], mpl_h)
         try:
             yield  # show
         finally:
@@ -431,11 +435,17 @@ def _window_ensure_minimum_sizes(self):
             self._interactor.setMinimumSize(0, 0)
             if adjust_mpl:
                 self._mplcanvas.canvas.setMinimumSize(0, 0)
+                self._mpl_dock.widget().setMinimumSize(0, 0)
             self._process_events()
             self._process_events()
-            # 4. Resize the window and interactor to the correct size
+            # 4. Compute the extra height required for dock decorations and add
+            win_h = win_sz.height()
+            if adjust_mpl:
+                win_h += max(
+                    self._mpl_dock.widget().size().height() - mpl_h, 0)
+            # 5. Resize the window and interactor to the correct size
             #    (not sure why, but this is required on macOS at least)
-            self._interactor.window_size = (win_sz.width(), win_sz.height())
+            self._interactor.window_size = (win_sz.width(), win_h)
             self._interactor.resize(ren_sz.width(), ren_sz.height())
             self._process_events()
             self._process_events()
@@ -513,9 +523,12 @@ def show(self):
 
 
 def _create_dock_widget(window, name, area):
-    dock = QDockWidget()
+    # create dock widget
+    dock = QDockWidget(name)
+    # add scroll area
     scroll = QScrollArea(dock)
     dock.setWidget(scroll)
+    # give the scroll area a child widget
     widget = QWidget(scroll)
     scroll.setWidget(widget)
     scroll.setWidgetResizable(True)


Test patch: