Repository: vispy/vispy

Issue:
Fix grid solver allowing for 0 sized widgets and causing errors
As discussed in #2093 it seems there are random cases (hard to reproduce) that produce widgets in a Grid widget with 0 size. This results in the PanZoomCamera sometimes getting a scale of `0` which then fails later inside the STTransform/MatrixTransform work.


Discussion:
@djhoese Just guessing, a few lines below that change, there is a newly introduced if clause which compares a difference to be <1e-4. Could that cause this? Sorry, can't link the exact line atm. 
Thanks @kmuehlbauer. That will be the next thing I play with. I also see there are other places where the constraint is `>= 0`.
@djhoese Looks like you had the right feeling. :+1:
Oh, one is still running. 
Since this is intermittent, let's try rerunning it a couple times. This may be a solution if it passes a couple times (will have to wait for @almarklein to test locally since I still haven't been able to get it to fail here). I don't really like this solution though. It is simple and obvious, but I think it means that any checks in the Grid for "is the size of the widget 0? Ok, then don't draw it" which I think there is at least one instance of.

I think the `if` statement you mentioned was added to avoid floating point equality issues. So it may not be an issue actually.
So for the record `>= 1` passed once.

...twice.

...three time.
Just tested these changes with my script. Sorry, still fails.
But even with these fixes ... why does the height become so small in this example sometimes? ... mmm
> But even with these fixes ... why does the height become so small in this example sometimes? ... mmm

Yeah this is definitely a workaround and definitely not the root of the cause. I'm not sure if kiwisolver has some psuedo-random stuff going on inside that would make it provide different solutions to some constraints depending on this randomness. That's the only thing I can think of. Oh, except maybe a race condition between the size stuff and the window not being opened yet (and vispy's size properties being updated).

Can you tell if when it fails it is on the first ".scale" call from the PanZoomCamera or after a couple of the Visuals/components have been initialized?
Full traceback:
```
  File "c:\dev\pylib\vispy\vispy\app\canvas.py", line 436, in show
    self._backend._vispy_set_visible(visible)
  File "c:\dev\pylib\vispy\vispy\app\backends\_qt.py", line 425, in _vispy_set_visible
    self.showNormal()
  File "c:\dev\pylib\vispy\vispy\app\backends\_qt.py", line 526, in event
    out = super(QtBaseCanvasBackend, self).event(ev)
  File "c:\dev\pylib\vispy\vispy\app\backends\_qt.py", line 526, in event
    out = super(QtBaseCanvasBackend, self).event(ev)
  File "c:\dev\pylib\vispy\vispy\app\backends\_qt.py", line 846, in paintGL
    self._vispy_canvas.events.draw(region=None)
  File "c:\dev\pylib\vispy\vispy\util\event.py", line 453, in __call__
    self._invoke_callback(cb, event)
  File "c:\dev\pylib\vispy\vispy\util\event.py", line 469, in _invoke_callback
    cb(event)
  File "c:\dev\pylib\vispy\vispy\scene\canvas.py", line 218, in on_draw
    self._draw_scene()
  File "c:\dev\pylib\vispy\vispy\scene\canvas.py", line 267, in _draw_scene
    self.draw_visual(self.scene)
  File "c:\dev\pylib\vispy\vispy\scene\canvas.py", line 305, in draw_visual
    node.draw()
  File "c:\dev\pylib\vispy\vispy\scene\visuals.py", line 99, in draw
    self._visual_superclass.draw(self)
  File "c:\dev\pylib\vispy\vispy\visuals\visual.py", line 599, in draw
    if self._prepare_draw(view=self) is False:
  File "c:\dev\pylib\vispy\vispy\scene\widgets\grid.py", line 197, in _prepare_draw
    self._update_child_widget_dim()
  File "c:\dev\pylib\vispy\vispy\scene\widgets\grid.py", line 499, in _update_child_widget_dim
    widget.rect = Rect(x, y, width, height)
  File "c:\dev\pylib\vispy\vispy\util\frozen.py", line 17, in __setattr__
    object.__setattr__(self, key, value)
  File "c:\dev\pylib\vispy\vispy\scene\widgets\widget.py", line 250, in rect
    self.events.resize()
  File "c:\dev\pylib\vispy\vispy\util\event.py", line 453, in __call__
    self._invoke_callback(cb, event)
  File "c:\dev\pylib\vispy\vispy\util\event.py", line 471, in _invoke_callback
    _handle_exception(self.ignore_callback_errors,
  << caught exception here: >>
  File "c:\dev\pylib\vispy\vispy\util\event.py", line 469, in _invoke_callback
    cb(event)
  File "c:\dev\pylib\vispy\vispy\scene\cameras\panzoom.py", line 195, in viewbox_resize_event
    self.view_changed()
  File "c:\dev\pylib\vispy\vispy\scene\cameras\base_camera.py", line 437, in view_changed
    self._update_transform()
  File "c:\dev\pylib\vispy\vispy\scene\cameras\panzoom.py", line 291, in _update_transform
    self.tf_mat.matrix = self.transform.as_matrix().matrix
  File "c:\dev\pylib\vispy\vispy\visuals\transforms\linear.py", line 219, in as_matrix
    m.scale(self.scale)
  File "c:\dev\pylib\vispy\vispy\visuals\transforms\linear.py", line 445, in scale
    self.matrix = np.dot(self.matrix, scale)
  File "c:\dev\pylib\vispy\vispy\visuals\transforms\linear.py", line 401, in matrix
    self.shader_imap()
  File "c:\dev\pylib\vispy\vispy\visuals\transforms\linear.py", line 389, in shader_imap
    fn['inv_matrix'] = self.inv_matrix  # uniform mat4
  File "c:\dev\pylib\vispy\vispy\visuals\transforms\linear.py", line 407, in inv_matrix
    self._inv_matrix = np.linalg.inv(self.matrix)
  File "<__array_function__ internals>", line 5, in inv
  File "c:\pythons\python39\lib\site-packages\numpy\linalg\linalg.py", line 546, in inv
    ainv = _umath_linalg.inv(a, signature=signature, extobj=extobj)
  File "c:\pythons\python39\lib\site-packages\numpy\linalg\linalg.py", line 88, in _raise_linalgerror_singular
    raise LinAlgError("Singular matrix")
```

Look like its in the draw event. All visuals are created beforehand.
How many iterations until it failed with both sets of `>= 1` changes? Is that at least more iterations than without the changes?

Are you just running the scatter_histogram.py example in a loop? Or what does your script look like? I'll see if I can get it to fail.

A summary of what we're seeing:

1. We know that the kiwisolver change is the problem.
2. We know that something is resulting in a matrix that can't be inverted:

```python
matrix = np.array([[2.52820244e+01, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00],
       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00],
       [0.00000000e+00, 0.00000000e+00, 9.99999997e-07, 0.00000000e+00],
       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.00000000e+00]])

np.linalg.inv(matrix)
```

I assume this is caused by a scale of 0.

### Next Steps

I suppose the next step is to print out various things that the GridWidget is controlling (like sizes of the other widgets) and seeing what changes between iterations.

Edit:

Theoretically somewhere inbetween these steps:

```
  File "c:\dev\pylib\vispy\vispy\scene\widgets\grid.py", line 197, in _prepare_draw
    self._update_child_widget_dim()
  File "c:\dev\pylib\vispy\vispy\scene\widgets\grid.py", line 499, in _update_child_widget_dim
    widget.rect = Rect(x, y, width, height)
```
With your original changing of two constraints it *seemed* to take more iterations for it to fail, but current `main` now also seem to take more iters than before. Might just be the alignment of the moon or something.

With the additional two constraints changed, I ran over 800 iters without getting an error, so I'm pretty sure that fixes the issue (well, avoids the error, anyway).
I run basically the scatter example in a loop yes:
<details>

    import numpy as np
    
    import vispy.plot as vp
    
    iter = 0
    while True:
        iter += 1
        print(f"iter {iter}")
    
        np.random.seed(2324)
        n = 100000
        data = np.empty((n, 2))
        lasti = 0
        for i in range(1, 20):
            nexti = lasti + (n - lasti) // 2
            scale = np.abs(np.random.randn(2)) + 0.1
            scale[1] = scale.mean()
            data[lasti:nexti] = np.random.normal(size=(nexti-lasti, 2),
                                                loc=np.random.randn(2),
                                                scale=scale / i)
            lasti = nexti
        data = data[:lasti]
    
    
        color = (0.3, 0.5, 0.8)
        n_bins = 100
    
        fig = vp.Fig(show=False)
        line = fig[0:4, 0:4].plot(data, symbol='o', width=0,
                                face_color=color + (0.02,), edge_color=None,
                                marker_size=4)
        line.set_gl_state(depth_test=False)
        fig[4, 0:4].histogram(data[:, 0], bins=n_bins, color=color, orientation='h')
        fig[0:4, 4].histogram(data[:, 1], bins=n_bins, color=color, orientation='v')
    
        fig.show()
</details>
This is what the example looks like (normally). Those margins seem a little big to me.
![image](https://user-images.githubusercontent.com/3015475/123163286-d7b09900-d471-11eb-8ee1-c6de9e434f07.png)

Reminder that during this switch to kiwisolver one of the tests failed because suddenly `.padding` (and maybe `.margin`) were being respected.
Ok so I added a print on line 499 of grid.py with `print(x, y, width, height)` and I was able to get failures.

https://github.com/vispy/vispy/blob/89e83cabea4e0e6ae08df5012985e816009924aa/vispy/scene/widgets/grid.py#L498-L502

This is what I got on one run:

```
iter 66
102.0 41.0 456.9999999999982 297.99999999999994
102.0 41.0 456.99999999999966 20.0
70.50000000000001 41.0 34.25000000000001 297.99999999999994
iter 67
102.0 41.0 456.9999999999982 297.99999999999994
102.0 41.0 456.99999999999966 19.999999999999996
70.50000000000003 41.0 34.25 297.99999999999994
iter 68
102.0 41.0 456.9999999999982 297.99999999999994
102.0 41.0 456.99999999999875 20.0
70.50000000000001 41.0 34.25000000000001 297.99999999999994
iter 69
102.0 41.0 456.9999999999982 297.99999999999994
102.0 41.0 456.99999999999955 0.0
<traceback>
```

On my system, this happens from the `fig.show()` call. Note how the failure happens when the second `widget` is being updated (there is no third entry). Note that the last value (height) is `0.0`. Also notice that for the previous iterations the precision of some of the values changes. My guess is this is why that `if` statement @kmuehlbauer brought up had to be changed from a regular `!=` to `(y - x) > epsilon`.

I hope to end my day now so I probably won't look at this much more. Tomorrow...we finish this. :crossed_swords: 

Resolving patch:
diff --git a/vispy/scene/widgets/grid.py b/vispy/scene/widgets/grid.py
--- a/vispy/scene/widgets/grid.py
+++ b/vispy/scene/widgets/grid.py
@@ -462,19 +462,15 @@ def _update_child_widget_dim(self):
 
         # we only need to remove and add the height and width constraints of
         # the solver if they are not the same as the current value
-        if abs(rect.height - self._var_h.value()) > 1e-4:
-            # if self._height_stay:
-            #     self._solver.remove_constraint(self._height_stay)
-
+        h_changed = abs(rect.height - self._var_h.value()) > 1e-4
+        w_changed = abs(rect.width - self._var_w.value()) > 1e-4
+        if h_changed:
             self._solver.suggestValue(self._var_h, rect.height)
-            # self._height_stay = self._solver.add_stay(self._var_h | 'strong')
-
-        if abs(rect.width - self._var_w.value()) > 1e-4:
-            # if self._width_stay:
-            #     self._solver.remove_constraint(self._width_stay)
 
+        if w_changed:
             self._solver.suggestValue(self._var_w, rect.width)
-            # self._width_stay = self._solver.add_stay(self._var_w | 'strong')
+        if h_changed or w_changed:
+            self._solver.updateVariables()
 
         value_vectorized = np.vectorize(lambda x: x.value())
 


Test patch: