Don't draw bogus panels

This fixes freezing when the window is scaled below the sizes allowed
by the Window hints.
Now it's possible to scale the window down to 1x1 without issues.
This will definitely resolve other issues as well.