PanelView: hide when appropriate instead of just making non-interactive

This ensures that those side highlight bars won't be visible when
there's for example a dialog open on top
Make BufferDelegate less prone for warnings