Don't reveal a non-existant panel through panning

If you keep panning to close an open panel, you'll see the panel on
the opposite side. That is fine if you have both panels. If you
have only a left panel or a right panel, but not both, there's
nothing to see on the opposite side. We restrict the gesture now so
you can't see beyond the center panel unless you have panels set
on both sides.