Only use gestures on main panel

Change-Id: I8f1e4402f84ce06275ef77978e0496db5d920f34