Don't allow swiping the side panels over the textfield

Ref: #15 - Improve side panels