Only remove arrow keys in normal mode
ack ack ack