Solved the instant jump problem!

Seams like the animation handler goes from the top right to the bottom
left, so when moving down or left it would run through the animation
sequence instantly. Added a guard to make sure it only moves once per
animation cycle.