Add unlisteners to directionspanel events on elements