Ensure event handlers are destroyed on unmount.

These can be left over if, for instance, the draggable item has a 'close' button which
closes on mousedown.