Ignore click events when ctrl is pressed

These are handled by the mousedown listener instead

Closes #184