Add deregister function to PortEventListener

I have done a little bit of testing of this but it probably needs a bit of a review to make sure that it doesn't break things.