Fix event handler leak

I noticed this while reading the code and have not tested it, but I think the wrong EventEmitter object was getting the `removeListener` call.