be specific about which cases to stop listening to `change` event

We were turning off listening to the change event so that we didn't set up a new event listener every time we called this method without turning them off, but this was having a side-effect of turning off the change event for EVERYTHING. This gets specific about when to stop listening.