Cancel listening to changes during teardown