Ensure callbacks are not called when nullifying adjustment originators

Fixes #3964