Ensure ensembles reconfigure before nodes exit

See specific commit messages for more details