Don't endlessly refresh pillars in multi-master

Closes #13187