Don't update bindings not affected by anchor layout algorithm

Fixes bug introduced in c739d422