Fix skipped sync when affiliation is only change.

When adding a new affilitaion was the only change, the sync would be
skiped. This should fix that.