Reorder and unify the config's stat modifier handling