Do override configs still go through the normal config validation system? If so, we could move this check there.