Limit scale_factor to target_scale_factor