Revise scoring for end-to-end model