An optimization which only adds path labels when needed