Before revision:  This simple modification, when combined with a novel mention clustering algorithm for selecting which examples to label, is extremely cost-efficient in terms of the performance obtained per annotation budget.
After revision:  This simple modification, when combined with a novel mention clustering algorithm for selecting which examples to label, is much more efficient in terms of the performance obtained per annotation budget.