don't rebuild the whole point set modifier when deleting a point