more robust token annotation support, allowing for corrections