Consolidated token annotations and language id

as discussed in email