corrected and added a test for token offsets