fixed token compaction bug which showed up when blocks had no tokens but some punctuation