I changed the method for zipping Marc’s tokenizer and the NLTK tokenizer for better matching, but they’re still not perfect.  Currently, when running the index_sentences() method from token_classifier_new.py, the tokenized sentences with mismatched length are printed out so it’s obvious where the tokenizations differ.