nltk/nltk — pull request #653 by arthurdarcet (2014-04-16 15:47:49+00:00)
Title: PunktSentenceTokenizer: accept the realign_boundaries argument in the span_tokenize method

Adapt `PunktSentenceTokenizer._realignboundaries` to work with slices instead of using the text directly

This also fix the failing test in the first commit: `"(Sent1. ) Sent2."` should be tokenized `['(Sent1. )', 'Sent2.']`, not `['Sent1.)', 'Sent2.']`.
Without this, the space between the dot and the closing parenthesis is missing in the tokenized sentences.