preprocessing for Collobert & Weston model - data is from wikipedia dataset

Have problem stopping after N tokens, to do with yield control flow, currently stopping after N sentences