now able to parse tokenised corpus.