sentence_prob function uses word iterator.
Main loop of perplexity comp. uses line iterator.
Not tested.