rhiever/reddit-analysis — pull request #8 by bboe (2013-03-02 22:58:40+00:00)
Title: Add threshold and single_occurance filtering.

The `parseText` `threshold` argument indicates that words are only added that appear below a certain relative threshold to prevent word spamming. I have this value set by default to `0.34` which I think is still pretty high.

The `single_occurence` argument when True indicates that the word count is only incremented at most once per text block. I wanted to set it to default True, however, that fairly dramatically changes the behavior. At some point it'd be good to make toggling that setting a command line option.