xiaoxu193/PyTeaser — pull request #30 by BorisVerk (2014-07-22 05:52:39+00:00)
Title: sped up Summarize() by about 8 times

Testing for membership in a list is linear time, while testing membership in a set in constant time. The draw back is that we introduce a one time linear overhead when converting a list to a set, but since we test for member ship multiple times for every word in an article, the conversion is worthwhile.

```
bv PyTeaser (master)$ python -m timeit -s "from pyteaser import Summarize; article_title = '{{title from test.py}}'; article_text = '{{283 words from test.py}}'" "Summarize(article_title, article_text)"
100 loops, best of 3: 16.2 msec per loop
bv PyTeaser (master)$ vi pyteaser.py #made the 5 character change
bv PyTeaser (master)$ git add ; git commit
bv PyTeaser (master)$ python -m timeit -s "from pyteaser import Summarize; article_title = '{{title from test.py}}'; article_text = '{{283 words from test.py}}'" "Summarize(article_title, article_text)"
1000 loops, best of 3: 1.88 msec per loop
```

Also minor readability fixes by using python's smarter />= statements.

Ps. I realize the majority of time this script spends its time on http requests, not summarization.