Not found a good string locality hash; but persuing alt line of attack; identifying popular retweets
and then matching up userids and times and only then looking for text similarity; will likely
use greatest common substring, note: not subsequence.
Some progress...getting there.
Adding support for filtering out adhoc retweets; need to double-check twitter month abbreviations;
will for now first consider time and userid/name then text for adhoc retweets; will have to eyeball
some and refine.
Progress...
More progress...
Progress...can combine 2 major functions into one to avoid 2 passes over bz2 file; will
do that soon and start testing on small runs, then larger.