Updated scheme, also added processed bool
Rehash is queued processing. Basic implementation.

Using the workers gem to provide functionality with jruby. Currently
modifiying it to do what I want it to.
Dropped jruby idea, basic functionality in place
Changed queries to get some better performance
Bugfix for nonunique words in sentences
Pool still had 20 workers, now defaults to 1
Added argument, first one is # threads
major bugfix for severing punctuation

There was a pretty huge bug where marko's thread that was processing
a line would just croak. I couldn't pinpoint it but knew it was
logging. Well, I found it occured when processing [.!?,]+ due to my
scan function. I've fixed it with a workaround.
Now limits based on threads, rate estimations too

Now longer printing "done" and just printing an estimate of rate
Chains weren't being inserted
pg connections are now done on the worker
exec_prepared was slow, now joining INSERT VALUES

Which is disgustingly faster.
Improved summary stats, cleaned up some
Broke random speech, but fast metadata logging
Added widhash function to simplify word => wid
Reimplemented random speech, fixed say

Random speech is now done separately from logging and processing. This
gives a decent speed boost to both of those two processes. However,
there's still a concern with the count(*)'s going on in speech
generation that are slowing everything down.
Added !queue
Some text like "\"" or "" would never process.

I added a catch to mark these as "good" and go on with life.
Updated schema, moved sever to its own file