Add basic Tokenizer
Add simple classifier
Fix test name
Use language name as hash key
Add sample gathering class
Dump classifier results
Don't crash if classifier data is missing
Improve tokenizing sgml tags
Rebuild classifier data
Merge branch 'master' into bayesian
Rebuild classifier data
GC classifier db
Rebuild classifier data
Recognise that PEG.js-generated parsers are in fact generated
accidental hard tabs
Merge pull request #180 from michaelficarra/pegjs

Recognise that PEG.js-generated parsers are in fact generated
Merge branch 'master' into bayesian
More tokenizer docs
Refactor tokenizer test helper
Doc sample class