initial commit
rename because code starts with the same letter as caroline so we wouldn't be able to lazily tab into that folder :D
setting up new java project. this is a whole new world (a new fantastic point of view)
setup.
Jeff did good things
caroline hates java with the passion of a thousand burning suns at the moment. anyhow, the program now reads in files and spews out POS (parts of speech, mind) and stemmed data via System.out.println
merge yo
adding forgotten files. also gitignore
with nifty, we had QBDBCB. here, we have LBTB, woot! (added skeleton for token/base and labeler/base classes woot
Jeff did lots of things (don't mind preprocessingtest directory :S)
HTML Stripping and Stoplist Removal should be working, feel free to test
changes from the past weekend that I forgot
Merge branch 'master' of github.com:snap-stanford/MetroMaps
Jeeef
Merge branch 'master' of github.com:snap-stanford/MetroMaps
Feeds POSLabeler and StemLabeler into TokenBase! Finished up basic implementation for TokenBase.
took nap. added important words (just first sentence right now b/c no way to determine paragraph. got rid of all warnings because they were driving me nuts
output to text files now working! test run on all 18k data points. this is gonna take a while.. :)
bug fix so it doesn't generate a 100GB output file ^^
Significant phrases base code, plus dat sexy LingPipe library