added code for tokenizing
counting tokens
done first exploratory analysis
fixed
using ngram packae
using ngram instead of weka

performance baseline for file  en_US.twitter.10.txt : 10 s on home PC
using tokenie -> 6s
added prob calculation
fixed
Merge branch 'master' of github.com:behrica/ds-capstone