sazlin/reTOracle — pull request #60 by xbobo (2014-07-24 10:18:55+00:00)
Title: MLalgos

Couldn't sleep tonight, but made some progress on the algos:

Unsupervised - added an additional component to normalize the PMI sentiment orientation score. It is now MUCH more accurate at least according to the eyeball test. More thorough testing is difficult for now as google shuts down api access after a series of queries in short succession.

Supervised - rewrote large portions of the code, now optimized with pandas and numpy and runs significantly faster. I also realized that the data in our test samples are almost always biased one way or the other. It now manually parses the data to grab an even number of positive and negative cases. Unfortunately, as expected this lowers the accuracy to the low 70's. Will work on more thorough feature engineering tomorrow