chbrown/lexicons — pull request #1 by iandennismiller (2014-05-28 01:30:29+00:00)
Title: patch for LIWC dictionary, fix divide-by-zero during scaling, wait longer to truncate floats

Hi - I found your code to be really useful, and I just wanted to share a few little tweaks I needed to get the code running on my setup. I am not sending this pull request because I want you to change your repo or anything, but rather because I wanted to thank you, and in the off chance any of these tweaks are helpful, I wanted to share.

There are three major features to this pull:

- patch the LIWC dictionary: I found that my copy of the LIWC dictionary included a few inconsistencies that interfered with parsing, so I created a reusable process for transforming the dictionary prior to processing.
- fix divide by zero during scaling: the LIWC algorithm weights frequency by sentence length, but when the sentence is empty (i.e. length is 0), I need to ensure the result still consists of floats (i.e. the data type expected). Divide by zero results in non-float values. I added a simple test that ensures floats are always returned.
- wait longer to truncate floats: I needed access to the LIWC results as floating point values, so I refactored the step that formats the results (because it was returning truncated strings rather than floats).

I think these changes are zero-impact insofar as they should not break any existing interfaces.

Anyway, feel free to do whatever you want; pull or don't. Thanks for sharing your code in the first place.