chrislo/sourceclassifier — pull request #1 by samsquire (2014-01-19 09:33:43+00:00)
Title: Keep getting invalid utf-8 sequence errror when training from shootout.

Encountered this error when trying to train from a CVS shootout checkout:
invalid byte sequence in UTF-8

Perhaps some of the shootout files have different encodings? It seems to error on the files containing symbols.

This is a workaround to force to force encoding then convert into utf-8.

From https://stackoverflow.com/questions/9607554/ruby-invalid-byte-sequence-in-utf-8