coerce text to correct encoding in tokenize() by using text.decode('ISO-8859-1')
tested langid.classify() to match language and confidence values against those generated by the python2 version
no testing of other options (server option not working)