nltk/nltk — Issue #47: character encoding problem in UDHR corpus
Author: alexrudnick
Created: 2012-01-17 06:24:47+00:00
State: open

For two of the UDHR files, doing list(udhr.words(f)) produces the following decoding errors:

Arabic_Alarabia-Arabic 'charmap' codec can't decode byte 0xde in position 18: character maps to
<undefined>
Czech_Cesky-UTF8 'utf8' codec can't decode byte 0x8a in position 1: unexpected code byte

Migrated from http://code.google.com/p/nltk/issues/detail?id=235