dcramer/chardet — pull request #5 by ablegrape (2012-09-10 19:39:54+00:00)
Title: Fixed latin1 detector in chardet

Hi David,

Following up on our exchange from a few weeks ago. I've commited the bug fix for the integer math bug (causes a file with even one "low confidence" character to have 0 confidence level) and also updated the confidence multiplier for latin 1 to one that works better (the original multiplier causes many files to be incorrectly detected as iso-latin-2). Testing on both the problem case mentioned ( http://www.lvo.com/GASTRONOMIE/VINS/VITI/VITI1F.HTML ) and the character set tables from http://www.columbia.edu/kermit/csettables.html suggests that the fixed code performs better.

Hope this is useful - it's certainly helped fix a few problem documents in the application I'm working on.

Best,

Doug