robmcmullen/peppy — Issue #872: error on opening utf-8 files containing characters with values beyond 0x7f
Author: robmcmullen
Created: 2012-02-21 06:16:45+00:00
State: open
Labels: UNCATEGORIZED

**[reported by alefcant@...]** *[Trac time 20110111 113114Z]* any utf file with BOM or containing characters such à,è, etc. cause the following error:

Traceback (most recent call last):
File "~/peppy-0.15.0/peppy/fileopener.py", line 176, in determineMajorMode
self.scanBufferForMajorMode()
File "~/peppy-0.15.0/peppy/fileopener.py", line 199, in scanBufferForMajorMode
self.modecls = MajorModeMatcherDriver.match(self.buffer, url=self.url)
File "~/peppy-0.15.0/peppy/majormodematcher.py", line 192, in match
mode = cls.matchFile(buffer, magic_size, url, header)
File "~/peppy-0.15.0/peppy/majormodematcher.py", line 273, in matchFile
likely = cls.scanLanguage(header, modes)
File "~/peppy-0.15.0/peppy/majormodematcher.py", line 532, in scanLanguage
how_likely = mode.verifyLanguage(header)
File "~/peppy-0.15.0/peppy/fundamental.py", line 237, in verifyLanguage
if line.startswith(cls.start_line_comment):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 7: ordinal not in range(128)

e.g.
-------------------------------
#!/usr/bin/env python
# -*- coding: utf-8 -*-

print("è")
-------------------------------

BTW I'm downloading the sources via svn to check if the problem is still there