robmcmullen/peppy — Issue #865: file load failed on fundamental mode when encoding not ascii and not marked
Author: robmcmullen
Created: 2012-02-21 06:15:58+00:00
State: open
Labels: fundamental

*[Trac time 20100909 233938Z]* Sample file with non-ascii characters (see attached) fails to open with:
```
Traceback (most recent call last):
File "/data2/home/rob/src/peppy/peppy/fileopener.py", line 176, in determineMajorMode
self.scanBufferForMajorMode()
File "/data2/home/rob/src/peppy/peppy/fileopener.py", line 199, in scanBufferForMajorMode
self.modecls = MajorModeMatcherDriver.match(self.buffer, url=self.url)
File "/data2/home/rob/src/peppy/peppy/majormodematcher.py", line 192, in match
mode = cls.matchFile(buffer, magic_size, url, header)
File "/data2/home/rob/src/peppy/peppy/majormodematcher.py", line 273, in matchFile
likely = cls.scanLanguage(header, modes)
File "/data2/home/rob/src/peppy/peppy/majormodematcher.py", line 532, in scanLanguage
how_likely = mode.verifyLanguage(header)
File "/data2/home/rob/src/peppy/peppy/fundamental.py", line 237, in verifyLanguage
if line.startswith(cls.start_line_comment):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 97: ordinal not in range(128)
```
but if the filename is renamed .txt instead of .dot, it works.