libgit2/libgit2 — pull request #902 by pwkelley (2012-08-27 15:17:45+00:00)
Title: Clar reads test files as utf-8 (fixes win32)

The various test files under the clar/ directory are UTF-8 encoded, but have no byte order marker indicating that. I believe this is standard practice on UNIX. On Windows, Python tries to read the files in the system code page (for the United States this is code page 1252), a single-byte encoding. This causes the following error:

Loading test suites...
attr_file (4 tests)
attr_flags (3 tests)
attr_lookup (5 tests)
attr_repo (7 tests)
buf_basic (2 tests)
checkout_checkout (12 tests)
clone_clone (5 tests)
commit_commit (1 tests)
commit_parent (1 tests)
commit_parse (4 tests)
commit_signature (7 tests)
commit_write (2 tests)
config_add (2 tests)
config_multivar (5 tests)
config_new (1 tests)
config_read (15 tests)
config_stress (3 tests)
config_write (6 tests)
core_buffer (15 tests)
core_copy (3 tests)
core_dirent (6 tests)
Traceback (most recent call last):
File "clar", line 311, in
main()
File "clar", line 49, in main
print_mode = options.print_mode)
File "clar", line 86, in __init__
self._process_test_file(test_name, f.read())
File "C:\Python32\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 1009:
character maps to

This change modifies the test driver script to specify utf-8 explicitly, to avoid this problem.

Thanks,
Philip
Microsoft Corporation