Repository: mesonbuild/meson

Issue:
UnicodeEncodeError in mesonlib.do_conf_file()
```python
Traceback (most recent call last):
  File "/usr/lib/python3.5/site-packages/mesonbuild/mesonmain.py", line 283, in run
    app.generate()
  File "/usr/lib/python3.5/site-packages/mesonbuild/mesonmain.py", line 171, in generate
    intr.run()
  File "/usr/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 1336, in run
    self.evaluate_codeblock(self.ast, start=1)
  File "/usr/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 1358, in evaluate_codeblock
    raise e
  File "/usr/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 1352, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/usr/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 1473, in evaluate_statement
    return self.function_call(cur)
  File "/usr/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 2494, in function_call
    return self.funcs[func_name](node, self.flatten(posargs), kwargs)
  File "/usr/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 66, in wrapped
    return f(self, node, args, kwargs)
  File "/usr/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 2209, in func_subdir
    self.evaluate_codeblock(codeblock)
  File "/usr/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 1358, in evaluate_codeblock
    raise e
  File "/usr/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 1352, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/usr/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 1473, in evaluate_statement
    return self.function_call(cur)
  File "/usr/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 2494, in function_call
    return self.funcs[func_name](node, self.flatten(posargs), kwargs)
  File "/usr/lib/python3.5/site-packages/mesonbuild/interpreter.py", line 2257, in func_configure_file
    mesonlib.do_conf_file(ifile_abs, ofile_abs, conf.held_object)
  File "/usr/lib/python3.5/site-packages/mesonbuild/mesonlib.py", line 292, in do_conf_file
    f.writelines(result)
UnicodeEncodeError: 'ascii' codec can't encode character '\xa9' in position 13: ordinal not in range(128)
```


Discussion:
I'm hitting this bug too with LANG=C. Workaround is export LANG=en_US.utf8
The problem is that you're trying to write a unicode character to a file when using a non-unicode encoding. We can fix this by setting the `encoding` kwarg while writing files. However, this will break in other ways, such as when using files with unicode filenames.

What's the output of:

```
python3 -c 'import locale; print(locale.getpreferredencoding())'
```

With `LANG=C` that will return `ANSI_X3.4-1968` or some similar ansi encoding. I don't think we can do anything about a broken `LANG` setting except while writing/reading files.

Resolving patch:
diff --git a/meson.py b/meson.py
--- a/meson.py
+++ b/meson.py
@@ -14,10 +14,21 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from mesonbuild import mesonmain
-import sys, os
+from mesonbuild import mlog, mesonmain
+import sys, os, locale
 
 def main():
+    # Warn if the locale is not UTF-8. This can cause various unfixable issues
+    # such as os.stat not being able to decode filenames with unicode in them.
+    # There is no way to reset both the preferred encoding and the filesystem
+    # encoding, so we can just warn about it.
+    e = locale.getpreferredencoding()
+    if e.upper() != 'UTF-8':
+        mlog.warning('You are using {!r} which is not a a Unicode-compatible '
+                     'locale.'.format(e))
+        mlog.warning('You might see errors if you use UTF-8 strings as '
+                     'filenames, as strings, or as file contents.')
+        mlog.warning('Please switch to a UTF-8 locale for your platform.')
     # Always resolve the command path so Ninja can find it for regen, tests, etc.
     launcher = os.path.realpath(sys.argv[0])
     return mesonmain.run(launcher, sys.argv[1:])
diff --git a/mesonbuild/mesonlib.py b/mesonbuild/mesonlib.py
--- a/mesonbuild/mesonlib.py
+++ b/mesonbuild/mesonlib.py
@@ -303,7 +303,7 @@ def do_conf_file(src, dst, confdata):
     replace_if_different(dst, dst_tmp)
 
 def dump_conf_header(ofilename, cdata):
-    with open(ofilename, 'w') as ofile:
+    with open(ofilename, 'w', encoding='utf-8') as ofile:
         ofile.write('''/*
  * Autogenerated by the Meson build system.
  * Do not edit, your changes will be lost.


Test patch: