Repository: python-pillow/Pillow

Issue:
ImageFont and non latin characters in a path to font
### What did you do?

```
>>> from PIL.ImageFont import truetype
>>> import os.path
>>> fname = 'C:\\temp\\Шрифты\\font.ttf'
>>> os.path.exists(fname)
True
>>> truetype(fname)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\jenyay\.virtualenvs\outwiker-lFR0QK22\lib\site-packages\PIL\ImageFont.py", line 261, in truetype
    return FreeTypeFont(font, size, index, encoding, layout_engine)
  File "C:\Users\jenyay\.virtualenvs\outwiker-lFR0QK22\lib\site-packages\PIL\ImageFont.py", line 144, in __init__
    self.font = core.getfont(font, size, index, encoding, layout_engine=layout_engine)
OSError: cannot open resource
```

If `fname` contains only Latin characters, then everything works.

### What versions of Pillow and Python are you using?

Pillow 5.0.1
Windows 7, Windows 10.
In Linux is OK.


Discussion:
Can't reproduce on MacOS either.
Info from duplicate https://github.com/python-pillow/Pillow/issues/3480: 

Reproducible with Chinese characters with:

* OS: Windows 10 Pro 1803
* Python: 3.6.5
* Pillow: 5.3.0

But not in CentOS.


Looks like FreeType's `FT_New_Face` doesn't support Unicode paths on Windows.

---

`core.getfont` is `getfont` in `_imagingft.c` (making it hard to debug on Windows).

`getfont` can raise if `error` is set, and they're all return values from `FT_` calls: `FT_New_Face`, `FT_New_Memory_Face`, `FT_Set_Pixel_Sizes`, etc.

https://github.com/python-pillow/Pillow/blob/080bfd3ee1412b401d520fe26c51e2f5515e3a65/src/_imagingft.c#L268-L302

And "cannot open resource" is a FreeType error:
```c
  FT_ERRORDEF_( Cannot_Open_Resource,                        0x01,
                "cannot open resource" )
```
https://github.com/aseprite/freetype2/blob/ae3afbc47184db5c7753318b36237af261c66b09/include/freetype/fterrdef.h#L61-L62

`FT_New_Face` is most likely, as it passes in the `filename`.

---

From https://stackoverflow.com/a/10075289/724176:

> Since freetype is meant to be portable, it only uses fopen, which can only accept const char* filenames. So, either load file to memory (or memory map it) and then use FT_New_Memory_Face to create font or convert wchar_t pszFileName into 8-bit encoding, potentially losing characters due to impossible conversion.
>
> **On linux**, you could attempt to use setlocale so fopen will accept UTF8 strings, convert wchar_t string to UTF8. However **on windows** [it won't work](http://msdn.microsoft.com/en-us/library/x99tb11d%28v=vs.90%29.aspx). So either load file to memory, or convert pszFileName to 8-bit encoding, then pass it to FT_New_Face.

Both of these are similar issues and contain suggestions to load the file and instead pass it to `FT_New_Memory_Face`
* https://freetype.nongnu.narkive.com/XIf73fcO/ft-ft-new-face
* https://github.com/Robmaister/SharpFont/issues/89

---

And Pillow already does something like this. `ImageFont.truetype`'s `font` parameter:

> *font* – A filename or file-like object containing a TrueType font. Under Windows, if the file is not found in this filename, the loader also looks in Windows `fonts/` directory.

If a path is passed in, it uses `FT_New_Face`. If bytes, it uses `FT_New_Memory_Face`.

---

I've not tested this on Windows (and it still works on Mac), please try something like this:

```python
from PIL import ImageFont
from io import BytesIO

FONT_PATH = "方正黄草简体.ttf"

def open_filename(filename):
    font = ImageFont.truetype(filename)
    return font

def open_data(filename):
    with open(filename, "rb") as f:
        font_bytes = BytesIO(f.read())
    font = ImageFont.truetype(font_bytes)
    return font

# print(open_filename(FONT_PATH))
print(open_data(FONT_PATH))
```

Does that help?

One of the threads above suggests not to use `FT_New_Memory_Face` by default, as it loads the whole font into RAM rather than a single font, so we wouldn't want to do that in all cases.

However, perhaps it could be done on Windows when the filename contains Unicode/non-ASCII.

@hugovk , it works on my Windows machine. Thanks~

Resolving patch:
diff --git a/src/PIL/ImageFont.py b/src/PIL/ImageFont.py
--- a/src/PIL/ImageFont.py
+++ b/src/PIL/ImageFont.py
@@ -140,13 +140,26 @@ def __init__(self, font=None, size=10, index=0, encoding="",
 
         self.layout_engine = layout_engine
 
+        def load_from_bytes(f):
+            self.font_bytes = f.read()
+            self.font = core.getfont(
+                "", size, index, encoding, self.font_bytes, layout_engine)
+
         if isPath(font):
+            if sys.platform == "win32":
+                font_bytes_path = font if isinstance(font, bytes) else font.encode()
+                try:
+                    font_bytes_path.decode('ascii')
+                except UnicodeDecodeError:
+                    # FreeType cannot load fonts with non-ASCII characters on Windows
+                    # So load it into memory first
+                    with open(font, 'rb') as f:
+                        load_from_bytes(f)
+                    return
             self.font = core.getfont(font, size, index, encoding,
                                      layout_engine=layout_engine)
         else:
-            self.font_bytes = font.read()
-            self.font = core.getfont(
-                "", size, index, encoding, self.font_bytes, layout_engine)
+            load_from_bytes(font)
 
     def _multiline_split(self, text):
         split_character = "\n" if isinstance(text, str) else b"\n"


Test patch:
diff --git a/Tests/test_imagefont.py b/Tests/test_imagefont.py
--- a/Tests/test_imagefont.py
+++ b/Tests/test_imagefont.py
@@ -7,6 +7,7 @@
 import sys
 import copy
 import re
+import shutil
 import distutils.version
 
 FONT_PATH = "Tests/fonts/FreeMono.ttf"
@@ -131,6 +132,15 @@ def test_font_with_open_file(self):
         with open(FONT_PATH, 'rb') as f:
             self._render(f)
 
+    def test_non_unicode_path(self):
+        try:
+            tempfile = self.tempfile("temp_"+chr(128)+".ttf")
+        except UnicodeEncodeError:
+            self.skipTest("Unicode path could not be created")
+        shutil.copy(FONT_PATH, tempfile)
+
+        ImageFont.truetype(tempfile, FONT_SIZE)
+
     def _render(self, font):
         txt = "Hello World!"
         ttf = ImageFont.truetype(font, FONT_SIZE,