Repository: kivy/kivy

Issue:
kivy isn't able to display certain unicode characters properly
To reproduce run the following code

```
# -*- coding: utf-8 -*-
from kivy.uix.textinput import TextInput
from kivy.base import runTouchApp
runTouchApp(TextInput(text=u'Hello éé çç तेरा मेरा'))
```



Discussion:
Ok, after deeper analysis, it appear that nothing is wrong on our side. No ttf that we are shipping is handling theses characters. If i use DevanagariMT.ttf, characters are rendered.

So i came to the conclusion that OS is building its own character table, and might use multiple font to be able to render all the characters. Dunno how to fit that behavior into kivy, specially because we are relying on underlaying fonts. I'll put is in standby, need more reflexion.

One could use the kitchen sink approach, one font with all the code-pages (like the GNUuni font  http://unifoundry.com/unifont.html), or one can always use a font that includes the characters that fits their use-case, (eg I'm currently using FreeSans.ttf).

I think the OS builds the missing code pages with methods like font-linking, where the missing code-pages in the font when accessed are replaced by the fonts that do have those code-pages.

There are many fonts that are created keeping this in mind and include only the code-page/s for a specific language/set of languages example indic.ttf, as it stand right now these font's cant be used in kivy for getting all the chars, so one has to find a font that has all the char-set that they are looking for. That's good enough right now but it would be even better if kivy supported these kind of fonts.

A bolder approach would be to ask the platform's graphics provider to render the line, e.g. use Quartz 2D on OS X and WPF on Windows. This way the platform can substitute glyphs from other fonts, plus do kerning, right-to-left scripts, and a few other typographical touches.

Actually, this is what we're doing. It's mostly depending from the backend, cairo/pygame/... are not doing that kind of stuff, while cocoa would do for example.
Even if i partially agreed about we could rely on platform, it would be harder to maintain that using external project (because all the platform specific code need platform specific toolchain in most case). Using an external lib is less pain.

If FreeSans.ttf could show all the characters... let's try to use it.

akshayaurora, could you do that test in kivy ?

FreeSans.ttf has the characters I need but not all, GNUuni font on the other hand has all the possible character sets and some more however it's 15.6 mb in size . [because of that trying to include it in kivy raises a IOError:
http://paste.kde.org/189074/]   scratch that, renaming the font works. It displays everything.
This is the code I'm using to test, it should help us see if most of the major languages are rendered properly.
http://paste.kde.org/189080/

On 19/01/2012 13:32, qua-non wrote:

> FreeSans.ttf has the characters I need but not all, GNUuni font on the other hand has all the possible character sets and some more however it's 15.6 mb in size because of that trying to include it in kivy raises a IOError:
> http://paste.kde.org/189074/
> This is the code I'm using to test, it should help us see if most of the major languages are rendered properly.
> http://paste.kde.org/189080/
> 
> ---
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/kivy/kivy/issues/332#issuecomment-3561979

Even my webbrowser can't see all the chars (Kannada, Telugu).
What's the best approach so?
1. Embed one font that cover everything, but will increase the size of 
   package, and change the default styling...
2. Write documentation about how to use another unicode font that could 
   render all the chars?
3. Write a layer that detect and use a default font for char not 
   supported in the first one ?

Right now, i would prefer to go to 2., for performance and size reason. 
Note that later, when theming will be implemented in Kivy, we could 
replace the default font everywhere more easily than today.

Mathieu

On Thu, Jan 19, 2012 at 7:15 PM, Mathieu Virbel <
reply@reply.github.com

> wrote:
> 
> On 19/01/2012 13:32, qua-non wrote:
> 
> > FreeSans.ttf has the characters I need but not all, GNUuni font on the
> > other hand has all the possible character sets and some more however it's
> > 15.6 mb in size because of that trying to include it in kivy raises a
> > IOError:
> > http://paste.kde.org/189074/
> > This is the code I'm using to test, it should help us see if most of the
> > major languages are rendered properly.
> > http://paste.kde.org/189080/
> > 
> > ---
> > 
> > Reply to this email directly or view it on GitHub:
> > https://github.com/kivy/kivy/issues/332#issuecomment-3561979
> 
> Even my webbrowser can't see all the chars (Kannada, Telugu).

Same here.

What's the best approach so?

> 1. Embed one font that cover everything, but will increase the size of
>    package, and change the default styling...
> 2. Write documentation about how to use another unicode font that could
>    render all the chars?
> 3. Write a layer that detect and use a default font for char not
>    supported in the first one ?
> 
> Right now, i would prefer to go to 2., for performance and size reason.

That makes sense as even if the huge font is included, it might not please
every-one as the glyphs might not be aesthetically pleasing. Providing the
documentation makes perfect sense in this case.

Note that later, when theming will be implemented in Kivy, we could

> replace the default font everywhere more easily than today.

Themes would be interesting :).

> Mathieu
> 
> ---
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/kivy/kivy/issues/332#issuecomment-3562738


Resolving patch:
diff --git a/kivy/uix/label.py b/kivy/uix/label.py
--- a/kivy/uix/label.py
+++ b/kivy/uix/label.py
@@ -132,8 +132,14 @@ def texture_update(self, *largs):
 
     .. warning::
 
-        Depending of your text provider, the font file can be ignored. But you
-        can mostly use this without trouble.
+        Depending of your text provider, the font file can be ignored. However,
+        you can mostly use this without trouble.
+
+        If the font used lacks the glyphs for the perticular language/symbols
+        you are using, you will see '[]' blank box characters instead of the
+        actual glyphs. The solution is to use a font that has the glyphs you
+        need to display. For Example to display 'क', use a font like
+        freesans.ttf that has the glyph.
 
     :data:`font_name` is a :class:`~kivy.properties.StringProperty`, default to
     'fonts/DroidSans.ttf'.
diff --git a/kivy/uix/textinput.py b/kivy/uix/textinput.py
--- a/kivy/uix/textinput.py
+++ b/kivy/uix/textinput.py
@@ -1192,8 +1192,14 @@ def _set_text(self, text):
 
     .. warning::
 
-        Depending of your text provider, the font file can be ignored. But you
-        can mostly use this without trouble.
+        Depending of your text provider, the font file can be ignored. However
+        you can mostly use this without trouble.
+
+        If the font used lacks the glyphs for the perticular language/symbols
+        you are using, you will see '[]' blank box characters instead of the
+        actual glyphs. The solution is to use a font that has the glyphs you
+        need to display. For Example to display 'क', use a font like
+        freesans.ttf that has the glyph in it.
 
     :data:`font_name` is a :class:`~kivy.properties.StringProperty`, default to
     'fonts/DroidSans.ttf'.


Test patch: