Repository: ArthurSonzogni/FTXUI (C++)

Issue:
Unicode combining characters do not show up
It looks like unicode combining characters such as https://www.fileformat.info/info/unicode/char/0303/index.htm do not render correctly.

Modifying the example starter project as a test (see https://gist.github.com/retupmoca/68a4852a28b7cf6d2c39a90d7670212f for the full modified file), I added:
```
  // plain UTF-8 output of a random combining character:
  // https://www.fileformat.info/info/unicode/char/0303/index.htm
  std::cout << "done\xcc\x83:\n";

  auto summary = [&] {
    auto content = vbox({
        // converted UTF-8
        hbox({text(to_wstring(std::string("- done\xcc\x83:   "))), text(L"3") | bold}) | color(Color::Green),
        // UTF-16 / UCS-2
        hbox({text(L"- active\x0303: "), text(L"2") | bold}) | color(Color::RedLight),
        hbox({text(L"- queue:  "), text(L"9") | bold}) | color(Color::Red),
    });
    return window(text(L" Summary "), content);
  };
```

When I run the modified program, I see:
![image](https://user-images.githubusercontent.com/1424544/121401431-b8b8ef80-c926-11eb-9f2f-8fcfbc0f320b.png)

That is, the combining character renders correctly in my terminal if I directly output it, but displaying it via FTXUI does not appear to work.


Discussion:
Thanks for reporting this. I will take a look.
A ```ftxui::Pixel``` is:
```cpp
struct Pixel {
  wchar_t character = U' ';
  bool blink = false;
  bool bold = false;
  bool dim = false;
  bool inverted = false;
  bool underlined = false;
  Color background_color = Color::Default;
  Color foreground_color = Color::Default;
};
```

So, a single `wchar_t` is not enough to store combining characters.
I am actually quite surprised this fallback so nicely. ;-)

Not fully sure how to overcome this issue.
Note that the character you are using can also be represented via a single character:
```cpp
  auto document = text(L"- activẽ: ");
  auto screen = Screen::Create(Dimension::Full(), Dimension::Fit(document));
  Render(screen, document);
  screen.Print();
```
output
![Capture d’écran du 2021-06-09 21-12-32](https://user-images.githubusercontent.com/4759106/121414955-7f54a400-c967-11eb-89ce-167089deccd8.png)

This doesn't solve this issue, but this might solve your particular problem?
Yeah, most of the common graphemes are available as a single codepoint - it's not currently a major issue for me, but I wanted to report it for completeness :)
I added a note into the README.

At first glance, this seems difficult to support. I will try to learn more about this unicode feature. If its specification is not too insane, there might be a way to support "multiple character per pixels" feature.
In case it may help: https://github.com/magiblot/tvision/issues/26
> In case it may help: magiblot/tvision#26

This helps a lot! Sounds like you had the exact same problem and you solved it. Thank you!



Patch:
diff --git a/include/ftxui/screen/screen.hpp b/include/ftxui/screen/screen.hpp
index 791b55465..7d0a5dcd5 100644
--- a/include/ftxui/screen/screen.hpp
+++ b/include/ftxui/screen/screen.hpp
@@ -18,9 +18,16 @@ using Element = std::shared_ptr<Node>;
 /// @brief A unicode character and its associated style.
 /// @ingroup screen
 struct Pixel {
+  // The graphemes stored into the pixel. To support combining characters,
+  // like: a⃦, this can potentially contains multiple codepoitns.
+  // Required: character.size() >= 1;
+  std::wstring character = L" ";
+
+  // Colors:
   Color background_color = Color::Default;
   Color foreground_color = Color::Default;
-  wchar_t character = U' ';
+
+  // A bit field representing the style:
   bool blink : 1;
   bool bold : 1;
   bool dim : 1;
diff --git a/src/ftxui/dom/text.cpp b/src/ftxui/dom/text.cpp
index a8ade9aee..e8d24f609 100644
--- a/src/ftxui/dom/text.cpp
+++ b/src/ftxui/dom/text.cpp
@@ -29,10 +29,15 @@ class Text : public Node {
     if (y > box_.y_max)
       return;
     for (wchar_t c : text_) {
-      if (x > box_.x_max)
-        return;
-      screen.at(x, y) = c;
-      x += wchar_width(c);
+      const int width = wchar_width(c);
+      if (width >= 1) {
+        if (x > box_.x_max)
+          return;
+        screen.PixelAt(x, y).character = c;
+      } else {
+        screen.PixelAt(x - 1, y).character += c;
+      }
+      x += std::max(width, 0);
     }
   }
 
diff --git a/src/ftxui/screen/screen.cpp b/src/ftxui/screen/screen.cpp
index 8e514440d..92a76f6fa 100644
--- a/src/ftxui/screen/screen.cpp
+++ b/src/ftxui/screen/screen.cpp
@@ -164,17 +164,14 @@ std::string Screen::ToString() {
     }
     for (int x = 0; x < dimx_;) {
       auto& pixel = pixels_[y][x];
-      wchar_t c = pixel.character;
       UpdatePixelStyle(ss, previous_pixel, pixel);
 
-      auto width = wchar_width(c);
-      if (width <= 0) {
-        // Avoid an infinite loop for non-printable characters
-        c = L' ';
-        width = 1;
+      int x_inc = 0;
+      for (auto& c : pixel.character) {
+        ss << c;
+        x_inc += wchar_width(c);
       }
-      ss << c;
-      x += width;
+      x += std::max(x_inc, 1);
     }
   }
 
@@ -191,7 +188,7 @@ void Screen::Print() {
 /// @param x The character position along the x-axis.
 /// @param y The character position along the y-axis.
 wchar_t& Screen::at(int x, int y) {
-  return PixelAt(x, y).character;
+  return PixelAt(x, y).character[0];
 }
 
 /// @brief Access a Pixel at a given position.


Test patch:
diff --git a/src/ftxui/dom/text_test.cpp b/src/ftxui/dom/text_test.cpp
index 92f4b2317..8c397ffd0 100644
--- a/src/ftxui/dom/text_test.cpp
+++ b/src/ftxui/dom/text_test.cpp
@@ -6,6 +6,7 @@
 #include "ftxui/dom/node.hpp"       // for Render
 #include "ftxui/screen/box.hpp"     // for ftxui
 #include "ftxui/screen/screen.hpp"  // for Screen
+#include "ftxui/screen/string.hpp"  // for to_string
 #include "gtest/gtest_pred_impl.h"  // for Test, EXPECT_EQ, TEST
 
 using namespace ftxui;
@@ -86,6 +87,23 @@ TEST(TextTest, CJK_3) {
       screen.ToString());
 }
 
+TEST(TextTest, CombiningCharacters) {
+  const std::wstring t =
+      // Combining above:
+      L"ā à á â ã ā a̅ ă ȧ ä ả å a̋ ǎ a̍ a̎ ȁ a̐ ȃ a̒ a̔ a̕ a̚ a̛ a̽ a̾ a̿ à á a͂ a͆ a͊ a͋ a͌ a͐ "
+      L"a͑ a͒ a͗ a͘ a͛ a͝ a͞ a͠ a͡ aͣ aͤ aͥ aͦ aͧ aͨ aͩ aͪ aͫ aͬ aͭ aͮ aͯ a᷀ a᷁ a᷃ a᷄ a᷅ a᷆ a᷇ a᷈ a᷉ a᷾ a⃐ a⃑ a⃔ "
+      L"a⃕ a⃖ a⃗ a⃛ a⃜ a⃡ a⃩ a⃰ a︠ a︡ a︢ a︣"
+      // Combining middle:
+      L"a̴ a̵ a̶ a̷ a̸ a⃒ a⃓ a⃘ a⃙ a⃚ a⃝ a⃞ a⃟ a⃥ a⃦"
+      // Combining below:
+      L"a̗ a̘ a̙ a̜ a̝ a̞ a̟ a̠ a̡ a̢ ạ ḁ a̦ a̧ ą a̩ a̪ a̫ a̬ a̭ a̮ a̯ a̰ a̱ a̲ a̳ a̹ a̺ a̻ a̼ aͅ a͇ a͈ a͉ a͍ "
+      L"a͎ a͓ a͔ a͕ a͖ a͙ a͚ a͜ a͟ a͢ a᷂ a᷊ a᷿ a⃨";
+  auto element = text(t);
+  Screen screen(290, 1);
+  Render(screen, element);
+  EXPECT_EQ(to_string(t), screen.ToString());
+}
+
 // Copyright 2020 Arthur Sonzogni. All rights reserved.
 // Use of this source code is governed by the MIT license that can be found in
 // the LICENSE file.