Repository: WeblateOrg/weblate

Issue:
Use of soft hyphen for better automatic line break control in translations
Is there a good way to deal with soft hyphen in weblate? (Or any other recommendation?)


Discussion:
The only related thing Weblate does right now is to check for existence of [zero width space](https://docs.weblate.org/en/latest/user/checks.html#zero-width-space).

Can you please clarify what are you looking for?
It's about resulting visualization of a translation in application's UI. E.g.
- English: Display Settings
- German: Anzeigeeinstellungen

Per it's nature, English is not dealing with very long words, constructed and concatenated as it is possible in german. Therfore in some places the according visualization widget needs to break the text into two lines. What works well for english above, since there two words are separated with white space, resulting in line 1: Display, line 2: Settings.

Whereas for german, the out of box result would be something like: line 1: Anzeigeinstel, line 2: lungen. Without a hyphen, just arbitrarily wrapped. 

With soft hyphen this could be controlled better.
- HTML style: `Anzeige&shy;einstellungen`
- Result: 
  - line 1: Anzeige-
  - line 2: einstellungen

Question is whether from weblate point of view there is a way to deal with this.
Drawback is, soft hyphen is technology specific (html5, xml, ...)
There is nothing what would prevent translators use it right now. I don't see what specific support would be needed from Weblate side here.
The problem starts with editing, how a soft hyphen could be entered. There is a unicode character for this: U+00AD, see also https://de.wikipedia.org/wiki/Weiches_Trennzeichen

In windows you can "write" this sign by press+hold : ALT + 0173. The following is an example where after each word a soft hyphen is entered:

This­Is­A­Very­Long­Sentence­Where­After­Each­Word­A­Soft­Hyphen­Is­Entered­So­That­By­Exceeding­A­Line­It­Automatically­Breaks­At­The­Desired­Position­And­Renders­A­Hyphen­Accordingly.

Drawback is that this is a non-displayable character unless we come to a line end where at the position of a soft hyphen the line breaks, and only then this character becomes visible as a hyphen.

For editing in weblate it could be helpful to treat soft hyphen specially, as:

![shy-editor](https://user-images.githubusercontent.com/8058166/64759698-3d02f680-d538-11e9-9a4c-30c021aee2e8.png)

So that we have besides the special char buttons an additional one for soft hyphen (shy). When pressing this button and only the unicode char is injected this still is invisible. So may to enrich the input field with a special visualization for the soft hyphen char.

Textually, this is sufficient. What I don't know is whether soft hyphen needs to be treated differently, depending on destination technology (to handle accordingly in importer/exporter) as for e.g. json, html, xml, xliff, resx, ...
Again, this is to start a discussion. I'm not fully clear whether this is really needed.
Pootle uses a web font called “Raw” to make invisible Unicode formatting marks, like SHY and NBSP, explicit in the editor.
Related to https://github.com/WeblateOrg/weblate/issues/1930

Custom font should be quite easy to implement, but probably would not help with double space and more complex issues. The good thing is that we could the [Pootle Raw font](https://github.com/translate/pootle/tree/master/pootle/static/fonts).

I was always considering bringing in more complex editor which would handle highlighting such things and possibly also help with editing format strings and markup.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Resolving patch:
diff --git a/weblate/trans/specialchars.py b/weblate/trans/specialchars.py
--- a/weblate/trans/specialchars.py
+++ b/weblate/trans/specialchars.py
@@ -273,6 +273,7 @@ def get_special_chars(language, additional="", source=""):  # noqa: C901
     yield get_quote(code, ALT_CLOSE, gettext("Alternative closing quote"))
 
     if code in HYPHEN_LANGS:
+        yield format_char("\u00AD")
         yield gettext("Hyphen"), "‐", "‐"
 
     if code in EN_DASH_LANGS:
diff --git a/weblate/trans/templatetags/translations.py b/weblate/trans/templatetags/translations.py
--- a/weblate/trans/templatetags/translations.py
+++ b/weblate/trans/templatetags/translations.py
@@ -67,7 +67,7 @@
 
 # This should match whitespace_regex in weblate/static/loader-bootstrap.js
 WHITESPACE_REGEX = (
-    r"(\t|\u00A0|\u1680|\u2000|\u2001|\u2002|\u2003|"
+    r"(\t|\u00A0|\u00AD|\u1680|\u2000|\u2001|\u2002|\u2003|"
     r"\u2004|\u2005|\u2006|\u2007|\u2008|\u2009|\u200A|"
     r"\u202F|\u205F|\u3000)"
 )


Test patch:
diff --git a/weblate/trans/tests/test_specialchars.py b/weblate/trans/tests/test_specialchars.py
--- a/weblate/trans/tests/test_specialchars.py
+++ b/weblate/trans/tests/test_specialchars.py
@@ -23,11 +23,11 @@ def check_chars(self, language, count, matches, *args, **kwargs):
 
     def test_af(self):
         chars = list(get_special_chars(Language(code="af")))
-        self.assertEqual(len(chars), 11)
+        self.assertEqual(len(chars), 12)
 
     def test_cs(self):
         chars = list(get_special_chars(Language(code="cs")))
-        self.assertEqual(len(chars), 10)
+        self.assertEqual(len(chars), 11)
 
     def test_brx(self):
         chars = list(get_special_chars(Language(code="brx")))
@@ -40,17 +40,17 @@ def test_brx_add(self):
     @override_settings(SPECIAL_CHARS=[chr(x) for x in range(256)])
     def test_settings(self):
         chars = list(get_special_chars(Language(code="cs")))
-        self.assertEqual(len(chars), 262)
+        self.assertEqual(len(chars), 263)
 
     def test_additional(self):
         self.check_chars(
-            Language(code="cs"), 14, ["a", "h", "o", "j"], additional="ahoj"
+            Language(code="cs"), 15, ["a", "h", "o", "j"], additional="ahoj"
         )
 
     def test_arrows(self):
-        self.check_chars(Language(code="cs"), 12, ["→", "⇒"], source="→⇒→⇒")
+        self.check_chars(Language(code="cs"), 13, ["→", "⇒"], source="→⇒→⇒")
 
     def test_arrows_rtl(self):
         self.check_chars(
-            Language(code="ar", direction="rtl"), 13, ["←", "⇐"], source="→⇒→⇒"
+            Language(code="ar", direction="rtl"), 14, ["←", "⇐"], source="→⇒→⇒"
         )