Repository: mozilla/pontoon

Issue:
Pretranslation issues with multiline strings
Pretranslation struggles to translate Fluent strings where line breaks are used to limit the line width, as it treats each line as a separate sentence.

```
rec-pw-1-2 =
  Make this password unique and different from any others you use.
  A good strategy to follow is to combine two or more unrelated
  words to create an entire pass phrase, and include numbers and symbols.
```

I don't think we can completely ignore line breaks, because there are some cases where these have an effect on display ([example](https://searchfox.org/mozilla-central/rev/169bf38e150667afac81ab73ef8b5ace8f1dfa8d/browser/locales/en-US/browser/downloads.ftl#244-248)). But maybe we can strip line breaks (i.e. replace them with a whitespace) when sending the translation to the machine translation engine, hoping that translators will catch edge cases when reviewing?

Cc @eemeli because that's a topic that we discussed at some point for the Fluent editor.


Discussion:
Should we [advise](https://mozilla-l10n.github.io/documentation/localization/dev_best_practices.html) against treating the line breaks as meaningful as in the said [example](https://searchfox.org/mozilla-central/rev/169bf38e150667afac81ab73ef8b5ace8f1dfa8d/browser/locales/en-US/browser/downloads.ftl#244-248)?

Line breaks are useful in resource files, but cause problems in tools like Pontoon. Not just when pretranslated, but also in the editor, where they are unnecessary and break lines at unexpected places.

I think tools should ignore them on import. And possibly even add them on serialization of long strings.
> Should we [advise](https://mozilla-l10n.github.io/documentation/localization/dev_best_practices.html) against treating the line breaks as meaningful as in the said [example](https://searchfox.org/mozilla-central/rev/169bf38e150667afac81ab73ef8b5ace8f1dfa8d/browser/locales/en-US/browser/downloads.ftl#244-248)?

We could, and ask them to use either `\n` or markup to signal the line break, or break it in multiple strings.

> And possibly even add them on serialization of long strings.

Are you suggesting to serialize strings with a max width (e.g. 120 characters)?

Resolving patch:
diff --git a/pontoon/pretranslation/pretranslate.py b/pontoon/pretranslation/pretranslate.py
--- a/pontoon/pretranslation/pretranslate.py
+++ b/pontoon/pretranslation/pretranslate.py
@@ -32,11 +32,15 @@ def visit_SelectExpression(self, node):
         return self.generic_visit(node)
 
     def visit_TextElement(self, node):
-        pretranslation, service = get_pretranslated_data(node.value, self.locale)
+        # Machine translation treats each line as separate sentence,
+        # hence we replace newline characters with spaces.
+        source = node.value.replace("\n", " ")
+
+        pretranslation, service = get_pretranslated_data(source, self.locale)
 
         if pretranslation is None:
             raise ValueError(
-                f"Pretranslation for `{node.value}` to {self.locale.code} not available."
+                f"Pretranslation for `{source}` to {self.locale.code} not available."
             )
 
         node.value = pretranslation


Test patch:
diff --git a/pontoon/pretranslation/tests/test_pretranslate.py b/pontoon/pretranslation/tests/test_pretranslate.py
--- a/pontoon/pretranslation/tests/test_pretranslate.py
+++ b/pontoon/pretranslation/tests/test_pretranslate.py
@@ -171,6 +171,37 @@ def test_get_pretranslations_fluent_whitespace(
     assert response == [(pretranslated_string, None, tm_user)]
 
 
+@pytest.mark.django_db
+def test_get_pretranslations_fluent_multiline(
+    fluent_resource, entity_b, locale_b, tm_user
+):
+    input_string = dedent(
+        """
+        multiline =
+            Multi
+            Line
+            Message
+    """
+    )
+    fluent_entity = EntityFactory(resource=fluent_resource, string=input_string)
+
+    # 100% TM match exists
+    tm = TranslationMemoryFactory.create(
+        entity=entity_b,
+        source="Multi Line Message",
+        target="TM: Multi Line Message",
+        locale=locale_b,
+    )
+
+    expected = f"multiline = {tm.target}"
+
+    # Re-serialize to match whitespace
+    pretranslated_string = serializer.serialize_entry(parser.parse_entry(expected))
+
+    response = get_pretranslations(fluent_entity, locale_b)
+    assert response == [(pretranslated_string, None, tm_user)]
+
+
 @patch("pontoon.pretranslation.pretranslate.get_google_translate_data")
 @pytest.mark.django_db
 def test_get_pretranslations_fluent_plural(