Repository: WeblateOrg/weblate

Issue:
Spellchecker doesn't work with new editor
`<textarea style="position: absolute; bottom: -1em; padding: 0px; width: 1px; height: 1em; outline: currentcolor none medium;" autocorrect="off" autocapitalize="off" spellcheck="false" tabindex="260" wrap="off"></textarea>`

Tried setting autocorrect="on", but then I couldn't edit the field at all.
Works elsewhere in Weblate, like the "Search…" in https://hosted.weblate.org/accounts/profile/#notifications
which is just `<input class="search-input" type="text" placeholder="Search…">`

FF 78.5.0esr (64-bit) on Linux with the standard dictionaries. Does this happen to anyone else?

Edit: No luck in Vivaldi either.


Discussion:
Probably yes because what you see is not actually a textarea, but styled content and the actual textarea is hidden. We're now using CodeMirror for editing text to support highting placeables, double space or syntax for some markup formats.
Can it be applied by Weblate then?
It's quite a major drawback... At least for me, as I don't have innate spelling skills, so I have to copy every other phrase to another tab and it's damn annoying...

I don't need a fancy WYSIWYG edit field, is there an option to revert this to a plain text editor?
PS: also it breaks pasting with middle button and copy to X.org's select buffer... It's probably caused by the same thing, but should I report it as a yet another bug?
Pasting with middle button should work (it works for me), but the X selection indeed does not work. Feel free to report this at https://github.com/codemirror/CodeMirror/issues (assuming you can reproduce it on their demo as well - https://codemirror.net/index.html).

As for spell checking, I agree that it's a major drawback. We will probably end up having own JS implementation.

On the other side we need to move towards more comfortable editor and that's not possible with simple textarea. The CodeMirror editor has been in place for comments for several releases and I've seen no complaints on that. If you know some editor which will not break browser spellcheck and allows highlighting of some strings, we are open to suggestions.
I'm not insisting, but still believe it would be nice to have an option to disable the CodeMirror and revert back to the plain old textarea as a workaround for the issue. I'm not a web developer, but I suppose it shouldn't be hard. I've just issued `$('.CodeMirror')[0].CodeMirror.toTextArea();` from the web console and it worked quite fine. The buttons to insert special characters on the toolbar malfunctioned and the char counter didn't worked but in general it went pretty smooth...

I can't suggest another editor, but I suppose that anything fancier than a textarea will prevent the use of the in-browser spell checker. I assume you may build in the spell checker into java script, but it may be tricky to guess the languages.

@nijel, thanks, those issues about selection buffer look like solely a firefox-related trouble (at least I can't reproduce them on chrome). I would rather not report them to CodeMirror upstream yet as far as I still use a rather old version of the browser (68.4.1) and it'd be not that easy for me to upgrade right now...
PS: About pasting with the middle button: if `middlemouse.contentLoadURL=true` in `about:config`, pasting a URL in a such way triggers a redirect. Yet that works fine for a normal textarea. I haven't tested this thoroughly on a first glance...
PPS: sorry for a bit of unrelated noise.
> I'm not insisting, but still believe it would be nice to have an option to disable the CodeMirror and revert back to the plain old textarea as a workaround for the issue.

I'd rather prefer a single solution that would work for all. Having this configurable will introduce two code paths for any client-side manipulation with the editor and it will break things over the time.

> it may be tricky to guess the languages.

We don't have to guess languages, we know into which language is user translating.

> I can't suggest another editor, but I suppose that anything fancier than a textarea will prevent the use of the in-browser spell checker. 

That doesn't seem to be the case. There seems to be solutions which do support highlighting and still are real textarea, for example https://github.com/kueblc/LDT. The question is whether it doesn't break other things (RTL support is usually problematic) and whether it's maintained in a way we could rely on it...
I've quickly tested LDT and it breaks with RTL and line wrapping seems to be messed up as well, so that one doesn't seem to be viable solution...
 The question is whether we can use some existing solution or reinvent the wheel and write something that works for us. There seem to be two approaches:

## Display overlay on top of textarea 

Uses native textarea editing and displays overlay with the result of highlighting. The user is still editing the actual textarea, but he sees something with the highlighting. The downside is that formatting cannot change the size of the text because it has to match to the textarea. We currently rely on this to display placeables, but it could be adjusted. Most of the editors also break on line wrapping - textarea does that, the overlay has to match, what doesn't work with LDT.

Some useful links:

* https://codersblock.com/blog/highlight-text-inside-a-textarea/
* https://www.horuskol.net/blog/2020-03-05/live-code-highlighting-in-the-browser-with-vanilla-javascript/
* https://github.com/satya164/react-simple-code-editor
* https://github.com/lonekorean/highlight-within-textarea
* https://github.com/kueblc/LDT
* https://softwebtuts.blogspot.com/2019/01/micro-code-editor-with-textarea-syntax.html
* https://github.com/PrismJS/live
* https://en.wikipedia.org/wiki/Comparison_of_JavaScript-based_source_code_editors

This suffers issue of using browser edit history, for which there doesn't seem to be a solution for Firefox, see https://bugzilla.mozilla.org/show_bug.cgi?id=1220696

## Using contenteditable

This is mostly for WYSIWIG, but we really do not need that. On the other side it's closer to what we are looking for - freely stylable editor. It will be probably hard to find solution that will work for us, as most the solutions seems to either focus on code editing (which will probably have troubles with RTL) or being WYSIWIG HTML editors. The tricky part here is also handling undo/redo as built-in gets broken on DOM manipulation (which are probably necessary for highlighting).

* https://medv.io/codejar/ 
* https://quilljs.com/

## What we are looking for

* Syntax highlighting editor
* Utilize spell checking from the browser
* Proper undo/redo controls from browser
* Seamless support for RTL languages
* Line wrapping
* Text completion (currently used for `@username` mentions, but might be extended)
* We probably do not need optimization tricks as the edited text is typically quite short
* Custom highlighting rules (to handle Weblate features like double space or placeables)
* Markup highlighting for HTML, Markdown and reStructuredText
* No auto indentation
* Native browser tab behaviour
* Automatic sizing
Doing some research on this and here are my findings:

* CodeMirror can only use own spell checker, see https://github.com/codemirror/CodeMirror/issues/1017. That is annoying as we would have to distribute all the dictionaries and it would be still broken with third-party browser extensions such as LanguageTool.
* Using transparent `<textarea>` and highlighting using rendered side element seems doable approach, but it limits highlighting capabilities (cannot use anything what would break alignment of overlaying elements).
* Using `contenteditable` would need quite some code to handle things textarea does natively - forcing plain text on paste, undo/redo implementation (because we will be changing DOM for syntax highlighting).

I will try to dig deeper into existing solutions for highlighting `<textarea>` and see if some of that matches our needs.

Resolving patch:
diff --git a/weblate/trans/forms.py b/weblate/trans/forms.py
--- a/weblate/trans/forms.py
+++ b/weblate/trans/forms.py
@@ -107,6 +107,16 @@
 COPY_TEMPLATE = 'data-checksum="{0}" data-content="{1}"'
 
 
+class MarkdownTextarea(forms.Textarea):
+    def __init__(self, **kwargs):
+        kwargs["attrs"] = {
+            "dir": "auto",
+            "class": "markdown-editor highlight-editor",
+            "data-mode": "markdown",
+        }
+        super().__init__(**kwargs)
+
+
 class WeblateDateInput(forms.DateInput):
     def __init__(self, datepicker=True, **kwargs):
         attrs = {"type": "date"}
@@ -293,12 +303,12 @@ def render(self, name, value, attrs=None, renderer=None, **kwargs):
         placeables = [hl[2] for hl in highlight_string(unit.source_string, unit)]
 
         # Need to add extra class
-        attrs["class"] = "translation-editor form-control"
+        attrs["class"] = "translation-editor form-control highlight-editor"
         attrs["tabindex"] = tabindex
         attrs["lang"] = lang.code
         attrs["dir"] = lang.direction
         attrs["rows"] = 3
-        attrs["maxlength"] = unit.get_max_length()
+        attrs["data-max"] = unit.get_max_length()
         attrs["data-mode"] = unit.edit_mode
         attrs["data-placeables"] = "|".join(re.escape(pl) for pl in placeables if pl)
         if unit.readonly:
@@ -326,7 +336,7 @@ def render(self, name, value, attrs=None, renderer=None, **kwargs):
                     fieldid,
                     label,
                     textarea,
-                    attrs["maxlength"],
+                    attrs["data-max"],
                     len(val),
                 )
             )
@@ -919,7 +929,7 @@ class CommentForm(forms.Form):
         ),
     )
     comment = forms.CharField(
-        widget=forms.Textarea(attrs={"dir": "auto", "class": "codemirror-markdown"}),
+        widget=MarkdownTextarea,
         label=_("New comment"),
         help_text=_("You can use Markdown and mention users by @username."),
         max_length=1000,
@@ -1004,7 +1014,10 @@ class ContextForm(forms.ModelForm):
     class Meta:
         model = Unit
         fields = ("explanation", "labels", "extra_flags")
-        widgets = {"labels": forms.CheckboxSelectMultiple()}
+        widgets = {
+            "labels": forms.CheckboxSelectMultiple(),
+            "explanation": MarkdownTextarea,
+        }
 
     doc_links = {
         "explanation": ("admin/translating", "additional"),
@@ -1023,7 +1036,7 @@ def __init__(self, data=None, instance=None, user=None, **kwargs):
         self.helper.disable_csrf = True
         self.helper.form_tag = False
         self.helper.layout = Layout(
-            Field("explanation", css_class="codemirror-markdown"),
+            Field("explanation"),
             Field("labels"),
             ContextDiv(
                 template="snippets/labels_description.html",
@@ -1733,7 +1746,7 @@ class Meta:
         )
         widgets = {
             "access_control": forms.RadioSelect(),
-            "instructions": forms.Textarea(attrs={"class": "codemirror-markdown"}),
+            "instructions": MarkdownTextarea,
         }
 
     def clean(self):
@@ -2058,7 +2071,7 @@ class Meta:
         fields = ["message", "category", "expiry", "notify"]
         widgets = {
             "expiry": WeblateDateInput(),
-            "message": forms.Textarea(attrs={"class": "codemirror-markdown"}),
+            "message": MarkdownTextarea,
         }
 
 
diff --git a/weblate/trans/models/unit.py b/weblate/trans/models/unit.py
--- a/weblate/trans/models/unit.py
+++ b/weblate/trans/models/unit.py
@@ -1122,17 +1122,17 @@ def all_flags(self):
 
     @cached_property
     def edit_mode(self):
-        """Returns edit mode for CodeMirror."""
+        """Returns syntax higlighting mode for Prismjs."""
         flags = self.all_flags
         if "rst-text" in flags:
-            return "text/x-rst"
+            return "rest"
         if "md-text" in flags:
-            return "text/x-markdown"
+            return "markdown"
         if "xml-text" in flags:
-            return "application/xml"
+            return "xml"
         if "safe-html" in flags:
-            return "text/html"
-        return "null"
+            return "html"
+        return "none"
 
     def get_secondary_units(self, user):
         """Return list of secondary units."""


Test patch:
diff --git a/weblate/trans/tests/test_selenium.py b/weblate/trans/tests/test_selenium.py
--- a/weblate/trans/tests/test_selenium.py
+++ b/weblate/trans/tests/test_selenium.py
@@ -103,13 +103,6 @@ def wait_for_page_load(self, timeout=30):
         yield
         WebDriverWait(self.driver, timeout).until(staleness_of(old_page))
 
-    def find_codemirror(self, htmlid):
-        return self.driver.find_element(
-            By.XPATH,
-            f"//textarea[@id='{htmlid}']"
-            "/following-sibling::div[contains(@class, 'CodeMirror')]",
-        )
-
     @classmethod
     def setUpClass(cls):
         # Screenshots storage
@@ -234,16 +227,9 @@ def click(self, element="", htmlid=None):
             self.actions.move_to_element(element).perform()
             element.click()
 
-    def clear_field(self, element, codemirror: bool = False):
-        if codemirror:
-            self.actions.move_to_element(element).click(element).perform()
-            # Wait for the CodeMirror transition completes
-            time.sleep(1)
-            self.actions.send_keys(Keys.CONTROL + "a").perform()
-            self.actions.send_keys(Keys.DELETE).perform()
-        else:
-            element.send_keys(Keys.CONTROL + "a")
-            element.send_keys(Keys.DELETE)
+    def clear_field(self, element):
+        element.send_keys(Keys.CONTROL + "a")
+        element.send_keys(Keys.DELETE)
         return element
 
     def do_login(self, create=True, superuser=False):
@@ -922,11 +908,10 @@ def test_weblate(self):
             element.submit()
 
         # Trigger check
-        element = self.find_codemirror("id_a2a808c8ccbece08_0")
-        self.clear_field(element, codemirror=True)
-        element = self.find_codemirror("id_a2a808c8ccbece08_1")
-        self.clear_field(element, codemirror=True)
-        self.actions.send_keys("několik slov").perform()
+        self.clear_field(self.driver.find_element(By.ID, "id_a2a808c8ccbece08_0"))
+        element = self.driver.find_element(By.ID, "id_a2a808c8ccbece08_1")
+        self.clear_field(element)
+        element.send_keys("několik slov")
         with self.wait_for_page_load():
             element.submit()
         self.screenshot("checks.png")