Repository: translate/pootle

Issue:
Make Markdown the default content format
At the moment we fall back to HTML for `POOTLE_MARKUP_FILTER`.  This seems like a safe default but means that most people then are required to write HTML and might not even know that they can use simpler Markdown if they configure things.

I'd like us to change this so that:
1. Markdown is the default and is installed by default.
2.  Choosing HTML is a concious decision.

This would mean that the default is sane and simple for anyone writing pages.  You do lose complete control, and that's for the HTML people, but you gain a very simple way of adding pages.



Discussion:
@translate/evernote can we have some thoughts on this?

I totally love markdown. Having said that, I believe HTML is a saner default because:
- It's more widespread than other markup formats, and more people are familiar with it.
- It requires no extra dependencies.

> people then are required to write HTML and might not even know that they can use simpler Markdown if they configure things

Arguably the same can be said about any default markup format. Is discoverability perhaps the real issue?

cc @iafan for extra thoughts.

@julen i think the problem is that atm users are by default encouraged to edit html in a non-html editor - which even for experienced coders can be a pita. Although  i can definitely understand the reasoning to offer html input.

Having markdown as the default seems to offer the right balance of user-editability

The editor highlights the markup mode you are on and the live preview is immediate (this is even an extra advantage for HTML as it requires no server round-trips). This has been a recent improvement; did you miss this feature perhaps?

no i see that - but im still not that keen on editing HTML outside my editor - and generally against asking end users to edit it without a WYSIWYG editor

@julen I think discoverability might be an issue.  But I think also lockin.  The pages use a format globally, not per page.  So you can't change at a later date.  Or use HTML when it makes sense and Markdown elsewhere.  If you make HTML and discover you can use Markdown you need to rewrite all the pages.

I guess having a live inline HTML editor ala Wordpress would probably address both worlds and we'd likely not even worry about Markdown.

Can we probably use both at the same time? Markdown itself allows for inline HTML. We could use Markdown by default, seamlessly adding HTML tags and CSS styles as we need them.

With that being said (and if everyone agrees that this is something we'd like to explore), I'd suggest the following:
1. Make Markdown->HTML conversion in JavaScript, to make sure there's no round-trip to the server during the preview.
2. Make this conversion always on, so that both `None` and `markdown` values for `POOTLE_MARKUP_FILTER` work the same; mark one of those options as obsolete. This will give a good config upgrade path.

With that, we might also want to discontinue the support for `textile` and `restructuredtext` options.

And also drop `textile`.

Both at the same time might be a good compromise: [check a working example](http://jonschlinkert.github.io/remarkable/demo/).

If along with this the rest of the options are gone (reST, Textile), I can foresee extra benefits:
- Zero configuration and sane defaults, hence no extra docs or discoverability issues
- No need for custom `MarkupField` (the rendered contents are directly stored in the DB)
- Instant preview always (no need for custom XHR endpoints for round-tripping)
- Simpler front-end code for the CodeMirror editor component.

An open question would remain for transitioning over from reST and Textile — however [we already questioned](https://github.com/translate/pootle/issues/4111) whether they were actually being used or not. If there's no intention for an abrupt change, it'd be wise to warn deployments that `POOTLE_MARKUP_FILTER` is being deprecated in version N, then removed in N+1.


Resolving patch:
diff --git a/pootle/checks.py b/pootle/checks.py
--- a/pootle/checks.py
+++ b/pootle/checks.py
@@ -263,44 +263,6 @@ def check_settings(app_configs=None, **kwargs):
             id="pootle.W010",
         ))
 
-    try:
-        markup_filter = settings.POOTLE_MARKUP_FILTER[0]
-    except AttributeError:
-        errors.append(checks.Warning(
-            _("POOTLE_MARKUP_FILTER is missing."),
-            hint=_("Set POOTLE_MARKUP_FILTER."),
-            id="pootle.W012",
-        ))
-    except (IndexError, TypeError, ValueError):
-        errors.append(checks.Warning(
-            _("Invalid value in POOTLE_MARKUP_FILTER."),
-            hint=_("Set a valid value for POOTLE_MARKUP_FILTER."),
-            id="pootle.W013",
-        ))
-    else:
-        if markup_filter is not None:
-            try:
-                if markup_filter == 'textile':
-                    import textile  # noqa
-                elif markup_filter == 'markdown':
-                    import markdown  # noqa
-                elif markup_filter == 'restructuredtext':
-                    import docutils  # noqa
-                else:
-                    errors.append(checks.Warning(
-                        _("Invalid markup in POOTLE_MARKUP_FILTER."),
-                        hint=_("Set a valid markup for POOTLE_MARKUP_FILTER."),
-                        id="pootle.W014",
-                    ))
-            except ImportError:
-                errors.append(checks.Warning(
-                    _("POOTLE_MARKUP_FILTER is set to '%s' markup, but the "
-                      "package that provides can't be found.", markup_filter),
-                    hint=_("Install the package or change "
-                           "POOTLE_MARKUP_FILTER."),
-                    id="pootle.W015",
-                ))
-
     if settings.POOTLE_TM_SERVER:
         tm_indexes = []
 
@@ -380,6 +342,69 @@ def check_settings(app_configs=None, **kwargs):
     return errors
 
 
+@checks.register()
+def check_settings_markup(app_configs=None, **kwargs):
+    from django.conf import settings
+
+    errors = []
+
+    try:
+        markup_filter = settings.POOTLE_MARKUP_FILTER[0]
+    except AttributeError:
+        errors.append(checks.Warning(
+            _("POOTLE_MARKUP_FILTER is missing."),
+            hint=_("Set POOTLE_MARKUP_FILTER."),
+            id="pootle.W012",
+        ))
+    except (IndexError, TypeError, ValueError):
+        errors.append(checks.Warning(
+            _("Invalid value in POOTLE_MARKUP_FILTER."),
+            hint=_("Set a valid value for POOTLE_MARKUP_FILTER."),
+            id="pootle.W013",
+        ))
+    else:
+        if markup_filter is not None:
+            try:
+                if markup_filter == 'textile':
+                    import textile  # noqa
+                elif markup_filter == 'markdown':
+                    import markdown  # noqa
+                elif markup_filter == 'restructuredtext':
+                    import docutils  # noqa
+                elif markup_filter == 'html':
+                    pass
+                else:
+                    errors.append(checks.Warning(
+                        _("Invalid markup in POOTLE_MARKUP_FILTER."),
+                        hint=_("Set a valid markup for POOTLE_MARKUP_FILTER."),
+                        id="pootle.W014",
+                    ))
+            except ImportError:
+                errors.append(checks.Warning(
+                    _("POOTLE_MARKUP_FILTER is set to '%s' markup, but the "
+                      "package that provides can't be found.", markup_filter),
+                    hint=_("Install the package or change "
+                           "POOTLE_MARKUP_FILTER."),
+                    id="pootle.W015",
+                ))
+        if markup_filter is None:
+                errors.append(checks.Warning(
+                    _("POOTLE_MARKUP_FILTER set to 'None' is deprecated."),
+                    hint=_("Set your markup to 'html' explicitly."),
+                    id="pootle.W025",
+                ))
+        if markup_filter in ('html', 'textile', 'restructuredtext'):
+                errors.append(checks.Warning(
+                    _("POOTLE_MARKUP_FILTER is using '%s' markup, which is "
+                      "deprecated and will be removed in future.",
+                      markup_filter),
+                    hint=_("Convert your staticpages to Markdown and set your "
+                           "markup to 'markdown'."),
+                    id="pootle.W026",
+                ))
+    return errors
+
+
 @checks.register('data')
 def check_users(app_configs=None, **kwargs):
     from django.contrib.auth import get_user_model
diff --git a/pootle/core/markup/filters.py b/pootle/core/markup/filters.py
--- a/pootle/core/markup/filters.py
+++ b/pootle/core/markup/filters.py
@@ -59,7 +59,8 @@ def get_markup_filter_display_name():
         'textile': u'Textile',
         'markdown': u'Markdown',
         'restructuredtext': u'reStructuredText',
-    }.get(name, u'HTML')
+        'html': u'HTML',
+    }.get(name)
 
 
 def get_markup_filter():
@@ -77,6 +78,8 @@ def get_markup_filter():
             import markdown  # noqa
         elif markup_filter == 'restructuredtext':
             import docutils  # noqa
+        elif markup_filter == 'html':
+            pass
         else:
             return (None, '')
     except Exception:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -318,9 +318,6 @@ def adjust_installation_command():
 # Pootle FS plugins
 extras_require['git'] = parse_requirements('requirements/_pootle_fs_git.txt')
 dependency_links += parse_dependency_links('requirements/_pootle_fs_git.txt')
-# Markdown
-extras_require['markdown'] = parse_requirements('requirements/_markup_markdown.txt')
-dependency_links += parse_dependency_links('requirements/_markup_markdown.txt')
 # Testing
 extras_require['travis'] = parse_requirements('requirements/travis.txt',
                                               recurse=True)


Test patch: