Repository: django-wiki/django-wiki

Issue:
Markdown Safe Mode is Deprecated
I was looking into the possible options that `markdown.Markdown` could take and I found this note about `safe_mode` being deprecated:
https://pythonhosted.org/Markdown/release-2.6.html#safe_mode-deprecated

I am generating wiki pages from models that have multiple WYSIWG edited fields that return HTML instead of Markdown. So to make that work I will have to disable `safe_mode` and probably use Bleach with a list of approved tags.

If that kind of change would be accepted, I could refactor my work from being project-specific into a PR to django-wiki. If so, it would be good to know the desired settings API.



Discussion:
Thanks for reporting! And great, let's get rid of this practice and introduce a better one.

That should also be the last straw in supporting Python 2.6, which has made us conditionally use an even older version of Markdown:

https://github.com/django-wiki/django-wiki/blob/master/setup.py#L67

Is there any way to achieve the previous behavior without introducing Bleach as a new dependency?

I've used `html.parser.HTMLParser` to strip tags before: http://stackoverflow.com/a/925630/1998376
This could probably be modified to replace tags w/ text, or to escape all tags as well.

I've been trying to force-fully insert this kind of Extension as suggested in Markdown's docs, however it breaks with the `extras` extension because it assumes that the `html_block` preprocessor exists.

```
from markdown.extensions import Extension

class EscapeHtml(Extension):
    def extendMarkdown(self, md, md_globals):
        del md.preprocessors['html_block']
        del md.inlinePatterns['html']
```
Regarding the `html.parser.HTMLParser` solution, IMO it becomes too complicated by having different implementations in Python 2 and Python 3. It also seems to escape `&amp;` like sequences and thus would need further modification in case people's implementations have been allowing this before.

I feel like opting for `bleach` right away because it seems to be what's recommended in Django for escaping html as well. It can also be made configurable in case we do want to allow certain tags down the road.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -28,6 +28,7 @@ def read_file(fname):
 
 requirements = [
     "Django>=1.8",
+    "bleach>=1.5,<2",
     "Pillow",
     "django-nyt>=1.0b1",
     "six",
diff --git a/wiki/conf/settings.py b/wiki/conf/settings.py
--- a/wiki/conf/settings.py
+++ b/wiki/conf/settings.py
@@ -1,5 +1,7 @@
 from __future__ import absolute_import, unicode_literals
 
+import bleach
+
 from django.conf import settings as django_settings
 from django.core.files.storage import default_storage
 from django.core.urlresolvers import reverse_lazy
@@ -26,13 +28,19 @@
         'codehilite',
         'sane_lists',
     ],
-    'safe_mode': 'replace',
     'extension_configs': {
         'toc': {
             'title': _('Table of Contents')}},
 }
 MARKDOWN_KWARGS.update(getattr(django_settings, 'WIKI_MARKDOWN_KWARGS', {}))
 
+# Allowed tags in Markdown article contents.
+MARKDOWN_HTML_WHITELIST = getattr(
+    django_settings,
+    'WIKI_MARKDOWN_HTML_WHITELIST',
+    bleach.ALLOWED_TAGS
+)
+
 # This slug is used in URLPath if an article has been deleted. The children of the
 # URLPath of that article are moved to lost and found. They keep their permissions
 # and all their content.
diff --git a/wiki/core/markdown/__init__.py b/wiki/core/markdown/__init__.py
--- a/wiki/core/markdown/__init__.py
+++ b/wiki/core/markdown/__init__.py
@@ -1,10 +1,12 @@
 from __future__ import absolute_import, unicode_literals
 
+import bleach
+import markdown
+
 from wiki.conf import settings
 from wiki.core.markdown.mdx.previewlinks import PreviewLinksExtension
 from wiki.core.markdown.mdx.responsivetable import ResponsiveTableExtension
 from wiki.core.plugins import registry as plugin_registry
-import markdown
 
 
 class ArticleMarkdown(markdown.Markdown):
@@ -27,6 +29,10 @@ def get_markdown_extensions(self):
         extensions += plugin_registry.get_markdown_extensions()
         return extensions
 
+    def convert(self, text, *args, **kwargs):
+        text = bleach.clean(text, tags=settings.MARKDOWN_HTML_WHITELIST)
+        return super(ArticleMarkdown, self).convert(text, *args, **kwargs)
+
 
 def article_markdown(text, article, *args, **kwargs):
     md = ArticleMarkdown(article, *args, **kwargs)


Test patch:
diff --git a/wiki/tests/test_markdown.py b/wiki/tests/test_markdown.py
--- a/wiki/tests/test_markdown.py
+++ b/wiki/tests/test_markdown.py
@@ -1,12 +1,16 @@
 from __future__ import absolute_import, unicode_literals
 
-from django.test import TestCase
 import markdown
+from django.test import TestCase
+from mock import patch
 from wiki.core.markdown import ArticleMarkdown
 from wiki.core.markdown.mdx.responsivetable import ResponsiveTableExtension
-from mock import patch
+from wiki.models import URLPath
+from wiki.tests.base import ArticleTestBase
+
+
+class ArticleMarkdownTests(ArticleTestBase):
 
-class ArticleMarkdownTests(TestCase):
     @patch('wiki.core.markdown.settings')
     def test_do_not_modify_extensions(self, settings):
         extensions = ['footnotes', 'attr_list', 'sane_lists']
@@ -15,7 +19,20 @@ def test_do_not_modify_extensions(self, settings):
         ArticleMarkdown(None)
         self.assertEqual(len(extensions), number_of_extensions)
 
+    def test_html_removal(self):
+
+        urlpath = URLPath.create_article(
+            self.root,
+            'html_removal',
+            title="Test 1",
+            content="</html>only_this"
+        )
+
+        self.assertEqual(urlpath.article.render(), "<p>&lt;/html&gt;only_this</p>")
+
+
 class ResponsiveTableTests(TestCase):
+
     def setUp(self):
         self.md = markdown.Markdown(extensions=[
             'extra',