Repository: lepture/mistune

Issue:
parsing html can fail to close tag correctly
noticed while debugging an issue in https://github.com/lektor/lektor/issues/241#issuecomment-231819694 i noticed that raw html handling can be broken if there is a space before the closing `>` of the opening tag. i.e.

```
In [38]: mistune.markdown('<iframe src="http://gfycat.com" ></iframe>', escape=False)
Out[38]: '<p><iframe src="http://gfycat.com" >&lt;/iframe&gt;</p>\n'
```

but

```
In [39]: mistune.markdown('<iframe src="http://gfycat.com"></iframe>', escape=False)
Out[39]: '<p><iframe src="http://gfycat.com"></iframe></p>\n'
```

in the case of the iframe, this causes the opening iframe to gobble up the rest of the page content. this seems to hold for any tag, though, e.g.
```
In [40]: mistune.markdown('<a href="http://gfycat.com" ></a>', escape=False)
Out[40]: '<p><a href="http://gfycat.com" >&lt;/a&gt;</p>\n'
```


Discussion:
nsfmc: Hello, i'm not sure what the protocol is for this. 

I ran into this bug from a strangely formed embed code i got from gyfcat, but i suspect others could run into it too.

the `test_safe_links` test is currently failing in master, but the test that this adds passes.

Let me know if you have any questions!
lepture: @nsfmc a test case failed.
nsfmc: Hi @lepture, I believe that test case is failing in master as well. If there is a branch I can fork off of that is building successfully I'm happy to rebase my pull request off that one. I'm afraid I don't understand well enough the changes needed to fix the link sanitization issues :(

Patch:
diff --git a/mistune.py b/mistune.py
index b341cf2..67b0880 100644
--- a/mistune.py
+++ b/mistune.py
@@ -74,7 +74,7 @@ def escape(text, quote=False, smart_amp=True):
 
 def escape_link(url, **kwargs):
     """Remove dangerous URL schemes like javascript: and escape afterwards."""
-    lower_url = url.lower()
+    lower_url = url.lower().strip('\x00\x1a \n\r\t')
     for scheme in _scheme_blacklist:
         if lower_url.startswith(scheme):
             return ''
@@ -157,7 +157,7 @@ class BlockGrammar(object):
     block_html = re.compile(
         r'^ *(?:%s|%s|%s) *(?:\n{2,}|\s*$)' % (
             r'<!--[\s\S]*?-->',
-            r'<(%s)((?:%s)*?)>([\s\S]+?)<\/\1>' % (_block_tag, _valid_attr),
+            r'<(%s)((?:%s)*?)\s*>([\s\S]+?)<\/\1>' % (_block_tag, _valid_attr),
             r'<%s(?:%s)*?\s*\/?>' % (_block_tag, _valid_attr),
         )
     )
@@ -447,7 +447,7 @@ class InlineGrammar(object):
     inline_html = re.compile(
         r'^(?:%s|%s|%s)' % (
             r'<!--[\s\S]*?-->',
-            r'<(\w+%s)((?:%s)*?)>([\s\S]*?)<\/\1>' % (_valid_end, _valid_attr),
+            r'<(\w+%s)((?:%s)*?)\s*>([\s\S]*?)<\/\1>' % (_valid_end, _valid_attr),
             r'<\w+%s(?:%s)*?\s*\/?>' % (_valid_end, _valid_attr),
         )
     )


Test patch:
diff --git a/tests/test_extra.py b/tests/test_extra.py
index 7318444..07c1ca3 100644
--- a/tests/test_extra.py
+++ b/tests/test_extra.py
@@ -77,6 +77,12 @@ def test_parse_inline_html():
     assert 'href' not in ret
 
 
+def test_block_html():
+    ret = mistune.markdown(
+        '<div ></div>', escape=False
+    )
+    assert '<div ></div>' in ret
+
 def test_parse_block_html():
     ret = mistune.markdown(
         '<div>**foo**</div>', parse_block_html=True, escape=False