Repository: Python-Markdown/markdown
Pull request #662 (2018-05-20 00:52:17)

Add missing comma in HTML_EMPTY
Because of the missing comma, the last two tags in the list, `meta` and `param` are interpreted as a single string literal `"metaparam"`, so the serializer fails to correctly produce empty elements for `<meta>` and `<param>` tags.

waylan: So it appears that the failing test is [caused][1] by the latest version of wheel not supporting Python 3.3. Otherwise this looks fine. Thanks.

[1]: https://travis-ci.org/Python-Markdown/markdown/jobs/381219613#L590
waylan: We probably should add a test for this.

Patch:
diff --git a/.travis.yml b/.travis.yml
index 3809135..01bdbd6 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -5,16 +5,20 @@ matrix:
   include:
     - python: '2.7'
       env: TOXENV=py27
-    - python: '3.3'
-      env: TOXENV=py33
     - python: '3.4'
       env: TOXENV=py34
     - python: '3.5'
       env: TOXENV=py35
     - python: '3.6'
       env: TOXENV=py36
+    - python: '3.7'
+      env: TOXENV=py37
+      dist: xenial
+      sudo: true
     - python: 'pypy'
       env: TOXENV=pypy
+    - python: 'pypy3'
+      env: TOXENV=pypy3
     - env: TOXENV=flake8
     - env: TOXENV=checkspelling
 
@@ -39,4 +43,3 @@ after_success:
 branches:
   only:
     - master
-    - md3
diff --git a/docs/extensions/admonition.md b/docs/extensions/admonition.md
index 2d2634d..22a6aea 100644
--- a/docs/extensions/admonition.md
+++ b/docs/extensions/admonition.md
@@ -72,8 +72,26 @@ results in:
 </div>
 ```
 
-rST suggests the following `types`, but you're free to use whatever you want:
-    attention, caution, danger, error, hint, important, note, tip, warning.
+You can also provide additional CSS class names separated by spaces. The first
+class should be the "type." For example:
+
+```md
+!!! danger highlight blink "Don't try this at home"
+    ...
+```
+
+will render:
+
+```html
+<div class="admonition danger highlight blink">
+<p class="admonition-title">Don't try this at home</p>
+<p>...</p>
+</div>
+```
+
+rST suggests the following "types": `attention`, `caution`, `danger`, `error`,
+`hint`, `important`, `note`, `tip`, and `warning`; however, you're free to use
+whatever you want.
 
 Styling
 -------
diff --git a/docs/extensions/fenced_code_blocks.md b/docs/extensions/fenced_code_blocks.md
index 96fe786..4c8058d 100644
--- a/docs/extensions/fenced_code_blocks.md
+++ b/docs/extensions/fenced_code_blocks.md
@@ -21,9 +21,9 @@ Thus, the following text (taken from the above referenced PHP documentation):
 ```md
 This is a paragraph introducing:
 
-~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~
 a one-line code block
-    ~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~
 ```
 
 Fenced code blocks can have a blank line as the first  and/or last line of a
diff --git a/docs/extensions/nl2br.md b/docs/extensions/nl2br.md
index 8c53d33..d561510 100644
--- a/docs/extensions/nl2br.md
+++ b/docs/extensions/nl2br.md
@@ -6,7 +6,7 @@ New-Line-to-Break Extension
 Summary
 -------
 
-The New-Line-to-Break (`nl2b`) Extension will cause newlines to be treated as
+The New-Line-to-Break (`nl2br`) Extension will cause newlines to be treated as
 hard breaks; like StackOverflow and [GitHub][] flavored Markdown do.
 
 [Github]: http://github.github.com/github-flavored-markdown/
diff --git a/docs/reference.md b/docs/reference.md
index 936d389..25c2058 100644
--- a/docs/reference.md
+++ b/docs/reference.md
@@ -181,10 +181,6 @@ __tab_length__{: #tab_length }:
 
 : Length of tabs in the source. Default: 4
 
-__enable_attributes__{: #enable_attributes}:
-
-: Enable the conversion of attributes. Defaults to `True`.
-
 __smart_emphasis__{: #smart_emphasis }:
 
 : Treat `_connected_words_` intelligently Default: True
diff --git a/markdown/core.py b/markdown/core.py
index f4111e4..0fcf71d 100644
--- a/markdown/core.py
+++ b/markdown/core.py
@@ -27,7 +27,6 @@ class Markdown(object):
 
     option_defaults = {
         'tab_length':            4,
-        'enable_attributes':     True,
         'smart_emphasis':        True,
         'lazy_ol':               True,
     }
@@ -53,7 +52,6 @@ class Markdown(object):
             * "xhtml": Outputs XHTML style tags. Default.
             * "html": Outputs HTML style tags.
         * tab_length: Length of tabs in the source. Default: 4
-        * enable_attributes: Enable the conversion of attributes. Default: True
         * smart_emphasis: Treat `_connected_words_` intelligently Default: True
         * lazy_ol: Ignore number of first item of ordered lists. Default: True
 
diff --git a/markdown/extensions/admonition.py b/markdown/extensions/admonition.py
index 8576638..b001957 100644
--- a/markdown/extensions/admonition.py
+++ b/markdown/extensions/admonition.py
@@ -41,7 +41,8 @@ class AdmonitionProcessor(BlockProcessor):
 
     CLASSNAME = 'admonition'
     CLASSNAME_TITLE = 'admonition-title'
-    RE = re.compile(r'(?:^|\n)!!! ?([\w\-]+)(?: +"(.*?)")? *(?:\n|$)')
+    RE = re.compile(r'(?:^|\n)!!! ?([\w\-]+(?: +[\w\-]+)*)(?: +"(.*?)")? *(?:\n|$)')
+    RE_SPACES = re.compile('  +')
 
     def test(self, parent, block):
         sibling = self.lastChild(parent)
@@ -80,11 +81,12 @@ class AdmonitionProcessor(BlockProcessor):
 
     def get_class_and_title(self, match):
         klass, title = match.group(1).lower(), match.group(2)
+        klass = self.RE_SPACES.sub(' ', klass)
         if title is None:
             # no title was provided, use the capitalized classname as title
             # e.g.: `!!! note` will render
             # `<p class="admonition-title">Note</p>`
-            title = klass.capitalize()
+            title = klass.split(' ', 1)[0].capitalize()
         elif title == '':
             # an explicit blank title should not be rendered
             # e.g.: `!!! warning ""` will *not* render `p` with a title
diff --git a/markdown/extensions/legacy_attrs.py b/markdown/extensions/legacy_attrs.py
new file mode 100644
index 0000000..b28223f
--- /dev/null
+++ b/markdown/extensions/legacy_attrs.py
@@ -0,0 +1,49 @@
+"""
+Legacy Attributes Extension
+===========================
+
+An extension to Python Markdown which implements legacy attributes.
+
+Prior to Python-Markdown version 3.0, the Markdown class had an `enable_attributes`
+keyword which was on by default and provided for attributes to be defined for elements
+using the format `{@key=value}`. This extension is provided as a replacement for
+backward compatability. New documents should be authored using attr_lists. However,
+numerious documents exist which have been using the old attribute format for many
+years. This extension can be used to continue to render those documents correctly.
+"""
+
+import re
+from markdown.treeprocessors import Treeprocessor, isString
+from markdown.extensions import Extension
+
+
+ATTR_RE = re.compile(r'\{@([^\}]*)=([^\}]*)}')  # {@id=123}
+
+
+class LegacyAttrs(Treeprocessor):
+    def run(self, doc):
+        """Find and set values of attributes ({@key=value}). """
+        for el in doc.iter():
+            alt = el.get('alt', None)
+            if alt is not None:
+                el.set('alt', self.handleAttributes(el, alt))
+            if el.text and isString(el.text):
+                el.text = self.handleAttributes(el, el.text)
+            if el.tail and isString(el.tail):
+                el.tail = self.handleAttributes(el, el.tail)
+
+    def handleAttributes(self, el, txt):
+        """ Set attributes and return text without definitions. """
+        def attributeCallback(match):
+            el.set(match.group(1), match.group(2).replace('\n', ' '))
+        return ATTR_RE.sub(attributeCallback, txt)
+
+
+class LegacyAttrExtension(Extension):
+    def extendMarkdown(self, md, md_globals):
+        la = LegacyAttrs(md)
+        md.treeprocessors.add('legacyattrs', la, '>inline')
+
+
+def makeExtension(**kwargs):  # pragma: no cover
+    return LegacyAttrExtension(**kwargs)
diff --git a/markdown/extensions/meta.py b/markdown/extensions/meta.py
index cafad4a..2c2c8e3 100644
--- a/markdown/extensions/meta.py
+++ b/markdown/extensions/meta.py
@@ -36,10 +36,15 @@ class MetaExtension (Extension):
 
     def extendMarkdown(self, md, md_globals):
         """ Add MetaPreprocessor to Markdown instance. """
+        md.registerExtension(self)
+        self.md = md
         md.preprocessors.add("meta",
                              MetaPreprocessor(md),
                              ">normalize_whitespace")
 
+    def reset(self):
+        self.md.Meta = {}
+
 
 class MetaPreprocessor(Preprocessor):
     """ Get Meta-Data. """
diff --git a/markdown/inlinepatterns.py b/markdown/inlinepatterns.py
index 18da73b..d67ef1f 100644
--- a/markdown/inlinepatterns.py
+++ b/markdown/inlinepatterns.py
@@ -154,17 +154,6 @@ def dequote(string):
         return string
 
 
-ATTR_RE = re.compile(r"\{@([^\}]*)=([^\}]*)}")  # {@id=123}
-
-
-def handleAttributes(text, parent):
-    """Set values of an element based on attribute definitions ({@id=123})."""
-    def attributeCallback(match):
-        parent.set(match.group(1), match.group(2).replace('\n', ' '))
-        return ''
-    return ATTR_RE.sub(attributeCallback, text)
-
-
 """
 The pattern classes
 -----------------------------------------------------------------------------
@@ -601,12 +590,7 @@ class ImageInlineProcessor(LinkInlineProcessor):
         if title is not None:
             el.set("title", title)
 
-        if self.markdown.enable_attributes:
-            truealt = handleAttributes(text, el)
-        else:
-            truealt = text
-
-        el.set('alt', self.unescape(truealt))
+        el.set('alt', self.unescape(text))
         return el, m.start(0), index
 
 
@@ -676,10 +660,6 @@ class ImageReferenceInlineProcessor(ReferenceInlineProcessor):
         el.set("src", href)
         if title:
             el.set("title", title)
-
-        if self.markdown.enable_attributes:
-            text = handleAttributes(text, el)
-
         el.set("alt", self.unescape(text))
         return el
 
diff --git a/markdown/serializers.py b/markdown/serializers.py
index 1e8d9dd..308cf7a 100644
--- a/markdown/serializers.py
+++ b/markdown/serializers.py
@@ -39,6 +39,7 @@
 
 from __future__ import absolute_import
 from __future__ import unicode_literals
+from xml.etree.ElementTree import ProcessingInstruction
 from . import util
 ElementTree = util.etree.ElementTree
 QName = util.etree.QName
@@ -46,32 +47,17 @@ if hasattr(util.etree, 'test_comment'):  # pragma: no cover
     Comment = util.etree.test_comment
 else:  # pragma: no cover
     Comment = util.etree.Comment
-PI = util.etree.PI
-ProcessingInstruction = util.etree.ProcessingInstruction
 
 __all__ = ['to_html_string', 'to_xhtml_string']
 
 HTML_EMPTY = ("area", "base", "basefont", "br", "col", "frame", "hr",
-              "img", "input", "isindex", "link", "meta" "param")
+              "img", "input", "isindex", "link", "meta", "param")
 
 try:
     HTML_EMPTY = set(HTML_EMPTY)
 except NameError:  # pragma: no cover
     pass
 
-_namespace_map = {
-    # "well-known" namespace prefixes
-    "http://www.w3.org/XML/1998/namespace": "xml",
-    "http://www.w3.org/1999/xhtml": "html",
-    "http://www.w3.org/1999/02/22-rdf-syntax-ns#": "rdf",
-    "http://schemas.xmlsoap.org/wsdl/": "wsdl",
-    # xml schema
-    "http://www.w3.org/2001/XMLSchema": "xs",
-    "http://www.w3.org/2001/XMLSchema-instance": "xsi",
-    # dublic core
-    "http://purl.org/dc/elements/1.1/": "dc",
-}
-
 
 def _raise_serialization_error(text):  # pragma: no cover
     raise TypeError(
@@ -79,13 +65,6 @@ def _raise_serialization_error(text):  # pragma: no cover
         )
 
 
-def _encode(text, encoding):
-    try:
-        return text.encode(encoding, "xmlcharrefreplace")
-    except (TypeError, AttributeError):  # pragma: no cover
-        _raise_serialization_error(text)
-
-
 def _escape_cdata(text):
     # escape character data
     try:
@@ -137,142 +116,73 @@ def _escape_attrib_html(text):
         _raise_serialization_error(text)
 
 
-def _serialize_html(write, elem, qnames, namespaces, format):
+def _serialize_html(write, elem, format):
     tag = elem.tag
     text = elem.text
     if tag is Comment:
         write("<!--%s-->" % _escape_cdata(text))
     elif tag is ProcessingInstruction:
         write("<?%s?>" % _escape_cdata(text))
+    elif tag is None:
+        if text:
+            write(_escape_cdata(text))
+        for e in elem:
+            _serialize_html(write, e, format)
     else:
-        tag = qnames[tag]
-        if tag is None:
+        namespace_uri = None
+        if isinstance(tag, QName):
+            # QNAME objects store their data as a string: `{uri}tag`
+            if tag.text[:1] == "{":
+                namespace_uri, tag = tag.text[1:].split("}", 1)
+            else:
+                raise ValueError('QName objects must define a tag.')
+        write("<" + tag)
+        items = elem.items()
+        if items:
+            items = sorted(items)  # lexical order
+            for k, v in items:
+                if isinstance(k, QName):
+                    # Assume a text only QName
+                    k = k.text
+                if isinstance(v, QName):
+                    # Assume a text only QName
+                    v = v.text
+                else:
+                    v = _escape_attrib_html(v)
+                if k == v and format == 'html':
+                    # handle boolean attributes
+                    write(" %s" % v)
+                else:
+                    write(' %s="%s"' % (k, v))
+        if namespace_uri:
+            write(' xmlns="%s"' % (_escape_attrib(namespace_uri)))
+        if format == "xhtml" and tag.lower() in HTML_EMPTY:
+            write(" />")
+        else:
+            write(">")
             if text:
-                write(_escape_cdata(text))
+                if tag.lower() in ["script", "style"]:
+                    write(text)
+                else:
+                    write(_escape_cdata(text))
             for e in elem:
-                _serialize_html(write, e, qnames, None, format)
-        else:
-            write("<" + tag)
-            items = elem.items()
-            if items or namespaces:
-                items = sorted(items)  # lexical order
-                for k, v in items:
-                    if isinstance(k, QName):
-                        k = k.text
-                    if isinstance(v, QName):
-                        v = qnames[v.text]
-                    else:
-                        v = _escape_attrib_html(v)
-                    if qnames[k] == v and format == 'html':
-                        # handle boolean attributes
-                        write(" %s" % v)
-                    else:
-                        write(" %s=\"%s\"" % (qnames[k], v))
-                if namespaces:
-                    items = namespaces.items()
-                    items.sort(key=lambda x: x[1])  # sort on prefix
-                    for v, k in items:
-                        if k:
-                            k = ":" + k
-                        write(" xmlns%s=\"%s\"" % (k, _escape_attrib(v)))
-            if format == "xhtml" and tag.lower() in HTML_EMPTY:
-                write(" />")
-            else:
-                write(">")
-                if text:
-                    if tag.lower() in ["script", "style"]:
-                        write(text)
-                    else:
-                        write(_escape_cdata(text))
-                for e in elem:
-                    _serialize_html(write, e, qnames, None, format)
-                if tag.lower() not in HTML_EMPTY:
-                    write("</" + tag + ">")
+                _serialize_html(write, e, format)
+            if tag.lower() not in HTML_EMPTY:
+                write("</" + tag + ">")
     if elem.tail:
         write(_escape_cdata(elem.tail))
 
 
-def _write_html(root,
-                encoding=None,
-                default_namespace=None,
-                format="html"):
+def _write_html(root, format="html"):
     assert root is not None
     data = []
     write = data.append
-    qnames, namespaces = _namespaces(root, default_namespace)
-    _serialize_html(write, root, qnames, namespaces, format)
-    if encoding is None:
-        return "".join(data)
-    else:
-        return _encode("".join(data))
+    _serialize_html(write, root, format)
+    return "".join(data)
 
 
 # --------------------------------------------------------------------
-# serialization support
-
-def _namespaces(elem, default_namespace=None):
-    # identify namespaces used in this tree
-
-    # maps qnames to *encoded* prefix:local names
-    qnames = {None: None}
-
-    # maps uri:s to prefixes
-    namespaces = {}
-    if default_namespace:
-        namespaces[default_namespace] = ""
-
-    def add_qname(qname):
-        # calculate serialized qname representation
-        try:
-            if qname[:1] == "{":
-                uri, tag = qname[1:].split("}", 1)
-                prefix = namespaces.get(uri)
-                if prefix is None:
-                    prefix = _namespace_map.get(uri)
-                    if prefix is None:
-                        prefix = "ns%d" % len(namespaces)
-                    if prefix != "xml":
-                        namespaces[uri] = prefix
-                if prefix:
-                    qnames[qname] = "%s:%s" % (prefix, tag)
-                else:
-                    qnames[qname] = tag  # default element
-            else:
-                if default_namespace:
-                    raise ValueError(
-                        "cannot use non-qualified names with "
-                        "default_namespace option"
-                        )
-                qnames[qname] = qname
-        except TypeError:  # pragma: no cover
-            _raise_serialization_error(qname)
-
-    # populate qname and namespaces table
-    try:
-        iterate = elem.iter
-    except AttributeError:
-        iterate = elem.getiterator  # cET compatibility
-    for elem in iterate():
-        tag = elem.tag
-        if isinstance(tag, QName) and tag.text not in qnames:
-            add_qname(tag.text)
-        elif isinstance(tag, util.string_type):
-            if tag not in qnames:
-                add_qname(tag)
-        elif tag is not None and tag is not Comment and tag is not PI:
-            _raise_serialization_error(tag)
-        for key, value in elem.items():
-            if isinstance(key, QName):
-                key = key.text
-            if key not in qnames:
-                add_qname(key)
-            if isinstance(value, QName) and value.text not in qnames:
-                add_qname(value.text)
-        text = elem.text
-        if isinstance(text, QName) and text.text not in qnames:
-            add_qname(text.text)
-    return qnames, namespaces
-
+# public functions
 
 def to_html_string(element):
     return _write_html(ElementTree(element).getroot(), format="html")
diff --git a/markdown/treeprocessors.py b/markdown/treeprocessors.py
index 7c37ae7..df5e748 100644
--- a/markdown/treeprocessors.py
+++ b/markdown/treeprocessors.py
@@ -366,26 +366,9 @@ class InlineProcessor(Treeprocessor):
                     stack.append((child, self.ancestors[:]))
 
             for element, lst in insertQueue:
-                if self.markdown.enable_attributes:
-                    if element.text and isString(element.text):
-                        element.text = inlinepatterns.handleAttributes(
-                            element.text, element
-                        )
-                i = 0
-                for obj in lst:
+                for i, obj in enumerate(lst):
                     newChild = obj[0]
-                    if self.markdown.enable_attributes:
-                        # Processing attributes
-                        if newChild.tail and isString(newChild.tail):
-                            newChild.tail = inlinepatterns.handleAttributes(
-                                newChild.tail, element
-                            )
-                        if newChild.text and isString(newChild.text):
-                            newChild.text = inlinepatterns.handleAttributes(
-                                newChild.text, newChild
-                            )
                     element.insert(i, newChild)
-                    i += 1
         return tree
 
 
diff --git a/setup.py b/setup.py
index 73c7ab3..9614a49 100755
--- a/setup.py
+++ b/setup.py
@@ -56,6 +56,7 @@ setup(
     maintainer_email='waylan.limberg@icloud.com',
     license='BSD License',
     packages=['markdown', 'markdown.extensions'],
+    python_requires='>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*',
     entry_points={
         'console_scripts': [
             '%s = markdown.__main__:run' % SCRIPT_NAME,
@@ -78,6 +79,7 @@ setup(
             'tables = markdown.extensions.tables:TableExtension',
             'toc = markdown.extensions.toc:TocExtension',
             'wikilinks = markdown.extensions.wikilinks:WikiLinkExtension',
+            'legacy_attrs = markdown.extensions.legacy_attrs:LegacyAttrExtension',
         ]
     },
     classifiers=[
@@ -88,11 +90,10 @@ setup(
         'Programming Language :: Python :: 2',
         'Programming Language :: Python :: 2.7',
         'Programming Language :: Python :: 3',
-        'Programming Language :: Python :: 3.2',
-        'Programming Language :: Python :: 3.3',
         'Programming Language :: Python :: 3.4',
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
+        'Programming Language :: Python :: 3.7',
         'Topic :: Communications :: Email :: Filters',
         'Topic :: Internet :: WWW/HTTP :: Dynamic Content :: CGI Tools/Libraries',
         'Topic :: Internet :: WWW/HTTP :: Site Management',
diff --git a/tox.ini b/tox.ini
index 7a415bf..42cdd50 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,5 +1,5 @@
 [tox]
-envlist = py27, py32, py33, py34, py35, py36, pypy, flake8, checkspelling
+envlist = py27, py34, py35, py36, py37, pypy, pypy3, flake8, checkspelling
 
 [testenv]
 deps = -rtest-requirements.txt


Test patch:
diff --git a/tests/extensions/admonition.html b/tests/extensions/admonition.html
index 511d71a..8812dcb 100644
--- a/tests/extensions/admonition.html
+++ b/tests/extensions/admonition.html
@@ -26,6 +26,15 @@
 <p>For something completely different.</p>
 <p>You can also use a custom CSS class name.</p>
 </div>
+<div class="admonition class1 class2 class3">
+<p class="admonition-title">And now...</p>
+<p>For something completely different.</p>
+<p>Several class names can be separated by space chars.</p>
+</div>
+<div class="admonition note anotherclass">
+<p class="admonition-title">Note</p>
+<p>The default title is the capitalized first class name.</p>
+</div>
 <div class="admonition tip">
 <p>An explicitly empty string prevents the title from being rendered.</p>
 </div>
diff --git a/tests/extensions/admonition.txt b/tests/extensions/admonition.txt
index ab8434c..03ff4e9 100644
--- a/tests/extensions/admonition.txt
+++ b/tests/extensions/admonition.txt
@@ -25,6 +25,14 @@ Not part of an Admonition!
 
     You can also use a custom CSS class name.
 
+!!! class1 class2    class3 "And now..."
+    For something completely different.
+
+    Several class names can be separated by space chars.
+
+!!! note anotherclass
+    The default title is the capitalized first class name.
+
 !!! tip ""
     An explicitly empty string prevents the title from being rendered.
 
diff --git a/tests/misc/attributes-image-ref.html b/tests/misc/attributes-image-ref.html
deleted file mode 100644
index 6974420..0000000
--- a/tests/misc/attributes-image-ref.html
+++ /dev/null
@@ -1,1 +0,0 @@
-<p><img alt="img" id="foo" src="http://example.com/i.jpg" /></p>
\ No newline at end of file
diff --git a/tests/misc/attributes-image-ref.txt b/tests/misc/attributes-image-ref.txt
deleted file mode 100644
index a216971..0000000
--- a/tests/misc/attributes-image-ref.txt
+++ /dev/null
@@ -1,4 +0,0 @@
-![img{@id=foo}][img]
-
-  [img]: http://example.com/i.jpg
-
diff --git a/tests/misc/attributes2.html b/tests/misc/attributes2.html
deleted file mode 100644
index b78fee0..0000000
--- a/tests/misc/attributes2.html
+++ /dev/null
@@ -1,6 +0,0 @@
-<p id="TABLE.OF.CONTENTS"></p>
-<ul>
-<li id="TABLEOFCONTENTS"></li>
-</ul>
-<p id="TABLEOFCONTENTS">Or in the middle of the text </p>
-<p id="tableofcontents"></p>
\ No newline at end of file
diff --git a/tests/misc/attributes2.txt b/tests/misc/attributes2.txt
deleted file mode 100644
index d635cb2..0000000
--- a/tests/misc/attributes2.txt
+++ /dev/null
@@ -1,10 +0,0 @@
-{@id=TABLE.OF.CONTENTS}
-
-
-* {@id=TABLEOFCONTENTS}
-
-
-Or in the middle of the text {@id=TABLEOFCONTENTS}
-
-{@id=tableofcontents}
-
diff --git a/tests/misc/bidi.html b/tests/misc/bidi.html
index ffe04dc..3cc0444 100644
--- a/tests/misc/bidi.html
+++ b/tests/misc/bidi.html
@@ -1,7 +1,7 @@
 <p><strong>Python</strong>（パイソン）は、<a href="http://en.wikipedia.org/wiki/Guido_van_Rossum">Guido van Rossum</a> によって作られたオープンソースのオブジェクト指向スクリプト言語。<a href="http://ja.wikipedia.org/wiki/Perl">Perl</a>とともに欧米で広く普及している。イギリスのテレビ局 BBC が製作したコメディ番組『空飛ぶモンティ・パイソン』にちなんで名付けられた。 (Pythonには、爬虫類のニシキヘビの意味があり、Python言語のマスコットやアイコンとして使われることがある。)</p>
 <p>|||||||||||||||||||||||||||||THIS SHOULD BE LTR|||||||||||||||||||||||||</p>
-<p dir="rtl">|||||||||||||||||||||||||||||THIS SHOULD BE RTL|||||||||||||||||||||||||  </p>
-<p dir="ltr">(<strong>بايثون</strong> لغة برمجة حديثة بسيطة، واضحة، سريعة ، تستخدم أسلوب البرمجة الكائنية (THIS SHOULD BE LTR ) وقابلة للتطوير    بالإضافة إلى أنها مجانية و مفتوح </p>
+<p>|||||||||||||||||||||||||||||THIS SHOULD BE RTL|||||||||||||||||||||||||  </p>
+<p>(<strong>بايثون</strong> لغة برمجة حديثة بسيطة، واضحة، سريعة ، تستخدم أسلوب البرمجة الكائنية (THIS SHOULD BE LTR ) وقابلة للتطوير  بالإضافة إلى أنها مجانية و مفتوح </p>
 <p>پایتون زبان برنامه‌نویسی تفسیری و سطح بالا ، شی‌گرا و یک زبان برنامه‌نویسی تفسیری سمت سرور قدرتمند است که توسط گیدو ون روسوم در سال ۱۹۹۰ ساخته شد. این زبان در ویژگی‌ها شبیه پرل، روبی، اسکیم، اسمال‌تاک و تی‌سی‌ال است و از مدیریت خودکار حافظه استفاده می‌کند</p>
 <p>Python，是一种面向对象的、直譯式的计算机程序设计语言，也是一种功能强大而完善的通用型语言，已经具有十多年的发展历史，成熟且稳定。</p>
 <p>ބްލޫ ވޭލްގެ ދޫ މަތީގައި އެއްފަހަރާ 50 މީހުންނަށް ތިބެވިދާނެވެ. ބޮޑު މަހުގެ ދުލަކީ އެހާމެ ބޮޑު އެއްޗެކެވެ.</p>
diff --git a/tests/misc/bidi.txt b/tests/misc/bidi.txt
index f11ff1c..7e6dbea 100644
--- a/tests/misc/bidi.txt
+++ b/tests/misc/bidi.txt
@@ -2,10 +2,10 @@
 
 |||||||||||||||||||||||||||||THIS SHOULD BE LTR|||||||||||||||||||||||||
 
-|||||||||||||||||||||||||||||THIS SHOULD BE RTL||||||||||||||||||||||||| {@dir=rtl} 
+|||||||||||||||||||||||||||||THIS SHOULD BE RTL|||||||||||||||||||||||||  
 
 
-(**بايثون** لغة برمجة حديثة بسيطة، واضحة، سريعة ، تستخدم أسلوب البرمجة الكائنية (THIS SHOULD BE LTR ) وقابلة للتطوير  {@dir=ltr}  بالإضافة إلى أنها مجانية و مفتوح 
+(**بايثون** لغة برمجة حديثة بسيطة، واضحة، سريعة ، تستخدم أسلوب البرمجة الكائنية (THIS SHOULD BE LTR ) وقابلة للتطوير  بالإضافة إلى أنها مجانية و مفتوح 
 
 
 
diff --git a/tests/misc/multi-test.html b/tests/misc/multi-test.html
index 2c8fe9e..9fe9648 100644
--- a/tests/misc/multi-test.html
+++ b/tests/misc/multi-test.html
@@ -1,5 +1,3 @@
-<h1 id="inthebeginning">Header </h1>
-<p>Now, let's try something <em class="special">inline</em>, to see if it works</p>
 <p>Blah blah blah <a href="http://www.slashdot.org">http://www.slashdot.org</a></p>
 <ul>
 <li>Basic list</li>
diff --git a/tests/misc/multi-test.txt b/tests/misc/multi-test.txt
index c4ab1c1..feaac31 100644
--- a/tests/misc/multi-test.txt
+++ b/tests/misc/multi-test.txt
@@ -1,9 +1,3 @@
-
-# Header {@id=inthebeginning}
-
-Now, let's try something *inline{@class=special}*, to see if it works
-
-
 Blah blah blah <http://www.slashdot.org>
 
 * Basic list
diff --git a/tests/misc/uche.html b/tests/misc/uche.html
index e62329d..9134e95 100644
--- a/tests/misc/uche.html
+++ b/tests/misc/uche.html
@@ -1,3 +1,3 @@
 <p><img alt="asif" src="http://fourthought.com/images/ftlogo.png" title="Fourthought logo" /></p>
-<p><a href="http://fourthought.com/"><img alt="" src="http://fourthought.com/images/ftlogo.png" style="float: left; margin: 10px; border: none;" title="Fourthought logo" /></a></p>
+<p><a href="http://fourthought.com/"><img alt="Alt text" src="http://fourthought.com/images/ftlogo.png" title="Fourthought logo" /></a></p>
 <p><a href="http://link.com/"><img alt="text" src="x" /></a></p>
\ No newline at end of file
diff --git a/tests/misc/uche.txt b/tests/misc/uche.txt
index a3dda1a..625d2ae 100644
--- a/tests/misc/uche.txt
+++ b/tests/misc/uche.txt
@@ -1,7 +1,6 @@
 ![asif](http://fourthought.com/images/ftlogo.png "Fourthought logo")
 
-[![{@style=float: left; margin: 10px; border:
-none;}](http://fourthought.com/images/ftlogo.png "Fourthought
+[![Alt text](http://fourthought.com/images/ftlogo.png "Fourthought
 logo")](http://fourthought.com/)
 
 [![text](x)](http://link.com/)
diff --git a/tests/test_apis.py b/tests/test_apis.py
index 251657b..2875c85 100644
--- a/tests/test_apis.py
+++ b/tests/test_apis.py
@@ -18,6 +18,7 @@ from logging import DEBUG, WARNING, CRITICAL
 import yaml
 import tempfile
 from io import BytesIO
+from xml.etree.ElementTree import ProcessingInstruction
 
 
 PY3 = sys.version_info[0] == 3
@@ -465,23 +466,47 @@ class testSerializers(unittest.TestCase):
     def testHtml(self):
         """ Test HTML serialization. """
         el = markdown.util.etree.Element('div')
+        el.set('id', 'foo<&">')
         p = markdown.util.etree.SubElement(el, 'p')
-        p.text = 'foo'
+        p.text = 'foo <&escaped>'
+        p.set('hidden', 'hidden')
         markdown.util.etree.SubElement(el, 'hr')
+        non_element = markdown.util.etree.SubElement(el, None)
+        non_element.text = 'non-element text'
+        script = markdown.util.etree.SubElement(non_element, 'script')
+        script.text = '<&"test\nescaping">'
+        el.tail = "tail text"
         self.assertEqual(
             markdown.serializers.to_html_string(el),
-            '<div><p>foo</p><hr></div>'
+            '<div id="foo&lt;&amp;&quot;&gt;">'
+            '<p hidden>foo &lt;&amp;escaped&gt;</p>'
+            '<hr>'
+            'non-element text'
+            '<script><&"test\nescaping"></script>'
+            '</div>tail text'
         )
 
     def testXhtml(self):
         """" Test XHTML serialization. """
         el = markdown.util.etree.Element('div')
+        el.set('id', 'foo<&">')
         p = markdown.util.etree.SubElement(el, 'p')
-        p.text = 'foo'
+        p.text = 'foo<&escaped>'
+        p.set('hidden', 'hidden')
         markdown.util.etree.SubElement(el, 'hr')
+        non_element = markdown.util.etree.SubElement(el, None)
+        non_element.text = 'non-element text'
+        script = markdown.util.etree.SubElement(non_element, 'script')
+        script.text = '<&"test\nescaping">'
+        el.tail = "tail text"
         self.assertEqual(
             markdown.serializers.to_xhtml_string(el),
-            '<div><p>foo</p><hr /></div>'
+            '<div id="foo&lt;&amp;&quot;&gt;">'
+            '<p hidden="hidden">foo&lt;&amp;escaped&gt;</p>'
+            '<hr />'
+            'non-element text'
+            '<script><&"test\nescaping"></script>'
+            '</div>tail text'
         )
 
     def testMixedCaseTags(self):
@@ -496,6 +521,68 @@ class testSerializers(unittest.TestCase):
             '<MixedCase>not valid <EMPHASIS>html</EMPHASIS><HR /></MixedCase>'
         )
 
+    def testProsessingInstruction(self):
+        """ Test serialization of ProcessignInstruction. """
+        pi = ProcessingInstruction('foo', text='<&"test\nescaping">')
+        self.assertIs(pi.tag, ProcessingInstruction)
+        self.assertEqual(
+            markdown.serializers.to_xhtml_string(pi),
+            '<?foo &lt;&amp;"test\nescaping"&gt;?>'
+        )
+
+    def testQNameTag(self):
+        """ Test serialization of QName tag. """
+        div = markdown.util.etree.Element('div')
+        qname = markdown.util.etree.QName('http://www.w3.org/1998/Math/MathML', 'math')
+        math = markdown.util.etree.SubElement(div, qname)
+        math.set('display', 'block')
+        sem = markdown.util.etree.SubElement(math, 'semantics')
+        msup = markdown.util.etree.SubElement(sem, 'msup')
+        mi = markdown.util.etree.SubElement(msup, 'mi')
+        mi.text = 'x'
+        mn = markdown.util.etree.SubElement(msup, 'mn')
+        mn.text = '2'
+        ann = markdown.util.etree.SubElement(sem, 'annotations')
+        ann.text = 'x^2'
+        self.assertEqual(
+            markdown.serializers.to_xhtml_string(div),
+            '<div>'
+            '<math display="block" xmlns="http://www.w3.org/1998/Math/MathML">'
+            '<semantics>'
+            '<msup>'
+            '<mi>x</mi>'
+            '<mn>2</mn>'
+            '</msup>'
+            '<annotations>x^2</annotations>'
+            '</semantics>'
+            '</math>'
+            '</div>'
+        )
+
+    def testQNameAttribute(self):
+        """ Test serialization of QName attribute. """
+        div = markdown.util.etree.Element('div')
+        div.set(markdown.util.etree.QName('foo'), markdown.util.etree.QName('bar'))
+        self.assertEqual(
+            markdown.serializers.to_xhtml_string(div),
+            '<div foo="bar"></div>'
+        )
+
+    def testBadQNameTag(self):
+        """ Test serialization of QName with no tag. """
+        qname = markdown.util.etree.QName('http://www.w3.org/1998/Math/MathML')
+        el = markdown.util.etree.Element(qname)
+        self.assertRaises(ValueError, markdown.serializers.to_xhtml_string, el)
+
+    def testQNameEscaping(self):
+        """ Test QName escaping. """
+        qname = markdown.util.etree.QName('<&"test\nescaping">', 'div')
+        el = markdown.util.etree.Element(qname)
+        self.assertEqual(
+            markdown.serializers.to_xhtml_string(el),
+            '<div xmlns="&lt;&amp;&quot;test&#10;escaping&quot;&gt;"></div>'
+        )
+
     def buildExtension(self):
         """ Build an extension which registers fakeSerializer. """
         def fakeSerializer(elem):
diff --git a/tests/test_extensions.py b/tests/test_extensions.py
index aee9bac..35eaf1d 100644
--- a/tests/test_extensions.py
+++ b/tests/test_extensions.py
@@ -454,6 +454,20 @@ The body. This is paragraph one.'''
         self.assertEqual(self.md.convert(text), '')
         self.assertEqual(self.md.Meta, {'title': ['No newline']})
 
+    def testMetaDataReset(self):
+        """ Test that reset call remove Meta entirely """
+
+        text = '''Title: A Test Doc.
+Author: Waylan Limberg
+        John Doe
+Blank_Data:
+
+The body. This is paragraph one.'''
+        self.md.convert(text)
+
+        self.md.reset()
+        self.assertEqual(self.md.Meta, {})
+
 
 class TestWikiLinks(unittest.TestCase):
     """ Test Wikilinks Extension. """
diff --git a/tests/test_legacy.py b/tests/test_legacy.py
index 7fca02a..26d08de 100644
--- a/tests/test_legacy.py
+++ b/tests/test_legacy.py
@@ -90,7 +90,7 @@ class TestPl2004(LegacyTestCase):
     location = os.path.join(parent_test_dir, 'pl/Tests_2004')
     normalize = True
     input_ext = '.text'
-    exclude = ['Yuri_Footnotes']
+    exclude = ['Yuri_Footnotes', 'Yuri_Attributes']
 
 
 class TestPl2007(LegacyTestCase):
diff --git a/tests/test_syntax/extensions/test_legacy_attrs.py b/tests/test_syntax/extensions/test_legacy_attrs.py
new file mode 100644
index 0000000..506fce8
--- /dev/null
+++ b/tests/test_syntax/extensions/test_legacy_attrs.py
@@ -0,0 +1,48 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+from markdown.test_tools import TestCase
+
+
+class TestLegacyAtrributes(TestCase):
+
+    maxDiff = None
+
+    def testLegacyAttrs(self):
+        self.assertMarkdownRenders(
+            self.dedent("""
+                # Header {@id=inthebeginning}
+
+                Now, let's try something *inline{@class=special}*, to see if it works
+
+                @id=TABLE.OF.CONTENTS}
+
+
+                * {@id=TABLEOFCONTENTS}
+
+
+                Or in the middle of the text {@id=TABLEOFCONTENTS}
+
+                {@id=tableofcontents}
+
+                [![{@style=float: left; margin: 10px; border:
+                none;}](http://fourthought.com/images/ftlogo.png "Fourthought
+                logo")](http://fourthought.com/)
+
+                ![img{@id=foo}][img]
+
+                [img]: http://example.com/i.jpg
+            """),
+            self.dedent("""
+                <h1 id="inthebeginning">Header </h1>
+                <p>Now, let's try something <em class="special">inline</em>, to see if it works</p>
+                <p>@id=TABLE.OF.CONTENTS}</p>
+                <ul>
+                <li id="TABLEOFCONTENTS"></li>
+                </ul>
+                <p id="TABLEOFCONTENTS">Or in the middle of the text </p>
+                <p id="tableofcontents"></p>
+                <p><a href="http://fourthought.com/"><img alt="" src="http://fourthought.com/images/ftlogo.png" style="float: left; margin: 10px; border: none;" title="Fourthought logo" /></a></p>
+                <p><img alt="img" id="foo" src="http://example.com/i.jpg" /></p>
+            """),  # noqa: E501
+            extensions=['legacy_attrs']
+        )