Repository: getpelican/pelican

Issue:
Markdown Summary metadata is parsed for additional metadata
The Markdown parser forces the "meta" extension for parsing metadata headers. This parser object is reused when formatting header values for any header listed in the `FORMATTED_FIELDS` setting.

Because header values like Summary are parsed for metadata, any metadata-like values are discarded from the final output. Example:

```
Summary: Test: output
```

The header value of `Test: output` will be parsed as metadata, recognized as a header, and discarded from the final formatted Markdown output, resulting in an empty summary.



Discussion:
afaict, you found a bug in https://github.com/waylan/Python-Markdown/blob/master/markdown/extensions/meta.py#L27 

You seem to want that plugin to disregard any colons after the first one, until a new line.  That seems like it makes sense.

You might also want to look at, and possibly even request clarification on their practices, https://github.com/fletcher/MultiMarkdown/wiki/MultiMarkdown-Syntax-Guide#metadata

I think it's more due to the double parsing. When the full document is parsed, the summary meta is correct:

```
{"summary": "Test: output"}
```

Pelican then re-parses the meta value ("Test: output") as Markdown, which is nice for bold/italic/links/etc., but not great when your value looks like Markdown meta. The resulting render has an empty body, with some meta:

```
{"test": "output"}
```

Python-Markdown correctly parses the value on the first pass.

So, reparsing should be done without the `meta` extension?

Exactly, @avaris. I don't see a way to disable meta on the parser once the extension is loaded, so the reader might need a dedicated Markdown parser for use with `FORMATTED_FIELDS`.

Working from #1573 could also potentially solve this (with some other long term gains).  How does `pelican_commonmark`'s metadata parser compare to the current core parser?

Hi Annika. Is something you would be willing to help us fix/implement?


Resolving patch:
diff --git a/pelican/readers.py b/pelican/readers.py
--- a/pelican/readers.py
+++ b/pelican/readers.py
@@ -302,6 +302,9 @@ def _parse_metadata(self, meta):
         """Return the dict containing document metadata"""
         formatted_fields = self.settings['FORMATTED_FIELDS']
 
+        # prevent metadata extraction in fields
+        self._md.preprocessors.deregister('meta')
+
         output = {}
         for name, value in meta.items():
             name = name.lower()


Test patch:
diff --git a/pelican/tests/content/article_with_markdown_and_nested_metadata.md b/pelican/tests/content/article_with_markdown_and_nested_metadata.md
new file mode 100644
--- /dev/null
+++ b/pelican/tests/content/article_with_markdown_and_nested_metadata.md
@@ -0,0 +1,5 @@
+Title: Article with markdown and nested summary metadata
+Date: 2012-10-30
+Summary: Test: This metadata value looks like metadata
+
+This is some content.
diff --git a/pelican/tests/test_generators.py b/pelican/tests/test_generators.py
--- a/pelican/tests/test_generators.py
+++ b/pelican/tests/test_generators.py
@@ -239,6 +239,8 @@ def test_generate_context(self):
             ['Article title', 'published', 'Default', 'article'],
             ['Article with markdown and summary metadata multi', 'published',
              'Default', 'article'],
+            ['Article with markdown and nested summary metadata', 'published',
+             'Default', 'article'],
             ['Article with markdown and summary metadata single', 'published',
              'Default', 'article'],
             ['Article with markdown containing footnotes', 'published',
@@ -554,6 +556,7 @@ def test_article_order_by(self):
             'Article title',
             'Article with Nonconformant HTML meta tags',
             'Article with an inline SVG',
+            'Article with markdown and nested summary metadata',
             'Article with markdown and summary metadata multi',
             'Article with markdown and summary metadata single',
             'Article with markdown containing footnotes',
diff --git a/pelican/tests/test_readers.py b/pelican/tests/test_readers.py
--- a/pelican/tests/test_readers.py
+++ b/pelican/tests/test_readers.py
@@ -645,6 +645,19 @@ def test_duplicate_tags_or_authors_are_removed(self):
         }
         self.assertDictHasSubset(metadata, expected)
 
+    def test_metadata_not_parsed_for_metadata(self):
+        settings = get_settings()
+        settings['FORMATTED_FIELDS'] = ['summary']
+
+        reader = readers.MarkdownReader(settings=settings)
+        content, metadata = reader.read(
+            _path('article_with_markdown_and_nested_metadata.md'))
+        expected = {
+            'title': 'Article with markdown and nested summary metadata',
+            'summary': '<p>Test: This metadata value looks like metadata</p>',
+        }
+        self.assertDictHasSubset(metadata, expected)
+
     def test_empty_file(self):
         reader = readers.MarkdownReader(settings=get_settings())
         content, metadata = reader.read(