Repository: getpelican/pelican

Issue:
Reader level cache results in TypeError with empty metadata in Markdown
- [x] I have read the [Filing Issues](https://docs.getpelican.com/en/latest/contribute.html#filing-issues) and subsequent “How to Get Help” sections of the documentation.
- [x] I have searched the [issues](https://github.com/getpelican/pelican/issues?q=is%3Aissue) (including closed ones) and believe that this is not a duplicate.

<!--
  Once the above boxes are checked, if you are able to fill in the following list
  with your information, it would be very helpful for maintainers.
-->

- **Python version**: 3.8.2
- **Pelican version**: 4.5.3

## Issue

Metadata that should be discarded ([`_DISCARD`](https://github.com/getpelican/pelican/blob/8033162ba4393db60791b201fb100d1be0f04431/pelican/readers.py#L27-L31)) is stored in the cache and upon retrieval new `object()` instance is created and [fails filtration](https://github.com/getpelican/pelican/blob/8033162ba4393db60791b201fb100d1be0f04431/pelican/readers.py#L575).

Relevant information for reproducing is below. Note the empty `Tag:` metadata. First run is without cache and succeeds. Second run retrieves from cache and errors due to not being filtered.
```console
$ cat content/article.md
Title: asd
Tags:

Content here
$ cat pelicanconf.py
DEFAULT_DATE = 'fs'

CACHE_CONTENT = True
LOAD_CONTENT_CACHE = True
CONTENT_CACHING_LAYER= 'reader'
$ pelican content -s pelicanconf.py
WARNING: Feeds generated without SITEURL set properly may not be valid
WARNING: No timezone information specified in the settings. Assuming your timezone is UTC for feed generation. Check https://docs.getpelican.com/en/latest/settings.html#TIMEZONE for more information
WARNING: Watched path does not exist: /home/avaris/ws/test/content/images
Done: Processed 1 article, 0 drafts, 0 pages, 0 hidden pages and 0 draft pages in 0.11 seconds.
$ pelican content -s pelicanconf.py
WARNING: Feeds generated without SITEURL set properly may not be valid
WARNING: No timezone information specified in the settings. Assuming your timezone is UTC for feed generation. Check https://docs.getpelican.com/en/latest/settings.html#TIMEZONE for more information
WARNING: Watched path does not exist: /home/avaris/ws/test/content/images
CRITICAL: 'object' object is not iterable
$ pelican content -s pelicanconf.py -D
DEBUG: Pelican version: 4.5.3
DEBUG: Python version: 3.8.2
[snip]
CRITICAL: 'object' object is not iterable
Traceback (most recent call last):
  File "/home/avaris/.venvs/ptest/bin/pelican", line 8, in <module>
    sys.exit(main())
  File "/home/avaris/.venvs/ptest/lib/python3.8/site-packages/pelican/__init__.py", line 523, in main
    pelican.run()
  File "/home/avaris/.venvs/ptest/lib/python3.8/site-packages/pelican/__init__.py", line 109, in run
    p.generate_context()
  File "/home/avaris/.venvs/ptest/lib/python3.8/site-packages/pelican/generators.py", line 659, in generate_context
    for tag in article.tags:
TypeError: 'object' object is not iterable

```


Discussion:
Hi @avaris, is this fix still wanted? 
I was looking into the [code](https://github.com/getpelican/pelican/blob/4.7.0/pelican/readers.py#L571-L575) and it seems fairly easy to fix it. 



From: 
```python
        content, reader_metadata = self.get_cached_data(path, (None, None))
        if content is None:
            content, reader_metadata = reader.read(path)
            self.cache_data(path, (content, reader_metadata))
        metadata.update(_filter_discardable_metadata(reader_metadata))
```

To:
```python
        content, reader_metadata = self.get_cached_data(path, (None, None))
        if content is None:
            content, reader_metadata = reader.read(path)
            reader_metadata = _filter_discardable_metadata(reader_metadata)  # discard before caching
            self.cache_data(path, (content, reader_metadata))
        metadata.update(reader_metadata)
```

I was wondering it's ok for me to submit a PR for it + unit tests covering that scenario.
Hi @jonasborges, yes this is still needed and you are more than welcome to submit a PR for it :). Thanks. Your suggested change is correct.

Resolving patch:
diff --git a/pelican/readers.py b/pelican/readers.py
--- a/pelican/readers.py
+++ b/pelican/readers.py
@@ -571,8 +571,9 @@ def read_file(self, base_path, path, content_class=Page, fmt=None,
         content, reader_metadata = self.get_cached_data(path, (None, None))
         if content is None:
             content, reader_metadata = reader.read(path)
+            reader_metadata = _filter_discardable_metadata(reader_metadata)
             self.cache_data(path, (content, reader_metadata))
-        metadata.update(_filter_discardable_metadata(reader_metadata))
+        metadata.update(reader_metadata)
 
         if content:
             # find images with empty alt


Test patch:
diff --git a/pelican/tests/content/article_with_markdown_and_empty_tags.md b/pelican/tests/content/article_with_markdown_and_empty_tags.md
new file mode 100644
--- /dev/null
+++ b/pelican/tests/content/article_with_markdown_and_empty_tags.md
@@ -0,0 +1,4 @@
+Title: Article with markdown and empty tags
+Tags:
+
+This is some content.
diff --git a/pelican/tests/test_generators.py b/pelican/tests/test_generators.py
--- a/pelican/tests/test_generators.py
+++ b/pelican/tests/test_generators.py
@@ -265,6 +265,8 @@ def test_generate_context(self):
             ['This is a super article !', 'published', 'yeah', 'article'],
             ['This is a super article !', 'published', 'Default', 'article'],
             ['Article with an inline SVG', 'published', 'Default', 'article'],
+            ['Article with markdown and empty tags', 'published', 'Default',
+             'article'],
             ['This is an article with category !', 'published', 'yeah',
              'article'],
             ['This is an article with multiple authors!', 'published',
@@ -569,6 +571,7 @@ def test_article_order_by(self):
             'Article title',
             'Article with Nonconformant HTML meta tags',
             'Article with an inline SVG',
+            'Article with markdown and empty tags',
             'Article with markdown and nested summary metadata',
             'Article with markdown and summary metadata multi',
             'Article with markdown and summary metadata single',
diff --git a/pelican/tests/test_readers.py b/pelican/tests/test_readers.py
--- a/pelican/tests/test_readers.py
+++ b/pelican/tests/test_readers.py
@@ -18,6 +18,7 @@ class ReaderTest(unittest.TestCase):
 
     def read_file(self, path, **kwargs):
         # Isolate from future API changes to readers.read_file
+
         r = readers.Readers(settings=get_settings(**kwargs))
         return r.read_file(base_path=CONTENT_PATH, path=path)
 
@@ -795,6 +796,23 @@ def test_typogrify_dashes_config(self):
         self.assertEqual(page.content, expected)
         self.assertEqual(page.title, expected_title)
 
+    def test_metadata_has_no_discarded_data(self):
+        md_filename = 'article_with_markdown_and_empty_tags.md'
+
+        r = readers.Readers(cache_name='cache', settings=get_settings(
+            CACHE_CONTENT=True))
+        page = r.read_file(base_path=CONTENT_PATH, path=md_filename)
+
+        __, cached_metadata = r.get_cached_data(
+            _path(md_filename), (None, None))
+
+        expected = {
+            'title': 'Article with markdown and empty tags'
+        }
+        self.assertEqual(cached_metadata, expected)
+        self.assertNotIn('tags', page.metadata)
+        self.assertDictHasSubset(page.metadata, expected)
+
 
 class HTMLReaderTest(ReaderTest):
     def test_article_with_comments(self):