Repository: getpelican/pelican

Issue:
Metadata field not recognized
I can't seem to get ARTICLE_ORDER_BY working.  I see this message in the log irrespective of the custom metadata field I try to sort by:

WARNING: There is no "date_parsed" attribute in the item metadata.Defaulting to slug order.

I know the metadata is present because I can use it in Jinja templates like so:  {{ article.date_parsed }}

I'm using Pelican 3.5.0 and generating RST files that contain the metadata.  Am I missing something obvious?



Discussion:
I managed to get the metadata warning to go away, but still can't seem to get ARTICLE_ORDER_BY working.  Just seems to ignore it.  Any ideas to help figure out why would be greatly appreciated!

For what it's worth, the relevant code is in `utils.process_translations`, lines 507-521: https://github.com/getpelican/pelican/blob/30b8a2069ee610b581c4120c40ebdbb6ec93a09d/pelican/utils.py#L507-521

Hmm… same problem here

`ARTICLE_ORDER_BY = 'modified'`
there is no error nor warning but index is as before.

This is because `ArticlesGenerator` [sorts articles](https://github.com/getpelican/pelican/blob/master/pelican/generators.py#L552) after `process_translations` is performed. 

I'm guessing, when this feature was added (#1348) main concern was `PAGE_ORDER_BY`. `ARTICLE_ORDER_BY` was an afterthought and wasn't tested properly. As a reviewer of that PR, I should share the blame :).

Basically, the sorting done afterwards should be moved into `process_translations` as a default for `ARTICLE_ORDER_BY`. That means, `*_ORDER_BY` has to be able to manage reversing and the way to do that right now is a custom function. I'll try to come up with a reasonable solution. Preferably something that'll handle reversing the order easier than a custom function. Maybe something like `'reversed-date'` vs `'date'`. Alternatively, the `*_ORDER_BY` settings can be changed to a `tuple`/`list` in the form of `(key, reversed)` pairs (e.g. `('date', True)`)

Just use `*_ORDER_DIRECTION`, pretty standard stuff... we need this but going to have to hack it I guess.

I have `reversed-date`/`date` pretty much implemented. I just need to add tests before submitting a PR. It should be ready in a day or two if my time permits.

I don't want to add additional settings. For one, settings is pretty overcrowded as is. And secondly, `ARTICLE_ORDER_DIRECTION` (or `ARTICLE_ORDER_REVERSED` as I'd prefer) would be `True` by default and may trip up some people that don't pay attention.


Resolving patch:
diff --git a/pelican/generators.py b/pelican/generators.py
--- a/pelican/generators.py
+++ b/pelican/generators.py
@@ -561,8 +561,7 @@ def generate_context(self):
                     self.tags[tag].append(article)
             for author in getattr(article, 'authors', []):
                 self.authors[author].append(article)
-        # sort the articles by date
-        self.articles.sort(key=attrgetter('date'), reverse=True)
+
         self.dates = list(self.articles)
         self.dates.sort(key=attrgetter('date'),
                         reverse=self.context['NEWEST_FIRST_ARCHIVES'])
diff --git a/pelican/settings.py b/pelican/settings.py
--- a/pelican/settings.py
+++ b/pelican/settings.py
@@ -65,7 +65,7 @@
     'OUTPUT_RETENTION': [],
     'ARTICLE_URL': '{slug}.html',
     'ARTICLE_SAVE_AS': '{slug}.html',
-    'ARTICLE_ORDER_BY': 'slug',
+    'ARTICLE_ORDER_BY': 'reversed-date',
     'ARTICLE_LANG_URL': '{slug}-{lang}.html',
     'ARTICLE_LANG_SAVE_AS': '{slug}-{lang}.html',
     'DRAFT_URL': 'drafts/{slug}.html',
diff --git a/pelican/utils.py b/pelican/utils.py
--- a/pelican/utils.py
+++ b/pelican/utils.py
@@ -541,16 +541,28 @@ def process_translations(content_list, order_by=None):
             try:
                 index.sort(key=order_by)
             except Exception:
-                logger.error('Error sorting with function {}'.format(order_by))
-        elif order_by == 'basename':
-            index.sort(key=lambda x: os.path.basename(x.source_path or ''))
-        elif order_by != 'slug':
-            try:
-                index.sort(key=attrgetter(order_by))
-            except AttributeError:
-                error_msg = ('There is no "{}" attribute in the item metadata.'
-                             'Defaulting to slug order.')
-                logger.warning(error_msg.format(order_by))
+                logger.error('Error sorting with function %s', order_by)
+        elif isinstance(order_by, six.string_types):
+            if order_by.startswith('reversed-'):
+                order_reversed = True
+                order_by = order_by.replace('reversed-', '', 1)
+            else:
+                order_reversed = False
+
+            if order_by == 'basename':
+                index.sort(key=lambda x: os.path.basename(x.source_path or ''),
+                           reverse=order_reversed)
+            # already sorted by slug, no need to sort again
+            elif not (order_by == 'slug' and not order_reversed):
+                try:
+                    index.sort(key=attrgetter(order_by),
+                               reverse=order_reversed)
+                except AttributeError:
+                    logger.warning('There is no "%s" attribute in the item '
+                        'metadata. Defaulting to slug order.', order_by)
+        else:
+            logger.warning('Invalid *_ORDER_BY setting (%s).'
+                'Valid options are strings and functions.', order_by)
 
     return index, translations
 


Test patch:
diff --git a/pelican/tests/test_generators.py b/pelican/tests/test_generators.py
--- a/pelican/tests/test_generators.py
+++ b/pelican/tests/test_generators.py
@@ -387,6 +387,65 @@ def test_standard_metadata_in_default_metadata(self):
                                 'パイソン', 'マック'])
         self.assertEqual(tags, tags_expected)
 
+    def test_article_order_by(self):
+        settings = get_settings(filenames={})
+        settings['DEFAULT_CATEGORY'] = 'Default'
+        settings['DEFAULT_DATE'] = (1970, 1, 1)
+        settings['CACHE_CONTENT'] = False   # cache not needed for this logic tests
+        settings['ARTICLE_ORDER_BY'] = 'title'
+
+        generator = ArticlesGenerator(
+            context=settings.copy(), settings=settings,
+            path=CONTENT_DIR, theme=settings['THEME'], output_path=None)
+        generator.generate_context()
+
+        expected = [
+            'An Article With Code Block To Test Typogrify Ignore',
+            'Article title',
+            'Article with Nonconformant HTML meta tags',
+            'Article with markdown and summary metadata multi',
+            'Article with markdown and summary metadata single',
+            'Article with markdown containing footnotes',
+            'Article with template',
+            'Rst with filename metadata',
+            'Test Markdown extensions',
+            'Test markdown File',
+            'Test md File',
+            'Test mdown File',
+            'Test mkd File',
+            'This is a super article !',
+            'This is a super article !',
+            'This is a super article !',
+            'This is a super article !',
+            'This is a super article !',
+            'This is a super article !',
+            'This is an article with category !',
+            'This is an article with multiple authors in lastname, firstname format!',
+            'This is an article with multiple authors in list format!',
+            'This is an article with multiple authors!',
+            'This is an article with multiple authors!',
+            'This is an article without category !',
+            'This is an article without category !',
+            'マックOS X 10.8でパイソンとVirtualenvをインストールと設定']
+
+        articles = [article.title for article in generator.articles]
+        self.assertEqual(articles, expected)
+
+        # reversed title
+        settings = get_settings(filenames={})
+        settings['DEFAULT_CATEGORY'] = 'Default'
+        settings['DEFAULT_DATE'] = (1970, 1, 1)
+        settings['CACHE_CONTENT'] = False   # cache not needed for this logic tests
+        settings['ARTICLE_ORDER_BY'] = 'reversed-title'
+
+        generator = ArticlesGenerator(
+            context=settings.copy(), settings=settings,
+            path=CONTENT_DIR, theme=settings['THEME'], output_path=None)
+        generator.generate_context()
+
+        articles = [article.title for article in generator.articles]
+        self.assertEqual(articles, list(reversed(expected)))
+
 
 class TestPageGenerator(unittest.TestCase):
     # Note: Every time you want to test for a new field; Make sure the test
@@ -473,6 +532,23 @@ def test_generate_sorted(self):
         pages = self.distill_pages(generator.pages)
         self.assertEqual(pages_expected_sorted_by_title, pages)
 
+        # sort by title reversed
+        pages_expected_sorted_by_title = [
+            ['This is a test page with a preset template', 'published',
+             'custom'],
+            ['This is a test page', 'published', 'page'],
+            ['This is a markdown test page', 'published', 'page'],
+            ['Page with a bunch of links', 'published', 'page'],
+            ['A Page (Test) for sorting', 'published', 'page'],
+        ]
+        settings['PAGE_ORDER_BY'] = 'reversed-title'
+        generator = PagesGenerator(
+            context=settings.copy(), settings=settings,
+            path=CUR_DIR, theme=settings['THEME'], output_path=None)
+        generator.generate_context()
+        pages = self.distill_pages(generator.pages)
+        self.assertEqual(pages_expected_sorted_by_title, pages)
+
     def test_tag_and_category_links_on_generated_pages(self):
         """
         Test to ensure links of the form {tag}tagname and {category}catname