Repository: pypi/warehouse

Issue:
Legacy and Warehouse RSS feeds differ
@andrew asked in https://github.com/librariesio/libraries.io/issues/2024#issuecomment-372638824 about Warehouse's RSS feeds:

> * https://pypi.org/rss/updates.xml
> * https://pypi.org/rss/packages.xml

> Which I expected the contents to match the old ones but currently don't:

> * https://pypi.python.org/pypi?%3Aaction=rss
> * https://pypi.python.org/pypi?%3Aaction=packages_rss

I've verified through visual inspection that the data in the legacy RSS feed and the data in the Warehouse RSS feeds differ.

This is a bug in the feeds or a bug in the docs. Currently our [feeds documentation](https://warehouse.readthedocs.io/api-reference/feeds/) and [Warehouse migration guide](https://warehouse.readthedocs.io/api-reference/integration-guide/) don't say anything about deliberate differences between the legacy and Warehouse RSS feeds. We can update the docs if there's a deliberate reason for the difference.


Discussion:
I don't believe there should be any differences between the two of them (other than the obvious difference of the URL changing), so it's likely a bug somewhere. Is it a difference in the structure of the data, or the data itself?
@dstufft the structure looks the same to me, looks like the order of the items in `<pubDate>` seems to be whats causing the bug
Ah, probably just https://github.com/pypa/warehouse/blob/master/warehouse/templates/rss/packages.xml#L11 and https://github.com/pypa/warehouse/blob/master/warehouse/templates/rss/updates.xml#L10 need adjusted. Which might be fixable just by adjusting https://github.com/pypa/warehouse/blob/54e59eab93be8aee21d96d993b64bf53272b9087/warehouse/i18n/filters.py#L35-L37. I haven't dug into it to figure out why there is a difference, but those would likely be the right places to do a fix.
Looks like the `all-projects` cache key is not set for these as well, so the results are as much as ~24 hours stale.
I think Warehouse is using the correct RFC 822 format here, it's legacy that's incorrect: https://validator.w3.org/feed/docs/error/InvalidRFC2822Date.html

We have some other issues though:
* for the `updates` feed, we're using `release.project.created` and not `release.created` for the `pubdate`
* The DTD we're using has been deprecated: https://validator.w3.org/feed/docs/error/DeprecatedDTD.html
Okay, so we'll need to either decide if we want to use the correct format, or maintain compatibility with legacy. I feel like I *probably* want to document that the format is changed to match the correct format, but I'm happy to be persuaded otherwise.

Resolving patch:
diff --git a/warehouse/rss/views.py b/warehouse/rss/views.py
--- a/warehouse/rss/views.py
+++ b/warehouse/rss/views.py
@@ -26,6 +26,7 @@
             1 * 24 * 60 * 60,                         # 1 day
             stale_while_revalidate=1 * 24 * 60 * 60,  # 1 day
             stale_if_error=5 * 24 * 60 * 60,          # 5 days
+            keys=["all-projects"],
         ),
     ],
 )
@@ -53,6 +54,7 @@ def rss_updates(request):
             1 * 24 * 60 * 60,                         # 1 day
             stale_while_revalidate=1 * 24 * 60 * 60,  # 1 day
             stale_if_error=5 * 24 * 60 * 60,          # 5 days
+            keys=["all-projects"],
         ),
     ],
 )


Test patch: