Repository: internetarchive/openlibrary

Issue:
Date format isn't correctly read off edition.
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->

### Evidence / Screenshot (if possible)
![Screenshot_2021-12-29 Kerstin Gier Open Library](https://user-images.githubusercontent.com/33803304/147663665-59810ef6-d97a-4fb4-bedd-5847039916c2.png)

### Relevant url?
<!-- `https://openlibrary.org/...` -->

[Affected work](https://openlibrary.org/works/OL26745647W)

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ...
2. Do ...

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual:
* Expected:

### Details

when adding the publication date to editions in the recommended format (YYYY-MM-DD), the website isn't able to display the line "First published in YYYY" on an author's page. "Sep 29th, 2021" would work on the other hand. This doesn't allow to correctly sort by most recent/first published.

### Proposal & Constraints
as both formats "YYYY" and "Sep 29th, 2021" work and are correctly sorted on a work's page, I can imagine those two are converted to the same format in order to be able to be compared to each other. If that's not the case for "YYYY-MM-DD", it would be worth investigating if that effects the sort order in the edition list as well.

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
@cdrini 


Discussion:
@HamsterDeveloper one consideration worth clarifying is that on the Author page, we're actually seeing a list of Works and not Editions.

This happens to be the result of technical limitations, not a our decision about the experience. This list is populated by our search engine with indexes works (not editions). This is something we have scheduled to work on in the 1st half of this year.

@cdrini as I understand it, because we're seeing works on the author's page, we're not easily able to show the edition level facet information (like date) that @HamsterDeveloper might prefer.
@mekarpeles it's really only about the format of the source data though, because the relevant information is already being taken from the edition and displayed correctly if another date format is used. 
When the date is entered into an edition in ISO 8601 format the SOLR document attribute "first_publish_year" doesn't get created for the indexed edition. This is the data that ultimately gets used by the SearchResultsWork macro and also the worksearch module. The reason it doesn't get indexed are lines 597-602 in update_work.py:

```python
pub_years = {
    m.group(1) for m in (re_year.search(date) for date in pub_dates) if m
}
if pub_years:
    add_list('publish_year', pub_years)
    add('first_publish_year', min(int(y) for y in pub_years))
```
re_year is a regular expression that matches 4-digit years but doesn't match ISO 8601 dates. See https://regex101.com/r/QUlBC0/1

I tested this in my local environment by creating some editions with ISO 8601 dates and examining the documents in SOLR for those editions and noticed the missing "first_publish_year" attribute.

Resolving patch:
diff --git a/openlibrary/solr/update_work.py b/openlibrary/solr/update_work.py
--- a/openlibrary/solr/update_work.py
+++ b/openlibrary/solr/update_work.py
@@ -43,9 +43,8 @@
 re_author_key = re.compile(r'^/(?:a|authors)/(OL\d+A)')
 re_bad_char = re.compile('[\x01\x0b\x1a-\x1e]')
 re_edition_key = re.compile(r"/books/([^/]+)")
-re_iso_date = re.compile(r'^(\d{4})-\d\d-\d\d$')
 re_solr_field = re.compile(r'^[-\w]+$', re.U)
-re_year = re.compile(r'(\d{4})$')
+re_year = re.compile(r'\b(\d{4})\b')
 
 # This will be set to a data provider; have faith, mypy!
 data_provider = cast(DataProvider, None)
@@ -491,9 +490,6 @@ def get_pub_year(self, e):
         """
         pub_date = e.get('publish_date', None)
         if pub_date:
-            m = re_iso_date.match(pub_date)
-            if m:
-                return m.group(1)
             m = re_year.search(pub_date)
             if m:
                 return m.group(1)
@@ -595,8 +591,9 @@ def add_list(name, values):
         pub_dates = {e[k] for e in editions if e.get(k)}
         add_list(k, pub_dates)
         pub_years = {
-            m.group(1) for m in (re_year.search(date) for date in pub_dates) if m
+            self.get_pub_year(e) for e in editions
         }
+        pub_years = pub_years - {None,}
         if pub_years:
             add_list('publish_year', pub_years)
             add('first_publish_year', min(int(y) for y in pub_years))


Test patch:
diff --git a/openlibrary/tests/solr/test_update_work.py b/openlibrary/tests/solr/test_update_work.py
--- a/openlibrary/tests/solr/test_update_work.py
+++ b/openlibrary/tests/solr/test_update_work.py
@@ -164,10 +164,12 @@ def test_publish_year(self):
         test_dates = [
             "2000",
             "Another 2000",
-            "2001-01-02",  # Doesn't seems to be handling this case
+            "2001-01-02",  # ISO 8601 formatted dates now supported
             "01-02-2003",
+            "2004 May 23",
             "Jan 2002",
             "Bad date 12",
+            "Bad date 123412314",
         ]
         work = make_work()
         update_work.data_provider = FakeDataProvider(
@@ -175,7 +177,7 @@ def test_publish_year(self):
         )
 
         d = build_data(work)
-        assert sorted(d['publish_year']) == ["2000", "2002", "2003"]
+        assert sorted(d['publish_year']) == ["2000", "2001", "2002", "2003", "2004"]
         assert d["first_publish_year"] == 2000
 
     def test_isbns(self):