Repository: internetarchive/openlibrary

Issue:
092 DDC shelf polluted with wrong works
In DDC, `92` or `920` is sometimes appended to a DDC to denote "Biography". We have a clause to handle this if it's part of the same DDC ( https://github.com/internetarchive/openlibrary/blob/master/openlibrary/utils/ddc.py#L27 ), but some books have it as an entirely separate DDC; e.g.:

- `[ "979.4/830046872073", "92" ]` https://openlibrary.org/books/OL3029363M
- `[ "813/.54", "B", "92" ]` https://openlibrary.org/books/OL2401343M

### Evidence / Screenshot (if possible)

### Relevant url?
https://openlibrary.org/explore

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to the 09X section

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual: A bunch of biographies!
* Expected: Not a bunch of biographies

### Details

- **Logged in (Y/N)?** Y
- **Browser type/version?** FF83
- **Operating system?** Win10
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
We can't handle this from the `normalize_ddc` function, since it only handles a string. We should add a `normalize_ddc_list` function to this file, and call it from https://github.com/internetarchive/openlibrary/blob/49603f38042f98699c05716d3946d0172f5e2d59/openlibrary/solr/update_work.py#L531-L540 . We should also add tests to make sure these new cases are handled.

### Related files

- https://github.com/internetarchive/openlibrary/blob/master/openlibrary/utils/ddc.py

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/solr/update_work.py b/openlibrary/solr/update_work.py
--- a/openlibrary/solr/update_work.py
+++ b/openlibrary/solr/update_work.py
@@ -568,9 +568,26 @@ def add_list(name, values):
             # Choose the... idk, longest for sorting?
             add("lcc_sort", sorted(lccs, key=len, reverse=True)[0])
 
+        def get_edition_ddcs(ed: dict):
+            ddcs = ed.get('dewey_decimal_class', [])  # type: List[str]
+            if len(ddcs) > 1:
+                # In DDC, `92` or `920` is sometimes appended to a DDC to denote
+                # "Biography". We have a clause to handle this if it's part of the same
+                # DDC (See utils/ddc.py), but some books have it as an entirely separate
+                # DDC; e.g.:
+                # * [ "979.4/830046872073", "92" ]
+                #   https://openlibrary.org/books/OL3029363M.json
+                # * [ "813/.54", "B", "92" ]
+                #   https://openlibrary.org/books/OL2401343M.json
+                ddcs = [
+                    ddcs[0],
+                    *[ddc for ddc in ddcs[1:] if ddc not in ('92', '920')]
+                ]
+            return ddcs
+
         raw_ddcs = set(ddc
                        for ed in editions
-                       for ddc in ed.get('dewey_decimal_class', []))
+                       for ddc in get_edition_ddcs(ed))
         ddcs = set(ddc
                    for raw_ddc in raw_ddcs
                    for ddc in normalize_ddc(raw_ddc))


Test patch:
diff --git a/openlibrary/tests/solr/test_update_work.py b/openlibrary/tests/solr/test_update_work.py
--- a/openlibrary/tests/solr/test_update_work.py
+++ b/openlibrary/tests/solr/test_update_work.py
@@ -409,6 +409,11 @@ def test_lccs(self, doc_lccs, solr_lccs, sort_lcc_index):
         'Ignores ISBNs/LCCs': (
             ['9781234123411', 'ML410', '132.3'],
             ['132.3'], 0),
+        'Ignores superfluous 920s': (['123.5', '920'], ['123.5'], 0),
+        'Ignores superfluous 92s': (['123.5', '92'], ['123.5'], 0),
+        'Ignores superfluous 92s (2)': (['123.5', 'B', '92'], ['123.5'], 0),
+        'Does not skip 920s': (['920', '123.5'], ['123.5', '920'], 0),
+        'Does not skip 92s': (['92', '123.5'], ['092', '123.5'], 1),
     }
 
     @pytest.mark.parametrize("doc_ddcs,solr_ddcs,sort_ddc_index",