Repository: internetarchive/openlibrary

Issue:
DDC normalizer should ignore dewey decimal catalogue edition number
A lot of books specify the version of the dewey decimal catalogue as part of the dewey decimal:

- `338.9/009171/7 19` https://openlibrary.org/works/OL15597238W/The_economics_of_feasible_socialism
- `332.6 21` https://openlibrary.org/books/OL7454161M/The_Random_Walk_Guide_To_Investing
- `[E] 21` https://openlibrary.org/works/OL14933747W/Geisterstunde_im_Kindergarten

The current ddc indexer treats these as 2 ddcs (e.g. `['332.6', '021']`), causing the 0* section to be polluted with entirely unrelated books.

At another time we should consider what exactly we want to do with the ddc edition, but for now we need to remove during normalization so that these sections are correct.

### Proposal & Constraints

- [ ] Add a clause at the end of `normalize_ddc` that removes 0* numbers if they (1) have no decimal component, and (2) are not the only number in the ddc
- [ ] Add tests for the examples above (and find some more)

### Related files

- https://github.com/internetarchive/openlibrary/blob/master/openlibrary/utils/ddc.py
- https://github.com/internetarchive/openlibrary/blob/master/openlibrary/utils/tests/test_ddc.py

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/utils/ddc.py b/openlibrary/utils/ddc.py
--- a/openlibrary/utils/ddc.py
+++ b/openlibrary/utils/ddc.py
@@ -93,6 +93,14 @@ def normalize_ddc(ddc):
             decimal = '.' + number_parts[1] if len(number_parts) > 1 else ''
 
             number = '%03d%s' % (int(integer), decimal)
+
+            # Discard catalog edition number
+            # At least one classification number available
+            if len(results) > 0:
+                # Check if number like '0*' without decimal component
+                if re.search(r'(^0\d*$)', parts['number']):
+                    continue
+
         # Handle [Fic] or [E]
         elif parts['fic']:
             number = '[%s]' % parts['fic'].title()


Test patch:
diff --git a/openlibrary/utils/tests/test_ddc.py b/openlibrary/utils/tests/test_ddc.py
--- a/openlibrary/utils/tests/test_ddc.py
+++ b/openlibrary/utils/tests/test_ddc.py
@@ -33,7 +33,7 @@
     ("-222.14", ['-222.14'], "Dewey numbers with minus signs."),
     ("-222.14 (927.5)", ['-222.14', '927.5 s'], "Dewey numbers with minus signs."),
     ("[320.9777]", ['320.9777'], "Dewey numbers in brackets."),
-    ("[016.3584] 012", ['016.3584', '012'], "Dewey numbers in brackets."),
+    ("[016.3584] 012", ['016.3584'], "Dewey numbers in brackets."),
     ("081s [370.19'342]", ['081 s', '370.19342'],
      "Dewey number followed by lowercase s and a second number in brackets."),
     ("C364/.971", ['364.971'], "Canadian CIP"),
@@ -43,6 +43,13 @@
     ('1', ['001'], 'whole number padding'),
     ('hello world!', [], 'junk'),
     ('978123412341', [], 'junk'),
+    ('338.9/009171/7 019', ['338.90091717'], 
+     'Dewey number with segmentation and edition number'),
+    ('332.6 021', ['332.6'], 'Dewey number and edition number'),
+    ('[E] 021', ['[E]'], 'Juvenile work and edition number'),
+    ('015', ['015'], 'Single Dewey number with edition number pattern'),
+    ('(015.73) 015.791 021', ['015.73 s', '015.791'], 
+     'Two Dewey numbers and one edition number'),
 ]