Repository: internetarchive/openlibrary

Issue:
Importer doesn't handle MARC 245$n volume/part subfield
These three imported Talis MARC records all end up with the same title, without any of the disambiguating information that the original record contains:

245 10 $aGuide to open learning materials.$nBooklet no. 3 & 4,$pAccountancy and book keeping. - [MARC](https://openlibrary.org/show-records/talis_openlibrary_contribution/talis-openlibrary-contribution.mrc:3026825531:453), [OpenLibrary](https://openlibrary.org/books/OL20216459M/Guide_to_open_learning_materials.)
245 10 $aGuide to open learning materials.$nBooklet no. 7,$pMotor vehicle engineering. - [MARC](https://openlibrary.org/show-records/talis_openlibrary_contribution/talis-openlibrary-contribution.mrc:3026826428:450), [OpenLibrary](https://openlibrary.org/books/OL20216469M/Guide_to_open_learning_materials.)
245 10 $aGuide to open learning materials.$nBooklet no. 13,$pChemistry. - [MARC](https://openlibrary.org/show-records/talis_openlibrary_contribution/talis-openlibrary-contribution.mrc:3026828647:435), [OpenLibrary](https://openlibrary.org/books/OL20216491M/Guide_to_open_learning_materials.) 

We should figure out a way to include this information in the OpenLibrary record so that it can be used, even if it just means appending it to the title.
Importer doesn't handle MARC 245$n volume/part subfield
These three imported Talis MARC records all end up with the same title, without any of the disambiguating information that the original record contains:

245 10 $aGuide to open learning materials.$nBooklet no. 3 & 4,$pAccountancy and book keeping. - [MARC](https://openlibrary.org/show-records/talis_openlibrary_contribution/talis-openlibrary-contribution.mrc:3026825531:453), [OpenLibrary](https://openlibrary.org/books/OL20216459M/Guide_to_open_learning_materials.)
245 10 $aGuide to open learning materials.$nBooklet no. 7,$pMotor vehicle engineering. - [MARC](https://openlibrary.org/show-records/talis_openlibrary_contribution/talis-openlibrary-contribution.mrc:3026826428:450), [OpenLibrary](https://openlibrary.org/books/OL20216469M/Guide_to_open_learning_materials.)
245 10 $aGuide to open learning materials.$nBooklet no. 13,$pChemistry. - [MARC](https://openlibrary.org/show-records/talis_openlibrary_contribution/talis-openlibrary-contribution.mrc:3026828647:435), [OpenLibrary](https://openlibrary.org/books/OL20216491M/Guide_to_open_learning_materials.) 

We should figure out a way to include this information in the OpenLibrary record so that it can be used, even if it just means appending it to the title.


Discussion:
cc: @hornc re: MARC ImportBot
The relevant code is here: https://github.com/internetarchive/openlibrary/blob/59f190c2017ae5c9b1ecc53f01c3ecbe304212e4/openlibrary/catalog/marc/parse.py#L175

Looks like $p "Name of part/section of a work" is considered, but not $n "Number of part/section of a work"

There does not appear to be any current test MARC records that have a 245$n to test against.

MARC title subfields reference:
https://www.loc.gov/marc/bibliographic/bd245.html
Here's another example: https://openlibrary.org/books/OL26605397M
All 400+ volumes of the National Union Catalog are missing the most obvious distinguishing feature, the volume number.

This should be straightforward for a first timer who wants to focus on backend / import stuff to address. A failing test needs to be written and then the 245$p code pattern can be repeated for 245$n.

@hornc Is this actually a Good First Issue??
I've updated @hornc's code reference to a permalink. The relevant MARC documentation is:

> $a - Title
> $b - Remainder of title
> [...]
> $n - Number of part/section of a work (R)
> $p - Name of part/section of a work (R)

which is formatted for OpenLibrary as $a[ : $b][ : $p]. I propose that we extend this to  $a[ : $b][ : $n][ : $p] since MARC subfields tend to be named in the logical order of presentation.

The MARC records in the original issue description (e.g. https://openlibrary.org/show-records/talis_openlibrary_contribution/talis-openlibrary-contribution.mrc:3026825531:453) can be used to create the necessary unit test.

If additional specification is needed to make this easy enough for a first timer, let's add it here.

I'm going to go out on a limb and say this is probably not a good first issue.
What additional detail would you like to see to make it a good first issue?
cc: @hornc re: MARC ImportBot
The relevant code is here: https://github.com/internetarchive/openlibrary/blob/59f190c2017ae5c9b1ecc53f01c3ecbe304212e4/openlibrary/catalog/marc/parse.py#L175

Looks like $p "Name of part/section of a work" is considered, but not $n "Number of part/section of a work"

There does not appear to be any current test MARC records that have a 245$n to test against.

MARC title subfields reference:
https://www.loc.gov/marc/bibliographic/bd245.html
Here's another example: https://openlibrary.org/books/OL26605397M
All 400+ volumes of the National Union Catalog are missing the most obvious distinguishing feature, the volume number.

This should be straightforward for a first timer who wants to focus on backend / import stuff to address. A failing test needs to be written and then the 245$p code pattern can be repeated for 245$n.

@hornc Is this actually a Good First Issue??
I've updated @hornc's code reference to a permalink. The relevant MARC documentation is:

> $a - Title
> $b - Remainder of title
> [...]
> $n - Number of part/section of a work (R)
> $p - Name of part/section of a work (R)

which is formatted for OpenLibrary as $a[ : $b][ : $p]. I propose that we extend this to  $a[ : $b][ : $n][ : $p] since MARC subfields tend to be named in the logical order of presentation.

The MARC records in the original issue description (e.g. https://openlibrary.org/show-records/talis_openlibrary_contribution/talis-openlibrary-contribution.mrc:3026825531:453) can be used to create the necessary unit test.

If additional specification is needed to make this easy enough for a first timer, let's add it here.

I'm going to go out on a limb and say this is probably not a good first issue.
What additional detail would you like to see to make it a good first issue?

Resolving patch:
diff --git a/openlibrary/catalog/marc/parse.py b/openlibrary/catalog/marc/parse.py
--- a/openlibrary/catalog/marc/parse.py
+++ b/openlibrary/catalog/marc/parse.py
@@ -217,22 +217,23 @@ def read_work_titles(rec):
 
 
 def read_title(rec):
-    STRIP_CHARS = r' /,;:='
+    # For cataloging punctuation complexities, see https://www.oclc.org/bibformats/en/onlinecataloging.html#punctuation
+    STRIP_CHARS = r' /,;:='  # Typical trailing punctuation for 245 subfields in ISBD cataloging standards
     fields = rec.get_fields('245') or rec.get_fields('740')
     if not fields:
         raise NoTitle('No Title found in either 245 or 740 fields.')
     # example MARC record with multiple titles:
     # https://openlibrary.org/show-marc/marc_western_washington_univ/wwu_bibs.mrc_revrev.mrc:299505697:862
-    contents = fields[0].get_contents(['a', 'b', 'c', 'h', 'p'])
-    b_and_p = [i for i in fields[0].get_subfield_values(['b', 'p']) if i]
+    contents = fields[0].get_contents(['a', 'b', 'c', 'h', 'n', 'p', 's'])
+    bnps = [i for i in fields[0].get_subfield_values(['b', 'n', 'p', 's']) if i]
     ret = {}
     title = None
     # MARC record with 245a missing:
     # https://openlibrary.org/show-marc/marc_western_washington_univ/wwu_bibs.mrc_revrev.mrc:516779055:1304
     if 'a' in contents:
         title = ' '.join(x.strip(STRIP_CHARS) for x in contents['a'])
-    elif b_and_p:
-        title = b_and_p.pop(0).strip(STRIP_CHARS)
+    elif bnps:
+        title = bnps.pop(0).strip(STRIP_CHARS)
     # talis_openlibrary_contribution/talis-openlibrary-contribution.mrc:183427199:255
     if title in ('See.', 'See also.'):
         raise SeeAlsoAsTitle('Title is: %s' % title)
@@ -243,9 +244,9 @@ def read_title(rec):
         if not title:  # ia:scrapbooksofmoun03tupp
             raise NoTitle('No title found from joining subfields.')
     ret['title'] = remove_trailing_dot(title)
-    if b_and_p:
+    if bnps:
         ret['subtitle'] = ' : '.join(
-            remove_trailing_dot(x.strip(STRIP_CHARS)) for x in b_and_p
+            remove_trailing_dot(x.strip(STRIP_CHARS)) for x in bnps
         )
     if 'c' in contents:
         ret['by_statement'] = remove_trailing_dot(' '.join(contents['c']))


Test patch:
diff --git a/openlibrary/catalog/marc/tests/test_data/bin_expect/equalsign_title.mrc b/openlibrary/catalog/marc/tests/test_data/bin_expect/equalsign_title.mrc
--- a/openlibrary/catalog/marc/tests/test_data/bin_expect/equalsign_title.mrc
+++ b/openlibrary/catalog/marc/tests/test_data/bin_expect/equalsign_title.mrc
@@ -3,7 +3,7 @@
     "Thedepartment = Yr adran"
   ],
   "pagination": "14p.",
-  "subtitle": "School budgets",
+  "subtitle": "School budgets : 1990/91",
   "title": "Cyllidebau ysgolion",
   "notes": "Welsh and English text = Testyn Cymraeg a Saesneg.",
   "number_of_pages": 14,