DOI: 10.5281/zenodo.15269300
title: A Versioned Literature Corpus derived from Biodiversity Heritage Library hash://md5/581141d3f4d1847c833d9e2aa64656f2

description: THIS DESCRIPTION IS A WORK IN PROGRESS, THE CONTENT OF THIS VERSIONED CORPUS IS FINAL.

Biodiversity Heritage Library (BHL, https://biodiversitylibrary.org) contains hundreds of thousands of digital works related to biodiversity. This publications contains a versioned snapshot of work metadata and the digital signatures of their associated pdfs as seen in period 2025-03-26/2025-04-22 . 

The associated metadata and their associated pdfs cover many publication titles, including, but not limited to (top 25 publication titles listed in decreasing order of frequency) - 

 


count
ContainerTitle


11601
The Canadian field-naturalist


10790
The Annals and magazine of natural history; zoology, botany, and geology


7152
ZooKeys


6477
The journal of the Bombay Natural History Society


6261
Bulletin du Muséum national d'histoire naturelle


5550
Proceedings of the Biological Society of Washington


5240
Rhodora


4794
The Biological bulletin


4755
The Auk


4744
Mosquito News


4679
Proceedings of the Entomological Society of Washington


4667
The Bulletin of zoological nomenclature


4641
Proceedings of the Zoological Society of London


4290
Phytologia


4195
Walter Deane correspondence. Senders A-Z


4008
The entomologist's record and journal of variation


3936
Journal of the New York Botanical Garden


3931
The Wilson bulletin


3781
Proceedings of the Linnean Society of New South Wales


3698
Bulletin de la Société botanique de France


3519
Journal of the New York Entomological Society


3435
Castlemaine Naturalist


3374
Revue suisse de zoologie


3357
Psyche


3349
Botanical gazette
description: Files included in this publication are:

HEAD - described the signature of this literature corpus as a MD5 hash.

title.txt - list of a the BHL journal titles (aka ContainerTitle)  considered in this corpus. List was derived from part.txt.

part.txt - list of publication parts considered in this corpus with digital signature hash://md5/f235166f8cb094f56063680af4d1328a as seen at https://biodiversitylibrary.org/data/part.txt .

bhlpart.ris.zip - publication part metadata expressed in RIS bibliographic records.  https://www.biodiversitylibrary.org/data/RIS/bhlpart.ris.zip

as well as the following files named after the content their contain using the md5 hash algorithm [1]:

016c9f11f52e6ce67c7399a4d588c2eb01ae1d4fc5dae07417d07ea15b1ebcbb0502e4ea67c725e22f71aef16b16430317428dc255c56dadd82f80a0a006b1c917589c09f48219cc210bb17233f905871beac84c691ca2c581d7c7126a6c9fa31fb70a2c683eb7a5545900bc6668950f27f552c25bc733d05a5cc67e9ba6385034e11bf61f64d9a9c841337c7091c31b352fa9f7e46576462cf7840b2992de0d3e437559b980d2e88daf3c05ec2f4d7745c0336103993c893d1ed56b9ddbcbd94c9cceb77db62729b04e0743a929cb344f56e60040ce1a8519b6b16c18f56c8053e144641ffded6800dea502a8bb47ed54c2199d04c207ac0a72ba70cbc033c554dcecd04cb55ac693b0dd700a83539d581141d3f4d1847c833d9e2aa64656f25f7f443f990afb27b7b2894aa5ccf62363af62d86d0f3d4d26bfd534bf7282bb70f6ac67e5fffa937106e6814a3364ad7290c3e847ed1d10e47e4fd12188b75277fbd090bbd649aa4f5f45a77543627c9379bb3c6b6f91b7da72231a2881dbb598fbf06892fcf919bc2107446147cc4a99cbba7ff89fed781368d2b557900bc3a250d215282393ad43494dd9de11da4aa28a615337c2c187a5d3286923642ad6a445e6eb1138710be4b93db45ab7b4a9a6dbf5d82868e0fece05f47c7e3a42e7ac44c9ac6746a9fe2babaac45ec58860ad6e37caf05deee6e0260b55564ecb87ad70be22c8150717f3edab08ff4727a8b061254939327dde119942947203f6bcb855da1ec4dd0600c8fd736a0f413ec2b9014b0b403516b1e852748d72633c7dbb263189781c35bc2ecf714e6f2e3bb9be94e9bd64eb83df76146e3091f176aec06de184c76c2a8a586c903a5e7ca582c4b9251c4f6db873a941b42544c1fc96c71376e752134950cf96e1e5467ce8d5c97bb72c0b15f3877c4e24e03ed423cace804fb3370465a27a89d5f731490da6d268444ddf44545c53e6bf4635b3ae57d86a121f36f8e6f081a20803444bc1aae19ad69093efab2b86c3ca04de8f95baf206133d0b42fc9126e36a30654748ecfeb6470b553aa9e104a6d7114a424a05
description: This part is a technical description that defines this corpus and their associated contents.

History

The (version) history of this corpus can be queried by:

preston history\
 --algo md5\
 --anchor hash://md5/581141d3f4d1847c833d9e2aa64656f2

to produce:

<hash://md5/581141d3f4d1847c833d9e2aa64656f2> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/17589c09f48219cc210bb17233f90587> .<hash://md5/17589c09f48219cc210bb17233f90587> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/c4b9251c4f6db873a941b42544c1fc96> .<hash://md5/c4b9251c4f6db873a941b42544c1fc96> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/be94e9bd64eb83df76146e3091f176ae> .<hash://md5/be94e9bd64eb83df76146e3091f176ae> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/53e144641ffded6800dea502a8bb47ed> .<hash://md5/53e144641ffded6800dea502a8bb47ed> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/b855da1ec4dd0600c8fd736a0f413ec2> .<hash://md5/b855da1ec4dd0600c8fd736a0f413ec2> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/7290c3e847ed1d10e47e4fd12188b752> .<hash://md5/7290c3e847ed1d10e47e4fd12188b752> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/3e437559b980d2e88daf3c05ec2f4d77> .<hash://md5/3e437559b980d2e88daf3c05ec2f4d77> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/352fa9f7e46576462cf7840b2992de0d> .<hash://md5/352fa9f7e46576462cf7840b2992de0d> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/ac44c9ac6746a9fe2babaac45ec58860> .<hash://md5/ac44c9ac6746a9fe2babaac45ec58860> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/d86a121f36f8e6f081a20803444bc1aa> .<hash://md5/d86a121f36f8e6f081a20803444bc1aa> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/1fb70a2c683eb7a5545900bc6668950f> .<hash://md5/1fb70a2c683eb7a5545900bc6668950f> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/63af62d86d0f3d4d26bfd534bf7282bb> .<hash://md5/63af62d86d0f3d4d26bfd534bf7282bb> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/b9014b0b403516b1e852748d72633c7d> .<hash://md5/b9014b0b403516b1e852748d72633c7d> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/f206133d0b42fc9126e36a30654748ec> .<hash://md5/f206133d0b42fc9126e36a30654748ec> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/98fbf06892fcf919bc2107446147cc4a> .<hash://md5/98fbf06892fcf919bc2107446147cc4a> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/a28a615337c2c187a5d3286923642ad6> .<hash://md5/a28a615337c2c187a5d3286923642ad6> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/e19ad69093efab2b86c3ca04de8f95ba> .<hash://md5/e19ad69093efab2b86c3ca04de8f95ba> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/feb6470b553aa9e104a6d7114a424a05> .<hash://md5/feb6470b553aa9e104a6d7114a424a05> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/a445e6eb1138710be4b93db45ab7b4a9> .<hash://md5/a445e6eb1138710be4b93db45ab7b4a9> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/016c9f11f52e6ce67c7399a4d588c2eb> .<hash://md5/016c9f11f52e6ce67c7399a4d588c2eb> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/a250d215282393ad43494dd9de11da4a> .<hash://md5/a250d215282393ad43494dd9de11da4a> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/0502e4ea67c725e22f71aef16b164303> .<hash://md5/0502e4ea67c725e22f71aef16b164303> <http://www.w3.org/ns/prov#wasDerivedFrom> <hash://md5/d268444ddf44545c53e6bf4635b3ae57> .<urn:uuid:0659a54f-b713-4f86-a917-5be166a14110> <http://purl.org/pav/hasVersion> <hash://md5/d268444ddf44545c53e6bf4635b3ae57> .

 

Volume

The total volume of this corpus can be estimated using

preston ls\
 --algo md5\
 --anchor hash://md5/581141d3f4d1847c833d9e2aa64656f2\
 | preston cat\
 | pv\
 > /dev/null

to produce:

Also, the number of unique pdfs associated with content referenced in the corpus can be estimated using

preston ls\
 --algo md5\
 --anchor hash://md5/581141d3f4d1847c833d9e2aa64656f2\
 | grep hasVersion\
 | grep partpdf\
 | grep -oE hash://md5/[a-f0-9]{32}\ | sort\ | uniq\
 | wc -l

to produce 375,763 items, or over 375k tracked pdfs. Note also that out of a total of  375,763  items, pdf content related to 38,826 part items, or about 10%, failed to be retrieved, even after multiple retries. This number can be estimated using:

diff\ <(preston ls\
 --algo md5\
 --anchor hash://md5/581141d3f4d1847c833d9e2aa64656f2\
 | grep hasVersion\
 | grep partpdf\
 | grep well-known\ | grep -Eo partpdf/[0-9]+\ | sort\ | uniq)\ <(preston ls\
 --algo md5\
 --anchor hash://md5/581141d3f4d1847c833d9e2aa64656f2\
 | grep hasVersion\
 | grep partpdf\
 | grep -v well-known\ | grep -Eo partpdf/[0-9]+\ | sort\ | uniq)\ | grep "^<"\ | wc -l