DOI: 10.5281/zenodo.3484554
title: A biodiversity dataset graph: BHL

description: A biodiversity dataset graph: BHL The intended use of this archive is to facilitate (meta-)analysis of the Biodiversity Heritage Library (BHL). The Biodiversity Heritage Library improves research methodology by collaboratively making biodiversity literature openly available to the world as part of a global biodiversity community. This dataset provides versioned snapshots of the BHL network as tracked by Preston [2] between 2019-05-13 and 2019-10-07 using "preston update -u https://biodiversitylibrary.org". The archive consists of 256 individual parts (e.g., preston-00.tar.gz, preston-01.tar.gz, ...) to allow for parallel file downloads. The archive contains three types of files: index files, provenance logs and data files. In addition, index files have been individually included in this dataset publication to facilitate remote access. Index files provide a way to links provenance files in time to establish a versioning mechanism. Provenance files describe how, when, what and where the BHL content was retrieved. For more information, please visit https://preston.guoda.bio or https://doi.org/10.5281/zenodo.1410543 . To retrieve and verify the downloaded BHL biodiversity dataset graph, first concatenate all the downloaded preston-*.tar.gz files (e.g., cat preston-*.tar.gz &gt; preston.tar.gz). Then, extract the archives into a "data" folder. Alternatively, you can use the preston[2] command-line tool to "clone" this dataset using: $ java -jar preston.jar clone --remote https://zenodo.org/record/3484555/files After that, verify the index of the archive by reproducing the following provenance log history: $ java -jar preston.jar history<br> &lt;0659a54f-b713-4f86-a917-5be166a14110&gt; &lt;http://purl.org/pav/hasVersion&gt; &lt;hash://sha256/89926f33157c0ef057b6de73f6c8be0060353887b47db251bfd28222f2fd801a&gt; .<br> &lt;hash://sha256/41b19aa9456fc709de1d09d7a59c87253bc1f86b68289024b7320cef78b3e3a4&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/89926f33157c0ef057b6de73f6c8be0060353887b47db251bfd28222f2fd801a&gt; .<br> &lt;hash://sha256/7582d5ba23e0d498ca4f55c29408c477d0d92b4fdcea139e8666f4d78c78a525&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/41b19aa9456fc709de1d09d7a59c87253bc1f86b68289024b7320cef78b3e3a4&gt; .<br> &lt;hash://sha256/a70774061ccded1a45389b9e6063eb3abab3d42813aa812391f98594e7e26687&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/7582d5ba23e0d498ca4f55c29408c477d0d92b4fdcea139e8666f4d78c78a525&gt; .<br> &lt;hash://sha256/007e065ba4b99867751d688754aa3d33fa96e6e03133a2097e8a368d613cd93a&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/a70774061ccded1a45389b9e6063eb3abab3d42813aa812391f98594e7e26687&gt; .<br> &lt;hash://sha256/4fb4b4d8f1ae2961311fb0080e817adb2faa746e7eae15249a3772fbe2d662a1&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/007e065ba4b99867751d688754aa3d33fa96e6e03133a2097e8a368d613cd93a&gt; .<br> &lt;hash://sha256/67cc329e74fd669945f503917fbb942784915ab7810ddc41105a82ebe6af5482&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/4fb4b4d8f1ae2961311fb0080e817adb2faa746e7eae15249a3772fbe2d662a1&gt; . To check the integrity of the extracted archive, confirm that each line produce by the command "preston verify" produces lines as shown below, with each line including "CONTENT_PRESENT_VALID_HASH". Depending on hardware capacity, this may take a while. $ java -jar preston.jar verify<br> hash://sha256/e0c131ebf6ad2dce71ab9a10aa116dcedb219ae4539f9e5bf0e57b84f51f22ca file:/home/preston/preston-bhl/data/e0/c1/e0c131ebf6ad2dce71ab9a10aa116dcedb219ae4539f9e5bf0e57b84f51f22ca OK CONTENT_PRESENT_VALID_HASH 49458087<br> hash://sha256/1a57e55a780b86cff38697cf1b857751ab7b389973d35113564fe5a9a58d6a99 file:/home/preston/preston-bhl/data/1a/57/1a57e55a780b86cff38697cf1b857751ab7b389973d35113564fe5a9a58d6a99 OK CONTENT_PRESENT_VALID_HASH 25745<br> hash://sha256/85efeb84c1b9f5f45c7a106dd1b5de43a31b3248a211675441ff584a7154b61c file:/home/preston/preston-bhl/data/85/ef/85efeb84c1b9f5f45c7a106dd1b5de43a31b3248a211675441ff584a7154b61c OK CONTENT_PRESENT_VALID_HASH 519892<br> hash://sha256/251e5032afce4f1e44bfdc5a8f0316ca1b317e8af41bdbf88163ab5bd2b52743 file:/home/preston/preston-bhl/data/25/1e/251e5032afce4f1e44bfdc5a8f0316ca1b317e8af41bdbf88163ab5bd2b52743 OK CONTENT_PRESENT_VALID_HASH 787414 Note that a copy of the java program "preston", preston.jar, is included in this publication. The program runs on java 8+ virtual machine using "java -jar preston.jar", or in short "preston". Files in this data publication: --- start of file descriptions --- -- description of archive and its contents (this file) --<br> README -- executable java jar containing preston[2] v0.1.8. --<br> preston.jar -- preston archives containing BHL data files, associated provenance logs and a provenance index --<br> preston-[00-ff].tar.gz -- individual provenance index files --<br> 2a5de79372318317a382ea9a2cef069780b852b01210ef59e06b640a3539cb5a<br> 2b1104cb7749e818c9afca78391b2d0099bbb0a32f2b348860a335cd2f8f6800<br> 4081bc59dff58d63f6a86c623cb770f01e9a355a42495b205bcb538cd526190f<br> 6f99a1388823fca745c9e22ac21e2da909a219aa1ace55170fa9248c0276903c<br> 82903464889fea7c53f53daedf4e41fa31092f82619edeb3415eb2b473f74af3<br> 9e8c86243df39dd4fe82a3f814710eccf73aa9291d050415408e346fa2b09e70<br> bcec6df2ea7f74e9a6e2830d0072e6b2fbe65323d9ddb022dd6e1349c23996e2 --- end of file descriptions --- <br> References [1] Biodiversity Heritage Library (BHL, https://biodiversitylibrary.org) accessed from 2019-05-13 to 2019-10-07 with provenance hash://sha256/4fb4b4d8f1ae2961311fb0080e817adb2faa746e7eae15249a3772fbe2d662a1.<br> [2] https://preston.guoda.bio, https://doi.org/10.5281/zenodo.1410543 . <br> This work is funded in part by grant NSF OAC 1839201 from the National Science Foundation.