DOI: 10.5281/zenodo.3849494
title: A biodiversity dataset graph: DataONE

description: A biodiversity dataset graph: DataONE The intended use of this archive is to facilitate (meta-)analysis of the Data Observation Network for Earth (DataONE). DataONE is a distributed infrastructure that provides information about earth observation data. This dataset provides versioned snapshots of the DataONE network as tracked by Preston [2] between 2018-11-06 and 2020-05-07 using "preston update -u https://dataone.org". The archive consists of 256 individual parts (e.g., preston-00.tar.gz, preston-01.tar.gz, ...) to allow for parallel file downloads. The archive contains three types of files: index files, provenance logs and data files. In addition, index files have been individually included in this dataset publication to facilitate remote access. Index files provide a way to links provenance files in time to establish a versioning mechanism. Provenance files describe how, when, what and where the DataONE content was retrieved. For more information, please visit https://preston.guoda.bio or https://doi.org/10.5281/zenodo.1410543 . To retrieve and verify the downloaded DataONE biodiversity dataset graph, first concatenate all the downloaded preston-*.tar.gz files (e.g., cat preston-*.tar.gz &gt; preston.tar.gz). Then, extract the archives into a "data" folder. Alternatively, you can use the preston[2] command-line tool to "clone" this dataset using: $ java -jar preston.jar clone --remote https://zenodo.org/record/3849494/files After that, verify the index of the archive by reproducing the following provenance log history: $ java -jar preston.jar history<br> &lt;0659a54f-b713-4f86-a917-5be166a14110&gt; &lt;http://purl.org/pav/hasVersion&gt; &lt;hash://sha256/8c67e0741d1c90db54740e08d2e39d91dfd73566ea69c1f2da0d9ab9780a9a9f&gt; .<br> &lt;hash://sha256/3ed3acaca7ac57f546d0b8877c1927ab5e08c23eccaa8219600c59c77a72c685&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/8c67e0741d1c90db54740e08d2e39d91dfd73566ea69c1f2da0d9ab9780a9a9f&gt; .<br> &lt;hash://sha256/857753997a7595a1b372b05641b58a25d9408b7ff08d557ce1fe8b73e4bd383f&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/3ed3acaca7ac57f546d0b8877c1927ab5e08c23eccaa8219600c59c77a72c685&gt; .<br> &lt;hash://sha256/7ee0376f4c3f7aeeda36927a5211395e5da8201e810e8c7e638a0fe23d001e88&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/857753997a7595a1b372b05641b58a25d9408b7ff08d557ce1fe8b73e4bd383f&gt; .<br> &lt;hash://sha256/68b4974d8ab7c4c7a7a4305065839b60ba460aaa862590b34c67877738feba90&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/7ee0376f4c3f7aeeda36927a5211395e5da8201e810e8c7e638a0fe23d001e88&gt; .<br> &lt;hash://sha256/060a76d56255bf9482c951748c91291fddeeb20f180632132be1344e081b2372&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/68b4974d8ab7c4c7a7a4305065839b60ba460aaa862590b34c67877738feba90&gt; .<br> &lt;hash://sha256/29357bdfab4548025f8a5743301f5c3c9146fa436c39e3c9e019fb9409ac9c42&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/060a76d56255bf9482c951748c91291fddeeb20f180632132be1344e081b2372&gt; .<br> &lt;hash://sha256/3669cd95100d1d533eb8953ff4ec5092cbd8addb8879b3e6262191148a8a3ebb&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/29357bdfab4548025f8a5743301f5c3c9146fa436c39e3c9e019fb9409ac9c42&gt; .<br> &lt;hash://sha256/8dc1663299359d271cb1b4c14ad521d0f1be67743689dd18016543dc1e097efb&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/3669cd95100d1d533eb8953ff4ec5092cbd8addb8879b3e6262191148a8a3ebb&gt; .<br> &lt;hash://sha256/dc4903e8afee651db1d9bf509f20503bf9c8e89679c4bcffb46d5b97440cb6de&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/8dc1663299359d271cb1b4c14ad521d0f1be67743689dd18016543dc1e097efb&gt; .<br> &lt;hash://sha256/f3bed9db3092c744604df5f50248a2ec36e564fe78a65f45c4190283bd61c807&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/dc4903e8afee651db1d9bf509f20503bf9c8e89679c4bcffb46d5b97440cb6de&gt; .<br> &lt;hash://sha256/e3c7b3b14b2b792e3e2e560a1b2bef059ac93f777dee616b836317bc9cbfcbf7&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/f3bed9db3092c744604df5f50248a2ec36e564fe78a65f45c4190283bd61c807&gt; .<br> &lt;hash://sha256/631a4531e7bb052816d28454bbeec3428d5e7bfd1f148c4f21ce63a6cf86c650&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/e3c7b3b14b2b792e3e2e560a1b2bef059ac93f777dee616b836317bc9cbfcbf7&gt; .<br> &lt;hash://sha256/87de0898919d2212977a586965e930ae45bdd1366073591c808c208a635e2814&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/631a4531e7bb052816d28454bbeec3428d5e7bfd1f148c4f21ce63a6cf86c650&gt; .<br> &lt;hash://sha256/79ec3ee370a0d38311bc352af07a36380cd3aa04dc98154cf723bbc73d12ee77&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/87de0898919d2212977a586965e930ae45bdd1366073591c808c208a635e2814&gt; .<br> &lt;hash://sha256/e54b360a4ca84a4503e4c10a8a8cca062c130be7429c8fe6ea1e0e82fe113e12&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/79ec3ee370a0d38311bc352af07a36380cd3aa04dc98154cf723bbc73d12ee77&gt; .<br> &lt;hash://sha256/2910f784f84e112f124a56ce54bd06b76e510f90276629d2d144ce29e326d80f&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/e54b360a4ca84a4503e4c10a8a8cca062c130be7429c8fe6ea1e0e82fe113e12&gt; .<br> &lt;hash://sha256/bcb0bdff0689cfb06f586d057703e41d1c6ba409867232217081dd8cb5053c87&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/2910f784f84e112f124a56ce54bd06b76e510f90276629d2d144ce29e326d80f&gt; .<br> &lt;hash://sha256/a12f8c7fbf4fbfa71536c7e1b2614a35454dac6a7fe9e1cc0b4df41ab2269bef&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/bcb0bdff0689cfb06f586d057703e41d1c6ba409867232217081dd8cb5053c87&gt; .<br> &lt;hash://sha256/2b5c445f0b7b918c14a50de36e29a32854ed55f00d8639e09f58f049b85e50e3&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/a12f8c7fbf4fbfa71536c7e1b2614a35454dac6a7fe9e1cc0b4df41ab2269bef&gt; . To check the integrity of the extracted archive, confirm that each line produce by the command "preston verify" produces lines as shown below, with each line including "CONTENT_PRESENT_VALID_HASH". Depending on hardware capacity, this may take a while. $ java -jar preston.jar verify<br> hash://sha256/e55c1034d985740926564e94decd6dc7a70f779a33e7deb931553739cda16945 file:/home/preston/preston-dataone/data/e5/5c/e55c1034d985740926564e94decd6dc7a70f779a33e7deb931553739cda16945 OK CONTENT_PRESENT_VALID_HASH 21580 hash://sha256/e55c1034d985740926564e94decd6dc7a70f779a33e7deb931553739cda16945<br> hash://sha256/d0ddcc2111b6134a570bcc7d89375920ef4d754130cecc0727c79d2b05a9f81f file:/home/preston/preston-dataone/data/d0/dd/d0ddcc2111b6134a570bcc7d89375920ef4d754130cecc0727c79d2b05a9f81f OK CONTENT_PRESENT_VALID_HASH 2035 hash://sha256/d0ddcc2111b6134a570bcc7d89375920ef4d754130cecc0727c79d2b05a9f81f<br> hash://sha256/472de9d1c9fd7e044aac409abfbfff9f12c6b69359df995d431009580ffb0f53 file:/home/preston/preston-dataone/data/47/2d/472de9d1c9fd7e044aac409abfbfff9f12c6b69359df995d431009580ffb0f53 OK CONTENT_PRESENT_VALID_HASH 1935 hash://sha256/472de9d1c9fd7e044aac409abfbfff9f12c6b69359df995d431009580ffb0f53<br> hash://sha256/b29879462cd43862129c5cf9b149c41ecd33ffef284a4dbea4ac1c0f90108687 file:/home/preston/preston-dataone/data/b2/98/b29879462cd43862129c5cf9b149c41ecd33ffef284a4dbea4ac1c0f90108687 OK CONTENT_PRESENT_VALID_HASH 1553 hash://sha256/b29879462cd43862129c5cf9b149c41ecd33ffef284a4dbea4ac1c0f90108687 <br> Note that a copy of the java program "preston", preston.jar, is included in this publication. The program runs on java 8+ virtual machine using "java -jar preston.jar", or in short "preston". Files in this data publication: --- start of file descriptions --- -- description of archive and its contents (this file) --<br> README -- executable java jar containing preston[2] v0.1.15. --<br> preston.jar -- preston archives containing DataONE data files, associated provenance logs and a provenance index --<br> preston-[00-ff].tar.gz -- individual provenance index files --<br> 2a5de79372318317a382ea9a2cef069780b852b01210ef59e06b640a3539cb5a<br> 2aecaf289def0e23a27058bf7715f226ef9189905f0be13228174825633125cf<br> 2f65ae542401d4c2daf1bca70de640211da6749188f67d28ea71acd7d8ba070b<br> 35eb1e17e2bf3e71212cde35bdb03e8a6545a57483ea3c1633929257b70cf637<br> 3d38b70198e448674be6a63d14b9817f3a956f48bba7418fa7baa086a56c05b7<br> 66ad3e5e904740f1e835ac6718dda4279e0c24b204ea0d1113cda1352a5072ba<br> 7466a35e42dea7e2be068060ec0c926f9a8686388ed504ef5c6c990c1ba4e8d0<br> 81161d9746c2a5823641c436e773fb4508516b055da85f4494b38c545349da39<br> 8bf062872ce958545d361e9d53a552ffb025ac29ab875caad1157c0995d34f66<br> a90eed8d70c54c8e554f2dfde4fceb434eda162d9615d62de96ded2344f88a78<br> c33ef5e29100b323412f1f3bc66908c8e01e4f0d1db4ea3685d2fffc47981dd6<br> c84dffef20fec958255e759db6445fc469d73695674a33ae6f7e567a088c9fe0<br> d362d599d72000c4feb464db5a669b12e15fc3ca1a49b1e7d4d6f7d6d5d15411<br> d9378616636be3686bbabd5bf29d50f0ef0e5ceb5ddd7dfce47f7e755b596b7d<br> da26fa6e7371385ed3f61af9a766221c833060d59dfd4869bbd7110f95f288db<br> e4103a75627857de3ee2e317429108611c244fc448c01d1d7bf652115c3b8a55<br> eb368fedb8f100210dd968edcf80f4d13cab3dd64135a6ab744102cf15e68c94<br> f13ab4bca04f894ae8eabb51fa01b4dfbc69f717eabc9896c728e2ba39c4db27<br> f493baf276892a199a0b0d078359f64a38fe8ad3f807921f8d41ef73f7343b1f<br> ff92b6c06ae5286bd2f1db679e0fcc4da294acb9bc01b2e9522378d99218c2e3 --- end of file descriptions --- <br> References [1] Data Observation Network for Earth (DataONE, https://dataone.org) accessed from 2018-11-06 to 2020-05-07 with provenance hash://sha256/2b5c445f0b7b918c14a50de36e29a32854ed55f00d8639e09f58f049b85e50e3.<br> [2] https://preston.guoda.bio, https://doi.org/10.5281/zenodo.1410543 . <br> This work is funded in part by grant NSF OAC 1839201 from the National Science Foundation.