DOI: 10.5281/zenodo.8327610
title: Catalogue of Life Repackaged and Sorted hash://sha256/35efd8089b3f86d6789ca75be70630e0e328f2c91b43c2636ecdc49f210b97bb hash://md5/c39e3cbe35f0ae86acf3e77baeb01bfb

description: # Introduction

The Catalogue of Life [@Banki_2023] is "\[...\] is an assembly ofexpert-based global species checklists with the aim to build acomprehensive catalogue of all known species of organisms on Earth.\[...\]"

This data publication contains a verifiable copy of the Catalogue ofLife as well as a reverse sorted version of the NameUsage.tsv table. Theaims of this publication are to:

1.  provide a signed citation [@Elliott_2023] for a copy of Catalogue of    Life2.  prepare Catalogue of Life to be included in the Nomer Corpus of    Taxonomic Resources [@Poelen_2023a].3.  pre-process the Catalogue of Life to facilitate optimized indexing    and offline taxonomic name alignments using tools like Nomer    [@Poelen_2023b].

Overall, the publications aims to facilitate taxonomic name alignmentusing the wealth of information provided by the Catalogue of Life tohelp enable fast, reproduceable, offline-enabled alignment of namelistswith taxonomic resources of known provenance (or origin).

An example of an application facilitated by this publication is theTaxonomic Name Alignment tool as provided throughhttps://github.com/globalbioticinteractions/name-alignment-template.This template repository implements an automated workflow using GitHubAction to align scientific names in csv/tsv files and darwin corearchive with common taxonomic name lists like Catalogue of Life, NCBITaxonomy, Integrated Taxonomic Information System (ITIS), and GBIFBackbone taxonomy.

## Methods

To capture and process the Catalogue of Life, the following steps weretaken:

1.  track and archive a copy of Catalogue of Life2.  reverse sort NameUsage.tsv3.  assign an alias to the processed resources

Steps 1-3 are captured and documented using Preston, a biodiversity datatracker. Preston not only helps to documents the steps, but alsoincludes the digital resources that were used and produced.

### Track and Archive

To track and archive a copy of Catalogue of Life, the following commandwas issued:

    preston track https://download.catalogueoflife.org/col/latest_coldp.zip

With this, a copy ofhttps://download.catalogueoflife.org/col/latest_coldp.zip is downloadedand their sha256 checksum (or hash) is calculated. Also, the downloadprocess is captured machine-readable rdf/nquads statement.

The content id (or sha256 hash) of the copy included in this publicationcan be found using:

    preston alias \      --anchor hash://sha256/35efd8089b3f86d6789ca75be70630e0e328f2c91b43c2636ecdc49f210b97bb \      --remote https://zenodo.org/deposit/18705780/files \       https://download.catalogueoflife.org/col/latest_coldp.zip \     | head -n1

and is

    <https://download.catalogueoflife.org/col/latest_coldp.zip> <http://purl.org/pav/hasVersion> <hash://sha256/ced8f3b1eb2ee451df4290e2c6135f08fdd4d5da0c697463b9091c75be809ea7> <urn:uuid:284325aa-c0b3-4ba3-9759-0bd5c388a0df> .

meaning that`hash://sha256/ced8f3b1eb2ee451df4290e2c6135f08fdd4d5da0c697463b9091c75be809ea7`is the content id (or sha256 hash) of the content produced by`https://download.catalogueoflife.org/col/latest_coldp.zip` at the timethis publication was compiled.

### Reverse Sort NameUsage.tsv

The second step was to pre-process (or transform) part of the trackedCatalogue of Life data package and reverse sort their content. We usethe following bash script to do this processing:

    (sed -u 1q; LC_ALL=C sort -r) | gzip

The script first prints the header (i.e., `sed -u 1q`), then reversesorts the following content (i.e., `LC_ALL=C sort -r`). Finally, theoutput is compressed using gzip.

This script is part of this data publication, and can be retrieved via:

    preston cat \     --remote https://zenodo.org/record/18705780/files/ \     hash://sha256/03292e3e40a04c565c83debd26ed521516f158a29c011b57a96d1c720c40b6cd

To help make it easier to reference the script, an alias was createdusing

    preston alias \     urn:example:reverse-sort.sh \     hash://sha256/03292e3e40a04c565c83debd26ed521516f158a29c011b57a96d1c720c40b6cd    ~~

    Following the authoring of the reverse sort script, and documenting their alias (i.e. 'urn:example:reverse-sort.sh`), we applied the script to the acquired copy of the Catalogue of Life using:

preston cat\--remote https://zenodo.org/record/18705780/files/\'zip:hash://sha256/ced8f3b1eb2ee451df4290e2c6135f08fdd4d5da0c697463b9091c75be809ea7!/NameUsage.tsv'\\| preston bash\--anchorhash://sha256/35efd8089b3f86d6789ca75be70630e0e328f2c91b43c2636ecdc49f210b97bb\--remote https://zenodo.org/record/18705780/files/\-c urn:example:reverse-sort.sh \~\~\~

The result of this process was the content identified by sha256 hash

    hash://sha256/1008433bfa5fe7fb059a720eddfe995e18b3e0e8f25ac0c990f1477c177d18cc

as documented in line 22 of the associated preston processing log in

    preston cat\      --remote https://zenodo.org/record/18705780/files/\     'line:hash://sha256/8ba35deafc847f0d5d69d357241a431b7fd9b6f2735189575b2b7168d523caa9!/L22' 

### Assign An Alias to Processed Resources

The alias 'col:NameUsage.tsv.gz' was defined to help make it easier topoint to the result using:

    preston alias\      col:NameUsage.tsv.gz\      hash://sha256/1008433bfa5fe7fb059a720eddfe995e18b3e0e8f25ac0c990f1477c177d18cc

With this, the following command was executed to list the first threelines of the produced resource:

    preston cat \     --anchor hash://sha256/35efd8089b3f86d6789ca75be70630e0e328f2c91b43c2636ecdc49f210b97bb \     --remote https://zenodo.org/record/18705780/files/ \     col:NameUsage.tsv.gz \     | gunzip \     | head -n3

Where `preston cat ...` prints the produced resource, `gunzip`uncompresses the result, and the first three lines are selected using`head -n3`.

The result of the operation is shown below:

  




col:ID
col:alternativeID
col:nameAlternativeID
col:sourceID
col:parentID
col:basionymID
col:status
col:scientificName
col:authorship
col:rank
col:notho
col:originalSpelling
col:uninomial
col:genericName
col:infragenericEpithet
col:specificEpithet
col:infraspecificEpithet
col:cultivarEpithet
col:combinationAuthorship
col:combinationAuthorshipID
col:combinationExAuthorship
col:combinationExAuthorshipID
col:combinationAuthorshipYear
col:basionymAuthorship
col:basionymAuthorshipID
col:basionymExAuthorship
col:basionymExAuthorshipID
col:basionymAuthorshipYear
col:namePhrase
col:nameReferenceID
col:publishedInYear
col:publishedInPage
col:publishedInPageLink
col:gender
col:genderAgreement
col:etymology
col:code
col:nameStatus
col:accordingToID
col:accordingToPage
col:accordingToPageLink
col:referenceID
col:scrutinizer
col:scrutinizerID
col:scrutinizerDate
col:extinct
col:temporalRangeStart
col:temporalRangeEnd
col:environment
col:species
col:section
col:subgenus
col:genus
col:subtribe
col:tribe
col:subfamily
col:family
col:superfamily
col:suborder
col:order
col:subclass
col:class
col:subphylum
col:phylum
col:kingdom
col:ordinal
col:branchLength
col:link
col:nameRemarks
col:remarks
col:modified
col:modifiedBy
clb:merged




_UATf344c37VElrQ7H85p0
 
 
 
KZX8B
 
accepted
Mystacocarida
Pennak & Zinn, 1943
class
 
 
Mystacocarida
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Boxshall, Geoff
 
2015-05-05
 
 
 
marine
 
 
 
 
 
 
 
 
 
 
 
 
 
Crustacea
Arthropoda
Animalia
 
 
https://www.marinespecies.org/aphia.php?p=taxdetails&id=1079
 
 
 
 
false


ZZZZ
 
 
1026
ZZZY
 
synonym
Cryptoripersia arizonensis
Ferris, 1919
species
 
 
 
Cryptoripersia
 
arizonensis
 
 
Ferris
 
 
 
1919
 
 
 
 
 
 
 
 
 
 
 
 
 
zoological
 
 
 
 
19d717dc-f553-4b71-adfc-7e420620aa50,9fe5b735-fcc8-4d9a-a00d-e0b62dbadecf,c17ead8c-57dd-46fd-b80d-6932443f3479,e14aa0cd-5eb3-4c10-87ec-d44cbfe837d3,c59aec86-4224-43a9-9958-f32a2459cfdc,35906e7a-4d3a-4f42-8b19-315dee356805,67a93281-7ff1-49cb-b8f6-a88c609a7e86,4af3819f-f1fd-4b27-b484-c69e0a73e2be,6b1fb685-e0d9-4423-a00f-499494115f50
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
false




# Results

As described in our methods, this publication derived the resource withalias col:NameUsage.tsv.gz and content id`hash://sha256/90cfb4edc5f7fa2b55e8bdd9b26d26aec135d1bc9dd11df77df46f732475d3fa`. This resource contains a reverse-sorted copy of the NameUsage.tsvprovided in the Catalogue of Life data package retrieved fromhttps://download.catalogueoflife.org/col/latest_coldp.zip with contentidentifier`hash://sha256/ced8f3b1eb2ee451df4290e2c6135f08fdd4d5da0c697463b9091c75be809ea7`.

The following tools were used to process the Catalogue of Life resource:

  




tool name




preston


bash


gzip


sed


head


sort




# Discussion

This publication is intended to facilitate re-use of the Catalogue ofLife data package in taxonomic name alignment workflows. While theprimary goal was to generate a resource for use in Nomer v0.6.2[@Poelen_2023b], other usage can be imagined such as:

1.  Lots of Copies Keeps Stuff Safe (LOCKSS [@Maniatis_2005]): keep an    identical copy of Catalogue of Life data package outside of the    Catalogue of Life infrastructure.2.  demonstrating how data transformation processes can be documented    using Preston3.  making a streamable copy of a reverse-sorted copy of Catalogue of    Life available via    https://zenodo.org/record/18705780/files/90cfb4edc5f7fa2b55e8bdd9b26d26aec135d1bc9dd11df77df46f732475d3fa    for use in workflows like looking up the first record that contain    *Enhydra lutris* (Sea otter):

<!-- -->

     curl -L 'https://zenodo.org/record/18705780/files/90cfb4edc5f7fa2b55e8bdd9b26d26aec135d1bc9dd11df77df46f732475d3fa'\       | gunzip\       | grep "Enhydra lutris"\       | head -n1

# Acknowledgements

This work stands on the shoulders of contributors to open sourcesoftware and openly accessible datasets. Thank you!