Package: rocrate
ro-crate-py is a Python library to create and consumeResearch Object Crates. It currently supports theRO-Crate 1.1specification.Installationro-crate-py requires Python 3.7 or later. The easiest way to install is viapip:pip install rocrateTo install manually from this code base (e.g., to try the latest development revision):git clone https://github.com/ResearchObject/ro-crate-py
cd ro-crate-py
pip install .UsageCreating an RO-CrateIn its simplest form, an RO-Crate is a directory tree with anro-crate-metadata.jsonfile at the top level that contains metadata about the other files and directories, represented bydata entities. These metadata consist both of properties of the data entities themselves and of other, non-digital entities calledcontextual entities(representing, e.g., a person or an organization).Suppose Alice and Bob worked on a research task together, which resulted in a manuscript written by both; additionally, Alice prepared a spreadsheet containing the experimental data, which Bob used to generate a diagram. Let's make an RO-Crate to package all this:fromrocrate.rocrateimportROCratecrate=ROCrate()paper=crate.add_file("exp/paper.pdf",properties={"name":"manuscript","encodingFormat":"application/pdf"})table=crate.add_file("exp/results.csv",properties={"name":"experimental data","encodingFormat":"text/csv"})diagram=crate.add_file("exp/diagram.svg",dest_path="images/figure.svg",properties={"name":"bar chart","encodingFormat":"image/svg+xml"})We've started by adding the data entities. Now we need contextual entities to represent Alice and Bob:fromrocrate.model.personimportPersonalice_id="https://orcid.org/0000-0000-0000-0000"bob_id="https://orcid.org/0000-0000-0000-0001"alice=crate.add(Person(crate,alice_id,properties={"name":"Alice Doe","affiliation":"University of Flatland"}))bob=crate.add(Person(crate,bob_id,properties={"name":"Bob Doe","affiliation":"University of Flatland"}))Next, we express authorship of the various files:paper["author"]=[alice,bob]table["author"]=alicediagram["author"]=bobFinally, we serialize the crate to disk:crate.write("exp_crate")Now theexp_cratedirectory should contain copies of the three files and anro-crate-metadata.jsonfile with a JSON-LD serialization of the entities and relationships we created, according to the RO-Crate profile. Note that we have chosen a different destination path for the diagram, while the other two files have been placed at the top level with their names unchanged (the default).Some applications and services support RO-Crates stored as archives. To save the crate in zip format, usewrite_zip:crate.write_zip("exp_crate.zip")You can also add whole directories. A directory in RO-Crate is represented by theDatasetentity:logs=crate.add_dataset("exp/logs")Note that the above adds all files and directories contained in"exp/logs"recursively to the crate, but only the top-level"exp/logs"dataset itself is listed in the metadata file (there is no requirement to represent every file and folder in the JSON-LD). To also add files and directory recursively to the metadata, useadd_tree(but note that it only works on local directory trees).Appending elements to property valuesWhat ro-crate-py entities actually store is their JSON representation:paper.properties(){"@id":"paper.pdf","@type":"File","name":"manuscript","encodingFormat":"application/pdf","author":[{"@id":"https://orcid.org/0000-0000-0000-0000"},{"@id":"https://orcid.org/0000-0000-0000-0001"},]}Whenpaper["author"]is accessed, a new list containing thealiceandbobentities is generated on the fly. For this reason, callingappendonpaper["author"]won't actually modify thepaperentity in any way. To add an author, use theappend_tomethod instead:donald=crate.add(Person(crate,"https://en.wikipedia.org/wiki/Donald_Duck"))paper.append_to("author",donald)Note thatappend_toalso works if the property to be updated is missing or has only one value:fornin"Mickey_Mouse","Scrooge_McDuck":p=crate.add(Person(crate,f"https://en.wikipedia.org/wiki/{n}"))donald.append_to("follows",p)Adding remote entitiesData entities can also be remote:input_data=crate.add_file("http://example.org/exp_data.zip")By default the file won't be downloaded, and will be referenced by its URI in the serialized crate:{"@id":"http://example.org/exp_data.zip","@type":"File"},If you addfetch_remote=Trueto theadd_filecall, however, the library (whencrate.writeis called) will try to download the file and include it in the output crate.Another option that influences the behavior when dealing with remote entities isvalidate_url, alsoFalseby default: if it's set toTrue, when the crate is serialized, the library will try to open the URL to add / update metadata bits such as the content's length and format (but it won't try to download the file unlessfetch_remoteis also set).Adding entities with an arbitrary typeAn entity can be of any type listed in theRO-Crate context. However, only a few of them have a counterpart (e.g.,File) in the library's class hierarchy (either because they are very common or because they are associated with specific functionality that can be conveniently embedded in the class implementation). In other cases, you can explicitly pass the type via thepropertiesargument:fromrocrate.model.contextentityimportContextEntityhackathon=crate.add(ContextEntity(crate,"#bh2021",properties={"@type":"Hackathon","name":"Biohackathon 2021","location":"Barcelona, Spain","startDate":"2021-11-08","endDate":"2021-11-12"}))Note that entities can have multiple types, e.g.:"@type"=["File","SoftwareSourceCode"]Modifying the crate from JSON-LD dictionariesTheadd_jsonldmethod allows to add a contextual entity directly from a
JSON-LD dictionary containing at least the@idand@typekeys:crate.add_jsonld({"@id":"https://orcid.org/0000-0000-0000-0000","@type":"Person","name":"Alice Doe"})Existing entities can be updated from JSON-LD dictionaries viaupdate_jsonld:crate.update_jsonld({"@id":"https://orcid.org/0000-0000-0000-0000","name":"Alice K. Doe"})There is also anadd_or_update_jsonldmethod that adds the entity if it's
not already in the crate and updates it if it already exists (note that, when
updating, the@typekey is ignored). This allows to "patch" an RO-Crate from
a JSON-LD file. For instance, suppose you have the followingpatch.jsonfile:{"@graph":[{"@id":"./","author":{"@id":"https://orcid.org/0000-0000-0000-0001"}},{"@id":"https://orcid.org/0000-0000-0000-0001","@type":"Person","name":"Bob Doe"}]}Then the following sets Bob as the author of the crate according to the above
file:crate=ROCrate("temp-crate")withopen("patch.json")asf:json_data=json.load(f)fordinjson_data.get("@graph",[]):crate.add_or_update_jsonld(d)Consuming an RO-CrateAn existing RO-Crate package can be loaded from a directory or zip file:crate=ROCrate('exp_crate')# or ROCrate('exp_crate.zip')foreincrate.get_entities():print(e.id,e.type)ro-crate-metadata.json CreativeWork
./ Dataset
paper.pdf File
results.csv File
images/figure.svg File
https://orcid.org/0000-0000-0000-0000 Person
https://orcid.org/0000-0000-0000-0001 PersonThe first two entities shown in the output are themetadata file descriptorand theroot data entity, respectively. These are special entities managed by theROCrateobject, and are always present. The other entities are the ones we added in thesection on RO-Crate creation. You can access data entities withcrate.data_entitiesand contextual entities withcrate.contextual_entities. For instance:foreincrate.data_entities:author=e.get("author")ifnotauthor:continueelifisinstance(author,list):print(e.id,[p["name"]forpinauthor])else:print(e.id,repr(author["name"]))paper.pdf ['Alice Doe', 'Bob Doe']
results.csv 'Alice Doe'
images/figure.svg 'Bob Doe'You can fetch an entity by its@idas follows:article=crate.dereference("paper.pdf")Command Line Interfacero-crate-pyincludes a hierarchical command line interface: therocratetool.rocrateis the top-level command, while specific functionalities are provided via sub-commands. Currently, the tool allows to initialize a directory tree as an RO-Crate (rocrate init) and to modify the metadata of an existing RO-Crate (rocrate add).$rocrate--helpUsage: rocrate [OPTIONS] COMMAND [ARGS]...Options:--help  Show this message and exit.Commands:addinitwrite-zipCrate initializationTherocrate initcommand explores a directory tree and generates an RO-Crate metadata file (ro-crate-metadata.json) listing all files and directories asFileandDatasetentities, respectively.$rocrateinit--helpUsage: rocrate init [OPTIONS]Options:--gen-preview-e, --exclude CSV-c, --crate-dir PATH--help                Show this message and exit.The command acts on the current directory, unless the-coption is specified. The metadata file is added (overwritten if present) to the directory at the top level, turning it into an RO-Crate.Adding items to the crateTherocrate addcommand allows to add workflows and other entity types (currentlytesting-related metadata) to an RO-Crate:$rocrateadd--helpUsage: rocrate add [OPTIONS] COMMAND [ARGS]...Options:--help  Show this message and exit.Commands:test-definitiontest-instancetest-suiteworkflowNote that data entities (e.g., workflows) must already be present in the directory tree: the effect of the command is to register them in the metadata file.Example# From the ro-crate-py repository rootcdtest/test-data/ro-crate-galaxy-sortchangecaseThis directory is already an RO-Crate. Delete the metadata file to get a plain directory tree:rmro-crate-metadata.jsonNow the directory tree contains several files and directories, including a Galaxy workflow and a Planemo test file, but it's not an RO-Crate since there is no metadata file. Initialize the crate:rocrateinitThis creates anro-crate-metadata.jsonfile that lists files and directories rooted at the current directory. Note that the Galaxy workflow is listed as a plainFile:{"@id":"sort-and-change-case.ga","@type":"File"}To register the workflow as aComputationalWorkflow:rocrateaddworkflow-lgalaxysort-and-change-case.gaNow the workflow has a type of["File", "SoftwareSourceCode", "ComputationalWorkflow"]and points to aComputerLanguageentity that represents the Galaxy workflow language. Also, the workflow is listed as the crate'smainEntity(see theWorkflow RO-Crate profile).To addworkflow testing metadatato the crate:rocrateaddtest-suite-itest1
rocrateaddtest-instancetest1http://example.com-rjobs-itest1_1
rocrateaddtest-definitiontest1test/test1/sort-and-change-case-test.yml-eplanemo-v'>=0.70'LicenseCopyright 2019-2023 The University of Manchester, UKCopyright 2020-2023 Vlaams Instituut voor Biotechnologie (VIB), BECopyright 2020-2023 Barcelona Supercomputing Center (BSC), ESCopyright 2020-2023 Center for Advanced Studies, Research and Development in Sardinia (CRS4), ITCopyright 2022-2023 École Polytechnique Fédérale de Lausanne, CHLicensed under the
Apache License, version 2.0https://www.apache.org/licenses/LICENSE-2.0,
see the fileLICENSE.txtfor details.Cite asThe above DOI corresponds to the latest versioned release aspublished to Zenodo, where you will find all earlier releases.
To citero-crate-pyindependent of version, usehttps://doi.org/10.5281/zenodo.3956493, which will always redirect to the latest release.You may also be interested in the paperPackaging research artefacts with RO-Crate.