PresQT Workshop 1:  Data Rescue: Packaging, Curation, Ingest, and Discovery by Elliot Metsger
Channel: Natalie Meyers
Date: 2017-05-08

A talk on Data Rescue: Packaging, Curation, Ingest, and Discovery with Fedora 4, API-X,and the Open Science Framework given by Elliot Metsger, from the Data Conservancy at Johns Hopkins University at the PresQT Workshop May 1, 2017 at University of Notre Dame. 

Grass roots efforts to identify, select, and archive “endangered” (e.g. politically sensitive) data are gaining traction.  This presentation presents an opinionated integration of discrete tools that enable the packaging, curation, ingest, and exposure of data.  

The Data Conservancy (DC) has developed the Data Conservancy Package Tool GUI which allows the user to create a package and apply semantic metadata to the files in the package payload.  Therefore, packages produced by this software contain semantic descriptions along with the package payload.

The Data Conservancy Package Ingest service is able to ingest these semantic packages into a Fedora 4 repository, exposing the binary content and semantic descriptions as web resources. The DC Package Ingest service is exposed at the repository layer as a Fedora API extension, which allows the service to integrate seamlessly with the repository and client using REST-based principles.

Finally, active development is underway to integrate Fedora with the Open Science Frameworkas an OSF Storage Add-On.  This allows users of the OSF to browse data in a Fedora repository using familiar (albeit limiting) filesystem semantics.

Data Conservancy packaging tools and services, together with the Fedora repository and Open Science Framework, provide a compelling technology stack for archiving and curating endangered data while providing access to the data using an open platform.