Package: datapackage-pipelines-ckan
Data Package
Pipelinesprocessors forCKAN.Install# clone the repo and install it with pip

git clone https://github.com/frictionlessdata/datapackage-pipelines-ckan.git
pip install -e .Usagedatapackage-pipelines-ckan contains several pipeline processors for
working with CKAN.ckan.add_ckan_resourceA processor to retrieve metadata about a CKAN resource from a CKAN
instance and add it as a datapackage resource.run:ckan.add_ckan_resourceparameters:ckan-host:http://demo.ckan.orgresource-id:d51c9bd4-8256-4289-bdd7-962f8572efb0ckan-api-key:env:CKAN_API_KEY# an env var defining a ckan user api keyckan-host: The base url (and scheme) for the CKAN instance (e.g.http://demo.ckan.org).resource-id: The id of CKAN resourceckan-api-key: Either a CKAN user api key or, if in the formatenv:CKAN_API_KEY_NAME, an env var that defines an api key.
Optional, but necessary for private datasets.ckan.dump.to_ckanA processor to save a datapackage and resources to a specified CKAN
instance.run:ckan.dump.to_ckanparameters:ckan-host:http://demo.ckan.orgckan-api-key:env:CKAN_API_KEYoverwrite_existing:truepush_resources_to_datastore:truedataset-properties:name:test-dataset-010203state:draftprivate:trueowner_org:my-test-orgckan-host: The base url (and scheme) for the CKAN instance (e.g.http://demo.ckan.org).ckan-api-key: Either a CKAN user api key or, if in the formatenv:CKAN_API_KEY_NAME, an env var that defines an api key.overwrite_existing: Iftrue, if the CKAN dataset already
exists, it will be overwritten by the datapackage. Optional, and
default isfalse.push_resources_to_datastore: Iftrue, newly created resources
will be pushed the CKAN DataStore. Optional, and default isfalse.push_resources_to_datastore_method: Value is a string, one of
‘upsert’, ‘insert’ or ‘update’. This will be the method used to add
data to the DataStore (seehttps://ckan.readthedocs.io/en/latest/maintaining/datastore.html#ckanext.datastore.logic.action.datastore_upsert).
Optional, the default is ‘insert’.dataset-properties: An optional object, the properties of which
will be used to set properties of the CKAN dataset.CKAN dataset from datapackageThe processor first creates a CKAN dataset from the datapackage
specification, using the CKAN api`package_create<http://docs.ckan.org/en/latest/api/#ckan.logic.action.create.package_create>`__.
If the dataset already exists, and parameteroverwrite_existingisTrue, the processor will attempt to update the CKAN dataset using`package_update<http://docs.ckan.org/en/latest/api/#ckan.logic.action.update.package_update>`__.
All existing resources and dataset properties will be overwritten.CKAN resources from datapackage resourcesIf the CKAN dataset was successfully created or updated, the dataset
resources will be created for each resource in the datapackage, using`resource_create<http://docs.ckan.org/en/latest/api/#ckan.logic.action.create.resource_create>`__.
If datapackage resource are marked for streaming (they have thedpp:streamed=Trueproperty), resource files will be uploaded to the
CKAN filestore. For example, remote resources may be marked for
streaming by the inclusion of thestream_remote_resourcesprocessor
earlier in the pipeline.Additionally, ifpush_resources_to_datastoreisTrue, the
processor will push resources marked for streaming to the CKAN DataStore
using`datastore_create<https://ckan.readthedocs.io/en/latest/maintaining/datastore.html#ckanext.datastore.logic.action.datastore_create>`__
and`datastore_upsert<https://ckan.readthedocs.io/en/latest/maintaining/datastore.html#ckanext.datastore.logic.action.datastore_upsert>`__.