Package: ckanext-attribution
ckanext-attributionA CKAN extension that adds support for complex attribution.OverviewThis extension standardises author/contributor attribution for datasets, enabling enhanced metadata
and greater linkage between datasets. It currently integrates with theORCIDandRORAPIs; contributors ('agents') can be added directly from these databases,
or manually.Contributors can be added and edited via actions or via a Vue app that can be inserted into
thepackage_metadata_fields.htmltemplate snippet.SchemaThe schema is (partially) based on
theRDA/TDWG recommendations. Three new models are
added:Agent(contributors),ContributionActivity, andAffiliation(plus small linking models
between these andPackagerecords).AgentDefinesoneagent.FieldTypeValuesNotesagent_typestring'person', 'org', 'other'family_namestringonly used for 'person' recordsgiven_namesstringonly used for 'person' recordsgiven_names_firstboolTrue, Falseonly used for 'person' records; if the given names should be displayed first according to the person's culture/language (default True)namestringused for non-'person' recordslocationstringused for non-person records, optional; a location to display for the organisation to help differentiate between similar names (e.g. 'Natural History Museum (_London_)' and 'Natural History Museum (Dublin)')external_id|string||an identifier from an external service like ORCID or RORexternal_id_scheme|string|'orcid', 'ror', other|the scheme for theexternal_id; currently only 'orcid' and 'ror' are fully supported, though basic support for others can be implemented by adding to theattribution_controlled_listsactionuser_id|string|User.idforeign key|link to a user account on the CKAN instanceContributionActivityDefinesoneactivity performed byoneagent ononespecific dataset.FieldTypeValuesNotesactivitystring[controlled vocabulary]the activity/role the agent is associated with, e.g. 'Editor', 'Methodology'; roles are defined in theattribution_controlled_listsaction, which currently lists theDataciteandCRediTrole taxonomies (but can be expanded)schemestring[controlled vocabulary]name of the defined scheme fromattribution_controlled_listslevelstring'Lead', 'Equal', 'Supporting'optional degree of contribution (fromCRediT)timedatetimeoptional date/time of the activityorderintegerorder of the agent within all who are associated with the same activity, e.g. 1st Editor, 3rd DataCollector (optional)A specialisedContributionActivityentry with a '[citation]' activity is used to define the order
in which contributors should be cited (and/or if they should be cited at all).AffiliationDefines a relationship betweentwoagents, either as a 'universal' (persistent) affiliation or for
a single package (e.g. a project affiliation).FieldTypeValuesNotesagent_a_idstringAgent.idforeign keyone of the two agents (a/b order does not matter)agent_b_idstringAgent.idforeign keyone of the two agents (a/b order does not matter)affiliation_typestringvery short description (1 or 2 words) of affiliation, e.g. 'employment' (optional)descriptionstringlonger description of affiliation (optional)start_datedatedate at which the relationship began, e.g. employment start date (optional)end_datedatedate at which the relationship ended (optional)package_idstringPackage.idforeign keylinks affiliation to a specific package/dataset (optional)InstallationPath variables used below:$INSTALL_FOLDER(i.e. where CKAN is installed), e.g./usr/lib/ckan/default$CONFIG_FILE, e.g./etc/ckan/default/development.iniInstalling from PyPIpipinstallckanext-attributionInstalling from sourceClone the repository into thesrcfolder:cd$INSTALL_FOLDER/src
gitclonehttps://github.com/NaturalHistoryMuseum/ckanext-attribution.gitActivate the virtual env:.$INSTALL_FOLDER/bin/activateInstall via pip:pipinstall$INSTALL_FOLDER/src/ckanext-attributionInstalling in editable modeInstalling from apyproject.tomlin editable mode (i.e.pip install -e) requiressetuptools>=64; however, CKAN 2.9 requiressetuptools==44.1.0. Seeour CKAN forkfor a version of v2.9 that uses an updated setuptools if this functionality is something you need.Post-install setupAdd 'attribution' to the list of plugins in your$CONFIG_FILE:ckan.plugins=... attributionInstalllesscglobally:npminstall-g"less@~4.1"Add this block topackage_metadata_fields.htmlto show the Vue app:{% block package_custom_fields_agent %}
     {{ super() }}
{% endblock %}Change theauthorsfield in your SOLRschema.xmlto set up faceting.<schema><fields><...><fieldname="author"type="string"indexed="true"stored="true"multiValued="true"/><...></fields><...><copyFieldsource="author"dest="text"/></schema>After making the changes, restart SOLR and reindex (ckan -c $CONFIG_FILE search-index rebuild).
You will also have to enable the config option (see below) to see this in the UI.ConfigurationThese are the options that can be specified in your .ini config file. NB:
settingckanext.attribution.debugtoTruemeans that the API
accessessandbox.orcid.orginstead oforcid.org.
Although both run by the ORCID organisation, these aredifferent websitesand you will need a
separate account/set of credentials for each. It is also worth noting that you will not have access
to the full set of authors on the sandbox.API credentials [REQUIRED]NameDescriptionOptionsckanext.attribution.orcid_keyYour ORCID API client ID/keyckanext.attribution.orcid_secretYour ORCID API client secretOptionalNameDescriptionOptionsDefaultckanext.attribution.debugIf true, use sandbox.orcid.org (for testing)True/FalseTrueckanext.attribution.enable_facetingEnable filtering by contributor name (requires change to SOLR schema)True/FalseFalseUsageActionsThis extension adds numerous new actions. These are primarily CRUD actions for managing models, with
inline documentation and predictable interactions. It's probably more helpful to only go over the
more "unusual" new actions here.agent_listSearch for agents by name or external ID, or just list all agents.data_dict={'q':'QUERY',# optional; searches in name, family_name, given_names, and external_id}toolkit.get_action('agent_list')({},data_dict)package_contributions_showShow all contribution records for a package, grouped by agent. Optionally provide a limit and offset
for pagination.data_dict={'id':'PACKAGE_ID','limit':'PAGE_SIZE','offset':'OFFSET'}toolkit.get_action('package_contributions_show')({},data_dict)Returns a dict:{'contributions':[{'agent':{# Agent.as_dict()},'activities':[# list of Activity.as_dict()],'affiliations':[{'affiliation':{# Affiliation.as_dict()},'other_agent':{# Agent.as_dict()}},# ...]},# ...],'total':total,'offset':offset,'page_size':limitortotal}agent_affiliationsShow all affiliations for a given agent, optionally limited to a specific dataset/package (plus '
global' affiliations).data_dict={'agent_id':'AGENT_ID','package_id':'PACKAGE_ID'# optional}toolkit.get_action('agent_affiliations')({},data_dict)Returns a list of records formatted as such:{'affiliation':{# Affiliation.as_dict()},'other_agent':{# Agent.as_dict()}}attribution_controlled_listsReturns collections of defined values (which can be modified by using@toolkit.chained_action).data_dict={'lists':['NAME1','NAME2']# optional; only return these lists}toolkit.get_action('attribution_controlled_lists')({},data_dict)There are four collections:agent_typesdescribes valid types for agents and adds additional detail;contribution_activity_typescontains role/activity taxonomies (i.e. Datacite and CRediT) and
lists the available activity values;contribution_activity_levelsis a list of contribution levels (i.e. 'lead', 'equal', and '
supporting', from CRediT);agent_external_id_schemesdescribes valid schemes for external IDs (currently, ORCID and ROR).These collections are useful for validation and frontend connectivity/standardisation. They are
contained within an action toa.enable frontend access via AJAX requests, andb.allow users to
override values as needed.agent_external_searchSearch external sources (ORCID and ROR) for agent data. Ignores records that already exist in the
database.data_dict={'q':'QUERY_STRING','sources':['SOURCE1','SOURCE2']# optional; only search these sources}toolkit.get_action('agent_external_search')({},data_dict)Results are returned formatted as such:{'SCHEME_NAME':{'records':[# list of agent dicts]'remaining':10000# number of other records found}}agent_external_readRead data from an external source like ORCID or ROR, either from an existing record or a new
external ID.# EITHERdata_dict_existing={'agent_id':'AGENT_ID','diff':False# optional; only show values that differ from the record's current values (default False)}# ORdata_dict_new={'external_id':'EXTERNAL_ID','external_id_scheme':'orcid'# or 'ror', etc.}toolkit.get_action('agent_external_read')({},data_dict)Commandsinitdbckan-c$CONFIG_FILEattributioninitdbInitialise database tables.syncckan-c$CONFIG_FILEattributionsync$OPTIONAL_ID$ANOTHER_OPTIONAL_IDRetrieve up-to-date information from external APIs for contributors with an external ID set.refresh-packagesckan-c$CONFIG_FILEattributionrefresh-packages$OPTIONAL_ID$ANOTHER_OPTIONAL_IDUpdate the author string for all (or the specified) packages.agent-external-searchckan-c$CONFIG_FILEattributionagent-external-search--limit10$OPTIONAL_ID$ANOTHER_OPTIONAL_IDSearch external APIs for contributors without an external ID set. Runrefresh-packagesand rebuild the search index after this command.merge-agentsckan-c$CONFIG_FILEattributionmerge-agents--q$SEARCH_QUERY--match-threshold75Find agents with similar names (optionally matching the search query) and merge them. Runrefresh-packagesand rebuild the search index after this command.migratedbckan-c$CONFIG_FILEattributionmigratedb--limit10--dry-run--no-search-apiAttempt to extract names of contributors from author fields and convert them to the new format.--limitwill only convert a certain number of packages at a time.--dry-runprevents saving to the database.--no-search-apijust extracts the names, without searching external APIs for contributors after.It is recommended to runmerge-agents,refresh-packages, and rebuild the search index after running this command.TestingThere is a Docker compose configuration available in this repository to make it easier to run tests. The ckan image uses the Dockerfile in thedocker/folder.To run the tests against ckan 2.9.x on Python3:Build the required images:docker-composebuildThen run the tests.
The root of the repository is mounted into the ckan container as a volume by the Docker compose
configuration, so you should only need to rebuild the ckan image if you change the extension's
dependencies.docker-composerunckan