Package: cldfbench
cldfbenchTooling to createCLDFdatasets from existing data.OverviewThis package provides tools to curate cross-linguistic data, with the goal of
packaging it asCLDFdatasets.In particular, it supports a workflow where:"raw" source data is downloaded to araw/subdirectory,and subsequently converted to one or more CLDF datasets in acldf/subdirectory, with the help of:configuration data in aetc/directory andcustom Python code (a subclass ofcldfbench.Datasetwhich implements the workflow actions).This workflow is supported via:a commandline interfacecldfbenchwhich calls the workflow actions assubcommands,acldfbench.Datasetbase class, which must be overwritten in a custom module
to hook custom code into the workflow.With this workflow and the separation of the data into three directories we want
to provide a workbench for transparently deriving CLDF data from data that has been
published before. In particular we want to delineate clearly:what forms part of the original or source data (raw),what kind of information is added by the curators of the CLDF dataset (etc)and what data was derived using the workbench (cldf).Further readingThis paper introducescldfbenchand uses an extended, real-world example:Forkel, R., & List, J.-M. (2020). CLDFBench: Give your cross-linguistic data a lift. In N. Calzolari, F. Béchet, P. Blache, K. Choukri, C. Cieri, T. Declerck, et al. (Eds.), Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020) (pp. 6995-7002). Paris: European Language Resources Association (ELRA).[PDF]Installationcldfbenchcan be installed viapip- preferably in avirtual environment- by running:pipinstallcldfbenchcldfbenchprovides some functionality that relies on python
packages which are not needed for the core functionality. These are specified asextrasand can be installed using syntax like:pipinstallcldfbench[<extras>]where<extras>is a comma-separated list of names from the following list:excel: support for reading spreadsheet data.glottolog: support to accessGlottolog data.concepticon: support to accessConcepticon data.clts: support to accessCLTS data.The command line interfacecldfbenchInstalling the python package will also install a commandcldfbenchavailable on
the command line:$cldfbench-h
usage:cldfbench[-h][--log-levelLOG_LEVEL]COMMAND...

optionalarguments:-h,--helpshowthishelpmessageandexit--log-levelLOG_LEVELloglevel[ERROR|WARN|INFO|DEBUG](default:20)availablecommands:Run"COMAMND -h"togethelpforaspecificcommand.COMMANDcheckRungenericCLDFchecks...As shown above, runcldfbench -hto get help, andcldfbench COMMAND -hto get
help on individual subcommands, e.g.cldfbench new -hto read about the usage
of thenewsubcommand.Dataset discoveryMostcldfbenchcommands operate on an existing dataset (unlikenew, which
creates a new one). Datasets can be discovered in two ways:Via the python module (i.e. the*.pyfile, containing theDatasetsubclass).
To use this mode of discovery, pass the path to the python module
asDATASETargument, when required by a command.Viaentry pointand
dataset ID. To use this mode, specify the name of the entry point as value of
the--entry-pointoption (or use the default namecldfbench.dataset) and
theDataset.idasDATASETargument.Discovery via entry point is particularly useful for commands that can operate
on multiple datasets. To selectalldatasets advertising a given entry point,
pass"_"(i.e. an underscore) asDATASETargument.WorkflowFor a full example of thecldfbenchcuration workflow, seethe tutorial.Creating a skeleton for a new dataset directoryA directory containing stub entries for a dataset can be created runningcldfbenchnewThis will create the following layout (where<ID>stands for the chosen dataset ID):<ID>/
├── cldf               # A stub directory for the CLDF data
│   └── README.md
├── cldfbench_<ID>.py  # The python module, providing the Dataset subclass
├── etc                # A stub directory for the configuration data
│   └── README.md
├── metadata.json      # The metadata provided to the subcommand serialized as JSON
├── raw                # A stub directory for the raw data
│   └── README.md
├── setup.cfg          # Python setup config, providing defaults for test integration
├── setup.py           # Python setup file, making the dataset "installable" 
├── test.py            # The python code to run for dataset validation
└── .github            # Integrate the validation with GitHub actionsImplementing CLDF creationcldfbenchprovides tools to make CLDF creation simple. Still, each dataset is
different, and so each dataset will have to provide its own custom code to do so.
This custom code goes into thecmd_makecldfmethod of theDatasetsubclass in
the dataset's python module.
(See also theAPI documentation ofcldfbench.Dataset.)Typically, this code will make use of one or morecldfbench.CLDFSpecinstances, which describes what kind of CLDF to create. ACLDFSpecalso gives access to acldfbench.CLDFWriterinstance, which wraps apycldf.Dataset.The main interfaces to these objects are:cldfbench.Dataset.cldf_specs: a method returning specifications of all CLDF datasets
that are created by the dataset,cldfbench.Dataset.cldf_writer: a method returning an initializedCLDFWriterassociated with a particularCLDFSpec.cldfbenchsupports several scenarios of CLDF creation:The typical use case is turning raw data into a single CLDF dataset. This would
require instantiating oneCLDFWriterwriter in thecmd_makecldfmethod, and
the defaults ofCLDFSpecwill probably be ok. Since this is the most common and
simplest case, it is supported with some extra "sugar": The initializedCLDFWriteris available asargs.writerwhencmd_makecldfis called.But it is also possible to create multiple CLDF datasets:For a dataset containing both, lexical and typological data, it may be appropriate
to create aẀordlistand aStructureDataset. To do so, one would have to
callcldf_writertwice, passing in an approriateCLDFSpec. Note that if
both CLDF datasets are created in the same directory, they can share theLanguageTable- but would have to specify distinct file names for theParameterTable, passing distinct values toCLDFSpec.data_fnames.When creating multiple datasets of the same CLDF module, e.g. to split a large  dataset into smaller chunks, care must be taken to also disambiguate the name
of the metadata file, passing distinct values toCLDFSpec.metadata_fname.When creating CLDF, it is also often useful to have standard reference catalogs
accessible, in particular Glottolog. See the section onCatalogsfor
a description of how this is supported bycldfbench.CatalogsLinking data to reference catalogs is a major goal of CLDF, thuscldfbenchprovides tools to make catalog access and maintenance easier. Catalog data must be
accessible in local clones of the data repository.cldfbenchprovides commands:catconfigto create the clones and make them known through a configuration file,catinfoto get an overview of the installed catalogs and their versions,catupdateto update local clones from the upstream repositories.See:https://cldfbench.readthedocs.io/en/latest//catalogs.htmlfor a list of reference catalogs which are currently supported incldfbench.Note:Cloningglottolog/glottolog- due to the
deeply nested directories of the language classification - results in long path names. On Windows
this may require disabling themaximum path length limitation.Curating a dataset on GitHubOne of the design goals of CLDF was to specify a data format that plays well with
version control. Thus, it's natural - and actually recommended - to curate a CLDF
dataset in a version controlled repository. The most popular way to do this in a
collaborative fashion is by using agitrepository hosted onGitHub.The directory layout supported bycldfbenchcaters to this use case in several ways:Each directory contains a fileREADME.md, which will be rendered as human readable
description when browsing the repository at GitHub.The file.travis.ymlcontains the configuration for hooking up a repository withTravis CI, to provide continuous consistency checking
of the data.Archiving a dataset with ZenodoCurating a dataset on GitHub also provides a simple way to archiving and publishing
released versions of the data. You can hook up your repository withZenodo(followingthis guide). Then, Zenodo will pick up any released package, assign a DOI to it, archive it and
make it accessible in the long-term.Some notes:Hook-up with Zenodo requires the repository to be public (not private).You should consider using an institutional account on GitHub and Zenodo to associate the repository with. Currently, only the user account registering a repository on Zenodo can change any metadata of releases lateron.Once released and archived with Zenodo, it's a good idea to add the DOI assigned by Zenodo to the release description on GitHub.To make sure a release is picked up by Zenodo, the version number must start with a letter, e.g. "v1.0" -not"1.0".Thus, with a setup as described here, you can make sure you createFAIR data.Extendingcldfbenchcldfbenchcan be extended or built-upon in various ways - typically by customizing core functionality in new python packages. To support particular types of raw data, you might want a customDatasetclass, or to support a particular type of CLDF data, you would customizeCLDFWriter.In addition to extendingcldfbenchusing the standard methods of object-oriented programming, there are two more ways of extendingcldfbench: Commands and dataset templates. Both are implemented usingentry ponits.
So packages which provide custom commands or dataset templates must declare these in metadata that is made known to other Python packages (in particular thecldfbenchpackage)upon installation.CommandsA python package (or a dataset) can provide additional subcommands to be run fromcldfbench.
For more info see thecommands.README.Custom dataset templatesA python package can provide alternative dataset templates to be run withcldfbench new.
Such templates are implemented by:a subclass ofcldfbench.Template,which is advertised using an entry pointcldfbench.scaffold:entry_points={'cldfbench.scaffold':['template_name=mypackage.scaffold:DerivedTemplate',],},