Package: csvw
csvwThis package providesa Python API to read and write relational, tabular data according to theCSV on the Webspecification andcommandline tools for reading and validating CSVW data.LinksGitHub:https://github.com/cldf/csvwPyPI:https://pypi.org/project/csvwIssue Tracker:https://github.com/cldf/csvw/issuesInstallationThis package runs under Python >=3.8, use pip to install:$pipinstallcsvwCLIcsvw2jsonConverting CSVW datato JSON$csvw2jsontests/fixtures/zipped-metadata.json{"tables":[{"url":"tests/fixtures/zipped.csv","row":[{"url":"tests/fixtures/zipped.csv#row=2","rownum":1,"describes":[{"ID":"abc","Value":"the value"}]},{"url":"tests/fixtures/zipped.csv#row=3","rownum":2,"describes":[{"ID":"cde","Value":"another one"}]}]}]}csvwvalidateValidating CSVW data$csvwvalidatetests/fixtures/zipped-metadata.jsonOKcsvwdescribeDescribing tabular-data files with CSVW metadata$csvwdescribe--delimiter"|"tests/fixtures/frictionless-data.csv{"@context":"http://www.w3.org/ns/csvw","dc:conformsTo":"data-package","tables":[{"dialect":{"delimiter":"|"},"tableSchema":{"columns":[{"datatype":"string","name":"FK"},{"datatype":"integer","name":"Year"},{"datatype":"string","name":"Location name"},{"datatype":"string","name":"Value"},{"datatype":"string","name":"binary"},{"datatype":"string","name":"anyURI"},{"datatype":"string","name":"email"},{"datatype":"string","name":"boolean"},{"datatype":{"dc:format":"application/json","base":"json"},"name":"array"},{"datatype":{"dc:format":"application/json","base":"json"},"name":"geojson"}]},"url":"tests/fixtures/frictionless-data.csv"}]}Python APIFind the Python API documentation atcsvw.readthedocs.io.A quick example for usingcsvwfrom Python code:importjsonfromcsvwimportCSVWdata=CSVW('https://raw.githubusercontent.com/cldf/csvw/master/tests/fixtures/test.tsv')print(json.dumps(data.to_json(minimal=True),indent=4))[{"province":"Hello","territory":"world","precinct":"1"}]Known limitationsWe readalldata which is specified as UTF-8 encoded using theutf-8-sigcodecs.
Thus, if such data starts withU+FEFFthis will be interpreted asBOMand skipped.Low level CSV parsing is delegated to thecsvmodule in Python's standard library. Thus, if acommentPrefixis specified in aDialectinstance, this will lead to skipping rows where the first value starts
withcommentPrefix,even if the value was quoted.Also, cell content containingescapecharmay not be round-tripped as expected (when specifyingescapecharor acsvw.DialectwithquoteCharbutdoubleQuote==False),
when minimal quoting is specified. This is due to inconsistentcsvbehaviour
across Python versions (seehttps://bugs.python.org/issue44861).CSVW conformanceWhile we use the CSVW specification as guideline, this package does not (and
probably never will) implement the full extent of this spec.When CSV files with a header are read, columns are not matched in order with
column descriptions in thetableSchema, but instead are matched based on the
CSV column header and the column descriptions'nameandtitlesatributes.
This allows for more flexibility, because columns in the CSV file may be
re-ordered without invalidating the metadata. A stricter matching can be forced
by specifying"header": falseand"skipRows": 1in the table's dialect
description.However,csvw.CSVWworks correctly for269 out of 270JSON tests,280 out of 282validation tests,10 out of 18non-normative testsfrom theCSVW Test suites.Compatibility withFrictionless Data SpecsA CSVW-described dataset is basically equivalent to a Frictionless DataPackage where allData ResourcesareTabular Data.
Thus, thecsvwpackage provides some conversion functionality. To
"read CSVW data from a Data Package", there's thecsvw.TableGroup.from_frictionless_datapackagemethod:fromcsvwimportTableGrouptg=TableGroup.from_frictionless_datapackage('PATH/TO/datapackage.json')To convert the metadata, theTableGroupcan then be serialzed:tg.to_file('csvw-metadata.json')Note that the CSVW metadata file must be written to the Data Package's directory
to make sure relative paths to data resources work.This functionality - together with the schema inference capabilities
offrictionless describe- provides
a convenient way to bootstrap CSVW metadata for a set of "raw" CSV
files, implemented in thecsvwdescribecommand described above.See alsohttps://www.w3.org/2013/csvw/wiki/Main_Pagehttps://csvw.orghttps://github.com/CLARIAH/COWhttps://github.com/CLARIAH/ruminatorhttps://github.com/bloomberg/pycsvwhttps://specs.frictionlessdata.io/table-schema/https://github.com/theodi/csvlint.rbhttps://github.com/ruby-rdf/rdf-tabularhttps://github.com/rdf-ext/rdf-parser-csvwhttps://github.com/Robsteranium/csvwrLicenseThis package is distributed under theApache 2.0 license.