Package: pyshacl
pySHACLA Python validator for SHACL.This is a pure Python module which allows for the validation ofRDFgraphs against Shapes Constraint Language (SHACL) graphs. This module uses therdflibPython library for working with RDF and is dependent on theOWL-RLPython module forOWL2 RL Profilebased expansion of data graphs.This module is developed to adhere to the SHACL Recommendation:Holger Knublauch; Dimitris Kontokostas.Shapes Constraint Language (SHACL). 20 July 2017. W3C Recommendation. URL:https://www.w3.org/TR/shacl/ED:https://w3c.github.io/data-shapes/shacl/Community for Help and SupportThe SHACL community has a discord server for discussion of topics around SHACL and the SHACL specification.Use this invitation link: https://discord.gg/RTbGfJqdKB to join the serverThere is a #pyshacl channel for discussion of this python library, and you can ask for general SHACL help too.InstallationInstall with PIP (Using the Python3 pip installerpip3)$pip3installpyshaclOr in a python virtualenv(these example commandline instructions are for a Linux/Unix based OS)$python3-mvirtualenv--python=python3--no-site-packages.venv
$source./.venv/bin/activate
$pip3installpyshaclTo exit the virtual enviornment:$deactivateCommand Line UseFor command line use:(these example commandline instructions are for a Linux/Unix based OS)$pyshacl-s/path/to/shapesGraph.ttl-m-irdfs-a-j-fhuman/path/to/dataGraph.ttlWhere-sis an (optional) path to the shapes graph to use-eis an (optional) path to an extra ontology graph to import-iis the pre-inferencing option-fis the ValidationReport output format (human= human-readable validation report)-menable the meta-shacl feature-aenable SHACL Advanced Features-jenable SHACL-JS Features (ifpyhsacl[js]is installed)System exit codes are:0= DataGraph is Conformant1= DataGraph is Non-Conformant2= The validator encountered a RuntimeError (check stderr output for details)3= Not-Implemented; The validator encountered a SHACL feature that is not yet implemented.Full CLI Usage options:$pyshacl-h
$python3-mpyshacl-h
usage:pyshacl[-h][-s[SHACL]][-e[ONT]][-i{none,rdfs,owlrl,both}][-m][-im][-a][-j][-it][--abort][--allow-info][-w][-d][-f{human,table,turtle,xml,json-ld,nt,n3}][-df{auto,turtle,xml,json-ld,nt,n3}][-sf{auto,turtle,xml,json-ld,nt,n3}][-ef{auto,turtle,xml,json-ld,nt,n3}][-V][-o[OUTPUT]]DataGraph

PySHACL0.25.0commandlinetool.

positionalarguments:DataGraphThefilecontainingtheTargetDataGraph.

optionalarguments:--serverIgnorealltherestoftheoptions,starttheHTTPServer.-h,--helpshowthishelpmessageandexit-s[SHACL],--shacl[SHACL]AfilecontainingtheSHACLShapesGraph.-e[ONT],--ont-graph[ONT]AfilepathorURLtoadocumentcontainingextraontologicalinformation.RDFSandOWLdefinitionsfromthisareusedtoinoculatetheDataGraph.-i{none,rdfs,owlrl,both},--inference{none,rdfs,owlrl,both}ChooseatypeofinferencingtorunagainsttheDataGraphbeforevalidating.-m,--metashaclValidatetheSHACLShapesgraphagainsttheshacl-shaclShapesGraphbeforevalidatingtheDataGraph.-im,--importsAllowimportofsub-graphsdefinedinstatementswithowl:imports.-a,--advancedEnablefeaturesfromtheSHACLAdvancedFeaturesspecification.-j,--jsEnablefeaturesfromtheSHACL-JSSpecification.-it,--iterate-rulesRunShape'sSHACLRulesiterativelyuntilthedata_graphreachesasteadystate.--abortAbortonfirstinvaliddata.--allow-info,--allow-infosShapesmarkedwithseverityofInfowillnotcauseresulttobeinvalid.-w,--allow-warning,--allow-warningsShapesmarkedwithseverityofWarningorInfowillnotcauseresulttobeinvalid.-d,--debugOutputadditionalruntimemessages.-f{human,table,turtle,xml,json-ld,nt,n3},--format{human,table,turtle,xml,json-ld,nt,n3}Chooseanoutputformat.Defaultis"human".-df{auto,turtle,xml,json-ld,nt,n3},--data-file-format{auto,turtle,xml,json-ld,nt,n3}ExplicitlystatetheRDFFileformatoftheinputDataGraphfile.Default="auto".-sf{auto,turtle,xml,json-ld,nt,n3},--shacl-file-format{auto,turtle,xml,json-ld,nt,n3}ExplicitlystatetheRDFFileformatoftheinputSHACLfile.Default="auto".-ef{auto,turtle,xml,json-ld,nt,n3},--ont-file-format{auto,turtle,xml,json-ld,nt,n3}ExplicitlystatetheRDFFileformatoftheextraontologyfile.Default="auto".-V,--versionShowPySHACLversionandexit.-o[OUTPUT],--output[OUTPUT]Sendoutputtoafile(defaultstostdout).Python Module UseFor basic use of this module, you can just call thevalidatefunction of thepyshaclmodule like this:frompyshaclimportvalidater=validate(data_graph,shacl_graph=sg,ont_graph=og,inference='rdfs',abort_on_first=False,allow_infos=False,allow_warnings=False,meta_shacl=False,advanced=False,js=False,debug=False)conforms,results_graph,results_text=rWhere:data_graphis an rdflibGraphobject or file path of the graph to be validatedshacl_graphis an rdflibGraphobject or file path or Web URL of the graph containing the SHACL shapes to validate with, or None if the SHACL shapes are included in the data_graph.ont_graphis an rdflibGraphobject or file path or Web URL a graph containing extra ontological information, or None if not required. RDFS and OWL definitions from this are used to inoculate the DataGraph.inferenceis a Python string value to indicate whether or not to perform OWL inferencing expansion of thedata_graphbefore validation.
Options are 'rdfs', 'owlrl', 'both', or 'none'. The default is 'none'.abort_on_first(optional)boolvalue to indicate whether or not the program should abort after encountering the first validation failure or to continue. Default is to continue.allow_infos(optional)boolvalue, Shapes marked with severity of Info will not cause result to be invalid.allow_warnings(optional)boolvalue, Shapes marked with severity of Warning or Info will not cause result to be invalid.meta_shacl(optional)boolvalue to indicate whether or not the program should enable the Meta-SHACL feature. Default is False.advanced: (optional)boolvalue to enable SHACL Advanced Featuresjs: (optional)boolvalue to enable SHACL-JS Features (ifpyshacl[js]is installed)debug(optional)boolvalue to indicate whether or not the program should emit debugging output text, including violations that didn't lead to non-conformance overall. So when debug is True don't judge conformance by absense of violation messages. Default is False.Some other optional keyword variables available on thevalidatefunction:data_graph_format: Override the format detection for the given data graph source file.shacl_graph_format: Override the format detection for the given shacl graph source file.ont_graph_format: Override the format detection for the given extra ontology graph source file.iterate_rules: Iterate SHACL Rules until steady state is found (only works with advanced mode).do_owl_imports: Enable the feature to allow the import of subgraphs usingowl:importsfor the shapes graph and the ontology graph. Note, you explicitly cannot use this on the target data graph.serialize_report_graph: Convert the report results_graph into a serialised representation (for example, 'turtle')check_dash_result: Check the validation result against the given expected DASH test suite result.check_sht_result: Check the validation result against the given expected SHT test suite result.Return value:a three-componenttuplecontaining:conforms: abool, indicating whether or not thedata_graphconforms to theshacl_graphresults_graph: aGraphobject built according to the SHACL specification'sValidation Reportstructureresults_text: python string representing a verbose textual representation of theValidation ReportPython Module CallYou can get an equivalent of the Command Line Tool using the Python3 executable by doing:$python3-mpyshaclIntegrated OpenAPI-3.0-compatible HTTP REST ServicePySHACL now has a built-in validation service, exposed via an OpenAPI3.0-compatible REST API.Due to the additional dependencies required to run, this feature is an optional extra.You must first install PySHACL with thehttpextra option enabled:$pip3install-Upyshacl[http]When that is installed, you can start the service using the by executing the CLI entrypoint:$pyshacl--server# or$pyshacl_server# or$python3-mpyshaclserver# or$dockerrun--rm-ePYSHACL_SERVER=TRUE-i-tdocker.io/ashleysommer/pyshacl:latestBy default, this will run the service on localhost address127.0.0.1on port8099.To view the SwaggerUI documentation for the service, navigate tohttp://127.0.0.1:8099/docs/swaggerand for the ReDoc version, go tohttp://127.0.0.1:8099/docs/redoc.To view the OpenAPI3 schema seehttp://127.0.0.1:8099/docs/openapi.jsonConfiguring the HTTP REST ServiceYou can force PySHACL CLI to start up in HTTP Server mode by passing environment variablePYSHACL_SERVER=TRUE. This is useful in a containerised service, where you willonlybe running PySHACL in this mode.PYSHACL_SERVER_LISTEN=1.2.3.4listen on a different IP Address or hostnamePYSHACL_SERVER_PORT=8080listen on given different TCP PORTPYSHACL_SERVER_HOSTNAME=example.orgwhen you are hosting the server behind a reverse-proxy or in a containerised environment, use this so PySHACL server knows what your externally facing hostname isErrorsUnder certain circumstances pySHACL can produce aValidation Failure. This is a formal error defined by the SHACL specification and is required to be produced as a result of specific conditions within the SHACL graph.
If the validator produces aValidation Failure, theresults_graphvariable returned by thevalidate()function will be an instance ofValidationFailure.
See themessageattribute on that instance to get more information about the validation failure.Other errors the validator can generate:ShapeLoadError: This error is thrown when a SHACL Shape in the SHACL graph is in an invalid state and cannot be loaded into the validation engine.ConstraintLoadError: This error is thrown when a SHACL Constraint Component is in an invalid state and cannot be loaded into the validation engine.ReportableRuntimeError: An error occurred for a different reason, and the reason should be communicated back to the user of the validator.RuntimeError: The validator encountered a situation that caused it to throw an error, but the reason does concern the user.UnlikeValidationFailure, these errors are not passed back as a result by thevalidate()function, but thrown as exceptions by the validation engine and must be
caught in atry ... exceptblock.
In the case ofShapeLoadErrorandConstraintLoadError, see thestr()string representation of the exception instance for the error message along with a link to the relevant section in the SHACL spec document.Windows CLIPyinstallercan beusedto create an
executable for Windows that has the same characteristics as the Linux/Mac
CLI program.
The necessary.specfile is already included inpyshacl/pyshacl-cli.spec.
Thepyshacl-cli.specPyInstaller spec file creates a.exefor the
pySHACL Command Line utility. See above for the pySHACL command line util usage instructions.Seethe PyInstaller installation guidefor info on how to install PyInstaller for Windows.Once you have pyinstaller, use pyinstaller to generate thepyshacl.exeCLI file like so:$cdsrc/pyshacl
$pyinstallerpyshacl-cli.specThis will outputpyshacl.exein thedistdirectory insrc/pyshacl.You can now run the pySHACL Command Line utility viapyshacl.exe.
See above for the pySHACL command line util usage instructions.DockerPull out the official docker image from Dockerhub:docker pull docker.io/ashleysommer/pyshacl:latestOr build the image yourself, from the PySHACL repository withdocker build . -t pyshacl.You can now run PySHACL inside a container; but you need to mount the data you want to validate.
For example, to validategraph.ttlagainstshacl.ttl, run :dockerrun--rm-i-t--mounttype=bind,src=`pwd`,dst=/datapyshacl-s/data/shacl.ttl/data/graph.ttlCompatibilityPySHACL is a Python3 library. For best compatibility use Python v3.7 or greater. Python3 v3.6 or below isnot supportedand this librarydoes not workon Python v2.7.x or below.PySHACL is now a PEP518 & PEP517 project, it usespyproject.tomlandpoetryto manage dependencies, build and install.For best compatibility when installing from PyPI withpip, upgrade to pip v18.1.0 or above.If you're on Ubuntu 16.04 or 18.04, you will need to runsudo pip3 install --upgrade pipto get the newer version.FeaturesA features matrix is kept in theFEATURES file.ChangelogA comprehensive changelog is kept in theCHANGELOG file.BenchmarksThis project includes a script to measure the difference in performance of validating the same source graph that has been inferenced using each of the four different inferencing options. Run it on your computer to see how fast the validator operates for you.LicenseThis repository is licensed under Apache License, Version 2.0. See theLICENSE deedfor details.ContributorsSee theCONTRIBUTORS file.CitationDOI:10.5281/zenodo.4750840(For all versions/latest version)ContactsProject Lead:Nicholas CarSenior Experimental ScientistCSIRO Land & Water, Environmental Informatics Group
Brisbane, Qld, Australianicholas.car@csiro.auhttp://orcid.org/0000-0002-8742-7730Lead Developer:Ashley SommerInformatics Software EngineerCSIRO Land & Water, Environmental Informatics Group
Brisbane, Qld, AustraliaAshley.Sommer@csiro.auhttps://orcid.org/0000-0003-0590-0131