Package: shaclapi
shaclAPIThe shaclAPI is a project meant to integrate theSHACLvalidation  into theSPARQLquery execution. Given a knowledge graph aSPARQLquery gives a set of solution mappings. TheSHACLvalidation of a SHACL shape schema over the knowledge graph gives validation results per pair of shape in the network and target entitiy assigned to the shape. The shaclAPI annotates the entities in the solution mappings with the validation result assigned by the SHACL validation. Further SHACL engine - agnostic heuristics are implemented, to reduce the overhead of the validation.SetupWebserviceDockerIn order to run the shaclAPI, build the Docker image from the source code:dockerbuild.-tsdmtib/shaclAPI:latestOnce the Docker image is built, you can start the shaclAPI:dockerrun--nameapi-d-p9999:5000sdmtib/shaclAPI:latestThe shaclAPI will be reachable athttp://localhost:9999. For the following examples execute:API=localhost:9999Docker ComposeA docker-compose file is provided, therefore the shaclAPI can be run using:docker-composeup--build-dThe shaclAPI will be accessible athttp://localhost:9999. For the following examples execute:API=localhost:9999PythonTo run the shaclAPI directly using python 3.8.11 with the flask development server, the python dependencies need to be installed:python-mpipinstall-rrequirements.txtDepending on the operating systemstart.batorstart.shcan now be used to start the shaclAPI.
The shaclAPI will be accessible athttp://localhost:5000. For the following examples execute:API=localhost:5000LibraryTo use the shaclAPI as a library it is recommended to use python 3.8.11, however other versions might work as well.
Install the python dependencies using:python-mpipinstall-rrequirements.txtUsageWebserviceYou can use the shaclAPI by making HTTP requests. The following describes the different API calls:GET: /ReturnsHello Worldand should be used to check whether the API is running or not.Example call:curl-XGET$API/POST: /multiprocessingThis is the main API call, used to send a query to the SPARQL endpoint and simultaneously execute the SHACL validation to validate the query results.
There are various options, which can be provided as parameters of the HTTP POST request. Additionally, a configuration file formatted as JSON can be provided with the config option. HTTP POST parameters will override the options configured in the configuration file.Example configuration:{"external_endpoint":"http://dbpedia.org/sparql","query":"PREFIX dbo:<http://dbpedia.org/ontology/>\nPREFIX dbr:<http://dbpedia.org/resource/>\nPREFIX dbp:<http://dbpedia.org/property/>\nSELECT DISTINCT ?x WHERE {\n    ?x a dbo:Film. \n ?x dbp:studio dbr:Walt_Disney_Pictures\n}","schemaDir":"./examples/dbpedia/shapes/","targetShape":"MovieShape","backend":"travshacl","output_format":"simple"}Example call:curl-XPOST-d"config=./examples/dbpedia/config.json"$API/multiprocessingExample output:[[{"?x":"dbr:Tex_(film)"},#ThequeryresultgivenintheformoftheoriginalSPARQLbindings[#TriplestoexplaintheSPARQLbindings["dbr:Tex_(film)","dbp:studio","dbr:Walt_Disney_Pictures"],["dbr:Tex_(film)","a","dbo:Film"],["dbr:Tex_(film)","dbo:imdbId","<0084783>"],["dbr:Tex_(film)","dbo:starring","dbr:Matt_Dillon"]],[#ThevalidationresultassignedtotheSPARQLbindings.EachentityoccuringintheSPARQLbindingscanviolate(ts:violatesShape)orsatisfy(ts:satisfiesShape)agiveninstance:["dbr:Tex_(film)","ts:violatesShape","MovieShape"]]],[{"?x":"dbr:Night_Crossing"},[["dbr:Night_Crossing","dbp:studio","dbr:Walt_Disney_Pictures"],["dbr:Night_Crossing","a","dbo:Film"],["dbr:Night_Crossing","dbo:imdbId","<0082810>"],["dbr:Night_Crossing","dbo:starring","dbr:Doug_McKeon"]],[["dbr:Night_Crossing","ts:violatesShape","MovieShape"]]],...]There are further examples provided in theexamplesdirectory.POST: /validationThis API call can be used to execute the SHACL validation  over the given SPARQL endpoint, while reducing the workload using the given heuristics and give the number of valid/invalid instances per Shape. There are various options, which can be provided as parameters of the HTTP POST request. Additionally, a configuration file formatted as JSON can be provided with the config option. HTTP POST parameters will override the options configured in the configuration file.Using the same configuration as with/multiprocessing:Example call:curl-XPOST-d"config=./examples/dbpedia/config.json"$API/validationExample output:{"MovieShape":{"invalid":50,"valid":0}}POST: /reduceThis API call can be used to retrieve the shape names of the reduced shape schema given a starting shape.
The options for this call can be provided as parameters of the HTTP POST request or as a JSON file using the config option.
HTTP POST parameters will override the options configured in the configuration file.Example call:curl-XPOST-d"schemaDir=./examples/lubm/shapes/"-d"targetShape=Department"$API/reduceExample output:{"shapes":["Department","University"]}LibrarySee the available sphinx documentation:https://sdm-tib.github.io/shaclAPI/html/index.htmlConfigurationThe shaclAPI is highly configurable and supports the following options.RequiredOptionDefaultDescriptionxquery-The query to be executed over the SPARQL endpoint.target_shape / targetShapeNoneThe target shape selected for the given query and the given shape schema.xexternal_endpoint-The URL of the SPARQL endpoint, which contains the data to be validated and retrieved.xschema_directory-The directory containing the shape files (.ttl or .json)configNoneThe path to a json formatted configuration file. Has to be absolute or relative w.r.t. run.py.output_directory./output/The directory which will be used by the SHACL engine and the shaclAPI to save the validation output and statistics to files (depending on the other configurations)shape_format / shapeFormatSHACLThe format of the shape files. Can be JSON or SHACLwork_in_parallel / workInParallelFalseWhether the SHACL engine should work in paralleluse_selective_queries / useSelectiveQueriesTrueWhether the SHACL engine should use more selective queries.maxSplit256The maximal number of entities in FILTER or VALUES clause of a SPARQL query, used by the SHACL engine.order_by_in_queriesTrueWhether the SHACL engine should use queries with a ORDER BY clause.backendtravshaclThe SHACL engine, which will be used by the shaclAPI.traversal_strategy / traversalStrategyDFSThe traversal strategy used by the backend to reduce the shape graph and by the backend to find the execution order. Can be DFS or BFS.heuristicTARGET IN BIGOnly if Trav-SHACL is used as backend. The heuristic used to determine the validation order of the shapes.replace_target_queryTrueWhether or not the shaclAPI should replace the target query of the target shape.start_with_target_shapeTrueWhether the SHACL engine is forced to start the validation process with the target shape.start_shape_for_validationNoneThe shape which is used as starting point for the validation in the backend. It will override the start point determined by the SHACL engine (in case of Trav-SHACL) and only applies if start_with_target_shape is false)merge_old_target_queryTrueWhether the shaclAPI should merge the query with the given target query in the target shape file. If this option is inactive the target query of the target shape is basically replaced with the star shaped query.remove_constraintsFalseWhether the shaclAPI should remove constraints of the target shape not mentioned in the query.output_formatsimpleWhich output format the api should use. This can be "test" or "simple".memory_size100000000Number of tuples, which can be stored in main memory during the join process.prune_shape_networkTrueWhether or not prune the shape schema to the shapes reachable from the target shapes.test_identifierrandom uuid1The test identifier will be used in output files identifing the run.run_in_serialFalseThis option can be turned on to force the steps of the shaclAPI to be executed in serial.reasoningTrueThis option will turn reasoning in terms of extended output on and off.use_pipesFalseWhether to use pipes during the multiprocessing. Otherwise the shaclAPI will use queues.collect_all_validation_resultsFalseWhether to collect all validation results for each mapping. Otherwise at least one validation result is collected for each given target_shape. Collecting all results will make the approach blocking.write_statsTrueWhether to write statistics to the output directory.outputsFalseWhether to save the validation output of the backend to a file.query_extension_per_target_shapeNoneFor each given target shape a query extension can be given. The given query is extended, when merged or replaced with the target definition of the target shape. The query is extended by replacing the last '}' in the query with the extension followed by a '}'.