Package: grlc
grlc, thegitrepositorylinked data APIconstructor, automatically builds Web APIs using shared SPARQL queries.http://grlc.io/If you use grlc in your work, please cite it as:@InProceedings{merono2016grlc,
 author = {Mero{\~{n}}o-Pe{\~{n}}uela, Albert and Hoekstra, Rinke},
 title = {{grlc Makes GitHub Taste Like Linked Data APIs}},
 booktitle = {The Semantic Web: ESWC 2016 Satellite Events, Heraklion, Crete, Greece, May 29 -- June 2,  2016},
 year = {2016},
 publisher = {Springer},
 pages = {342--353},
 isbn = {978-3-319-47602-5},
 doi = {10.1007/978-3-319-47602-5_48}
}What is grlc?grlc is a lightweight server that takes SPARQL queries (stored in a GitHub or GitLab repository, in your local filesystem, or listed in a URL), and translates them to Linked Data Web APIs. This enables universal access to Linked Data. Users are not required to know SPARQL to query their data, but instead can access a web API.Quick tutorialFor a quick usage tutorial check out our wikiwalkthroughandlist of features.Usagegrlc assumes that you have a collection of SPARQL queries as .rq files (likethis). grlc will create one API operation for each SPARQL query/.rq file in the collection.Your queries can add API parameters to each operation by using theparameter mappingsyntax. This allows your query to define query variables which will be mapped to API parameters for your API operation (see herefor an example).Your queries can include specialdecoratorsto add extra functionality to your API.Query locationgrlc can load your query collection from different locations: from a GitHub repository (api-git), from a GitLab repository (api-gitlab), from local storage (api-local), and from a specification file (api-url). Each type of location has specific features and is accessible via different paths. However all location types produce the same beautiful APIs.From a GitHub repositoryAPI path:http://grlc-server/api-git/<user>/<repo>grlc can build an API from any Github repository, specified by the GitHub user name of the owner (<user>) and repository name (<repo>).For example, assuming your queries are stored on a Github repo:https://github.com/CLARIAH/grlc-queries/, point your browser to the following locationhttp://grlc.io/api-git/CLARIAH/grlc-queries/grlc can make use of git's version control mechanism to generate an API based on a specific version of queries in the repository. This can be done by including the commit sha in the URL path (http://grlc-server/api-git/<user>/<repo>/commit/<sha>), for example:http://grlc.io/api-git/CLARIAH/grlc-queries/commit/79ceef2ee814a12e2ec572ffaa2f8212a22bae23grlc can also use a subdirectory inside your Github repo. This can be done by including a subdirectory in the URL path (http://grlc-server/api-git/<user>/<repo>/subdir/<subdir>).From a GitLab repositoryAPI path:http://grlc-server/api-gitlab/<user>/<repo>grlc can build an API from any GitLab repository, specified by the GitLab user name of the owner (<user>) and repository name (<repo>).For example, assuming your queries are stored on a GitLab repo:https://gitlab.com/c-martinez/grlc-queries, point your browser to the following locationhttp://grlc.io/api-gitlab/c-martinez/grlc-queries/grlc can make use of git's version control mechanism to generate an API based on a specific version of queries in the repository. This can be done by including the name of a branch in the URL path (http://grlc-server/api-gitlab/<user>/<repo>/branch/<branch>), for example:http://grlc.io/api-gitlab/c-martinez/grlc-queries/branch/mastergrlc can also use a subdirectory inside your GitLab repo. This can be done by including a subdirectory in the URL path (http://grlc-server/api-gitlab/<user>/<repo>/subdir/<subdir>), for example:http://grlc-server/api-gitlab/c-martinez/grlc-queries/subdir/subdir.From local storageAPI path:http://grlc-server/api-local/grlc can generate an API from a local directory in the computer where your grlc server runs. You can configure the location of this directory in yourgrlc server configuration file. See alsohow to install and run your own grlc instance.When the API is generated from a local directory, API information can be loaded from a configuration file in that folder. This file must be calledlocal-api-config.iniand it has the following format:[repo_info]repo_title=Some titleapi_description=Description of my APIcontact_name=My namecontact_url=https://mypage/licence_url=https://mylicence/From a specification fileAPI path:http://grlc-server/api-url/?specUrl=<specUrl>grlc can generate an API from a yaml specification file accessible on the web.For example, assuming your queries are listed on spec file:https://raw.githubusercontent.com/CLARIAH/grlc-queries/master/urls.yml, point your browser to the following locationhttp://grlc.io/api-url?specUrl=https://raw.githubusercontent.com/CLARIAH/grlc-queries/master/urls.ymlSpecification file syntaxA grlc API specification file is a YAML file which includes the necessary information to create a grlc API, most importantly a list of URLs to decorated and HTTP-dereferenceable SPARQL queries. This file should contain the following fieldstitle: Title of my APIdescription: API descriptioncontact: Contact details of the API owner. This should include thenameandurlproperties.licence: A URL pointing to the licence file for the API.queries: A list of URLs of SPARQL queries (with header decorators). Alternatively a query can be defined as a dictionary with anameand aurl.For example:title:Title of my APIdescription:Description of my APIcontact:name:Contact Nameurl:https://www.mywebsite.orglicence:http://example.org/licence.htmlqueries:-https://www.mywebsite.org/query1.rq-https://www.mywebsite.org/query2.rq-https://www.otherwebsite.org/query3.rq-name:QueryFoururl:https://www.mywebsite.org/query4.rqgrlc generated APIThe API paths of all location types point to the generatedswagger-uistyle API documentation. On the API documentation page, you can explore available API calls and execute individual API calls.You can also view the swagger spec of your API, by visiting<API-path>/swagger, for example:http://grlc.io/api-git/CLARIAH/grlc-queries/swaggergrlc query executionWhen you call an API endpoint, grlc executes the SPARQL query for that endpoint by combining supplied parameters and decorators.There are 4 options to specify your own endpoint:Add asparql_endpointon yourconfig.iniAdd aendpointparameter to your request: 'http://grlc.io/user/repo/query?endpoint=http://sparql-endpoint/'. You can add a#+ endpoint_in_url: Falsedecorator if you DO NOT want to see theendpointparameter in the swagger-ui of your API.Add the#+ endpoint:decorator.Add the URL of the endpoint on a single line in anendpoint.txtfile within the GitHub repository that contains the queries.The endpoint call will return the result of executing the query as a json representation of rdflib.query.QueryResult (for other result formats, you can use content negotiation via HTTPAcceptheaders). For json responses, the schema of the response can be modified by using the#+ transform:decorator.Decorator syntaxSpecial decorators are available to make your swagger-ui look nicer and to increase functionality. These are provided as comments at the start of your query file, making it still syntactically valid SPARQL. All decorators start with#+, for example:#+ decorator_1: decorator value#+ decorator_1: decorator valueSELECT*WHERE{?s?p?o.}The following is a list of available decorators and their function:summaryCreates a summary of your query/operation. This is shown next to your operation name in the swagger-ui.Syntax:#+ summary: This is the summary of my query/operationExamplequeryand the equivalentAPI operation.descriptionCreates a description of your query/operation. This is shown as the description of your operation in the swagger-ui.Syntax:#+ description: Extended description of my query/operation.Examplequeryand the equivalentAPI operation.endpointSpecifies a query-specific endpoint.Syntax:#+ endpoint: http://example.com/sparqlExamplequeryand the equivalentAPI operation.paginationPaginates the results in groups of (for example) 100. Links to previous, next, first, and last result pages are provided as HTTP response headers to avoid polluting the payload (see detailshere)Syntax:#+ pagination: 100Examplequeryand the equivalentAPI operation.methodIndicates the HTTP request method (GETandPOSTare supported).Syntax:#+ method: GETExamplequeryand the equivalentAPI operation.tagsAssign tags to your query/operation. Query/operations with the same tag are grouped together in the swagger-ui.Syntax:#+ tags:
#+   - firstTag
#+   - secondTagExamplequeryand the equivalentAPI operation.defaultsSet the default value in the swagger-ui for a specific parameter in the query.Syntax:#+ defaults:
#+   - param_name: default_valueExamplequeryand the equivalentAPI operation.enumerateIndicates which parameters of your query/operation should get enumerations (and get dropdown menus in the swagger-ui) using the given values from the SPARQL endpoint. The values for each enumeration variable can also be specified into the query decorators to save endpoint requests and speed up the API generation.Syntax:#+ enumerate:
#+   - var1:
#+     - value1
#+     - value2Examplequeryand the equivalentAPI operation.Notice that these should be plain variable names without SPARQL/BASIL conventions (sovar1instead of?_var1_iri)endpoint_in_urlAllows/disallows theendpointparameter from being provided as a URL parameter (allowed by default).Syntax:#+ endpoint_in_url: FalseExamplequeryand the equivalentAPI operation.transformAllows  query results to be converted to the specified JSON structure, by usingSPARQLTransformersyntax. Notice that the response content type must be set toapplication/jsonfor the transformation to take effect.Syntax:#+ transform: {
#+     "key": "?p",
#+     "value": "?o",
#+     "$anchor": "key"
#+   }Examplequeryand the equivalentAPI operation.endpoint-methodAllows the query to be sent from the grlc server to the SPARQL endpoint using eitherGETorPOSThttp method. (Default:POST)Syntax:#+ endpoint-method: GETExamplequeryand the equivalentAPI operation.Example APIsCheck these out:http://grlc.io/api-git/CLARIAH/grlc-querieshttp://grlc.io/api-gitlab/c-martinez/grlc-querieshttp://grlc.io/api-url?specUrl=https://raw.githubusercontent.com/CLARIAH/grlc-queries/master/urls.ymlhttp://grlc.io/api-git/CLARIAH/wp4-queries-hiscohttp://grlc.io/api-git/albertmeronyo/lodapihttp://grlc.io/api-git/albertmeronyo/lsq-apihttps://grlc.io/api-git/CEDAR-project/QueriesYou'll find the sources of these and many more inGitHubUsethis GitHub searchto see examples from other grlc users.Install and runYou can use grlc in different ways:Via grlc.io: you can use thegrlc.io serviceVia Docker: you can use thegrlc docker imageand start your own grlc server.Via pip: you can install thegrlc Python packageand start your own grlc server or use grlc as a Python library.More details for each of these options are given below.grlc.ioThe easiest way to use grlc is by visitinggrlc.ioand using this service to convert SPARQL queries into a RESTful API. Your queries can bestored on a github repoor can belisted on a specification file.DockerTo run grlc viadocker, you'll need a working installation of docker. To deploy grlc, just pull thelatest image from Docker hub. :dockerrun-it--rm-p8088:80clariah/grlcThe docker image allows you to setup several environment variable such asGRLC_SERVER_NAMEGRLC_GITHUB_ACCESS_TOKEN,GRLC_GITLAB_ACCESS_TOKENandGRLC_SPARQL_ENDPOINT:dockerrun-it--rm-p8088:80-eGRLC_SERVER_NAME=grlc.io-eGRLC_GITHUB_ACCESS_TOKEN=xxx-eGRLC_GITLAB_ACCESS_TOKEN=yyy-eGRLC_SPARQL_ENDPOINT=http://dbpedia.org/sparql-eDEBUG=trueclariah/grlcPipIf you want to run grlc locally or use it as a library, you can install grlc on your machine. Grlc isregistered in PyPiso you can install it using pip.Prerequisitesgrlc has the following requirements:Python3development files (depending on your OS):sudoapt-getinstalllibevent-devpython-all-devpip installOnce the base requirements are satisfied, you can install grlc like this:pipinstallgrlcOnce grlc is installed, you have several options:Stand alone serverUsing a WSGI serverAs a python libraryStandalone servergrlc includes a command line tool which you can use to start your own grlc server:grlc-serverUsing a WSGI serverYou can run grlc using a WSGI server such as gunicorn as follows:gunicorngrlc.server:appIf you want to use your own gunicorn configuration, for examplegunicorn_config.py:workers=5worker_class='gevent'bind='0.0.0.0:8088'Then you can run it as:gunicorn-cgunicorn_config.pygrlc.server:appNote:Sincegunicorndoes not work under Windows, you can usewaitressinstead:waitress-serve--port=8088grlc.server:appIf you want to run grlc at system boot as a service, you can find example upstart scripts atupstart/grlc libraryYou can use grlc as a library directly from your own python script. See theusage exampleto find out more.grlc server configurationRegardless of how you are running your grlc server, you will need to configure it using theconfig.inifile. Have a look at theexample config fileto see how it this file is structured.The configuration file contains the following variables:github_access_tokenaccess tokento communicate with Github API.gitlab_access_tokenaccess tokento communicate with GitLab API.local_sparql_dirlocal storage directory wherelocal queriesare located.server_namename of the server (e.g. grlc.io)sparql_endpointdefault SPARQL endpointuserandpasswordSPARQL endpoint default authentication (if required, specify'none'if not required)debugenable debug level logging.gitlab_urlto specify the base url of your GitLab instance.Git access tokenIn order for grlc to communicate with GitHub and/or GitLab, you'll need to tell grlc what your access token is:Get aGitHub personal access tokenorGitLab personal access token.You'll get an access token string, copy it and save it somewhere safe.Edit yourconfig.ini(github_access_tokenandgitlab_access_tokenrespectively) and/ordocker-compose.yml(GRLC_GITHUB_ACCESS_TOKENandGRLC_GITLAB_ACCESS_TOKENenvironment variables).Contribute!grlc needsyouto continue bringing Semantic Web content to developers, applications and users. No matter if you are just a curious user, a developer, or a researcher; there are many ways in which you can contribute:File in bug reportsRequest new featuresSet up your own environment and start hackingCheck ourcontributingguidelines for these and more, and join us today!If you cannot code, that's no problem! There's still plenty you can contribute:Share your experience at using grlc in Twitter (mention the handle@grlcldapi)If you are good with HTML/CSS,let us knowRelated toolsSPARQL2Gitis a Web interface for editing SPARQL queries and saving them in GitHub as grlc APIs.grlcRis a package for R that brings Linked Data into your R environment easily through grlc.Hay's toolslists grlc as a Wikimedia-related tool :-)This is what grlc users are sayingFlavour your Linked Data with grlc, by Carlos MartinezConverting any SPARQL endpoint to an OpenAPIby Egon WillighagenQuotes from grlc users:A cool project that can convert a random SPARQL endpoint into an OpenAPI endpointIt enables us to quickly integrate any new API requirements in a matter of seconds, without having to worry about configuration or deployment of the systemYou can store your SPARQL queries on GitHub and then you can run your queries on your favourite programming language (Python, Javascript, etc.) using a Web API (including swagger documentation) just as easily as loading data from a web pageContributors:Albert Meroño,Rinke Hoekstra,Carlos MartínezCopyright:Albert Meroño, Rinke Hoekstra, Carlos MartínezLicense:MIT License (seeLICENSE.txt)Academic publicationsAlbert Meroño-Peñuela, Carlos Martinez-Ortiz. “grlc: the git repository linked data API constructor.“ Journal of Open Source Software, 6(67), 2731 (2021),https://doi.org/10.21105/joss.02731Albert Meroño-Peñuela, Pasquale Lisena, Carlos Martínez-Ortiz. “Web Data APIs for Knowledge Graphs: Easing Access to Semantic Data for Application Developers”. Synthesis Lectures on Data, Semantics, and Knowledge, 12(1), pp.1-118 (2021) (Morgan & Claypool)https://doi.org/10.2200/S01114ED1V01Y202107DSK021Albert Meroño-Peñuela, Rinke Hoekstra. “grlc Makes GitHub Taste Like Linked Data APIs”. The Semantic Web – ESWC 2016 Satellite Events, Heraklion, Crete, Greece, May 29 – June 2, 2016, Revised Selected Papers. LNCS 9989, pp. 342-353 (2016). (PDF)Albert Meroño-Peñuela, Rinke Hoekstra. “SPARQL2Git: Transparent SPARQL and Linked Data API Curation via Git”. In: Proceedings of the 14th Extended Semantic Web Conference (ESWC 2017), Poster and Demo Track. Portoroz, Slovenia, May 28th – June 1st, 2017 (2017). (PDF)Albert Meroño-Peñuela, Rinke Hoekstra. “Automatic Query-centric API for Routine Access to Linked Data”. In: The Semantic Web – ISWC 2017, 16th International Semantic Web Conference. Lecture Notes in Computer Science, vol 10587, pp. 334-339 (2017). (PDF)Pasquale Lisena, Albert Meroño-Peñuela, Tobias Kuhn, Raphaël Troncy. “Easy Web API Development with SPARQL Transformer”. In: The Semantic Web – ISWC 2019, 18th International Semantic Web Conference. Lecture Notes in Computer Science, vol 11779, pp. 454-470 (2019). (PDF)