Knowledge Graph App in 15min
By: Szymon Klarman
Tags: GraphQL, Mongodb, Prototyping, Knowledge Graph, Json

Prototyping a simple knowledge graph application with JSONs, MongoDB and automatically generated GraphQL API

This post is a result of joint work with Anna Konieczna and Artur Haczek.

TL;DR

Simple knowledge graph applications can be easily built using JSON data managed entirely via a GraphQL layer. We describe a quick recipe for prototyping one such demo, Noble Connections, which involves:

JSON documents stored in a free-tier MongoDB Atlas instance Staple API — a GraphQL-based semantic abstraction middleware for accessing and virtualizing that data as a knowledge graph test deployment on Repl.it IDE

The code of this demo and deployment instructions are available at:

Introduction: walking on knowledge graphs, one step at a time

So you want to build a simple knowledge graph-driven application? Great! You know what it’s going to be about, you’ve drafted the core data schema that should underpin your knowledge graph, and you already have the initial data set to populate that structure — nothing very sophisticated to start with, say some people, places and some basic relationships between them. Something like this Noble Connections demo that allows one to explore different family links among the European nobility. You think it shouldn’t be too difficult to put it all together and expose via some basic UI, but suddenly you start making your way through yet another tutorial and get convinced that your next major challenge is the choice of an appropriate tech stack. What should it be? Probably some “native” graph database implementing the labelled property graph model — but then which one? The range of possibilities is broad and quickly growing. Or maybe you should go down the RDF route, using W3C standards to represent your data and manage it in an RDF triple store? Or perhaps a multi-model database might be that safest “catch-all” alternative..?

The knowledge graph space abounds in detailed comparisons and recommendations of most appropriate advanced tooling for building core graph-driven applications. In our case, however, what they ultimately seem to promise is a relatively steep learning curve in exchange for gains that might not be so clear at the point of prototyping that first knowledge graph app.

At the end it all comes down to the use-case and, arguably, simple use-cases can be effectively addressed with simple means.

It’s often stated that a knowledge graph is merely an abstract view of the application data, not directly tied to a concrete database, data architecture or implementation. Knowledge graphs are there in relational databases, document stores, data warehouses, on the Web itself, and across REST endpoints.

It’s really about recognizing that different types of entities in your domain of interest are inherently connected with each other via diverse relationships.

It does not imply that one needs to be running complex network analytics over that structure, or logical inference over its type hierarchy, or parallel path-searching algorithms. These are indeed all valid tasks, and ones in which the advanced tooling, mentioned above, fully delivers on its differentiating value proposition.

But sometimes it’s really just about taking a walk on a knowledge graph one step at a time.

In this blog post, we describe a quick recipe for prototyping one such simple knowledge graph app, Noble Connections, which makes use of:

plain JSON documents, stored in a free-tier MongoDB Atlas instance, to represent the application data;

GraphQL-based Staple API service for automatically structuring and virtualizing that data as a graph.

The app, with a vis.js-powered front-end graph visualization, is finally deployed live using the Repl.it IDE environment.

The architecture and deployment of the Noble Connections app.

I. Encoding graph data in JSONs

JSON is a simple yet extremely flexible data serialization format, which due to its immense popularity probably doesn’t need to be introduced to anyone. Formally speaking, it’s aimed for representing tree-shaped structures, i.e.: objects, their attributes and outgoing relations to other objects with their attributes, and so on. With a bit of systematic convention, however, it’s also perfectly suited for capturing graph-shaped models. Essentially, it’s all about consistent use of unique identifiers on all JSON objects and noting which properties are used to refer to such identifiers. For instance, the following two JSON objects describe the person Queen Elizabeth II and the country United Kingdom. The identifiers, which in this case are all URIs, are always associated with the key _id , (hence: http://dbpedia.org/resource/Elizabeth_II and http://dbpedia.org/resource/United_Kingdom , respectively) and further referenced from properties: birthCountry , deathCountry , parent , child and spouse .

The entity “Queen Elizabeth II” represented in JSON.

The entity “United Kingdom” represented in JSON.

Those more familiar with linked data standards, might immediately recognize that this is basically the modeling principle behind the JSON-LD format. The only missing component here is a so-called JSON-LD context, which formally enforces the very convention described above, as in:

{

"_id": "@id",

"_type": "@type",

"@vocab": "http://example.com/",

"birthCountry": {

"@type": "@id"

},

"deathCountry": {

"@type": "@id"

},

"parent": {

"@type": "@id"

},

"child": {

"@type": "@id"

},

"spouse": {

"@type": "@id"

}

}

By adopting this representation approach, individual JSON objects can naturally become pieces of arbitrarily large and complex graph puzzles, which is the whole point. The two JSONs in the example above, state together that a person called “Elizabeth II” was born in country named “United Kingdom”

Graph structures can be easily encoded in JSON format by consistent use of unique identifiers.

Another great thing about JSONs is the existing tool ecosystem around it — for instance, MongoDB Atlas — a cloud-based version of the popular MongoDB JSON store, which offers a free-tier storage of up to 512MB data with zero maintenance costs. In order to deploy the data for our Noble Connections app (originally extracted from DBpedia), we create one such free instance with a dedicated document collection, upload the data using python client, and configure a guest user with read-only access to enable application queries.

A web dashboard of MongoDB Atlas instance with Noble Connections data.

II. GraphQL — a graph abstraction middleware

A lot has been said about GraphQL, its diverse data management capabilities and the transformative impact it might have on how data-driven APIs can be built. Interestingly, GraphQL is also being increasingly considered as a convenient data abstraction and access layer over knowledge graphs, and that’s mostly because of its strictly typed schema and query languages, which help navigate semantic knowledge graphs, coupled with the flexibility of fetching data from different sources. Simply put, GraphQL is very well-suited for managing connected data from differently structured sources. In particular it’s a very natural fit for exposing graph data serialized in the form of JSON objects, as discussed above.

Staple API virtualizes data from different back-ends as a (linked data) knowledge graph and exposes it via an automatically generated GraphQL service.

To simplify the implementation of a suitable GraphQL endpoint over MongoDB we use Staple API — a lightweight GraphQL-based API for managing knowledge graphs on top of different data storage back-ends, including MongoDB. In a nutshell, Staple API takes a specified RDF ontology (in Noble Connections example this one is sufficient), a configuration of the back-end source, and automatically spins-up a ready GraphQL service, with a a corresponding schema and resolvers. The automation of this process is what in practice saves enormous amount of development time, as typically GraphQL resolvers are tied to data structures based on highly repetitive patterns, which have to be all coded against manually.

Spinning up a Staple API server from a provided ontology and a MongoDB data source configuration.

The GraphQL schema enforces a rigid structure over the data it manages, so we have to be sure that the data we inserted into MongoDB indeed matches the schema generated from the ontology. Alternatively, we could have used the corresponding mutations created by Staple API in order to insert data to MongoDB and let GraphQL validate its proper structuring.

GraphQL schema for Noble Connections app.

Once initiated, Staple API exposes a set of queries that enable connecting individual data points into larger fragments of the entire knowledge graph. Admittedly, the expressiveness of GraphQL query language allows predominantly to “walk on the graph”, i.e., to fetch objects and their selected neighbors, and the neighbors of those, etc. But if walking is what we are after, then GraphQL just gets the job done. For instance, the following prototypical query fetches a person with the identifier http://dbpedia.org/resource/Elizabeth_II along with her country of birth, and her children, with their parents and countries of birth, respectively.

Querying the Nobel Connection data with GraphQL.

Queries of exactly this type is all that is used to feed the UI of the Noble Connections demo, enabling controlled exploration of the entire graph.

III. Deployment on Repl.it

One of the quickest ways of test-deploying a simple web application online is by using Repl.it IDE environment.

Go to https://repl.it. Press + new repl to create a new Repl environment and choose Import From GitHub. Paste the URL of the demo repository (https://github.com/epistemik-co/staple-api-kg-demo) and import it. Press the Run button and wait for the app to start at the automatically generated URL of a form: https://***--five-nine.repl.co.

Importing a GitHub project into Repl.it.

The application (run from demo.js file) exposes two graphical interfaces:

the front-end UI at https://***--five-nine.repl.co

the Apollo playground of the underlying Staple API at https://***--five-nine.repl.co/graphql,

Running Noble Connections in Repl.it.

Done. The knowledge graph app is running and ready to play with, as the connections among the European nobility are about to be uncovered! :)

The core graph visualization on the front-end UI is created using a popular vis.js library, which is sufficient for rendering small- to mid-sized networks and supports easily customizable presentation, behavior and interaction patterns. The Apollo playground, which wouldn’t be normally exposed as part of the app itself, is included to enable further exploration of the data set on the GraphQL query-level.

Summary

There’s obviously no one-size-fits-all recipe for building and managing knowledge graphs in the real world. Demanding use-cases, with complex querying and analytics requirements, will often require advanced tooling such as graph/triple stores, semantic data description models, inference engines, or graph computing frameworks. There are other, however, in which simpler solutions, based on the use of plain JSONs, MongoDB and GraphQL will be perfectly valid and will support the intended usage scenario just fine, while greatly reducing the development effort and the cost of deploying and maintaining the application.