Graph to graph with Elixir
By: Tony Hammond
Tags: Elixir, Cypher, Linked Data, Graph Database, Sparql

3. Lay out the project

So let’s get a quick overview of the project.

a. Project layout

Here’s the project module layout for the TestGraph and NeoSemantics modules.

% tree lib

lib

├── application.ex ├── neo_semantics

│ ├── extension.ex

│ ├── inference.ex

│ └── mapping.ex

├── neo_semantics.ex ├── test_graph

│ ├── graph.ex

│ ├── lpg

│ │ └── cypher

│ │ └── client.ex

│ ├── lpg.ex

│ ├── query.ex

│ ├── rdf

│ │ └── sparql

│ │ └── client.ex

│ ├── rdf.ex

│ └── utils.ex

└── test_graph.ex 6 directories, 13 files

b. Data layout

Here’s the data layout for graphs and queries.

Note that graph models and queries are both expressed in Cypher ( .cypher ) for LPG databases, while for RDF databases graph models are expressed in an RDF serialization (e.g. Turtle, . ttl ) and queries are expressed in the SPARQL Query language ( .rq ) or the SPARQL Update language ( .ru ). (The GraphGist format ( .adoc ) is used to document its embedded Cypher queries.)

:test_graph data layout for LPG and RDF graphs and queries.

We’ll define %TestGraph.Graph{} and %TestGraph.Query{} structs to help with access to the data files.

c. Graph struct

Let’s define a %TestGraph.Graph{} struct to keep some related data fields together. One of the fields :type will track whether this is a property graph ( :lpg ) or a semantic graph ( :rdf ). The other fields are straightforward data access fields:

:data – graph data

– graph data :file – name of graph file (within the graphs dir)

– name of graph file (within the graphs dir) :path – absolute path of graph file

– absolute path of graph file :uri – absolute URI of graph file (a file: scheme URI)

We can define the struct using the sigil syntax ~w for bare words, and the trailing a character marks these as atoms:

defmodule TestGraph.Graph do

@moduledoc """

Module providing a struct for graphs.

""" defstruct ~w[data file path type uri]a end

We can test that this does the right thing by instantiating a vanilla struct:

iex> %TestGraph.Graph{}

%TestGraph.Graph{data: nil, file: nil, path: nil, type: nil, uri: nil}

And we can now define a %TestGraph.Graph{} constructor function TestGraph.Graph.new/3 which takes graph_data , graph_file , and graph_type as arguments for the :data , :file and :type fields, respectively. The :path and :uri fields are computed from the :file field.

def new(graph_data, graph_file, graph_type) do graphs_dir =

case graph_type do

:lpg -> @lpg_dir <> "/graphs/"

:rdf -> @rdf_dir <> "/graphs/"

_ -> raise "! Unknown graph_type: " <> graph_type

end %__MODULE__{

data: graph_data,

file: graph_file,

path: graphs_dir <> graph_file,

type: graph_type,

uri: "file://" <> graphs_dir <> graph_file,

} end

d. Query struct

We likewise define a %TestGraph.Query{} struct to keep some related data fields together.

defmodule TestGraph.Query do

@moduledoc """

Module providing a struct for queries.

""" defstruct ~w[data file path type uri]a end

And again we can define a %TestGraph.Query{} constructor function TestGraph.Query.new/3 .

def new(query_data, query_file, query_type) do queries_dir =

case query_type do

:lpg -> @lpg_dir <> "/queries/"

:rdf -> @rdf_dir <> "/queries/"

_ -> raise "! Unknown query_type: " <> query_type

end %__MODULE__{

data: query_data,

file: query_file,

path: queries_dir <> query_file,

type: query_type,

uri: "file://" <> queries_dir <> query_file,

} end

e. Data access

Now that we have our graph structs we can define data access functions to read and write the underlying data files.

We define a read/write pair read_graph/2 and write_graph/2 in TestGraph.LPG to access the data files and return a %TestGraph{} struct.

def read_graph(graph_file \\ graph_file()) do

graphs_dir = @graphs_dir

graph_data = File.read!(graphs_dir <> graph_file) TestGraph.Graph.new(graph_data, graph_file, :lpg)

end def write_graph(graph_data, graph_file \\ temp_graph_file()) do

graphs_dir = @graphs_dir

File.write!(graphs_dir <> graph_file, graph_data) TestGraph.Graph.new(graph_data, graph_file, :lpg)

end

To read the saved LPG graph "books.cypher" we can use this call:

iex> graph = TestGraph.LPG.read_graph("books.cypher")

%TestGraph.Graph{

data: "CREATE

(book:Book {

iri: \"urn:isbn:978-1-68050-252-7\",

date: \"2018-03-14\",

format: \"Paper\",

title: \"Adopting Elixir\"

}),

(author1:Author { iri: \"https://twitter.com/bgmarx\" }),

(author2:Author { iri: \"https://twitter.com/josevalim\" }),

(author3:Author { iri: \"https://twitter.com/redrapids\" }),

(publisher:Publisher { iri: \"https://pragprog.com/\" })



CREATE

(book)-[:AUTHORED_BY { role: \"first author\" }]->(author1),

(book)-[:AUTHORED_BY { role: \"second author\" }]->(author2),

(book)-[:AUTHORED_BY { role: \"third author\" }]->(author3),

(book)-[:PUBLISHED_BY]->(publisher)



;

",

file: "default.cypher",

path: ".../test_graph/priv/lpg/graphs/default.cypher",

type: :lpg,

uri: "file:///.../test_graph/priv/lpg/graphs/default.cypher"

} iex> graph.data

"CREATE

(book:Book {

iri: \"urn:isbn:978-1-68050-252-7\",

date: \"2018-03-14\",

format: \"Paper\",

title: \"Adopting Elixir\"

}),

(author1:Author { iri: \"https://twitter.com/bgmarx\" }),

(author2:Author { iri: \"https://twitter.com/josevalim\" }),

(author3:Author { iri: \"https://twitter.com/redrapids\" }),

(publisher:Publisher { iri: \"https://pragprog.com/\" })



CREATE

(book)-[:AUTHORED_BY { role: \"first author\" }]->(author1),

(book)-[:AUTHORED_BY { role: \"second author\" }]->(author2),

(book)-[:AUTHORED_BY { role: \"third author\" }]->(author3),

(book)-[:PUBLISHED_BY]->(publisher)



;

" iex> graph.uri

"file:///.../test_graph/priv/lpg/graphs/default.cypher"

To write a new LPG graph "new.cypher" with the same data:

iex> graph.data |> TestGraph.LPG.write_graph("new.cypher")

%TestGraph.Graph{

data: "CREATE

(book:Book {

iri: \"urn:isbn:978-1-68050-252-7\",

date: \"2018-03-14\",

format: \"Paper\",

title: \"Adopting Elixir\"

}),

(author1:Author { iri: \"https://twitter.com/bgmarx\" }),

(author2:Author { iri: \"https://twitter.com/josevalim\" }),

(author3:Author { iri: \"https://twitter.com/redrapids\" }),

(publisher:Publisher { iri: \"https://pragprog.com/\" })



CREATE

(book)-[:AUTHORED_BY { role: \"first author\" }]->(author1),

(book)-[:AUTHORED_BY { role: \"second author\" }]->(author2),

(book)-[:AUTHORED_BY { role: \"third author\" }]->(author3),

(book)-[:PUBLISHED_BY]->(publisher)



;

",

file: "new.cypher",

path: ".../test_graph/priv/lpg/graphs/new.cypher",

type: :lpg,

uri: "file:///.../test_graph/priv/lpg/graphs/new.cypher"

}

And similarly for LPG queries with a read/write pair read_query/2 and write_query/2 in TestGraph.LPG .

And then we replicate this pattern for RDF graphs and queries in TestGraph.RDF .

And to make these easier to access we define some top-level delegates:

defdelegate read_lpg_graph(arg), to: TestGraph.LPG, as: :read_graph

defdelegate read_lpg_query(arg), to: TestGraph.LPG, as: :read_query

Now we can simply call read_lpg_graph/1 and read_lpg_query/1 instead of TestGraph.LPG.read_graph/1 and TestGraph.LPG.read_query/1 . And likewise for LPG writes and for RDF reads/writes.

And to list the graphs and queries we have some helper functions together with corresponding top-level delegates:

iex> list_lpg_graphs()

["movies.cypher", "books.cypher", "default.cypher"] iex> list_rdf_graphs()

["books.ttl", "urn_isbn_978-1-68050-252-7.ttl", "http___dbpedia.org_resource_London.ttl", "london100.ttl", "london.ttl", "nobelprizes.ttl", "bibo.ttl", "tony.ttl", "temp.ttl", "http___example.org_Elixir.ttl", "elixir.ttl", "default.ttl", "cypher.ttl", "neo4j.ttl", "hello.ttl"] iex> list_lpg_queries()

... iex> list_rdf_queries()

...

f. Data query

Likewise for our query structs we have read/write access functions which this time return a %TestGraph.Query{} struct:

def read_query(query_file \\ query_file()) do

queries_dir = @queries_dir

query_data = File.read!(queries_dir <> query_file) TestGraph.Query.new(query_data, query_file, :lpg)

end def write_query(query_data, query_file \\ temp_query_file()) do

queries_dir = @queries_dir

File.write!(queries_dir <> query_file, query_data) TestGraph.Query.new(query_data, query_file, :lpg)

end

To read the saved LPG query "default.cypher" we can use this call:

iex> query = TestGraph.LPG.read_query("default.cypher")

%TestGraph.Query{

data: "match (n) return n limit 1

",

file: "default.cypher",

path: ".../test_graph/priv/lpg/queries/default.cypher",

type: :lpg,

uri: "file:///.../test_graph/priv/lpg/queries/default.cypher"

}

Query writes are the same as we earleir discussed for graph writes.

Functions to dispatch the queries are defined in the TestGraph.LPG.Cypher.Client module which we’ve aliased to Cypher_Client . There are two forms for making the remote query: rquery/2 and rquery!/2 .

iex> Cypher_Client.rquery! query.data

[

%{

"n" => %Bolt.Sips.Types.Node{

id: 26703,

labels: ["Book"],

properties: %{

"date" => "2018-03-14",

"format" => "Paper",

"iri" => "urn:isbn:978-1-68050-252-7",

"title" => "Adopting Elixir"

}

}

}

]

We can simplify this with the top-level delegates cypher and cypher! :

iex> cypher! query.data

[

%{

"n" => %Bolt.Sips.Types.Node{

id: 26703,

labels: ["Book"],

properties: %{

...

}

}

}

]

And, as before, we have the same parallel functions and delegate declarations for RDF:

iex> TestGraph.RDF.read_query("default.rq").data |> SPARQL_Client.rquery!

#RDF.Graph{name: nil

~I<http://dbpedia.org/resource/London>

~I<http://www.w3.org/2003/01/geo/wgs84_pos#lat>

%RDF.Literal{value: 51.50722122192383, lexical: "5.150722122192383E1", datatype: ~I<http://www.w3.org/2001/XMLSchema#double>}}

Or in delegate form:

iex> read_rdf_query("default.rq").data |> sparql! |> RDF.Turtle.write_string!

"<http://dbpedia.org/resource/London>

<http://www.w3.org/2003/01/geo/wgs84_pos#lat> 5.15072212219238E1 .

"