Project: GraphScope (App Definition and Development / Database)
File: grin.md

GRIN is a proposed standard graph retrieval interface in GraphScope. Its goal is to provide a common way for graph computing engines to retrieve graph data stored in different storage engines within GraphScope, and to simplify the integration of these engines with each other. GRIN is defined in C, making it portable to systems written in different programming languages, such as C++, Rust, and Java. It provides a set of common operations and data structure handlers that can be used to access graph data, regardless of the underlying storage engine. Traversal: navigating the graph structure to explore relationships between vertices Retrieval: retrieving the data and properties of vertices and edges Filter: filtering data structures based on partitioning or property conditions The latest version of headers can be found in . A Vineyard implementation for both computing and storage can be found in . The motivations behind GRIN are driven by the need for a common standard for accessing graph data for the computing engines of GraphScope. There are several factors that have contributed to this need: Complexity of integrating multiple graph computing engines and storage engines: GraphScope consists of numerous graph computing engines and storage engines, each with its own data model, query language, and API. This can make it challenging to integrate these engines with one another, as each engine requires a separate integration effort. Need for interoperability and collaboration: To fully realize GraphScope's potential, greater interoperability and collaboration between different components are necessary. A common standard for accessing graph data could help accelerate GraphScope's development and make it more accessible to a broader range of users. Growing demand for large-scale graph processing: As the volume of data being generated increases, there is a growing demand for graph processing systems that can efficiently analyze large-scale graph data. A common standard for accessing graph data could help accelerate the development of new features for engines in GraphScope. By defining the GRIN interfaces in C, it can be easily integrated with various programming languages, including Java, Python, and Rust, using foreign function interfaces (FFIs) that enable these languages to call C functions. Moreover, GRIN becomes a language-agnostic standard that can be utilized across different programming environments without requiring each environment to have its own implementation of the standard. This greatly simplifies the development and maintenance of graph computing systems relying on GRIN, as it allows developers to use their preferred programming languages and tools while still being able to access and process graph data using a common standard. As previously mentioned, GRIN offers computing engines a consistent method for accessing graphs stored in various storage engines through a set of common operations for graph traversal, data retrieval, and conditional filtering. These operations are defined as C functions, and their return values and parameters are typically handlers for graph concepts, such as vertices and edges. Here is an example showing how to handle a graph query using GRIN APIs. The data types with the prefix `GRIN` are handlers, while the functions with the prefix `grin` are the APIs in GRIN. Moreover, macros that start with `GRIN_` are provided by GRIN to reflect storage features. Each storage engine that implements GRIN APIs can set up these macros based on their own features, such as graph partition strategies and list retrieval styles. The storage provider for this example is Vineyard. The example demonstrates how to synchronize property values of vertices associated with a specific edge

The input parameters are `partitionedgraph`, the local `partition`, `edgetypename` (e.g., likes), and `vertexproperty_name` (e.g., features). The task is to find all the destination vertices of "boundary edges" with a type named "likes", and the vertices must have a property named "features". Here, a boundary edge is an edge whose source vertex is a master vertex, and the destination is a mirror vertex, given the context of the "edge-cut" partition strategy that the underlying storage uses. For each of these vertices, we send the value of the "features" property to its master partition. ```CPP void syncproperty(GRINPARTITIONEDGRAPH partitionedgraph, GRINPARTITION partition, const char* edgetypename, const char* vertexproperty_name) { GRINGRAPH g = gringetlocalgraphfrompartition(partitioned_graph, partition); // get local graph of partition GRINEDGETYPE etype = gringetedgetypebyname(g, edgetype_name); // get edge type from name GRINVERTEXTYPELIST srcvtypes = gringetsrctypesfromedgetype(g, etype); // get related source vertex type list GRINVERTEXTYPELIST dstvtypes = gringetdsttypesfromedgetype(g, etype); // get related destination vertex type list sizet srcvtypesnum = gringetvertextypelistsize(g, src_vtypes); sizet dstvtypesnum = gringetvertextypelistsize(g, dst_vtypes); assert(srcvtypesnum == dstvtypesnum); // the src & dst vertex type lists must be aligned for (sizet i = 0; i < srcvtypes_num; ++i) { // iterate all pairs of src & dst vertex type GRINVERTEXTYPE srcvtype = gringetvertextypefromlist(g, src_vtypes, i); // get src type GRINVERTEXTYPE dstvtype = gringetvertextypefromlist(g, dst_vtypes, i); // get dst type GRINVERTEXPROPERTY dstvp = gringetvertexpropertybyname(g, dstvtype, vertexproperty_name); // get the property called "features" under dst type if (dstvp == GRINNULLVERTEXPROPERTY) continue; // select out the pairs whose dst type does NOT have such a property called "features" GRINVERTEXPROPERTYTABLE dstvpt = gringetvertexpropertytablebytype(g, dst_vtype); // prepare property table of dst vertex type for later use GRINDATATYPE dstvpdt = gringetvertexpropertydatatype(g, dst_vp); // prepare property type for later use GRINVERTEXLIST srcvl = gringetvertexlist(g); // get the vertex list GRINVERTEXLIST srcvl = grinselecttypeforvertexlist(g, srcvtype, src_vl); // select the vertex of source type GRINVERTEXLIST srcvl = grinselectmasterforvertexlist(g, srcvl); // select master vertices under source type sizet srcvlnum = gringetvertexlistsize(g, srcvl); for (sizet j = 0; j < srcvl_num; ++j) { // iterate the src vertex GRINVERTEX v = gringetvertexfromlist(g, srcvl, j); GRINADJACENTLIST adjlist = gringetadjacentlist(g, GRINDIRECTION::OUT, v); // get the outgoing adjacent list of v GRINADJACENTLIST adjlist = grinselectedgetypeforadjacentlist(g, etype, adj_list); // select edges under etype GRINADJACENTLIST adjlsit = gringetadjacentlist(g, GRIN_DIRECTION::OUT, v); // get the outgoing adjacent list of v sizet alsz = gringetadjacentlistsize(g, adj_list); for (sizet k = 0; k < alsz; ++k) { GRINEDGE edge = gringetedgefromadjacentlist(g, adj_list, k); GRINEDGETYPE edgetype = gringetedgetype(g, edge); if (!grinequaledgetype(g, edgetype, etype)) continue; GRINVERTEX u = gringetneighborfromadjacentlist(g, adj_list, k); // get the dst vertex u const void* value = gringetvaluefromvertexpropertytable(g, dstvpt, u, dstvp); // get the property value of "features" of u GRINVERTEXREF uref = gringetvertexreffor_vertex(g, u); // get the reference of u that can be recognized by other partitions GRINPARTITION umasterpartition = gringetmasterpartitionfromvertex_ref(g, uref); // get the master partition for u sendvalue(umasterpartition, uref, dstvpdt, value); // the value must be casted to the correct type based on dstvp_dt before sending } } } } void run(vineyard::Client& client, const grape::CommSpec& comm_spec, vineyard::ObjectID fragmentgroupid) { LOG(INFO) << "Loaded graph to vineyard: " << fragmentgroupid; auto pg = getpartitionedgraphbyobjectid(client, fragmentgroup_id); auto localpartitions = gringetlocalpartition_list(pg); sizet pnum = gringetpartitionlistsize(localpartitions); assert(pnum > 0); // we only sync the first partition as example auto partition = gringetpartitionfromlist(local_partitions, 0); sync_property(pg, partition, "likes", "features"); } ``` This section explains the key concepts in GRIN to help users understand GRIN APIs more easily. GRIN provides a set of predefined C macros for storage engines to describe their

When a storage engine implements the GRIN APIs, it should first set up the macros to present its features such as partition strategies and enabled indices. The benefit is two-fold: On the computing engine side, developers can implement their methods using alternative logic and GRIN APIs when some features are claimed or disclaimed by different storage engines. This makes the code switch based on storage feature claims happens in the compiling stage rather than in runtime. Thus, it extends the versatility of methods implemented by GRINs without sacrificing efficiency. On the storage engine side, the predefined macros can filter out a large number of unnecessary APIs to avoid developers implementing them with boilerplate or inefficient code, since the design of storage engines may differ enormously from each other. What follows is an example of predefined macros regarding the locally completeness of vertex properties. Four macros are provided: `GRINASSUMEALLVERTEXPROPERTYLOCALCOMPLETE` `GRINASSUMEMASTERVERTEXPROPERTYLOCALCOMPLETE` `GRINASSUMEBYTYPEALLVERTEXPROPERTYLOCALCOMPLETE` `GRINASSUMEBYTYPEMASTERVERTEXPROPERTYLOCALCOMPLETE` Some assumptions may dominate others, which means that some assumptions apply in a wider range than others. Therefore, storage providers should be careful when setting these assumptions. Here, 1 dominates the others, which means that 2 to 4 are undefined when 1 is defined. Additionally, 2 dominates 4, and 3 dominates 4. GRIN provides different APIs under different assumptions. Suppose only 3 is defined; it means that vertices of certain types have all the properties locally complete, regardless of whether the vertex is master or mirror. In this case, GRIN provides an API to return these locally complete vertex types. In the case that none of these four macros is defined, GRIN will provide a per-vertex API to tell whether the vertex property is locally complete. GRIN provides two types (i.e., edge-cut and vertex-cut) of predefined partition strategies. Each strategy can be seen as a set of granular predefined macros based on the common understanding of the partition strategy. For storages using hybrid partition strategy, developers can set up the granular macros one after another. Vertex data is locally complete for master vertices. Edge data is locally complete for all edges. Neighbors are locally complete for master vertices. Vertex properties are locally complete for master vertices. Edge properties are locally complete for all edges. Vertex data is locally complete for all vertices. Edge data is locally complete for all edges. Mirror partition list is available for master vertices to broadcast messages. Vertex properties are locally complete for all vertices. Edge properties are locally complete for all edges. GRIN makes the following assumptions for its property graph model. Vertices have types, as do edges. The relationship between edge types and pairs of vertex types is many-to-many. Properties are bound to vertex and edge types, but some properties may have the same name. Labels can be assigned to vertices and edges (NOT their types) primarily for query filtering, and labels have no properties. Get GRIN APIs from Implement a wrapper class (normally `grinfragment` or `gringraph`) using GRIN APIs. When you find some function is hard or inefficient to implement, discuss with GRIN designers. Write or modify apps using the wrapper class (e.g., `grin_fragment`). Find a storage implementation to set up an end-to-end test. Make a grin folder in your system, and navigate into the grin folder. Add GRIN as a submodule and copy the `predefine.template` out as the `predefine.h` ```console $ git submodule add https://github.com/GraphScope/GRIN.git include $ cp include/predefine.template predefine.h ``` Modify the StorageSpecific part in the `predefine.h` based on the features of the

Implement the headers as much as possible in another folder (e.g., src) under grin. If you find the time complexity of some function is NOT sub-linear to the graph size, discuss with GRIN designers. Write a storage-specific method to get a graph handler from your storage. Run a graph traversal test. GRIN provides a series of handlers for graph concepts, such as vertex, edge and graph itself. Since almost everything in GRIN are handlers except of only a few string names, the type for a graph concept and its handler is always mixed-used in GRIN. For example, GRIN uses the type Vertex to represent the type of a vertex handler, instead of using VertexHandler for clean code. For any directed edge `(u, v)` where the direction goes from `u` to `v`, `u` is the source vertex, whereas `v` is the destination vertex. GRIN provides two alternative approaches, namely array-style retrieval and iterator, for list retrieval of `GRINVERTEXLIST`, `GRINEDGELIST`, and `GRINADJACENTLIST`. For other schema-level lists like `GRINVERTEXTYPELIST` or `GRINPARTITIONLIST`, GRIN generally assume the array-style list retrieval, and does **NOT** provide a `GRINENABLE_` macro for these lists. The array-style retrieval of a list handler is available to the user only if the storage can provide the size of the list, and an element retrieval API by position (i.e., index of array). Otherwise, the storage should provide a list iterator, see next section. Usually the array-style retrieval is controlled by the macros ended with `LISTARRAY` A vertex list array-style retrieval example ```CPP / grin/topology/vertexlist.h / GRINVERTEXLIST gringetvertexlist(GRINGRAPH g); // get the vertexlist of a graph sizet gringetvertexlistsize(GRINGRAPH g, GRINVERTEXLIST vl); // implement the API to return the size of vertexlist GRINVERTEX gringetvertexfromlist(GRINGRAPH g, GRINVERTEXLIST vl, size_t idx); // implement the API to return the element of vertexlist by position / run.cc / { auto vertexlist = gringetvertex_list(g); // with a graph (handler) g auto sz = gringetvertexlistsize(g, vertexlist); for (auto i = 0; i < sz; ++i) { auto v = gringetvertexfromlist(g, vertexlist, i); } } ``` A list iterator handler is provided to caller if the list size is unknown or for sequential scan efficiency. Usually the iterators are enabled by macros ended with `LISTITERATOR` A vertex list iterator example ```CPP / grin/topology/vertexlist.h / GRINVERTEXLISTITERATOR gringetvertexlistbegin(GRINGRAPH g, GRINVERTEXLIST vl); // get the begin iterator of the vertexlist GRINVERTEXLISTITERATOR gringetnextvertexlistiter(GRINGRAPH g, GRINVERTEXLISTITERATOR vli); // get next iterator bool grinisvertexlistend(GRINGRAPH g, GRINVERTEXLISTITERATOR vli); // check if reaches the end GRINVERTEX gringetvertexfromiter(GRINGRAPH g, GRINVERTEXLIST_ITERATOR vli); // get the vertex from the iterator / run.cc / { auto iter = gringetvertexlistbegin(g, vl); // with a graph (handler) g and vertexlist vl while (!grinisvertexlistend(g, iter)) { auto v = gringetvertexfromiter(g, iter); iter = gringetnextvertexlist_iter(g, iter); } } ``` Properties are bound to vertex and edge types. It means even some properties may have the same name, as long as they are bound to different vertex or edge types, GRIN will provide distinct handlers for these properties. This is because, although properties with the same name usually provide the same semantic in the graph, they may have different data types in the underlying storage for efficiency concerns (e.g., short date and long date). To avoid the incompatibility with storage engines, we made the design choice to bind properties under vertex and edge types. Meanwhile, GRIN provides an API to get all the property handlers with the (same) given property name. ```CPP / grin/property/type.h / GRINVERTEXTYPE gringetvertextypebyname(GRINGRAPH g, const char* name); /

/ GRINVERTEXPROPERTY gringetvertexpropertybyname(GRINGRAPH g, GRINVERTEXTYPE vtype, const char* name); GRINVERTEXPROPERTYLIST gringetvertexpropertiesbyname(GRIN_GRAPH g, const char* name); / run.cc / { auto vtype = gringetvertextypeby_name(g, "Person"); // get the vertex type of Person auto vprop = gringetvertexpropertyby_name(g, vtype, "Name"); // get the Name property bound to Person auto vpl = gringetvertexpropertiesby_name(g, "Name"); // get all the properties called Name under all the vertex types (e.g., Person, Company) in g } ``` GRIN does NOT distinguish label on vertices and edges, that means a vertex and an edge may have a same label. However the storage can tell GRIN whether labels are enabled in vertices or edges separatedly with macros of `WITHVERTEXLABEL` and `WITHEDGELABEL` respectively. GRIN provides sorted vertex list assumptions. GRIN also assumes that if a vertex list sorted, then there is complete ordering for the vertices from the list. Sorted vertex list facilitates computations like vertex list join and data structures like vertex array which uses vertex as the index of the array. GRIN introduces the reference concept in partitioned graph. It stands for the reference of an instance that can be recognized in partitions other than the current partition where the instance is accessed. For example, a `GRINVERTEXREF` is a reference of a `GRIN_VERTEX` that can be recognized in other partitions. ```CPP / grin/partition/partition.h / GRINVERTEXREF gringetvertexrefforvertex(GRINGRAPH, GRIN_VERTEX); const char* grinserializevertexref(GRINGRAPH, GRINVERTEXREF); GRINVERTEXREF grindeserializetovertexref(GRIN_GRAPH, const char*); GRINVERTEX gringetvertexfromvertexref(GRINGRAPH, GRINVERTEX_REF); / run.cc in machine 1 / { auto vref = gringetvertexreffor_vertex(g, v); // get v's vertex ref which can be recognized in machine 2 const char* msg = grinserializevertex_ref(g, vref); // serialize into a message // send the message to machine 2... } / run.cc in machine 2 / { // receive the message from machine 1... auto vref = grindeserializetovertexref(g, msg); // deserialize back to vertex ref auto v = gringetvertexfromvertex_ref(g, vref); // cast to vertex if g can recognize the vertex ref } ``` Master & mirror vertices are the concept borrowed from vertexcut partition strategy. When a vertex is recognized in several partitions, GRIN refers one of them as the master vertex while others as mirrors. This is primarily for data aggregation purpose to share a common central node for every one. While in edgecut partition, the concept becomes inner & outer vertices. GRIN uses master & mirror vertices to represent inner & outer vertices respectively to unify these concepts. The concept of local complete is with respect to whether a graph component adhere to a vertex or an edge is locally complete within the partition. Take vertex and properties as example. GRIN considers the vertex is "property local complete" if it can get all the properties of the vertex locally in the partition. There are concepts like "edge property local complete", "vertex neighbor local complete" and so on. GRIN does NOT assume any local complete on master vertices. Since in some extremely cases, master vertices may NOT contain all the data or properties locally. GRIN currently provides vertex-level/edge-level local complete judgement APIs, while the introduction of type-level judgement APIs is open for discussion. Concepts represent the schema of the graph, such as vertex type and properties bound to a certain edge type, are usually numbered naturally from `0` to its `num - 1` in many storage engines. To facilitate further optimizations in the upper computing engines, GRIN provides the natural number ID trait. A storage can provide such a trait if it also uses the natural numbering for graph schema concepts.