Under the hood of CREATE on AGE cypher
Tags: database, postgres, apacheage, graphdb
Published: 2023-08-13 21:04:45

## Hola

Greetings, **continuing** the series of **under the hood** of **Apache AGE**, last article was **introductory** to **create_graph** which is the first **entry** of any **graph** operation we are going to play with, so what is next is having nodes and something to work with **inside that graph** so we need to understand what is under the hood of the CREATE 
Before going into that I would like to **recommend** the following articles which are **good** to **understand** 
- **How AGE is executed on top of PostgreSQL**:
https://dev.to/rrrokhtar/how-age-is-executed-on-top-of-postgresql-4mpf
- **What's behind scenes of PostgreSQL from Apache AGE**?:  https://dev.to/rrrokhtar/whats-behind-scenes-of-apache-age-23f2

## Overview

Apache AGE is a layer that is being extended on top of PostgreSQL through the nature of the PostgreSQL's enabled extensions and one of them which are being executed after the parsing stage of PostgreSQL we have a Parse Tree from parser and the Query tree from the Analyzer and Re-writer stage so Apache AGE is being executed at the Query tree level, it takes Query Tree and returns tuned Query Tree, 

## What is that tuning exactly?

From the side of AGE it starts parsing the query and looking for every function call at the query tree once it finds a function call, it checks if it is a cypher or not, if it is a cypher it starts converting it into SUB-QUERY 

Our mission today is going through CREATE statement and getting know what is the equivalent SUB-QUERY it is being converted to during the conversion stage of AGE.

## Let's get started

Start by looking into transform a cypher_clause function which is responsible for that
```c

Query *transform_cypher_clause(cypher_parsestate *cpstate,
                               cypher_clause *clause)
```

It looks into the cypher clauses after being parsed into cypher nodes and transforms each node base on its type,
the type we are looking into is CREATE, so at some point in the function we have that
```c
    else if (is_ag_node(self, cypher_create))
    {
        result = transform_cypher_create(cpstate, clause);
    }
```

That's our target is learning what is happening inside transform_cypher_create

Diving into that 
```c

static Query *transform_cypher_create(cypher_parsestate *cpstate,
                                      cypher_clause *clause)
{
    ParseState *pstate = (ParseState *)cpstate;
    cypher_create *self = (cypher_create *)clause->self;
    cypher_create_target_nodes *target_nodes;
    Const *null_const;
    List *transformed_pattern;
    FuncExpr *func_expr;
    Query *query;
    TargetEntry *tle;

    target_nodes = make_ag_node(cypher_create_target_nodes);
    target_nodes->flags = CYPHER_CLAUSE_FLAG_NONE;
    target_nodes->graph_oid = cpstate->graph_oid;

    query = makeNode(Query);
    query->commandType = CMD_SELECT;
    query->targetList = NIL;

    if (clause->prev)
    {
        handle_prev_clause(cpstate, query, clause->prev, true);

        target_nodes->flags |= CYPHER_CLAUSE_FLAG_PREVIOUS_CLAUSE;
    }

    null_const = makeNullConst(AGTYPEOID, -1, InvalidOid);
    tle = makeTargetEntry((Expr *)null_const, pstate->p_next_resno++,
                          AGE_VARNAME_CREATE_NULL_VALUE, false);
    query->targetList = lappend(query->targetList, tle);

    /*
     * Create the Const Node to hold the pattern. skip the parse node,
     * because we would not be able to control how our pointer to the
     * internal type is copied.
     */
    transformed_pattern = transform_cypher_create_pattern(cpstate, query,
                                                          self->pattern);

    target_nodes->paths = transformed_pattern;
    if (!clause->next)
    {
        target_nodes->flags |= CYPHER_CLAUSE_FLAG_TERMINAL;
    }


    func_expr = make_clause_func_expr(CREATE_CLAUSE_FUNCTION_NAME,
                                      (Node *)target_nodes);

    // Create the target entry
    tle = makeTargetEntry((Expr *)func_expr, pstate->p_next_resno++,
                          AGE_VARNAME_CREATE_CLAUSE, false);
    query->targetList = lappend(query->targetList, tle);

    query->rtable = pstate->p_rtable;
    query->rteperminfos = pstate->p_rteperminfos;
    query->jointree = makeFromExpr(pstate->p_joinlist, NULL);

    return query;
}
```
### Function breakdown
- Extending the nodes of PostgreSQL using `DEFINE_NODE_METHODS` through `DEFINE_NODE_METHODS(cypher_create_target_nodes)`
And `cypher_create` is as the following:
```c
typedef struct cypher_create_target_nodes
{
    ExtensibleNode extensible;
    List *paths;
    uint32 flags;
    uint32 graph_oid;
} cypher_create_target_nodes;
```
So that we can say that we are having a node of type cypher_create is attached to the PostgreSQL nodes and We can have a Node from that type

- Assigning the target nodes' graphoid to the ParseState's graphoid and flags to NONE|NULLPTR

- Make a Query node of type CMD SELECT, used in handling previous clauses 
```c
    query = makeNode(Query);
    query->commandType = CMD_SELECT;
    query->targetList = NIL;

    if (clause->prev)
    {
        handle_prev_clause(cpstate, query, clause->prev, true);

        target_nodes->flags |= CYPHER_CLAUSE_FLAG_PREVIOUS_CLAUSE;
    }
```

- Create the Const Node to hold the pattern. skip the parse node, because we would not be able to control how our pointer to the internal type is copied. [From code comments]
```
    transformed_pattern = transform_cypher_create_pattern(cpstate, query, self->pattern);

target_nodes->paths = transformed_pattern;
    if (!clause->next)
    {
        target_nodes->flags |= CYPHER_CLAUSE_FLAG_TERMINAL;
    }
```

- Creates the function expression that represents the clause. Adds the extensible node that represents the metadata that the clause needs to handle the clause in the execution phase. [From code comments], That's mean that we convert that to intermediate function expression that represents it in the final query, i.e. the final query is split up into subquries and those subquries are function calls
```c
  func_expr = make_clause_func_expr(CREATE_CLAUSE_FUNCTION_NAME,
                                      (Node *)target_nodes);

    // Create the target entry
    tle = makeTargetEntry((Expr *)func_expr, pstate->p_next_resno++,
                          AGE_VARNAME_CREATE_CLAUSE, false);
    query->targetList = lappend(query->targetList, tle);
```

- Finish up the query and return it
```c
    query->rtable = pstate->p_rtable;
    query->rteperminfos = pstate->p_rteperminfos;
    query->jointree = makeFromExpr(pstate->p_joinlist, NULL);

    return query;
```

More explanation will be provided on part 2 about what is going on the backend in terms of the database 

All functions are there:
- [src/backend/parser/cypher_clause.c](https://github.com/apache/age/blob/master/src/backend/parser/cypher_clause.c)

- [src/backend/executor/cypher_create.c](https://github.com/apache/age/blob/master/src/backend/executor/cypher_create.c)


## References and resources
- https://github.com/apache/age
- https://age.apache.org/