What is going in AGE PG16
Tags: database, postgres, apacheage, pg16
Published: 2023-07-08 20:21:01

### Hi PG16 AGE TEAM, 

In our last meeting we have discussing the problems we got and one of them was that issue happens when there is a join when we use OPTIONAL MATCH that was existing in the cypher_match.sql regression tests, after digging into that issue **Panagiotis** has stated that in the meeting: 
You will figure out the first failed script is:  
`not ok 8     - cypher_match                              792 ms`  

so that we et started with

Here is the function we are trying to solve the issue within:  
`src/backend/parser/cypher_clause.c`   

```C

/*
 * transform_cypher_optional_match_clause
 *      Transform the previous clauses and OPTIONAL MATCH clauses to be LATERAL LEFT JOIN
 *   transform_cypher_optional_match_clause   to construct a result value.
 */
static RangeTblEntry *transform_cypher_optional_match_clause(cypher_parsestate *cpstate,
                                                             cypher_clause *clause)
{
    cypher_clause *prevclause;
    RangeTblEntry *l_rte, *r_rte;
    ParseNamespaceItem *l_nsitem, *r_nsitem;
    ParseState *pstate = (ParseState *) cpstate;
    JoinExpr* j = makeNode(JoinExpr);
    List *res_colnames = NIL, *res_colvars = NIL;
    Alias *l_alias, *r_alias;
    ParseNamespaceItem *jnsitem;
    int i = 0;

    j->jointype = JOIN_LEFT;

    l_alias = makeAlias(PREV_CYPHER_CLAUSE_ALIAS, NIL);
    r_alias = makeAlias(CYPHER_OPT_RIGHT_ALIAS, NIL);

    j->larg = transform_clause_for_join(cpstate, clause->prev, &l_rte,
                                        &l_nsitem, l_alias);
    pstate->p_namespace = lappend(pstate->p_namespace, l_nsitem);

    /*
     * Remove the previous clause so when the transform_clause_for_join function
     * transforms the OPTIONAL MATCH, the previous clause will not be transformed
     * again.
     */
    prevclause = clause->prev;
    clause->prev = NULL;

    //set the lateral flag to true
    pstate->p_lateral_active = true;

    j->rarg = transform_clause_for_join(cpstate, clause, &r_rte,
                                        &r_nsitem, r_alias);

    // we are done transform the lateral left join
    pstate->p_lateral_active = false;

    /*
     * We are done with the previous clause in the transform phase, but
     * reattach the previous clause for semantics.
     */
    clause->prev = prevclause;

    pstate->p_namespace = NIL;

    // get the colnames and colvars from the rtes
    get_res_cols(pstate, l_nsitem, r_nsitem, &res_colnames, &res_colvars);

    jnsitem = addRangeTableEntryForJoin(pstate,
                                        res_colnames,
                                        NULL,
                                        j->jointype,
                                        0,
                                        res_colvars,
                                        NIL,
                                        NIL,
                                        j->alias,
                                        NULL,
                                        false);

    j->rtindex = jnsitem->p_rtindex;

    for (i = list_length(pstate->p_joinexprs) + 1; i < j->rtindex; i++)
    {
        pstate->p_joinexprs = lappend(pstate->p_joinexprs, NULL);
    }
    pstate->p_joinexprs = lappend(pstate->p_joinexprs, j);
    Assert(list_length(pstate->p_joinexprs) == j->rtindex);

    pstate->p_joinlist = lappend(pstate->p_joinlist, j);

    /* add jrte to column namespace only */
    addNSItemToQuery(pstate, jnsitem, false, false, true);

    return jnsitem->p_rte;
}
```

The problem is a **dependency problem** we need that index which is being evaluated in the end we need that at first to be correctly set

```C
    jnsitem = addRangeTableEntryForJoin(pstate,
                                        res_colnames,
                                        NULL,
                                        j->jointype,
                                        0,
                                        res_colvars,
                                        NIL,
                                        NIL,
                                        j->alias,
                                        NULL,
                                        false);

    j->rtindex = jnsitem->p_rtindex;

```

The index is expected to be evaluated before that line:
```C
    j->rarg = transform_clause_for_join(cpstate, clause, &r_rte,
                                        &r_nsitem, r_alias);

```
because inside that function (transform_clause_for_join) the **rtindex** is being **set** the line before return statement 

```C
static Node *transform_clause_for_join(cypher_parsestate *cpstate,
                                       cypher_clause *clause,
                                       RangeTblEntry **rte,
                                       ParseNamespaceItem **nsitem,
                                       Alias* alias)
{
    RangeTblRef *rtr;

    *nsitem = transform_cypher_clause_as_subquery(cpstate,
                                               transform_cypher_clause,
                                               clause, alias, false);
    *rte = (*nsitem)->p_rte;

    rtr = makeNode(RangeTblRef);
    rtr->rtindex = (*nsitem)->p_rtindex;

    return (Node *) rtr;
}
```

"Overall we are trying to fix that first and make it work don't take into consideration any hassle may be done it can be refactored after we get it work" **PL said**

References: 
- https://github.com/postgres/postgres/tree/REL_16_STABLE
- https://github.com/panosfol/age/tree/PG16
- https://github.com/apache/age/
- https://github.com/muhammadshoaib/age/