Getting started with Sylver - Part2/3 : Intuitive JSON AST queries
Tags: linter, parser, query, json
Published: 2022-08-20 11:20:00

In [Part 1](https://dev.to/geoffreycopin/httpsblogsylverdevgetting-started-with-sylver-part1-building-a-json-parser-in-49-lines-of-code-bj5),
we used [Sylver](https://sylver.dev)'s meta language to build a specification for the
JSON format. But an AST, by itself, is not of much use.
In this next tutorial, we'll continue building our JSON configuration validator.
To this end, we'll learn how to use Sylver's query REPL (Read Eval Print Loop) to
identify the parts of our JSON code that do not comply with a set of increasingly complex
rules. In the next and last part, we'll learn how to package queries into a rule set
to share and reuse them easily.

If you have already installed `sylver` and `sylver --version` doesn't output
a version number >= `0.1.3`, please go to <https://sylver.dev> to download a fresh copy of the software.

## Prelude

We will reuse the `json.syl` spec file built in the previous tutorial.
Since the `config.json` file that we used to test our parser contains a valid
configuration, we'll need to create a new JSON document (`invalid_config.json`) containing 
an incorrect configuration:

```json
{
    "variables": [
        {
            "name": "date of birt`",
            "description": "Customer's date of birth",
            "type": "datetime"
        },
        {
            "name": "activity",
            "description": "A short text describing the customer's profession",
            "type": "string"
        },
        {
            "name": "country",
            "description": "Customer's country of residence",
            "type": "string",
            "values": ["us", "fr", "it" ]
        }
    ]
}
```

This file represents the configuration of an imaginary software handling a database of
customers. Each customer profile is described using a configurable set of variables.

We want to validate these variable declarations using the following rules:
1. Variable descriptions must contain at most 35 characters
2. The name of a variable must be a single lowercase word
3. If the `values` field is set, the `type` field should be absent (as it
matches the type of the values)

Let's parse this file and start a REPL to query our AST!

## Basic REPL usage

Loading the AST of `invalid_config.json` is as simple as invoking:
```
sylver query --spec=json.syl --files=invalid_config.json
```

The `files` argument accepts one or several file names or quoted glob patterns.

You can exit the REPL by typing `:quit` at the prompt.

`:print invalid_config.json` and `:print_ast invalid_config.json` can be used to visualize
one of the loaded files, or the corresponding AST.


## Query language basics

Syntax queries are of the form `match <NodeType> <binding>? (when <boolean expression>)?`.
The node binding and the `when [...]` clause are optional.
`NodeType` represents either a node type as it appears when printing the AST or a
placeholder (`_`) that matches every node. The whole part following the `match` keyword
is called a query pattern.

In the REPL, queries must be followed by a `;`

The most straightforward query, returning every node in the AST, is written as follows:
```
match _;
```

A slightly more advanced query to return every `String` node in our document:
```
match String;
```

If we only wish to retrieve the string literals above a certain length, we can
add a `when` clause: 
```
match String str when str.text.length > 35;
```
This query matches only the string literals whose text representation (quotes included)
contains more than 35 characters. In our document, there is only one match on line 10.

The node type can be tested using the `is` keyword. For instance,
to retrieve any node whose direct parent is an `Object`:
```
match _ node when node.parent is Object;
```
Returns the members list of all the objects in our document.

The `is` keyword can also test a node against a full query pattern surrounded by curly braces. So, for example, we can retrieve every node whose parent
is a member with key `name`. 
```
match _ node when node.parent is {
    Member m when m.key.text == '"name"'
};
```
String literals can be single or double quoted.

Now that we know how to write basic queries, let's try to find the nodes that violate our rules.

## Rule 1: variable descriptions should be shorter than 35 characters

Except for the `&&` operator in boolean expressions, this rule only
uses features that appeared in the previous section so you can test
yourself by trying to write it without looking at the following block of code!
```
match String desc when desc.text.length > 37 && desc.parent is {
   Member m when m.key.text == '"description"'
};
``` 
This query should return a single node on line 10 that is indeed longer than the 
specified length.
Note that we check that the text length is above 37 instead of 35 because of the surrounding quotes.

## Rule 2: variable names should be a single lowercase word

```
match String s when !s.text.matches(`"[a-z]+"`) && s.parent is {
   Member m when m.key.text == '"name"'
};
```
Returns a single node corresponding to the invalid `date of birth` name.

Apart from the boolean `!` operator, this rule demonstrates the use of the `matches`
method on text values. Unsurprisingly, it returns `true` when the text matches the 
regex literal given as an argument. As in spec files, regex literals are delimited by
backticks.


## Rule 3: fields `type` and `values` should be mutually exclusive

For this rule, we'll use array quantifying expressions of the form:
```
<quantifier> <array value> match <query pattern>
```
Where the quantifier is any of the following keywords: `no`, `any`, `all`.
Array quantifying expressions return true when any, all, or none of the values
in the given array match the query pattern.
Using this new tool, we can find the `Object` nodes for witch
there is at least one child member with key `type` and one child member with key `values`:

```
match Object n when
    any n.members.children match {  
        Member m when m.key.text == '"type"' 
    }
    && any n.members.children match { 
        Member m when m.key.text == '"values"' 
    };
```

## Conclusion
We now have queries to identify violations of our business rules, but opening the REPL
and pasting the queries whenever we want to validate a document isn't very practical.
So, in the final part of this series, we'll learn how to package queries into a Sylver rule
set to consume, distribute and share them more conveniently!