Package: hephaestus
HephaestusIntroductionHephaestus is a library to statically which parses a SQL query and return an AST.This is useful for different purposes like:statistic informations, i.e. table or field used, tipe of joins, filter conditions, etcpartial or full query comparisonquery type detectionThe SQL grammar is specific forExasol, but it can be easily ported to support other grammars as well.Hephaestus's parser is built inRustusing thelalrpopcrate for grammar's definition andvervolgused as a base for the SQL grammar.InstallationTo install the package you need to install thenightlyversion of the Rust compiler:curlhttps://sh.rustup.rs-sSf|sh-s---y--default-toolchain=nightlythen install the package straight from Github:pipinstallgit+https://github.com/expobrain/hephaestus@developNote: until version 0.8 ofpyo3you need to have thepythonbinary available in yourPATH; on some systems it will fail because onlypython3is present. In this case install the Python 2 package.UsageParsingHephaestus exposes a singleparse()function which given a SQl string returns a dictionary representing the SQL in an AST format:>>>importhephaestus>>>hephaestus.parse("SELECT 1"){'_type':'SelectStatement','common':[],'expr':{'_type':'QuerySetExpression','mode':{'_type':'SelectAll'},'columns':{'_type':'ListResultColumn','values':[{'_type':'ExprResultColumn','expr':{'_type':'NumericLiteral','num':'1'},'rename':None}]},'from':[],'where_expr':None,'group_by':None},'order_by':[],'limit':None}Every node of the dictionary has a_typekey which indicate the type of the node, i.e.SelectStatement, and a series of keys specific for the node's type which can contains scalars values likestr,int,None, a dictionary representing a node or a list of scalar values or nodes.For a full list of node types seesrc/ast.rsandsrc/ast_py.rs.TraversingParsing the incoming SQL expression is only the first step and its not very useful without the ability to traverse the tree and inspect the nodes.Instead of writing from scratch the code to traverse the dictionary nested structure by yourself Hephaestus provides you a class to be used to walk through the tree and call a method for every node it encouters during the traverse.Just inherith from theVisitorclass and implement methods in the form ofvisit_<node_type>where<node_type>is the type of the node in the_typekey.This example shows how to check if the given SQL statement is querying from theDUALtable:fromtypingimportDictfromhephaestus.walkimportVisitorclassMyVisitor(Visitor):has_dual_table=Falsedefvisit_NamedTableExpression(self,attr:str,node:Dict):self.has_dual_table=node["name"]==["dual"]visitor=MyVisitor("SELECT 1 FROM dual")visitor.walk()assertvisitor.has_dual_tableisTrueWhen theNamedTableExpressionnode is reached thevisit_NamedTableExpressionmethod is called and receives the parent node's attribute nameattrand the node itself.TestingThere are two set of tests, one for the Rust and one for the Python code.Note: Unfortunately at the moment it's not possible to execute both test suites with the sameCargo.toml(looks like that theextension-modulefeature inpyo3interfere with the way the tests are linked with the Rust library), so it's necessary to switch to a different config before running the them.To run the Rust tests:maketest_rustTo run the Python tests:maketest_pyContributeIf you want to contribute to the Hephaestus project to extend the grammar follow this steps:switch to a pure RustCargo.tomlwithmake rustwrite a test insrc/teststo cover the casesextend the grammar insrc/sql.lalrpop; please refer to theLALR Bookadd the relative AST note tosrc/ast.rsin some cases, like defining a new Rust enumerator, it's necessary to add an explicit conversion form the Rust type to a Python-compatible type; in this add the relevant code intosrc/ast_py.rsCaveatsnot all the grammar of Exasol is supported, most of the functions and statements needs to be implementeddespite the fact that the AST returned by theparse()function can be manipulated at will at the moment there's no way to get back a SQL string