Package: excel-sql-engine
excel-sql-engine: a tiny SQL engine for Excelexcel-sql-engine is a command-line tool that implements a Pyxl-SQL engine for Excel workbooks.Pyxl-SQL is a small implementation of SQL, for Excel workbooks.for Excel file format read/write, uses Openpyxlhttps://pypi.org/project/openpyxl/applies to Excel sheets that follow a 'table' format with unique column headerSQL queries are written inside the 'Pyxl SQL' sheetSQL supports SELECT INTO, UPDATE, JOIN, FROM, AS, WHERE, WHEN, WITH, GROUP BY, HAVING, ORDER BYuses python expressionsuses Excel formulas, with column namesis typically used to routinely merge/join information from several Excel sheets into a new one.excel-sql-engine can be used as a command-line tool, either python or standalone windows executable.
It looks for Excel files in the directory and executes commands found on the 'Pyxl SQL'
sheet. Command-line options allow to change the behavior.excel-sql-engine can also be used as a python libraryexcel-sql-engine is tested with tox and cov. See the "tests" directory in the source tree,
and the test coverage section of the docexcel-sql-engine is documented. See the "doc" directory in the source treeHow to get excel-sql-enginePython module on Pypi :https://pypi.org/project/excel-sql-engine/Python source on gitlab:https://gitlab.com/fabien_battini/pyxlsqlWindows on-file exe on gitlab artifacts:https://fabien_battini.gitlab.io/pyxlsql/html/download.htmlHow to use excel-sql-engineOne or several Excel workbooks contain 'origin' database sheets. For instance, the Northwind databaseAn additional 'destination' Excel file contains Pyxl-SQL statements and templates for the resulting databases. For instance, test_pyxlsql.xlsxrun the engine:pyxlSql.exe [options] if the windows executable has been downloaded and is reachable in the PATpython -m PyxlSql [options] if the python module has been installed through pipThe Excel file is read by the engine, and Pyxl-SQL statements are executedStatements are typically:SELECT INTO, UPDATE etc statements that read data in the 'origin' databases, write in the databases of the 'destination' workbooka 'DELETE SHEET Pyxl SQL' statement, because the Pyxl-SQL code is seldom useful in the final filesome 'SAVE' statement that write the updated 'destination' fileAs a result, several 'origin' Excel files are merged in a final 'destination' Excel file, according to the Pyxl-SQL statementsDifferences with standard SQL: restrictions and additionsPyxl-SQL syntax doesnotmanage intermediate (virtual) tables, so no (yet) sub-commands to a command
(e.g. SELECT INTO applied to a SELECT)Only a subset of SQL commands are managedcolumn names can contain space, and enclosed into {} when used in an expression or formulaby default, tables (i.e. Excel sheets) are numbered following their appearance in the command,
starting at 0,
so that the syntax @1{FIELD NAME} indicates "FIELD NAME" in the table/sheet number 1,
i.e. an alias ("AS") is created automaticallya SET clause is used to specify the fields being created in a SELECT command (or equivalent),
in a way similar to the SET clause in an UPDATE commandin a SET clause, the right member is NOT restricted to a field name, it can be an arbitrary PYTHON expression,
starting with ':=' for instance:SET	Priority	=	':= "Major" if @0{Range} in ("High", "Medium") and #0{Potential 2021} > 100000 else "Minor"'Excel formulas are also accepted, starting with '=', for instance:SET Mean = '=SUM(M2:Q2)/SUM(R2:V2)'the SET syntax in incremented with a WHEN clause, similar to WHERE, but executed for each SET clause individually.
For instance, the following clause will assign the value for column 2019 either in columns 'Basic 2019' or 'Medium 2019'
depending on the value of the KPI row, thus transforming a line segmentation into a row segmentation.SET	Basic 2019	=	2019	WHEN	:= @1{KPI} == "Basic"SET	Medium 2019	=	2019	WHEN	:= @1{KPI} == "Medium"the AGGREGATES clause allows a python expression as a generalization of MAX, MIN etc ...the IMPORT command allows importing python modulesthe FORMAT command specified an Excel format for a columnthe COMMENTS command introduces commentsPyxl-SQL file formatPyxl-SQL assumes that:Excel files are readable by OpenPyxlsheets used by Pyxl-SQL statements are formatted as 'tables', i.e.1st line (default) is a column headercolumn headers are all differentother sheets should be simply readable by openPyxlone file holds Pyxl-SQL commands and clauses that will be executedkeywords are CAPITALIZEDIt is possible to change the default location of a database within a sheet by using the DATABASE statement,
so that, for instance column headers are not read from line 1.In the case of an empty sheet (which will be filled-in by Pyxl-SQL statements using "SELECT INTO"),
thefirst lineafter the header may hold "examples", which can hold formats or formulas
that will be replicated for each line. This example line will be erased by "SELECT INTO"
See 'Basic' examplePyxl-SQL commandsPyxl-SQL commands and clauses are stored in a special sheet (by default "Pyxl SQL")
A command is stored on one line
A command is completed by the clauses that follow on next linesCommands are split among several columns:COMMAND: holds the command name, e.g. "LEFT JOIN", or the name of an associated Clause e.g. "WHERE"Target: holds the name of the target sheet (or column in the case of CLAUSES )KEY: holds a keyword that completes the definition of the command. e.g. "LEFT JOIN" is always completed by "FROM"Source: holds the value that will be assigned to each element in the target.
Depending on the command or clause, 'Source' can be a field name, an Excel formula, a Python expression...For some CLAUSES, a 2nd set (CONDITION, Target test, COMPARE, Source Test), on the same line, completes the definitionThe remaining columns are for COMMENTS, which can be any arbitrary cell, and are not parsedSHEET descriptorSheet are described by their names. e.g.'USA commands' : for sheets in the file being executed (i.e. which holds the 'Pyxl SQL' sheet )'Northwind.xlsx[Orders Details]' : for sheets in a different Excel fileFIELD descriptorsa Field may be denoted with the following syntax:'field name' only, if no ambiguity'(@|#)N{field name}', if there is an ambiguity. N is the number of the sheet'(@|#)alias{field name} alias is the alias of the sheetin the last 2 examples:@ denotes a string# denotes a numberThis type indication simplifies the management of n empty cells,because Excel considers empty as being 0 in a number context and "" in a string context,
whereas Python considers it as None. So the 'type' indication allows the conversion of None into
either 0 or "", which simplifies the expressionsexamplesExamples are taken fromhttps://www.w3schools.com/sql/default.aspand use the well-known Northwind sample database (V1 and V2)Testing and exampleTesting is done with tox and cov. to run the test, just run 'tox' in the root directory(venv) PyxlSQL> toxSee the tests/ directory in the source for more details.The current coverage is 93%, most of the non-covered code is defensivepyxl-sql as a toolpyxl-sql is also a command-line tool, with command-line options.
for more information:pyxlsql.py --helpDocumentationDocumentation is built using Sphinxhttps://www.sphinx-doc.org/Documentation is generated inhttps://fabien_battini.gitlab.io/pyxlsql/html/This process is achieved automatically through gitlab CI pipelines.gitlab repository:https://gitlab.com/fabien_battini/pyxlsqlpypi:https://pypi.org/project/excel-sql-engine/LinkedIN:https://www.linkedin.com/in/fabien-battini-supelec/