Package: hyaml
HYAMLHYAML is an expression-oriented language which serves the purpose of writing consice config files without introducing all the power of a fully-featured scripting language. In a nutshell, it's an extensible DSL for writing configs.The syntax is inspired by the Python programming language. At the same time, it lacks a lot of features Python has since they're not relevant to the common configuration tasks. For example, you won't see list expressions or lambdas here.Initially, HYAML was built with the library namedCodeTalker. That lib got the job done, however, its main purpose was to be as fast as possible. To be the fastest language parser around CodeTalker used Cython underneath that is essentially C with Python-like syntax. Since Cython, just as C, is a compiled language, it compiles CodeTalker's sources on installation. As time went by, this became a major pain point for maintenance. In addition to that, HYAML wasn't meant to bethatfast because of the way it's used: after reading a config file all HYAML-expressions are translated to regular Python functions. This is done once on startup and it's usuallyfast enough. Eventually, HYAML's backend was switched toANTLR.InstallationUse pip to install HYAML. It's tested against Python 3.7 and work with earlier versions is not guaranteed.pip install hyamlWork-in-progress versionTo install a version that hasn't yet been published to the main index run the following command:pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple hyamlLanguageHYAML is an expression-oriented language or, to put it another way, it's a one-liner-oriented language. It doesn't support statements or things like assignments (directly, though it can be enhanced in certain ways).FeaturesPrimitivesThe exhaustive list of supported primitive values:Numbers, including integers and floats (use regular notation, as in1,-1,1.0).Strings, enclosed by single or double quotes ('foo',"bar").Boolean values are represented astrueandfalse.Variables ($var_name).Basic mathAddition, subtraction, multiplication, and division are backed by Python. Pay special attention to division since it's translated to/which is float point division rather than integer one (namely,//). It works like this for historical reasons only.1 + 5  # addition
-3--5  # subtraction, translated to "-3 - -5", use if you don't like spaces
5 * 2  # multiplication
10 / 2 # divisionLogical operatorsLogical operators areand,or, andnot, just as in Python:$a and $b           # conjunction
$b or $c            # disjunction
not $a              # negation
$a and ($b or $c)   # grouping
$foo and not false  # negation precedes other operations, conjunction precedes disjunctionComparsisonAgain, as in Python and most modern programming languages1 > 2  # > stands for greater
1 >= 2 # greater or equal
1 < 2  # less
1 <= 2 # less or equal
1 == 2 # equal
1 != 2 # not equalRange comparisons are supported :tada:10 > $a > 5 # checks whether $a less than 10 and greater than 5Lists and dictsCompared to the previous version of HYAML lists (aka arrays) and dicts (aka associative arrays, maps, and hashmaps) are first-level citizens :tada::[1, 2, 3] + [$x, $y, $z] # concatenation of two lists
{abc: 123}               # creates a dict in python {"abc: 123"}Note that dict keys don't require quotes and support hyphens and colons as non-terminating symbols. In other words,{Sub-Attr:1: 500}is translated to{"Sub-Attr:1": 500}.Accessing attributesA common task when working with HYAML is accessing nested values in a dict. This is what.does:$dict_var.Dict-Key-Name.Nested-Key-Name:1The expression above corresponds to the following Python codedict_var["Dict-Key-Name"]["Nested-Key-Name:1"]-,_,:are special characters allowed for attribute names. When a dict doesn't have the key requested, it will produce a runtime error.Calling methodsValues in HYAML can have methods. Essentially, they are not methods at all but regular functions. There's nothing special about method calls except for adding parentheses is required even if the method doesn't take arguments."abc".substring(1)
$var.upper()
'0xfade114'.to_i(16)Methods returning booleans usually end with?:1.odd?()Work around nullnullvalue (known in Python asNone) is a common source of errors when the value you're working with is not set. It's a burden for both static (such as C or C++) and dynamic (such as Python or JavaScript) languages. Nowadays, many of them offer a concise form of checks for value presence (null-checks). HYAML follows the trend and has support for thesafe navigation operator:$dict_var?.Key-Name:1In the example above, if$dict_varisNoneor doesn't containKey-Name:1this won't produce an error. Of course, you can have chains when accessing nested attributes:$dict_var?.Key?.Nested-Key # this is safe now, yay!The operators works for methods as well:$dict_var?.substring(4)?.size() or 0APITranslator APITranslator takes a (presumably) valid string HYAML and generates a (most likely) valid Python expression:fromhyaml.translatorimporttranslatetranslate("$foo")# get(variables, 'foo')translate("1.odd?()")# is_odd(1)Compiler APICompiler takes one step further and makes a function from the given expression:fromhyaml.compilerimportcompilenine_plus_five=compiler("9 + 5")nine_plus_five()# 9For using variables in expressions you'll need to add bindings to functions:fromhyaml.compilerimportCompilercompile=Compiler(bindings=("variables",))inc=compile("$var + 1")inc({"var":1})# 2inc({"var":10})# 11In order to work with methods you'll need to provide a table of functions to the compiler:fromhyaml.compilerimportCompilermethods={"square":lambdax:x**2,"is_like":lambdax,y:x.startswith(y)}compile=Compiler(method_name=methods)square=compile("5.square()")square()# 25like=compiler("'abcdef'.like?('ab')")like()# TrueFinally, using variables and methods together:fromhyaml.compilerimportCompilermethods={"square":lambdax:x**2,"is_like":lambdax,y:x.startswith(y)}compile=Compiler(method_name=methods,bindings=("variables",))square=compile("$x.square()")square({"x":10})# 100square({"x":7})# 49like=compile("'abcdef'.like?($str)")like({"str":"ab"})# Truelike({"str":"abc"})# TrueFor those who curious, there's a module named "prelude" which holds globally available methods. They are used for implementing some language features. Specifically, the subscription operator (aka square brackets) relies onget, safe navigation to attributes and method calls relies onsafe_getandsafe_callrespectively. You can override those methods with your own variants usingmethod_table(but you cannot remove them, why would you anyway?).Running testspython-munittesttests/test_*PlaygroundHYAML is provided with interactive shell for testing out the language, fire it up withhyaml$ hyaml

>>> evaluate("$x.starts_with?('foo')", x="foobar")
True
>>>ANTLRTL;DR: ANTLR is a tool for building language parsers. Be sure you visited itswebsiteand checked out thesource code. ANTLR is written in Java but can generate parsers in Python. Compared to CodeTalker it has plenty ofexamplesand a ton of answered question at StackOverflow.In order to build a new parser, follow the instructions (taken from the ANTLR's githubpage):Install ANTLR fromhttps://www.antlr.org/.Define your language grammar in the .g4 format (there's an extension for VSCode!).Generate the output by runningantlr4-Dlanguage=Python3MyGrammar.g4Add all generated/updated files to commit.You're all set!G4One can the documentation or even read a book about ANTLR but building a new language can be done by examples. At least, this is the way Hyaml.g4 was written.Lexer/Parser APIA simple example of using generated lexer and parser follows:importsysfromantlr4import*fromHyamlLexerimportHyamlLexerfromHyamlParserimportHyamlParserdefmain(argv):input=FileStream(argv[1])### Create a lexer instance and pass an IO object to its constructorlexer=HyamlLexer(input)stream=CommonTokenStream(lexer)### Create a parser and feed it with tokensparser=HyamlParser(stream)tree=parser.prog()lisp_tree_str=tree.toStringTree(recog=parser)### Print a simple tree representationprint(lisp_tree_str)if__name__=='__main__':main(sys.argv)Running this script withpython parser-example.py sample.txtprints something like:(prog (expr (expr $cdr) (callChain (link (methodCall . some (arguments ( (exprList (expr (listLiteral [ ]))) )))))) \n)For transforming parsed structures you'll need a listener.Listener APIAlong with a lexer and a parser, ANTLR generates a listener, a class which can be used to track the parsing process. In order to use it, create a subclass for the generated listener and implement necessary hook methods:classListener(MyGrammarListener):def__init__(self):self._initial_state=MyState()defenterMyExpression(self,ctx):self._modifyStateAsNeeded()defexitMyExpression(self,ctx):self._modifyStateAsNeeded()Check outtranslator.pyto see a real example, it's rather straightforward.DevelopmentHYAML works with Python 3.7+. Install Python and pip then runpipinstall-e.[dev]Packaging and publishingUpdate the version number in setup.py, set a new tag (-sis not required if you don't sign your commits):git tag -a -s -m 'Version x.y.z' vx.y.zAccording to thedocs, run the following commands:pip install --upgrade setuptools wheel twine
python3 setup.py sdist bdist_wheelYou may want to publish the package to the Test PyPi repo first:twine upload --repository-url https://test.pypi.org/legacy/ dist/*If you're sure and ready, publish it to the main index:twine upload dist/*Note that for publishing you'll be asked for your credentials for access to PyPi repositories. See additionalinstructionson managing credentials.ContributingBug reports and pull requests are welcome on GitHub athttps://github.com/latera/hyaml.