Package: yasha
YashaYasha is a code generator based onJinja2template engine. At its simplest, a command-line callyasha-vvariables.yamltemplate.txt.j2will rendertemplate.txt.j2into a new file named astemplate.txt. See how the created file name is derived from the template name. The template itself remains unchanged.The tool was originally written to generate code for the zinc.rs'I/O register interfacefrom theCMSIS-SVDdescription file, and was used to interface withthe peripherals of Nordic nRF51ARM Cortex-M processor-based microcontroller. Yasha has since evolved to be flexible enough to be used in any project where the code generation is needed. The tool allows extending Jinja by domain specific filters, tests and extensions, and it operates smoothly with the commonly used build automation software like Make, CMake and SCons.InstallationAs a regular user:pipinstallyashaor if you like to get the latest development version:pipinstallgit+https://github.com/kblomqvist/yasha.gitor if you would like to take part into the development process:gitclonehttps://github.com/kblomqvist/yasha.git
pipinstall-eyashaUsageUsage: yasha [OPTIONS] [TEMPLATE_VARIABLES]... TEMPLATE

  Reads the given Jinja TEMPLATE and renders its content into a new file.
  For example, a template called 'foo.c.j2' will be written into 'foo.c' in
  case the output file is not explicitly given.

  Template variables can be defined in a separate file or given as part of
  the command-line call, e.g.

      yasha --hello=world -o output.txt template.j2

  defines a variable 'hello' for a template like:

      Hello {{ hello }} !

Options:
  -o, --output FILENAME         Place the rendered template into FILENAME.
  -v, --variables FILENAME      Read template variables from FILENAME. Built-
                                in parsers are JSON, YAML, TOML, INI, XML and CSV.
  -e, --extensions FILENAME     Read template extensions from FILENAME. A
                                Python file is expected.
  -c, --encoding TEXT           Default is UTF-8.
  -I, --include_path DIRECTORY  Add DIRECTORY to the list of directories to be
                                searched for the referenced templates.
  --no-variable-file            Omit template variable file.
  --no-extension-file           Omit template extension file.
  --no-trim-blocks              Load Jinja with trim_blocks=False.
  --no-lstrip-blocks            Load Jinja with lstrip_blocks=False.
  --remove-trailing-newline     Load Jinja with keep_trailing_newline=False.
  --mode [pedantic|debug]       In pedantic mode Yasha becomes extremely picky
                                on templates, e.g. undefined variables will
                                raise an error. In debug mode undefined
                                variables will print as is.
  -M                            Outputs Makefile compatible list of
                                dependencies. Doesn't render the template.
  -MD                           Creates Makefile compatible .d file alongside
                                the rendered template.
  --version                     Print version and exit.
  -h, --help                    Show this message and exit.Template variablesTemplate variables can be defined in a separate file. By defaultJSON,YAML,TOML,INI,XMLandCSVformats are supported.yasha-vvariables.yamltemplate.j2In case multiple files are given, the variables redefined in later files will take precedence.yasha-vvariables.yaml-vadditional_variables.yamltemplate.j2Additionally you may define variables as part of the command-line call. A variable defined via command-line will overwrite a variable defined in a file.yasha--foo=bar-vvariables.yamltemplate.j2CSV filesto use the data stored in a csv variable file in templates, the name of the variable in the template has to match the name of the csv variable file.For example, consider the following template and variable filestemplate.j2
mydata.csvAnd the following contents inmydata.csvcell1,cell2,cell3
cell4,cell5,cell6to access the rows of cells, you use the following syntax in your template (note that 'mydata' here matches the file name of the csv file){% for row in mydata %}
cell 1's value is {{row[0]}},
cell 2's value is {{row[1]}}
{% endfor %}By default, each row in the csv file is accessed in the template as a list of values (row[0],row[1], etc).
You can make each row accessible instead as a mapping by adding a header to the csv file.For example, consider the following contents ofmydata.csvfirst_column,column2,third column
cell1,cell2,cell3
cell4,cell5,cell6and the following Jinja template{% for row in mydata %}
cell 1's value is {{row.first_column}},
cell 2's value is {{row.column2}},
cell 3's value is {{row['third column']}}
{% endfor %}As you can see, cells can be accessed by column name instead of column index.
If the column name has no spaces in it, the cell can be accessed with 'dotted notation' (ierow.first_column) or 'square-bracket notation' (ierow['third column'].
If the column name has a space in it, the cell can only be accessed with 'square-bracket notation'Automatic file variables look upIf no variable file is explicitly given, Yasha will look for one by searching for a file named in the same way than the corresponding template but with the file extension either.json,.yaml,.yml,.toml, or.xml.For example, consider the following template and variable filestemplate.j2
template.yamlBecause of automatic file variables look up, the command-line callyashatemplate.j2is equal toyasha-vtemplate.yamltemplate.j2In case you want to omit the file variables in spite of its existence, use--no-variable-fileoption flag.Shared template file variablesImagine that you would be writing C code and have the following two templates in two different foldersroot/
    include/foo.h.j2
    source/foo.c.j2and you would like to share the same file variables between these two templates. So instead of creating separatefoo.h.yamlandfoo.c.yamlyou can createfoo.yamlunder the root folder:root/
    include/foo.h.j2
    source/foo.c.j2
    foo.yamlNow when you callcdroot
yashainclude/foo.h.j2
yashasource/foo.c.j2the variables defined infoo.yamlare used within both templates. This works because subfolders will be checked for the variable file until the current working directory is reached —rootin this case. For instance, variables are looked forfoo.h.j2in following order:include/foo.h.yaml
include/foo.yaml
foo.h.yaml
foo.yamlTemplate extensionsYou can extend Yasha by custom Jinjaextensions,testsandfiltersby defining those in a separate Python source file given via command-line option-e, or--extensionsas shown belowyasha-eextensions.py-vvariables.yamltemplate.j2Like for variable file, Yasha supports automatic extension file look up and sharing too. To avoid file collisions consider using the following naming convention for your template, extension, and variable files:template.py.j2
template.py.py
template.py.yamlNow the command-line callyashatemplate.py.j2is equal toyasha-etemplate.py.py-vtemplate.py.yamltemplate.py.j2TestsFunctions intended to work as a test have to be either prefixed bytest_deftest_even(number):returnnumber%2==0of defined inTESTSdictionarydefis_even(number):returnnumber%2==0TESTS={'even':is_even,}FiltersFunctions intended to work as a filter have to be either prefixed byfilter_deffilter_replace(s,old,new):returns.replace(old,new)or defined inFILTERSdictionarydefdo_replace(s,old,new):returns.replace(old,new)FILTERS={'replace':do_replace,}ClassesAll classes derived fromjinja2.ext.Extensionare considered as Jinja extensions and will be added to the environment used to render the template.ParsersIf none of the built-in parsers fit into your needs, it's possible to declare your own parser within the extension file. Either create a function named asparse_+<file extension>, or define the parse-function inPARSERSdictionary with the key indicating the file extension. Yasha will then pass the variable file object for the function to be parsed and expects to get dictionary as a return value.For example, below is shown an example XML file and a custom parser for that.<!-- variables.xml --><persons><person><name>Foo</name><address>FooValley</address></person><person><name>Bar</name><address>BarValley</address></person></persons># extensions.pyimportxml.etree.ElementTreeasetdefparse_xml(file):assertfile.name.endswith('.xml')tree=et.parse(file.name)root=tree.getroot()persons=[]foreleminroot.iter('person'):persons.append({'name':elem.find('name').text,'address':elem.find('address').text,})returndict(persons=persons)Template syntaxYou may change the template syntax via file extensions by redefining
the Jinja parser / lexer. The example below mimics the LaTeX environment.# extensions.pyBLOCK_START_STRING='<%'BLOCK_END_STRING='%>'VARIABLE_START_STRING='<<'VARIABLE_END_STRING='>>'COMMENT_START_STRING='<#'COMMENT_END_STRING='#>'Built-in filtersenvReads system environment variable in a template likesqlalchemy:url:{{'POSTGRES_URL'|env}}Params:default=NoneshellAllows to spawn new processes and connect to their standard output. The output is decoded and stripped by default.os:type:{{"lsb_release -a | grep Distributor | awk '{print $3}'"|shell}}version:{{'cat /etc/debian_version'|shell}}os:type:Debianversion:9.1Requires:Python >= 3.5Params:strip=True, check=True, timeout=2subprocessAllows to spawn new processes, but unlikeshellbehaves like Python's standard library.{%setr="uname"|subprocess(check=False)%}{# Returns either CompletedPorcess or CalledProcessError instance #}{%ifr.returncode-%}platform: Unknown{%else-%}platform:{{r.stdout.decode()}}{%-endif%}platform:LinuxRequires:Python >= 3.5Params:stdout=True, stderr=True, check=True, timeout=2Tips and tricksWorking with STDIN and STDOUTYasha can render templates from STDIN to STDOUT. For example, the below command-line call will render template from STDIN to STDOUT.cattemplate.j2|yasha-vvariables.yaml-Python literals as part of the command-line callVariables given as part of the command-line call can be Python literals, e.g. a list would be defined like thisyasha--lst="['foo', 'bar', 'baz']"template.j2The following is also interpreted as a listyasha--lst=foo,bar,baztemplate.j2Note that in case you like to pass a string with commas as a variable you have to quote it asyasha--str='"foo,bar,baz"'template.j2Other possible literals are:-1,0,1,2(an integer)2+3j,0+5j(a complex number)3.5,-2.7(a float)(1,),(1, 2)(a tuple){'a': 2}(a dict){1, 2, 3}(a set)True,False(boolean)Common file extensionsSometimes it would make sense to have common extensions over multiple templates, e.g. for the sake of filters. This can be achieved by settingYASHA_EXTENSIONSenvironment variable.exportYASHA_EXTENSIONS=$HOME/.yasha/extensions.py
yasha-vvariables.yaml-ooutput.txttemplate.j2Append search path for referenced templatesBy default the referenced templates, i.e. files referred to via Jinja'sextends,includeorimportstatements, are searched in relation to the template location. To extend the search path you can use the command-line option-I— like you would do with GCC to include C header files.yasha-vvariables.yaml-I$HOME/.yashatemplate.j2{%extends"skeleton.j2"%}{# 'skeleton.j2' is searched also from $HOME/.yasha #}{%blockmain%}{{super()}}...{%endblock%}Variable pre-processing before template renderingIf you need to pre-process template variables before those are passed into the template, you can do that via file extensions by wrapping the built-in parsers.# extensions.pyfromyasha.parsersimportPARSERSdefwrapper(parse):defpostprocess(file):variables=parse(file)variables['foo']='bar'# foo should always be barreturnvariablesreturnpostprocessforname,functioninPARSERS.items():PARSERS[name]=wrapper(function)Using tests and filters from AnsibleAnsibleis an IT automation platform that makes your applications and systems easier to deploy. It is based on Jinja2 and offers a large set ofcustom tests and filters, which can be easily taken into use via Yasha's file extensions.pipinstallansible# extensions.pyfromansible.plugins.test.coreimportTestModulefromansible.plugins.filter.coreimportFilterModuleFILTERS=FilterModule().filters()FILTERS.update(TestModule().tests())# Ansible tests are filter likeUsing Python objects of any type in YAMLFor security reasons, the built-in YAML parser is using thesafe_loadofPyYaml. This limits variables to simple Python objects like integers or lists. To work with a Python object of any type, you can overwrite the built-in implementation of the parser.# extensions.pyimportyamldefparse_yaml(file):assertfile.name.endswith(('.yaml','.yml'))variables=yaml.load(file)returnvariablesifvariableselsedict()defparse_yml(file):returnparse_yaml(file)Build automationYasha command-line options-Mand-MDreturn the list of the template dependencies in a Makefile compatible format. The later creates the separate.dfile alongside the template rendering instead of printing to stdout. These options allow integration with the build automation tools. Below are given examples for C files using CMake, Make and SCons.CMake# CMakeList.txtcmake_minimum_required(VERSION2.8.7)project(yasha)file(GLOBsources"src/*.c")file(GLOBtemplates"src/*.jinja")foreach(template${templates})string(REGEXREPLACE"\\.[^.]*$"""output${template})execute_process(COMMANDyasha-M${template}WORKING_DIRECTORY${CMAKE_SOURCE_DIR}OUTPUT_VARIABLEdeps)string(REGEXREPLACE"^.*: """deps${deps})string(REPLACE" "";"deps${deps})add_custom_command(OUTPUT${output}COMMANDyasha-o${output}${template}DEPENDS${deps})list(APPENDsources${output})endforeach()add_executable(a.out${sources})GNU Make# Makefile# User variablesSOURCES=$(wildcardsrc/*.c)TEMPLATES=$(wildcardsrc/*.jinja)EXECUTABLE=build/a.out# Add rendered .c templates to sources listSOURCES+=$(filter%.c,$(basename$(TEMPLATES)))# Resolve build dir from executableBUILDDIR=$(dir$(EXECUTABLE))# Resolve object filesOBJECTS=$(addprefix$(BUILDDIR),$(SOURCES:.c=.o))# Resolve .d files which list what files the object# and template files depend onOBJECTS_D=$(OBJECTS:.o=.d)TEMPLATES_D=$(addsuffix.d,$(basename$(TEMPLATES)))$(EXECUTABLE):$(OBJECTS)$(CC)$^-o$@$(BUILDDIR)%.o:%.c|$(filter%.h,$(basename$(TEMPLATES)))@mkdir-p$(dir$@)$(CC)-MMD-MP$<-c-o$@%.c:%.c.jinjayasha-MD$<-o$@%.h:%.h.jinjayasha-MD$<-o$@# Make sure that the following built-in implicit rule is cancelled%.o:%.c# Pull in dependency info for existing .o and template files-include $(OBJECTS_D) $(TEMPLATES_D)# Prevent Make to consider rendered templates as intermediate file.secondary:$(basename$(TEMPLATES))clean:ifeq ($(BUILDDIR),./)-rm-f$(EXECUTABLE)-rm-f$(OBJECTS)-rm-f$(OBJECTS_D)else-rm-rf$(BUILDDIR)endif-rm-f$(TEMPLATES_D)-rm-f$(basename$(TEMPLATES)).phony:cleanSCons# SConstructimportosimportyasha.sconsenv=Environment(ENV=os.environ,BUILDERS={"Yasha":yasha.scons.CBuilder()})sources=["main.c"]sources+=env.Yasha(["foo.c.jinja","foo.h.jinja"])# foo.h not appended to sourcesenv.Program("a.out",sources)Another example with separatebuildandsrcdirectories.# SConstructimportosimportyasha.sconsenv=Environment(ENV=os.environ,BUILDERS={"Yasha":yasha.scons.CBuilder()})sources=["build/main.c"]duplicate=0# See how the duplication affects to the file pathsenv.VariantDir("build","src",duplicate=duplicate)ifduplicate:tmpl=["build/foo.c.jinja","build/foo.h.jinja"]sources+=env.Yasha(tmpl)else:tmpl=["src/foo.c.jinja","src/foo.h.jinja"]sources+=env.Yasha(tmpl)env.Program("build/a.out",sources)