Package: metapensiero.tool.tinject
Author:Lele GaifaxContact:lele@metapensiero.itlicense:GNU General Public License version 3 or laterThis is a simple tool that takes aYAMLdeclarative configuration ofactions, each
describing the set ofstepsneeded to accomplish it, and realizes one or more actions
possibly interpolating a set of dynamic values inserted by the user.I’ve written it with the goal of taking away, as much as possible, the tediousness of creating
new content in aPatchDBsetup.There is a myriad of similar tools around, but most of them are either targeted at
bootstrapping a hierarchy of files and directories (that is, aone-shotoperation, for
example to create theskeletonof a new Python package, likecookiecutterto mention one) or
belong to a very specific domain (an example of this category may beZopeSkel).To fulfill my goal I needed a tool able to perform an arbitrary list of operations, primarily:create a directorycreate a file with a particular contentchange the content of an existing fileIt obviously needs to beparametricin almost every part: the pathnames and the actual
content of the files must be determined by atemplateinterpolated with a set ofvalues.This is whatJinja2excels at.Since the values are mostlyvolatile(that is, almost all of them change from one invocation
to the other), I first used the excellentinquirerlibrary to prompt the user, but at some
point I was asked about compatibility with Windows and thus I switched towhaaaaat, that
being based onprompt_toolkitwas more promising on that. More recently, due to a long
inactivity on the port to version 2 of prompt_toolkit (seewhaaaaat’s issue 23), I replaced
it withquestionary: thankfully they all expose a very similar interface, so that was very
easily achieved (hint: the latter two effectively inherit from the former).ContentsConfigurationGlobal sectionJinja2 environmentFile headersInitial stepsActionsStepsSample sessionBatch modeConfigurationThe YAML configuration file may be composed by one or twodocuments: an optionalglobal
sectionand a mandatory one containing the list of availableactions, each of them being
basically a list ofsteps.Global sectionThe firstdocumentcontains a set of optional global settings, in particular:the options to instantiate the Jinja2environmenta set of fileheaders, one for each kind of filea list ofstepsto be performed at every invocationJinja2 environmentTo avoid clashes with the syntax used by Python’sformat()minilanguage, thedefault
environment settingsare the following:block_start_stringThe start of a block is<<, instead of the default{%block_end_stringThe end of a block is>>, instead of the default%}variable_start_stringA variable is introduced by«, instead of the classic{{variable_end_stringA variable ends with», and not with the classic}}extensionsThe extensionjinja2_time.TimeExtensionis automatically selectedkeep_trailing_newlineBy default the trailing newline isnotdiscardedObviously any Jinja2 template defined by the configuration must adhere to these settings. If
you don’t like them, or if you want define different ones, you can put an entry like the
following in the globals section of the configuration:jinja:
  block_start_string: "[["
  block_end_string: "]]"
  variable_start_string: "<<"
  variable_end_string: ">>"File headersThis is a map between a filesuffixand a template that will be automatically injected in the
render’s context as the variableheader. For example, should you need to generate a Pascal
unit, you could insert the following:headers:
  pas: |
    (*« "*" * 70 »
     * Project: «package_name»
     * Created: «timestamp»
     *« "*" * 70 »)Initial stepsThis is a list of steps to be executed unconditionally at startup. In particular it may be used
to gather some values from arbitrary places to initialize theanswers, to prompt the user for
a common set of values and to define customsteps. As an example, you could say:steps:
  - python:
      script: |
        import os, pwd, sys

        class InitCustomAnswersWithoutPrompt(Step):
            def __init__(self, state, config):
                super().__init__(state, config)
                self.name = config['name']
                self.value = config['value']

            def announce(self):
                self.state.announce('*', "Inject %s=%s", self.name, self.value)

            def __call__(self, *args, **kwargs):
                return {self.name: self.value, python_version: sys.version}

        register_step('initcustom', InitCustomAnswersWithoutPrompt)

        myself = pwd.getpwuid(os.getuid())
        state.answers['author_username'] = myself.pw_name
        state.answers['author_fullname'] = myself.pw_gecos.split(',')[0]

  ## Here you can execute the new kind of operation defined above

  - initcustom:
      name: "myextravar"
      value: "thevalue"ActionsAnactionis identified by a unique name and carries an optional description, an optional set
of prompts specific to the action and a list of one or more steps.The following is a complete example:create_top_level_setup_py:
  description: Create top level setup.py

  prompt:
    - package_name:
        message: The name of the Python package

  steps:
    - createdir:
        directory: src

    - createfile:
        directory: src
        filename: setup.py
        content: |
          # Hi, I'm the setup.py file for «package_name»StepsAstepis some kind ofoperationthat must be carried out. The name of the step identifies
the kind of operation, and its value is used to specify the needed parameters. So, in the
example above, we have two steps,createdirandcreatefile, each requiring its specific
arguments map.A step may be conditionally skipped specifying an expression in itswhensetting: if
present, the operation will be performed only when the expression evaluates to true.This is the list of available operation kinds:changefilePerform some quite simple changes to the content of an existing file.Required configuration:directoryThe directory containing the file to be changedfilenameThe name of the existing file within the given directorychangesA list of tweaks: there are currently just three types, one thataddsome contentbeforea givenmarker, one to add the contentafterit and one thatinsertsome
contentbetweena markerandanother marker keeping the block sortedExample:- changefile:
    directory: src
    filename: listofitems.txt
    changes:
      - add: "«newitemname»\n"
        before: "\n;; items delimiter\n"
      - add: "«newitemname»\n"
        after: "\n;; reversed insertion order\n"

- changefile:
    directory: src
    filename: __init__.py
    changes:
      - insert: "from .«table_name» import «table_name»\n"
        between: "\n## ⌄⌄⌄ tinject import marker ⌄⌄⌄, please don't remove!\n"
        and: "\n## ⌃⌃⌃ tinject import marker ⌃⌃⌃, please don't remove!\n"createdirCreate a directory and its parents.Required configuration:directoryThe directory to be createdExample:- createdir:
    directory: src/my/new/packagecreatefileCreate a new file with a particular content.Required configuration:directoryThe directory contained the file to be createdfilenameThe name of the new filecontentA Jinja2 template that will be rendered and written to the new fileExample:- createfile:
    directory: "«docs_dir»/«schema_name»/tables"
    filename: "«table_name».sql"
    description: Structure of table «schema_name».«table_name»
    ## The template may be either inline or included from an external file
    content: !include 'table.sql'promptAsk the user for some information bits.Required configuration: a list of dictionaries, each representing aquestionary’s
question.Example:- prompt:
    - name_of_the_variable:
        message: Tell me the value
        default: "default value"

    - different_kind_of_input:
        message: Select the variant
        kind: list
        choices:
          - Big
          - Medium
          - SmallpythonExecute an arbitrary Python script.Required configuration:scriptThe code of the scriptThe script is executed with a context containing the classStep, the functionregister_stepand the globalstateof the program.See theinitial stepsabove for an example.repeatRepeat a list of substeps.Required configuration:stepsThe list of substeps to repeatOptional configuration:descriptionA message string, emitted at the start, if givenanswersThe name of variable holding a list of answers, when one substep is apromptcountThe number of iterationswhenA Jinja boolean expression: if given it’s evaluated once before the loop begins, that
gets executed only when it expression’s value is true, otherwise no repetition happens at
all; the expression may refer to previous answers, even those collected while looping
(that is, the variable specified by theanswersoption)untilA Jinja boolean expression: if given (andcountisnot), then the loop is
terminated when the condition is falseagain_messageWhen neithercountnoruntilare specified, the step will explicitly ask
confirmation about looping again, at the end of all substeps executionSeeexamples/repeat.ymlfor an example.Sample sessionCreate a new schema with a new table:$ tinject --verbose apply examples/patchdb.yml new_schema new_table

* Execute Python script
[?] Author fullname (author_fullname): Lele Gaifax
[?] Author username (author_username): lele
[?] Author email (author_email): «author_username»@example.com
[?] Fully qualified package name (package_name): package.qualified.name
[?] Timestamp (timestamp): << now 'local', '%a %d %b %Y %H:%M:%S %Z' >>
[?] Year (year): << now 'local', '%Y' >>
[?] Distribution license (license): GNU General Public License version 3 or later
[?] Copyright holder (copyright): © «year» «author_fullname»
[?] Root directory of Sphinx documentation (docs_dir): docs/database
[?] Root directory of SQLAlchemy model sources (model_dir): src/«package_name|replace(".","/")»

=====================
 Create a new schema
=====================
[?] Name of the new schema (schema_name): public

* Create directory docs/database/public

* Create file docs/database/public/index.rst

* Create directory docs/database/public/tables

* Create file docs/database/public/tables/index.rst

* Create directory src/package/qualified/name/entities/public

* Create file src/package/qualified/name/entities/public/__init__.py

* Create directory src/package/qualified/name/tables/public

* Create file src/package/qualified/name/tables/public/__init__.py

====================
 Create a new table
====================
[?] Schema name of the new table (schema_name): public
[?] Name of the new table (table_name): things
[?] Description of the new table (table_description): The table ``«schema_name».«table_name»`` contains...
[?] Name of the corresponding entity (entity_name): Thing

* Create file docs/database/public/tables/things.rst

* Create file docs/database/public/tables/things.sql

* Create file src/package/qualified/name/entities/public/thing.py

* Change file src/package/qualified/name/entities/public/__init__.py

  - insert “from .thing import T…” between “## ⌄⌄⌄ tinject impo…” and “## ⌃⌃⌃ tinject impo…”

  - add “mapper(Thing, t.thi…” after “## ⌃⌃⌃ tinject impo…”

* Create file src/package/qualified/name/tables/public/things.py

* Change file src/package/qualified/name/tables/public/__init__.py

  - insert “from .things import …” between “## ⌄⌄⌄ tinject impo…” and “## ⌃⌃⌃ tinject impo…”Verify:$ cat src/package/qualified/name/entities/public/__init__.py
# -*- coding: utf-8 -*-
# :Project:   package.qualified.name -- Entities in schema public
# :Created:   mer 15 giu 2016 13:24:54 CEST
# :Author:    Lele Gaifax <lele@example.com>
# :License:   GNU General Public License version 3 or later
# :Copyright: © 2016 Lele Gaifax
#

from sqlalchemy.orm import mapper

from ...tables import public as t

## ⌄⌄⌄ tinject import marker ⌄⌄⌄, please don't remove!
from .thing import Thing

## ⌃⌃⌃ tinject import marker ⌃⌃⌃, please don't remove!

mapper(Thing, t.things, properties={
})Add another table:$ tinject --verbose apply examples/patchdb.yml new_table

* Execute Python script
[?] Author fullname (author_fullname): Lele Gaifax
[?] Author username (author_username): lele
[?] Author email (author_email): «author_username»@example.com
[?] Fully qualified package name (package_name): package.qualified.name
[?] Timestamp (timestamp): << now 'local', '%a %d %b %Y %H:%M:%S %Z' >>
[?] Year (year): << now 'local', '%Y' >>
[?] Distribution license (license): GNU General Public License version 3 or later
[?] Copyright holder (copyright): © «year» «author_fullname»
[?] Root directory of Sphinx documentation (docs_dir): docs/database
[?] Root directory of SQLAlchemy model sources (model_dir): src/«package_name|replace(".","/")»

====================
 Create a new table
====================
[?] Schema name of the new table (schema_name): public
[?] Name of the new table (table_name): thangs
[?] Description of the new table (table_description): The table ``«schema_name».«table_name»`` contains...
[?] Name of the corresponding entity (entity_name): Thang

* Create file docs/database/public/tables/thangs.rst

* Create file docs/database/public/tables/thangs.sql

* Create file src/package/qualified/name/entities/public/thang.py

* Change file src/package/qualified/name/entities/public/__init__.py

  - insert “from .thang import T…” between “## ⌄⌄⌄ tinject impo…” and “## ⌃⌃⌃ tinject impo…”

  - add “mapper(Thang, t.tha…” after “## ⌃⌃⌃ tinject impo…”

* Create file src/package/qualified/name/tables/public/thangs.py

* Change file src/package/qualified/name/tables/public/__init__.py

  - insert “from .thangs import …” between “## ⌄⌄⌄ tinject impo…” and “## ⌃⌃⌃ tinject impo…”Verify:$ cat src/package/qualified/name/entities/public/__init__.py
# -*- coding: utf-8 -*-
# :Project:   package.qualified.name -- Entities in schema public
# :Created:   mer 15 giu 2016 13:24:54 CEST
# :Author:    Lele Gaifax <lele@example.com>
# :License:   GNU General Public License version 3 or later
# :Copyright: © 2016 Lele Gaifax
#

from sqlalchemy.orm import mapper

from ...tables import public as t

## ⌄⌄⌄ tinject import marker ⌄⌄⌄, please don't remove!
from .thang import Thang
from .thing import Thing

## ⌃⌃⌃ tinject import marker ⌃⌃⌃, please don't remove!

mapper(Thang, t.thangs, properties={
})

mapper(Thing, t.things, properties={
})Batch modeThere are cases when all thevariablesare already known and thus there’s no need to
interactively prompt the user to get the job done.Theapplyaction accepts the options--prompt-onlyand--answers-fileto make that
possible.The former can be used to collect needed information and print that back, or write them into a
YAML file with--output-answers:$ tinject apply -p -o pre-answered.yml patchdb.yml new_schema
? Author fullname (author_fullname) Lele Gaifax
? Author username (author_username) lele
? Author email (author_email) «author_username»@example.com
? Fully qualified package name (package_name) package.qualified.name
? Timestamp (timestamp) << now 'local', '%a %d %b %Y %H:%M:%S %Z' >>
? Year (year) << now 'local', '%Y' >>
? Distribution license (license) GNU General Public License version 3 or later
? Copyright holder (copyright) © «year» «author_fullname»
? Root directory of Sphinx documentation (docs_dir) docs/database
? Root directory of SQLAlchemy model sources (model_dir) src/«package_name|replace(".","/")»
? Name of the new schema (schema_name) publicIf you put those settings into a YAML file, you can then execute the action in batch mode:$ tinject -v apply -a pre-answered.yml patchdb.yml new_schema

* Execute Python script

=====================
 Create a new schema
=====================

* Create directory docs/database/public

* Create file docs/database/public/index.rst

* Create directory docs/database/public/tables

* Create file docs/database/public/tables/index.rst

* Create directory src/package/qualified/name/entities/public

* Create file src/package/qualified/name/entities/public/__init__.py

* Create directory src/package/qualified/name/tables/public

* Create file src/package/qualified/name/tables/public/__init__.py