Package: moban
Author:C.W. and its contributors (See contributors.rst)Issues:http://github.com/moremoban/moban/issuesLicense:MITAnnouncementIn version 0.8.0,moban.plugins.jinja2.tests.filesis moved to moban-ansible
package.moban.plugins.jinja2.filters.githubis moved to moban-jinja2-github
package Please install them for backward compatibility.Quick start$exportHELLO="world"$moban"{{HELLO}}"worldOr$exportHELLO="world"$echo"{{HELLO}}"|mobanOr simply$HELLO="world"moban"{{HELLO}}"A bit formal example:$moban-cdata.yml-tmy.templateworldGiven data.yml as:hello:worldand my.template as:{{hello}}Please note that data.yml will take precedence over environment variables.Template inheritance and custom template directoriesSuppose there existsshared/base.jj2, and two templateschild1.jj2andchild2.jj2derives from it. You can do:$moban-tchild1.jj2-tdshared-ochild1$moban-tchild2.jj2-tdshared-ochild2Data overload and custom data directoriesEffectively each data file you give to moban, it overrides environment variables.
Still you can have different layers of data. For example, you can haveshared/company_info.yml,  useproject1.ymlfor project 1 andproject2.ymlfor project 2. In each of the derived data file, simply mention:overrides:company_info.yml...Here is the command line to use your data:$moban-cdshared-cproject1.yaml-tREADME.jj2Custom jinja2 extensionmoban allows the injection of user preferred jinja2 extensions:$moban-ejj2=jinja2_time.TimeExtension...Well, can I nick some existing functions as filters, tests? Or create a global from another library?Sure, you can use the same ‘-e’ syntax:$moban-ejinja2=filter:module.path.filter_function\jinja2=test:module.path.test_function\jinja2=global:identifier=module.path.variableIn this case, you would have to include the external library in your own requirements.txtHere is an example:$moban-ejinja2=filter:moban.externals.file_system.url_join\jinja2=test:moban.externals.file_system.exists\jinja2=global:description=moban.constants.PROGRAM_DESCRIPTION\-t"{{ 'a'|url_join('b')}} {{'b' is exists}}"Can I write my own jinja2 test, filter and/or globals?moban allows the freedom of craftsmanship. Please refer to the docs for more
details. Here is an example:importsysimportbase64frommoban.plugins.jinja2.extensionsimportJinjaFilter@JinjaFilter()defbase64encode(string):ifsys.version_info[0]>2:content=base64.b64encode(string.encode("utf-8"))content=content.decode("utf-8")else:content=base64.b64encode(string)returncontentAnd you can use it within your jinja2 template,mytest.jj2:{{'abc'|base64encode}}Assume that the custom example was saved incustom-jj2-plugin$moban-pdcustom-jj2-plugin-tmytest.jj2...Moban will then load your custom jinja2 functionsSlim template syntax for jinja2withmoban-sliminstalled,Given a data.json file with the following content{
  "person": {
    "firstname": "Smith",
    "lastname": "Jones",
  },
}$moban--template-typeslim-cdata.json"{{person.firstname}} {{person.lastname}}"SmithJonesHandlebars.js templateWithmoban-handlebarsinstalled,Given a data.json file with the following content{
  "person": {
    "firstname": "Yehuda",
    "lastname": "Katz",
  },
}$moban--template-typehandlebars-cdata.json"{{person.firstname}} {{person.lastname}}"YehudaKatzForhandlebars.jsusers, yes, the example was copied from handlebarjs.com. The
aim is to show off what we can do.Let’s continue with a bit more fancy feature:$moban--template-typehandlebars-cdata.json"{{#with person}}{{firstname}} {{lastname}} {{/with}}"Moban’s way ofpybar3 usage:Let’s save the following file ascript.pyunderhelper_and_partialfolder:frommoban_handlebars.apiimportHelper,register_partialregister_partial('header','<h1>People</h1>')@Helper('list')def_list(this,options,items):result=[u'<ul>']forthinginitems:result.append(u'<li>')result.extend(options['fn'](thing))result.append(u'</li>')result.append(u'</ul>')returnresultAnd givendata.jsonreads as the following:{
    "people":[
        {"name": "Bill", "age": 100},
        {"name": "Bob", "age": 90},
        {"name": "Mark", "age": 25}
    ]
}Let’s invoke handlebar template:$moban--template-typehbs-pdhelper_and_partial-cdata.json"{{>header}}{{#list people}}{{name}} {{age}}{{/list}}"Handlebars-ing{{>header}...tomoban.outputHandlebarsed1file.$catmoban.output<h1>People</h1><ul><li>Bill100</li><li>Bob90</li><li>Mark25</li></ul>Velocity templateWithmoban-velocityinstalled,Given the following data.json:{"people":
    [
        {"name": "Bill", "age": 100},
        {"name": "Bob", "age": 90},
        {"name": "Mark", "age": 25}
    ]
}And given the following velocity.template:Old people:
#foreach ($person in $people)
 #if($person.age > 70)
  $person.name
 #end
#end

Third person is $people[2].namemobancan do the template:$moban--template-typevelocity-cdata.json-tvelocity.templateOldpeople:BillBobThirdpersonisMarkCan I write my own template engine?Yes and please check formore details.Given the following template type function, and saved in custom-plugin dir:frommoban.core.content_processorimportContentProcessor@ContentProcessor("de-duplicate","De-duplicating","De-duplicated")defde_duplicate(content:str,options:dict)->str:lines=content.split(b'\n')new_lines=[]forlineinlines:iflinenotinnew_lines:new_lines.append(line)returnb'\n'.join(new_lines)You can start using it like this:$moban--template-typede-duplicate-pdcustom-plugin-tduplicated_content.txtTOML data formatmoban-anyconfigshould be installed first.Given the following toml file, sample.toml:title = "TOML Example"
[owner]
name = "Tom Preston-Werner"You can do:$moban-csample.toml"{{owner.name}} made {{title}}"TomPreston-WernermadeTOMLExampleNot limited to toml, you can supply moban with the following data formats:Always supported formats, quoting from python-anyconfigFormatTypeRequirementJSONjsonjson(standard lib) orsimplejsonIni-likeiniconfigparser(standard lib)Picklepicklepickle(standard lib)XMLxmlElementTree(standard lib)Java propertiespropertiesNone (native implementation with standard lib)B-shshellvarsNone (native implementation with standard lib)For any of the following data formats, you elect to install by yourself.Supported formats by pluggable backend modulesFormatTypeRequired backendAmazon Ionionanyconfig-ion-backendBSONbsonanyconfig-bson-backendCBORcboranyconfig-cbor-backendoranyconfig-cbor2-backendConifgObjconfigobjanyconfig-configobj-backendMessagePackmsgpackanyconfig-msgpack-backendOr you could choose to install all:$pipinstallmoban-anyconfig[all-backends]Why not to use python-anyconfig itself, but yet another package?moban gives you a promise of any location whichpython-anyconfigdoes not support.Why do it mean ‘any location’?Thanks topyfilesystem 2,
moban is able to read data back fromgit repo,pypipackage,http(s), zip,
tar, ftp,s3oryou name it.Templates and configuration files over HTTP(S)httpfsshould be installed first.With httpfs,mobancan access any files over http(s) as its
template or data file:$moban-t'https://raw.githubusercontent.com/moremoban/pypi-mobans/dev/templates/_version.py.jj2'\-c'https://raw.githubusercontent.com/moremoban/pypi-mobans/dev/config/data.yml'\-o_version.pyIn an edge case, if github repo’s public url is given,
this github repo shall not have sub repos. This library will fail to
translate sub-repo as url. No magic.Templates and configuration files in a git repogitfs2is optional since v0.7.0 but was
installed by default since v0.6.1You can do the following with moban:$moban-t'git://github.com/moremoban/pypi-mobans.git!/templates/_version.py.jj2'\-c'git://github.com/moremoban/pypi-mobans.git!/config/data.yml'\-o_version.pyInfo:Foundrepoin/Users/jaska/Library/Caches/gitfs2/repos/pypi-mobansTemplatinggit://github.com/moremoban/pypi-mobans.git!/templates/_version.py.jj2to_version.pyTemplated1file.$cat_version.py__version__="0.1.1rc3"__author__="C.W."Templates and configuration files in a python packagepypifsis optional since v0.7.0 but
was installed by default since v0.6.1You can do the following with moban:$moban-t'pypi://pypi-mobans-pkg/resources/templates/_version.py.jj2'\-c'pypi://pypi-mobans-pkg/resources/config/data.yml'\-o_version.pyCollectingpypi-mobans-pkg....Installingcollectedpackages:pypi-mobans-pkgSuccessfullyinstalledpypi-mobans-pkg-0.0.7Templatingpypi://pypi-mobans-pkg/resources/templates/_version.py.jj2to_version.pyTemplated1file.$cat_version.py__version__="0.1.1rc3"__author__="C.W."Work with S3 and other cloud based file systemsPlease installfs-s3fs:$ pip install fs-s3fsThen you can access your files in s3 bucket:$moban-cs3://${client_id}:${client_secrect}@moremoban/s3data.yml\-o'zip://my.zip!/moban.output'{{hello}}$unzipmy.zip$catmoban.outputworldWhere the configuration sits in a s3 bucket, the output is a file in a zip. The content of s3data.yaml is:So what can I do with itHere is a list of other usages:Django Mobans, templates for django, docker etc.Math Sheets, generate custom math sheets in pdfAt scale, continous templating for open source projectsmobanenabledcontinuous templatinginpyexcelandcoalaproject to keep
documentation consistent across the documentations of individual libraries in the same
organisation. Here is the primary use case of moban, as of now:Usage beyond command lineAll use cases aredocumentedSupportIf you like moban, please support me on github,patreonorbounty sourceto maintain
the project and develop it further.With your financial support, I will be able to invest
a little bit more time in coding, documentation and writing interesting extensions.VisionAny template, any data in any locationmobanstarted with bringing the high performance template engine (JINJA2) for web
into static text generation.mobancan use other python template engine: mako, handlebars, velocity,
haml, slim and tornado, can read other data format: json and yaml, and can access both
template file and configuration file in
any location: zip, git, pypi package, s3, etc.Creditjinja2-fsloaderis the key component to enable PyFilesystem2 support in moban
v0.6x. Please show your stars there too!InstallationYou can install it via pip:$pipinstallmobanor clone it and install it:$gitclonehttp://github.com/moremoban/moban.git$cdmoban$pythonsetup.pyinstallCLI documentationusage:moban[-h][-cCONFIGURATION][-tTEMPLATE][-oOUTPUT][-td[TEMPLATE_DIR[TEMPLATE_DIR...]]][-pd[PLUGIN_DIR[PLUGIN_DIR...]]][-cdCONFIGURATION_DIR][-mMOBANFILE][-gGROUP][--template-typeTEMPLATE_TYPE][-dDEFINE[DEFINE...]][-eEXTENSION[EXTENSION...]][-f][--exit-code][-V][-v][template]Statictextgeneratorusinganytemplate,anydataandanylocation.positionalarguments:templatestringtemplatesoptionalarguments:-h,--helpshowthishelpmessageandexit-cCONFIGURATION,--configurationCONFIGURATIONthedatafile-tTEMPLATE,--templateTEMPLATEthetemplatefile-oOUTPUT,--outputOUTPUTtheoutputfileAdvancedoptions:Forbettercontrol-td[TEMPLATE_DIR[TEMPLATE_DIR...]],--template_dir[TEMPLATE_DIR[TEMPLATE_DIR...]]addmoredirectoriesfortemplatefilelookup-cdCONFIGURATION_DIR,--configuration_dirCONFIGURATION_DIRthedirectoryforconfigurationfilelookup-pd[PLUGIN_DIR[PLUGIN_DIR...]],--plugin_dir[PLUGIN_DIR[PLUGIN_DIR...]]addmoredirectoriesforpluginlookup-mMOBANFILE,--mobanfileMOBANFILEcustommobanfile-gGROUP,--groupGROUPasubsetoftargets--template-typeTEMPLATE_TYPEthetemplatetype,defaultisjinja2-dDEFINE[DEFINE...],--defineDEFINE[DEFINE...]tosupplyadditionaloroverridepredefinedvariables,format:VAR=VALUEs-eEXTENSION[EXTENSION...],--extensionEXTENSION[EXTENSION...]totoTEMPLATE_TYPE=EXTENSION_NAME-fforcemobantotemplateallfilesdespiteof.moban.hashesDeveloperoptions:Fordebugginganddevelopment--exit-codebydefault,existcode0meansnoerror,1meanserroroccured.Ittellsmobantochange1forchanges,2forerroroccured-V,--versionshowprogram'sversionnumberandexit-vshowverbose,try-v,-vv,-vvv8 contributorsIn alphabetical order:Andrew SchellerAyan BanerjeeCharlie LiuJohn VandenbergJoshua ChungPRAJWAL MsalotzSerekKiriChange log0.8.2 - 04.09.2020FixedUse any functions, any data structure of any python packages as jinja2
filters, tests, globals0.8.1 - 04.09.2020Fixed#399: content processor
should be called only oncecontent processor shall pass on options to content processors0.8.0 - 02.09.2020Removedmoban.plugins.jinja2.tests.files is moved to moban-ansible packagemoban.plugins.jinja2.filters.github is moved to moban-jinja2-github packageFixed#396: custom jinja2
plugins(filters, tests and globals) are not visible if a template is passed
as a string.0.7.10 - 16.08.2020Updated#393: Rendered content
output to stdout once0.7.9 - 06.08.2020Updated#390: single render action
will print to stdout by default0.7.8 - 09.06.2020Added#313: Non-textual source
files should default to copy0.7.7 - 24.5.2020Added-pdfor command line to include custom plugin directoriesFixedstrip did not work in 0.7.60.7.6 - 22.5.2020Added#38: finally be able strip
the rendered content0.7.5 - 21.5.2020Added#167: reverse what moban
have done: delete0.7.4 - 13.5.2020Fixed#378: suppress stdout
message from deprecated pip install. but please do not use and migrate
deprecated`requires` syntax.0.7.3 - 2.5.2020AddedAdded continuous check in travis for setup.py descriptions. No impact to
moban user.0.7.2 - 1.5.2020AddedSupport for templates and configuration files over HTTP(S) protocol with
httpfs! Yepee!0.7.1 - 25.04.2020Fixed#365: regression was
introduced by v0.6.5. If you uses mobanfile as data configuration file, you
are very likely to have this show stopper. Please upgrade to this version.0.7.0 - 18.01.2020Removed#360: make gitfs2 and
pypifs optional.#303: python 2.7 support
is dropped.Updated#360: show friendlier
error when unknown protocol exception was raised.0.6.8 - 7.12.2019Updatedsince version 0.5.0, when rendering a single file or string, moban would
report ‘Templated 1 of 0 files’, which should have been ‘Templated 1 file.’Removedpython 3.4 support is gone because colorama requires Python ‘>=2.7, !=3.0.*,
!=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*’0.6.7 - 1.12.2019Updatedno verbose for error, -v for warning, -vv for warning+info, -vvv for
warning+info+debug#351, show template plugin
name, i.e. ‘copying’ for copy instead of ‘templating’RemovedMessage: ‘Warning: Attempting to use environment vars as data…’ became
warning logMessage: ‘Warning: Both data.yml and /…/.moban.cd/data.yml does not exist’
became warning logwith -v, you would see them in such a situation0.6.6 - 10.11.2019Addedsupport moban file inheritance. one base moban file and child repos can
inherit and override0.6.5 - 13.10.2019Added#335: support intermediate
targets in moban file0.6.4 - 4.10.2019UpdatedCommand options have been grouped. –template_type became –template-typeIncrement gitfs2 to version 0.0.2.gitfs#40.6.3 - 25.09.2019Added#260: jinja-cli parity:
support command line pipe stream.0.6.2 - 15.09.2019Added#322: Implicit targets
with template extensions default to copy#257: ‘-e’ to load
extensions for template engines, i.e. jinja2#333: command line
template fails with version 0.6.10.6.1 - 10.09.2019Fixed#328: update backward
compatibility0.6.0 - 10.09.2019Added#205: supportpyFilesystem2#185: -v will enable moban
application logging for development. And -V is for version.#325: -vv show debug trace#126: Allow mobanfile to
include data from arbitrary config files#256: jinja2-cli parity:
‘-d hello=world’ to define custom variable on cliUpdated#275: fix moban 0.4.5 test
failures on openSUSE Tumbleweed0.5.0 - 14.07.2019Updated#277: Restored dependency
git-url-parse, replacing incompatible giturlparse which was used during moban
0.4.x#281: Fixed unicode
support on Python 2.7#274: Updated ruamel.yaml
dependency pins to restore support for Python 3.4, and prevent installation
of versions that can not be installed on Python 3.7#285: Fixed CI testing of
minimum requirements#271: Fixed repository
caching bug preventing branch switching#292: Reformatted YAML
files according to yamllint rules#291: Fixed filename typos
in README#280: Added CI to ensure
repository is in sync with upstream#280: sync setup.py from
pypi-mobans0.4.5 - 07.07.2019Updated#271: support git branch
change in later run.0.4.4 - 26.05.2019Updated#265: Use simpleread
binaryto read instead of encoding0.4.3 - 16.03.2019Removed#253: symbolic link in
regression pack causes python setup.py to do recursive includeAdded#209: Alert moban user
whengitis not available and is used.Updated#261: since moban group
template files per template type, this fill use first come first register to
order moban group0.4.2 - 08.03.2019Added#234: Define template
parameters on the fly insidetargetssection#62: select a group target
to runUpdated#180: No longer two
statistics will be shown in v0.4.x. legacy copy targets are injected into a
normal targets. cli target is made a clear priority.#244: version 0.4.2 is
first version which would work perfectly on windows since 17 Nov 2018. Note
that: file permissions are not used on windows. Why the date? because
samefile is not avaiable on windows, causing unit tests to fail hence it lead
to my conclusion that moban version between 17 Nov 2018 and March 2019 wont
work well on Windows.0.4.1 - 28.02.2019Added#235: user defined
template types so that custom file extensions, template configurations can be
controlled by moban user#232: the package
dependencies have been fine tuning to lower versions, most of them are dated
back to 2017.0.4.0 - 20.02.2019Added#165: Copy as pluginsUpdated#219: git clone depth set
to 2#186: lowest dependecy on
ruamel.yaml is 0.15.5, Jun 20170.3.10 - 03.02.2019Added#174: Store git cache in
XDG_CACHE_DIR#107: Add -v to show
current moban version#164: support additional
data formatsUpdated#178: UnboundLocalError:
local variable ‘target’ referenced before assignment#169: uses GitPython
instead of barebone git commands0.3.9 - 18-1-2019Updated#90: allow adding extra
jinja2 extensions.jinja2.ext.do,jinja2.ext.loopcontrolsare included by
default. what’s more, any other template enigne are eligible for extension
additions.#158: Empty file
base_engine.py is finally removed0.3.8 - 12-1-2019Updated#141: disable file
permissions copy feature and not to check file permission changes on windows.#154: introduce first ever
positional argument for string base template.#157: the exit code
behavior changed. for backward compactibility please use –exit-code.
Otherwise, moban will not tell if there is any changes.0.3.7 - 6-1-2019Updated#146: added a low-setup
usage mode via environment variables to moban#148: include test related
files in the package for package validation when distributing via linux
system, i.e. OpenSuse0.3.6 - 30-12-2018Updated#143: moban shall report
permission error and continue the rest of the copying task.#122: Since 0.3.6, moban
is tested on windows and macos too, using azure build pipelines. It is
already tested extensively on travis-ci on linux os.0.3.5 - 10-12-2018Updated#37: moban will report line
number where the value is empty and the name of mobanfile. Switch from pyyaml
to ruamel.yaml.0.3.4.1 - 28-11-2018Updated#137: missing
contributors.rst file0.3.4 - 18-11-2018Addedglobal variables to store the target and template file names in the jinja2
enginemoban-handlebars is tested to work well with this version and aboveUpdatedTemplate engine interface has been clarified and documented0.3.3 - 05-11-2018Addedalternative and expanded syntax for requires, so as to accomendate github
submodule recursive0.3.2 - 04-11-2018Addedconfiguration dirs may be located byrequires, i.e. configuration files may
be in a python package or git repository.0.3.1 - 02-11-2018Added#97: requires will clone a
repo if given. Note: only github, gitlab, bitbucket for now0.3.0 - 27-18-2018Added#89: Install pypi-hosted
mobans through requires syntaxUpdated#96: Fix for
FileNotFoundError for pluginsvarious documentation updatesRemoved#88: removed python 2.6
supportremoved python 3.3 support0.2.4 - 14-07-2018Added#32: option 1 copy a
directory without its subdirectories.#30: command line template
option is ignore when a moban file is present0.2.3 - 10-07-2018Added#76: running moban as a
module from python command#32: copy a directory
recusively#33: template all files in
a directory0.2.2 - 16-06-2018Added#31: create directory if
missing during copyingUpdated#28: if a template has been
copied once before, it is skipped in the next moban call0.2.1 - 13-06-2018Updatedtemplates using the same template engine will be templated as a groupupdate lml dependency to 0.0.30.2.0 - 11-06-2018Added#18: file exists test#23: custom jinja plugins#26: repr filter#47: allow the expansion of
template engine#58: allow template type
per templateUpdated#34: fix plural message if
single file is processed0.1.4 - 29-May-2018Updated#21: targets become
optional#19: transfer symlink’s
target file’s file permission under unix/linux systems#16: introduce copy key
word in mobanfile0.1.3 - 12-Mar-2018Updatedhandle unicode on python 20.1.2 - 10-Jan-2018Added#13: strip off new lines in
the templated file0.1.1 - 08-Jan-2018Addedthe ability to present a long text as multi-line paragraph with a custom
upper limitspeical filter expand github references: pull request and issues#15: fix templating syntax
to enable python 2.60.1.0 - 19-Dec-2017Added#14, provide shell exit
code0.0.9 - 24-Nov-2017Added#11, recognize .moban.yaml
as well as .moban.yml.#9, preserve file
permissions of the source template.-moption is added to allow you to specify a custom moban file. kinda
related to issue 11.Updateduse explicit version name:moban_file_spec_versionso thatversioncan be
used by users.#10Please
note: moban_file_spec_version is reserved for future file spec upgrade. For
now, all files are assumed to be ‘1.0’. When there comes a new version i.e.
2.0, new moban file based on 2.0 will have to include
‘moban_file_spec_version: 2.0’0.0.8 - 18-Nov-2017Added#8, verify the existence of
custom template and configuration directories. default .moban.td, .moban.cd
are ignored if they do not exist.UpdatedColorize error messages and processing messages. crayons become a dependency.0.0.7 - 19-Jul-2017AddedBring the visibility of environment variable into jinja2 templating process:#70.0.6 - 16-Jun-2017Addedadded ‘-f’ flag to force moban to template all files despite of .moban.hashesUpdatedmoban will not template target file in the situation where the changes
occured in target file than in the source: the template file + the data
configuration after moban has been applied. This new release will remove the
change during mobanization process.0.0.5 - 17-Mar-2017AddedCreate a default hash store when processing a moban file. It will save
unnecessary file write to the disc if the rendered content is not changed.Added summary reports0.0.4 - 11-May-2016UpdatedBug fix#5, should detect
duplicated targets in.moban.ymlfile.0.0.3 - 09-May-2016UpdatedBug fix#4, keep trailing
new lines0.0.2 - 27-Apr-2016UpdatedBug fix#1, failed to save
utf-8 characters0.0.1 - 23-Mar-2016AddedInitial release