Package: yacfg
yacfg - YAML ConfiguratorThis tool can generate a set of configuration files mainly created for
ActiveMQ Artemis, but it is not limited to only generating configuration files.It has a user facing Command Line Tool for quick and easy command line usage.
Furthermore, it is possible to use its API in your python code.Getting startedPython 3.7+Python PoetryFrom gitgitclonegit@github.com:rh-messaging-qe/yacfg.git
poetryinstall
yacfg--helpFrom PyPIpipinstallyacfg
yacfg--helpUser (CLI) guideyacfg--help# Set path for profiles and templates# For example the ActiveMQ Artemis template and profilesgitclonehttps://github.com/rh-messaging-qe/yacfg_artemis.git./yacfg_artemis# Currently is needed to setup profiles and templates paths as environment variablesexportYACFG_PROFILES=./yacfg_artemis/profilesexportYACFG_TEMPLATES=./yacfg_artemis/templates

yacfg--list-profiles
yacfg--list-templates# perform a generation of a default profileyacfg--profileartemis/default.yaml.jinja2# also save result to [OUTDIR] directoryyacfg--profile[PROFILE]--output[OUTDIR]CustomizationQuickest way to customize data is to use hot-variables, basically variables
that the profile itself provides for tuning. Next step is to write (modify) custom
profile with completely custom values.
If that does not satisfy your needs, then a custom template might be required.Profile tuningSimply export tuning values from profile you want to tune and change those you
need to change. Then supply the custom tuning file(s) when generating the profile.yacfg--profile[PROFILE]--export-tuningmy_values.yaml
vimmy_values.yaml
yacfg--profile[PROFILE]--tunemy_values.yaml# multiple tuning files can be overlaid# they are updated in sequence, only values present are overwrittenyacfg--profile[PROFILE]--tunemy_values.yaml--tunemachine_specific.yaml\--tunelogging_debug.yaml--output[OUTDIR]Custom profilesWrite your own, or simply export an existing profile and modify that.You can export dynamic version with includes of some modules, that would still
work. Either you can use imports from package, or your own local files.Or you can export completely rendered profile file without any includes or
variables and modify that as you like.# export profile with dynamic includes still active jinja2 filesyacfg--profile[PROFILE]--new-profilemy_new_profile.yaml.jinja2# export completely generated profile without any jinja2 fields, plain yamlyacfg--profile[PROFILE]--new-profile-staticmy_new_profile.yaml
vimmy_new_profile.yaml
yacfg--profilemy_new_profile.yamlProfile is just another jinja2 file that enables customization of profile data
-- that is tuning. Because of that we recommend keeping the extension.yaml.jinja2unless it is static profile without any jinja2 capabilities, in that case it could
be named.yaml. That way we can run yaml lint against static profiles and verify
that they are correct.All profiles have to be used to generate files without any tuning. That means,
if they are tune-able, they have to contain all default values in_defaultssection.
That section is also used for tuning, so any variable in there will be exported as tuning.Custom templatesThe last resort is to export a template and modify that. But remember a template,
or more correctly a template set is a directory containing a set of main
templates that subsequently generate_via_tuning_files a new file.Of course feel free to write your own templates. Especially when you need to
generate_via_tuning_files for something that is not packaged.Just remember for a template set to be identified the directory must contain
a file named '_template' and then main templates ending with '.jinja2'.yacfg--template[TEMPLATE]--new-templatemy_new_template
vimmy_new_template/[MAIN_TEMPLATES].jinja2
yacfg--templatemy_new_template--profile[PROFILE]Jinja2 filters:We use Jinja2 filters from Ansible project, read more about it here:
[https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_filters.html](Ansible filters documentation)API guideDirect use of API is to usegenerate()nearly the same as the CLI.
With option to use tuning values directly.Tuning data will be overlaid in order of appearance, using python
dict.update(), so values that will appear later will overwrite previous
values. We recommend that tuning values are always flat, because update
is not recursive. The same applies for data from tuning files as well
as the directly provided data.Data application order:profile defaultsdata from tuning files (in order of appearance)tuning_files_listdata provided directly (in order of appearance)tuning_data_listimportyacfg# generating only broker.xml config using default values from profile,# no tuning, writing output to a target pathyacfg.generate(profile='artemis/default.yaml.jinja2',output_filter=['broker.xml'],output_path='/opt/artemis-i0/etc/',)# using both files and direct values, and writing generated configs to# a target directoryyacfg.generate(profile='artemis/default.yaml.jinja2',tuning_files_list=['my_values.yaml','machine_specific.yaml','logging_debug.yaml'],tuning_data_list=[{'name':'custom name','config':'option_a'},{'address':'10.0.0.1'},{'LOG_LEVEL':'debug'},],output_path='/opt/artemis-i0/etc/',)# just get generated data for further processing, using just tuning filesdata=yacfg.generate(profile='artemis/default.yaml.jinja2',tuning_files_list=['my_values.yaml','machine_specific.yaml','logging_debug.yaml'],)print(data['broker.xml'])Batch configurationsIn case you have multiple services to configure in your environment,
that you probably will have at some point, there is a tool for that
as well. The tool is called yacfg-batch. It has only yaml input, and
it uses yacfg to generate configurations as you are already used to.Input yaml file defines all services you need to generate, what
profiles to use, and what tuning to provide toyacfg.
It allows you to configure defaults and common for services.Batch profile fileAs said it is YAML. It has two special sections:_defaultand_common.
As the name suggests,_defaultvalues are used when values are not
defined per specific section. Where_commonis added to the values
of all sections. The important thing here is that_defaulthas lower
priority than_commonand that has lower priority than specific section
values.Every section has 4 values:profile,template,tuning_files,
andtuning. As the name suggests,profiledefines what generation profile
to select, and it directly correlates withyacfg's--profile.templatedefines what generation template to use
(overrides one in the profile if defined), and it directly correlates with--templatefromyacfg.tuning_filesoption is a list of tuning
files to use, when combining defaults, commons, and specific values,
tuning_files list is concatenated. Finallytuningis a map of
specific tuning values, correlates with--optofyacfg. When combining
defaults, commons, and specifics, it will be updated over using python
dict.update() and it will work only on first level, so it is recommended
to use flat values for tuning only.Example_default:profile:artemis/default.yaml.jinja2tuning_files:-defaults/broker_default.yaml_common:tuning_files:-common/security.yaml-common/logging.yamltuning_values:LOG_LEVEL_ALL:INFObrokerA/opt/artemis/etc:pass:truebrokerB/opt/artemis/etc:profile:artemis/AIOBasic.yaml.jinja2tuning_files:-brokerB/queues.yaml---_default:profile:artemis/default.yaml.jinja2tuning_files:-defaults/broker_default.yamlbrokerC/opt/amq/etc:tuning:LOG_LEVEL_ALL:DEBUGAs you can see,yacfg-batchsupports multiple sections, in single
batch profile file, that allows you to generate multiple groups using
separated_defaultand_commonsections for that.Executing batchWhen you have defined all tuning files you need, and in the root of this
batch configuration you have your batch profile file, you can now simply
runyacfg-batch:yacfg-batch--input[batch_profile_file]--output[output_path]You can use multiple input files and all of those will be generated
consecutively. In the output path, new subdirectories will be created
for every item you configure (every section), section key will be used
for that subdirectory. If the section name resembles a path, whole
path will be created. For example forbrokerA/opt/artemis/etcthe configuration will be generated into[output_path]/brokerA/opt/artemis/etc/.DocumentationFormatted documentation can be viewed atrh-messaging-qe.github.io/yacfg/.ContributingIf you find a bug or room for improvement, submit either a ticket or PR.ContributorsAlphabetically orderedDominik Lenochdlenoch@redhat.com(maintainer)Michal Tóthmtoth@redhat.comOtavio Piskeopiske@redhat.comSean Daveysdavey@redhat.comZdenek Krauszkraus@redhat.com(author)LicenseCopyright 2018-2021 Red Hat Inc.Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.Acknowledgmentsjinja2-- awesome templating engineyaml-- very convenient user readable formatlearn_yaml-- great YAML cheat sheetpyyaml-- python YAML parserjq-- great tool for working with structured data (JSON)yq-- YAML variant of jqgithub templates examples-- Nice set of ISSUE_TEMPLATE.md and PULL_REQUESTS_TEMPLATE.md examplescontributing example-- example/template of CONTRIBUTING.mdFedora Project code-of-conduct-- the inspiration for CODE_OF_CONDUCT.md