Package: terrabuddy
terrabuddyTerrabuddy is a templating engine built on top ofterraform.  Terrabuddy allows terraform to be used in a way that is moreDRY, moreauditable, and moremodular.Terrabuddy Featureseasily install and update terraform binarieseasily manage component interdependencies via bundleseasily inject variables into your modulesbuilt in git workflow supportInstallationSystem Requirementspython3 with pip3 in your $PATHpip install terrabuddyor install using setup.pygit clone https://github.com/jumidev/terrabuddy.git
cd terrabuddy/tb
make install             # installs the tb CLI tool with python requirements

tb --setup               # downloads and installs terraform
tb --setup-terraformrc   # (optional) installs useful terraform default settings
tb --setup-shell         # (optional) installs useful tb shell aliasesInstalling & using terraform modulesterrabuddy can work with any terraform code.A repo with modules for Azure is providedhereFor AWS,here. (WIP)Backgroundtbis the terrabuddy command line interface.  It facilitates setting up your machine (see installation), allows you to list components and run terraform commands such asplan,apply,destroyand manages shared variables.tb introduces three notions for managing resoureces:projects,componentsandbundles.  A project is a git repo with a specific purpose.  Components are individual objects that you create on your cloud provider.  Bundles are sets of components that depend upon one another (and tb knows how to create them in the correct order).Anatomy of a project:Below is an example project encompassing three environmentsprep/
prod/
sbx/
.envrc.tpl
.gitignore
project.yml
README.md
tfstate_store.hcltThe top level directories in the project correspond to the environments,sbx(sandbox),prep(pre-prod), andprod..envrc.tpl contains a template for required environment variables, notably TF_MODULES_ROOT which points to the repo in to which terraform-modules-azure has been clonedproject.yml contains project-specific variables.  Other .yml files within the project containtfstate_store.hclt is an hcl template that will be applied in all componentsAnatomy of a component:Components are hclt files, e.g. hcl templates.HCLis a json-like declarative language used by terraform.cat component.hclt

source  {
    # local path
    path = "/path/to/terraform/files"

    ....
    # OR a git repo
    repo = "https://github.com/jumidev/terraform-modules-aws.git"
}
inputs {
    foo = "bar"
}theinputsblock contains the inputs which will be injected into the terraform modulethesourceblock tells tb which terraform module to use with this component, can be a local path or a git repo.hclt files contain variables, formatted**${like_this}**Component parsing and variablesWhentbis run on a component, it:lints/parses all hclt files, fails if there are syntax errorsloads all yml files in the component and all parent directories as variables.searches in the component directory and all parent directories for hclt files and combines them into a single hclt file.replaces all variables in the hclt.  If variables are left unreplaced, the parser stops with an error message.if all variables are replaced, it performs the requested terraform action (plan,apply,destroy, etc)Component parsing in detailcombines hclt files in the component directory with those in its parent directories.  For example, at the root of the project there is a tfstate_store.hclt file.  The contents of this file will be included inall components.Another example is to refactor the source.hclt in a situation where a folder contains lots of components that use the same terraform module.az-platform-infra$ ll prep/network_security_groups/*
prep/network_security_groups/bastion:
component.hclt

prep/network_security_groups/db:
component.hclt

prep/network_security_groups/db-apps:
component.hclt

prep/network_security_groups/public-webserver:
component.hcltAll of the above component.hclt contain the samesourceblockWe can add an hclt file with asourceblock in the top level folder, thusly:az-platform-infra$ ll prep/network_security_groups/*

prep/network_security_groups:
source.hclt

prep/network_security_groups/bastion:
component.hclt

prep/network_security_groups/db:
component.hclt

prep/network_security_groups/db-apps:
component.hclt

prep/network_security_groups/public-webserver:
component.hcltTemplate Overridinghclt files in upper directoryies can be overriddden by placing files with the same name deeper in the filesystem.  For example, if you have a component that requires a specific tfstate_store configuration, you can override the one in the root folder, thusly:prep/network_security_groups/public-webserver-other-remote-state:
component.hclt
tfstate_store.hclt  # overrides tfstate_store.hclt in project rootComponent variables in detailtb loads variables in a cascade process, starting at the project root and moving down the filesystem to the component.  For example forprep/bastion/managed_disk, the following yml files are loaded:project.ymlin the project root, contains various key/value pairsprep/env.ymlenv: "prep"prep/bastion/appname.ymlappname: "bastion"If, for example,project.ymlcontainsappname, its value will be overridden byprep/bastion/appname.yml.You can use tb to display component variables with theshowvarscommand$ tb showvars prep/bastion/managed_disk

COMPONENT_DIRNAME=managed_disk
COMPONENT_PATH=prep/bastion/managed_disk
PROJECT_ROOT=/home/user/myprojects/az-platform-infra
TB_INSTALL_PATH=/home/user/wf/terrabuddy/tb
appname=bastion
env=prep
location=westeurope
organization=wforg
private_subnet_cidr=172.16.4.0/22
project_name=platform-infra
public_subnet_cidr=172.16.2.0/24
vnet_cidr=172.16.0.0/19Special VariablesIn addition to variables loaded in .yml files, tb also provides special variablesCOMPONENT_PATHfull path to component, relative to projectCOMPONENT_DIRNAMEinnermost component directoryPROJECT_ROOTabsolute path to projectBundlesAnywhere in a project, components can be bound together as a bundle, simply by placing a bundle.yml file with anorderobject.For example,prep/bastion/bundle.ymlorder:
    - network_interface
    - a_record
    - managed_disk
    - virtual_machineThe above bundle tells tb that when the user runstb <command> prep/bastionit will in fact run four components:prep/bastion/network_interface,prep/bastion/a_record, etc... in the specified order.  When running the destroy command, this order is reversed.You can use the--dryargument on a bundle to see its components:$ tb show prep/bastion --dry

tb show prep/bastion/network_interface
tb show prep/bastion/a_record
tb show prep/bastion/managed_disk
tb show prep/bastion/virtual_machineBundles also support wildcards and other bundles, for exampleprep/bundle.ymlorder:
    - resource_group
    - dns/zone/*                    # all components in this dir, alphabetical order
    - virtual_network
    - subnets/*                     # all components in this dir, alphabetical order
    - application_security_groups/* # all components in this dir, alphabetical order
    - network_security_groups/*     # all components in this dir, alphabetical order
    - bastion                       # this is another bundletb <command> prepis thus a single "monster" bundle that runs the entire prep environmentListing Components and bundlestb uses the same commands as terraform:plan, apply, destroy, refresh, etc.
Components in a project can be listed with thetb plan|apply|showcommand.$ pwd
~/az-platform-infra$ tb plan
OOPS, no component specified, try one of these (bundles are bold underlined):

tb plan sbx
tb plan sbx/application_security_groups/db-apps
tb plan sbx/application_security_groups/public-webserver
tb plan sbx/application_security_groups/manager
tb plan sbx/application_security_groups/db
tb plan sbx/application_security_groups/bastion
tb plan sbx/virtual_network
tb plan sbx/storage_account/std
tb plan sbx/storage_account/premium
tb plan sbx/dns/zone/sbx.weatherforce.net
tb plan sbx/dns/zone/sbx.prv
tb plan prod/keybaseca
tb plan prod/keybaseca/network_interface
tb plan prod/keybaseca/managed_disk
tb plan prod/keybaseca/virtual_machine
tb plan prod/keybaseca/a_record
tb plan prod/application_security_groups/db-apps
tb plan prod/application_security_groups/public-webserver
tb plan prod/application_security_groups/db
tb plan prod/bastion
tb plan prod/bastion/network_interface
tb plan prod/bastion/virtual_machine
tb plan prod/bastion/a_record
tb plan prod/resource_group
tb plan prod/network_security_groups/db-apps
tb plan prod/network_security_groups/public-webserver
tb plan prod/network_security_groups/db
...runningtbcommandsEach of the above lines is a component.  Runningtb plan <component>will run the plan command on the component in question.# TODO REDOThe above result means that the component already exists in Azure and is up to date with the component.Git workflow integrationtbwas designed to take git workflow considerations into account.  When working with terrabuddy components, special care must be taken so ensure that developers working on separate components do not clobber each other's work.  tb includes git checking functions to inform developers if their local git repository is behind remote changes.For instance, developers A and B work on two unrelated components.Developer A is working on an application VM inprep/testappA/virtual_machine.  Developer A sees that network security rules do not allow their application to access required resources.  They amend the security groupprep/network_security_groups/db-appsto add the required security rules.  They apply their changes and push to remote.Developer B is working on an unrelated component.  They too need to amendprep/network_security_groups/db-appsto add their own required security rules (different from those that Developer A added).  They have forgotten togit pullso the changes made by Developer A are not on their machine.If they runtb apply prep/network_security_groups/db-appsthey will clobber developer A's changes.Howevertb does a git fetch and compares branchesbeforeeach command.Since Developer A has pushed their changes, tb on Developer B'a machine will show this message:GIT ERROR: You are on branch master and are behind the remote.  Please git pull and/or merge before proceeding.  Below is a git status:...The above also works for feature branches.  If developer B is working on a feature branch that was made prior to developer A's changes (pushed to master branch), tb will detect that Developer B's FB is behind master and prompt them to merge before proceeding.TODOremove terragrunt dependencypredictable temp dirs in home dirauto cleanup previous runs older than x daysauto manage remote states,enrypted remote statesexample spoke and hub architecture