Package: agileupstate
AgileUp StatePython 3.8+ project to manage AgileUP pipeline states with the following features:Linux and Windows compatible project.Defines state model.Saves and fetches states from vault.Exports private key for Linux SSH connections.Exports client PKI data for Windows WinRM connections.Creates cloud init zip file for mTLS connection data to Windows WinRM hosts.Exports ansible inventories for both Linux(SSH) and Windows(WinRM) connections.Provides simple connectivity tests.PrerequisitesThis project uses poetry is a tool for dependency management and packaging in Python. It allows you to declare the
libraries your project depends on, it will manage (install/update) them for you.Use the installer rather than pipinstalling-with-the-official-installer.poetryselfaddpoetry-bumpversionpoetry-V
Poetry(version1.2.0)Windows PathInstall poetry from powershell in admin mode.(Invoke-WebRequest-Urihttps://install.python-poetry.org-UseBasicParsing).Content|py-The path will beC:\Users\<YOURUSER>\AppData\Roaming\Python\Scripts\poetry.exewhich you will need to add to your system path.Windows GitBashWhen using gitbash you can setup an alias for the poetry command:aliaspoetry="\"C:\Users\<YOURUSER>\AppData\Roaming\Python\Scripts\poetry.exe\""Getting StartedpoetryupdatepoetryinstallDevelopmentThis project uses thehvacpython module and to develop locally you can run vault
as a docker service as detailed herelocal docker vault. For local development vault
setup follow theVAULTguide for information.Check your connection with the following command, note in development mode vault should not be sealed.exportVAULT_ADDR='http://localhost:8200'exportVAULT_TOKEN='8d02106e-b1cd-4fa5-911b-5b4e669ad07a'poetryrunagileupstatecheckRequired Environment State VariablesVariableDescriptionSIAB_IDUnique environment ID.SIAB_CLOUDCloud vendor API mnemonic.SIAB_LOCATION1Cloud vendor defined cloud location, uksouth, etc.SIAB_LOCATION2Cloud vendor defined cloud location, UK South, etc.SIAB_CONTEXTEnvironment context, e.g. dev. test, prod.SIAB_VALUES_PATHVault path to common environment values to be exported.SIAB_DOMAINOptional public domain that overrides cloud provided DNS names.SIAB_LOCATION: Azure has a different location string between "accounts" and "resources" and onlyuksouthis useful
to the automation, but we must also provideUK Southfor resource groups.FIXME: Needs to be verified.SIAB_VALUES_PATH: Rather than load variables into the delivery platform environment, there can be many, a better option
is to define a YML file that contains all the required common variables for a specific environment and have the user upload
that to vault. This application can then download the YML data file and convert it into an exports file that can be sourced
by the pipelines. These environment values that are exported can then be used by this project and other utilities such as
terraform, ansible and powershell.SIAB_DOMAIN: Cloud DNS services might in some cases provide a DNS domain that is not the same as the public internet
domain required by the project, for example server1.uksouth.cloudapp.azure.com might optionally be server1.meltingturret.io.username/password: These values are common across the environment, for example Ubuntu Azure images use ausername=azureuser,
and so it simplifies configuration if the same credentials are used for Linux and Windows environments running in Azure for
administration access, client administration access as well as PFX certificate files used on Windows for WinRM certificate
authentication. For AWSubuntuis the username for Ubuntu images the same approach can be taken there.Required environment inputs:These values should be setup in your CD platforms environment variables.exportSIAB_ID=001exportSIAB_CLOUD=armexportSIAB_LOCATION1=uksouthexportSIAB_LOCATION2="UK South"exportSIAB_CONTEXT=devexportSIAB_VALUES_PATH=siab-state/001-arm-uksouth-dev/siab-values/siab-values.ymlRequired values inputs (stored in vault pathSIAB_VALUES_PATH):connection:url:https://server1.meltingturret.io:5986username:azureuserpassword:mypasswordca_trust_path:siab-client/chain.meltingturret.io.pemcert_pem:siab-client/azureuser@meltingturret.io.pemcert_key_pem:siab-client/azureuser@meltingturret.io.keycloud:group_owner:Paul Gilligangroup_department:DevOpsgroup_location:uksouthRequired Supporting DataSome data is generated only once and thus can be uploaded to vault manually.Uploading values file:base64./siab-values.yml|vaultkvputsecret/siab-state/001-arm-uksouth-dev/siab-values/siab-values.ymlfile=-Uploading pfx files:base64./server1.meltingturret.io.pfx|vaultkvputsecret/siab-pfx/server1.meltingturret.io.pfxfile=-
base64./azureuser@meltingturret.io.pfx|vaultkvputsecret/siab-pfx/azureuser@meltingturret.io.pfxfile=-Uploading pki files:base64./chain.meltingturret.io.pem|vaultkvputsecret/siab-client/chain.meltingturret.io.pemfile=-
base64./azureuser@meltingturret.io.key|vaultkvputsecret/siab-client/azureuser@meltingturret.io.keyfile=-
base64./azureuser@meltingturret.io.pem|vaultkvputsecret/siab-client/azureuser@meltingturret.io.pemfile=-Provision Use CaseExample steps required for the Windows terraform provision use case shown below.agileupstatecloud-init--server-path=siab-pfx/ags-w-arm1.meltingturret.io.pfx--client-path=siab-pfx/devops@meltingturret.io.pfx
agileupstatecreatesource./siab-state-export.shterraforminit
terraformapply-auto-approve
agileupstatesaveExample steps required for the Linux terraform provision use case shown below.agileupstatecreatesource./siab-state-export.shterraforminit
terraformapply-auto-approve
agileupstatesaveDestroy Use CaseExample steps required for recovering system state use case shown below which might be for example to destroy an environment.agileupstateloadsource./siab-state-export.shterraforminit
terraformdestroy-auto-approveAnsible Use CaseExample steps required for the Windows ansible use case shown below.agileupstateloadsource./siab-state-export.shagileupstateinventory-windows--ca-trust-path=siab-client/chain.meltingturret.io.pem--cert-pem=siab-client/azureuser@meltingturret.io.pem--cert-key-pem=siab-client/devops@meltingturret.io.key
ansible-inventory-iinventory.ini--list
ansible-iinventory.ini"${TF_VAR_siab_name_underscore}"-mwin_pingExample steps required for the Linux ansible use case shown below.agileupstateloadsource./siab-state-export.shagileupstateinventory-linux
ansible-inventory-iinventory.ini--listANSIBLE_HOST_KEY_CHECKING=Trueansible-iinventory.ini--user"${TF_VAR_admin_username}""${TF_VAR_siab_name_underscore}"-mpingExports Use CaseTheymlfile fromSIAB_VALUES_PATHis exported to the filesiab-state-export.shwith the contents as shown in the
example below which can then be used by downstream utilities.exportSIAB_URL=https://server1.meltingturret.io:5986exportSIAB_USERNAME=azureuserexportSIAB_PASSWORD=mypasswordexportSIAB_CA_TRUST_PATH=siab-client/chain.meltingturret.io.pemexportSIAB_CERT_PEM=siab-client/azureuser@meltingturret.io.pemexportSIAB_CERT_KEY_PEM=siab-client/azureuser@meltingturret.io.keyexportTF_VAR_group_owner=PaulGilliganexportTF_VAR_group_department=DevOpsexportTF_VAR_group_location=UKSouthexportTF_VAR_admin_username=azureuserexportTF_VAR_admin_password=mypasswordexportTF_VAR_siab_name=001-arm-uksouth-devexportTF_VAR_siab_name_underscore=001_arm_uksouth_devsource./siab-state-export.shCloud Init Data Use CaseExample cloud init command that generates the zip file that is loaded onto Windows machines for WimRM certificate authentication.poetryrunagileupstatecloud-init--server-path=siab-pfx/ags-w-arm1.meltingturret.io.pfx--client-path=siab-pfx/azureuser@meltingturret.io.pfxAnsible Windows Inventory Use Caseinventory.iniis generated with the target(s) and configuration information for a successful SSH connection from Ansible.Whenexport SIAB_DOMAIN=meltingturret.io:[001_arm_uksouth_dev]ags-w-arm1.meltingturret.io[001_arm_uksouth_dev:vars]ansible_user=azureuseransible_password=heTgDg!J4buAv5kcansible_connection=winrmansible_port=5986ansible_winrm_ca_trust_path=chain.meltingturret.io.pemansible_winrm_cert_pem=azureuser@meltingturret.io.pemansible_winrm_cert_key_pem=azureuser@meltingturret.io.keyansible_winrm_transport=certificateAnsible Linux Inventory Use Caseinventory.iniis generated with the target(s) and configuration information for a successful SSH connection from Ansible.Whenexport SIAB_DOMAIN=meltingturret.io:[001_arm_uksouth_dev]ags-w-arm1.meltingturret.io ansible_ssh_private_key_file=vm-rsa-private-key.pemRunpoetryrunagileupstateLintpoetryrunflake8TestpoetryrunpytestPublishBy default we are usingPYPI packages.Create yourself an access token for PYPI and then follow the instructions.exportPYPI_USERNAME=__token__exportPYPI_PASSWORD=<YourAPIToken>
poetrypublish--build--username$PYPI_USERNAME--password$PYPI_PASSWORDVersioningWe useSemVerfor versioning. For the versions available, see thetags on this repository.ReleasingWe are usingpoetry-bumpversionto manage release versions.poetryversionpatchDependencyOnce the release has been created it is now available for you to use in other python projects via:pipinstallagileupstateAnd also for poetry projects via:poetryaddagileupstateContributingPlease readCONTRIBUTING.mdfor details on our code of conduct, and the process for submitting pull requests to us.LicenseThis project is licensed under the Apache License, Version 2.0 - see theLICENSEfile for details