Package: build-magic
build-magic 🔨✨An un-opinionated build automation tool.IntroductionBuild-magic is a command-line application for automating build/test/install/deploy tasks. It aims to provide a common means for automating tasks for developers and DevOps engineers. Build-magic does this by abstracting away how and where the commands are executed. Simply tell build-magic what commands to run and where to run them.Developing a C++ application for Linux on a Windows laptop? Build-magic can build your project in a Linux container or virtual machine. Have a dedicated build server? Build-magic can SSH into the remote server and start the build.Build-magic allows you to specify commands and the environment from the command-line or from a Config File.ContributingBuild-magic is still under active development, with new features being added regularly.Want to help make build-magic better? Pull Requests are welcome. Please review thecontributing guideon how you can contribute to build-magic.ExamplesArchive two files on a local Linux or Mac OS machine:>build-magictar-czfmyfiles.tar.gzfile1.txtfile2.txtThe following commands execute the same command on the local system, on a remote system, in a Docker container, and in a virtual machine:>build-magicechohelloworld

>build-magic--runnerremote-euser@myhostechohelloworld

>build-magic--runnerdocker-ealpine:latestechohelloworld

>build-magic--runnervagrant-eVagrantfileechohelloworldDesignBuilding software is hard enough, and managing shell scripts, documentation, and environments across a project for multiple users and operating systems can become a pain. Build-magic was designed to be the automation tool that works for you. It aims to do for builds what Ansible has done for installs, and Docker has done for deployment. It's time to ditch the shell scripts and complex Makefiles.These are the three features that distinguish build-magic:Un-opinionatedAs much as possible, build-magic strives to setup environments and execute commands the same way, whether commands are being executed locally, remotely, in a container, or in a VM. Also, unlike similar automation tools that have builtin directives, build-magic command directives don't impart any special meaning or track state -- they are simply descriptive. This keeps things simple, and what you see in a build-magic config file or in command-line arguments is exactly what's executed. Aside from the simple syntax, build-magic doesn't care how you run commands.DeclarativeBuild-magic uses a simple syntax that lets you describe what commands to run, the environment to use, and what files to include. There's no need for conditional statements that you might need when automating with a shell or batch script. Build-magic will take care of the details and execute the given commands in the specified environment.ReproducibleBecause build-magic lets you execute commands in a container or virtual machine, build-magic jobs are reproducible and portable across systems. This solves the"Well it works on my machine"problem.InstallationMacOSBuild-magic can be installed with Homebrew using the following commands:>brewtapcmmorrow/build-magic
>brewinstallbuild-magicWindowsYou can find an installer for Windows 10 and 11here.LinuxYou can find instructions for installing build-magic for Fedora based and Debian based distros from the build-magic Linux installationinstructions.PythonIf you have Python3 installed, the recommended way to install build-magic is into an isolated environment viapipx.>pipxinstallbuild-magicAlternatively, you can install build-magic from pypi into a virtual environment. Build-magic requires Python 3.6+. You can install build-magic with the following command:>pipinstallbuild-magicWarning: It isn't recommend to usepip install build-magicto install into a base Python environment. This is because build-magic uses pinned dependencies that might cause conflicts with package versions already installed in the base Python environment.Optional InstallsTo use the Docker command runner, Docker must be installed. You can find out more about installing Dockerhere.To use the Vagrant command runner, Vagrant must be installed. You can find out more about installing Vagranthere.To use the Remote command runner, OpenSSH must be installed. If you are using Linux or MacOS, OpenSSH is likely already installed. If you are using Windows 10+ OpenSSH can be enabled followingtheseinstructions.DocumentationYou can view the documentation athttps://cmmorrow.github.io/build-magic/.UsageBuild-magic can execute arbitrary shell commands from its command-line interface or by specifying commands in a Config File.Command-lineHere is a basic example of how to use build-magic from the command-line:>build-magic"echo hello world > hello.txt"Multiple commands can be executed as a sequence by using the--commandoption:>build-magic\--verbose\--commandexecute"echo hello >file1.txt"\--commandexecute"echo world >file2.txt"\--commandbuild"tar -czf myfiles.tar.gz file1.txt file2.txt"\--commandexecute"rm file1.txt file2.txt"\--commandinstall"tar -xzf myfiles.tar.gz"\--commandtest"cat file1.txt file2.txt"\--commandexecute"rm file1.txt file2.txt"\--commandexecute"rm myfiles.tar.gz"When using the--commandoption, a directive that describes the command must be provided. The directiveexecutecan be used as a catch-all for describing any command. The--verboseoption will print anything written to stdout by an individual command.To run the commands in a Linux container, add the--runnerand--environmentoptions:>build-magic\--runnerdocker\--environmentubuntu:latest\--verbose\--commandexecute"echo hello > file1.txt"\--commandexecute"echo world > file2.txt"\--commandbuild"tar -czf myfiles.tar.gz file1.txt file2.txt"\--commandexecute"rm file1.txt file2.txt"\--commandinstall"tar -xzf myfiles.tar.gz"\--commandtest"cat file1.txt file2.txt"\--commandexecute"rm file1.txt file2.txt"\--commandexecute"rm myfiles.tar.gz"Runbuild-magic --helpfor a list of all the available command-line options.Config FileSequences of commands can be grouped into stages and executed from a Config File written in YAML. The contents of the Config Fileconfig.yamlwill describes the same steps broken up into stages:build-magic:-stage:name:Setuprunner:dockerenvironment:ubuntu:latestcommands:-execute:echo hello > file1.txt-execute:echo world > file2.txt-stage:name:Archiverunner:dockerenvironment:ubuntu:latestcommands:-build:tar -czf myfiles.tar.gz file1.txt file2.txtlabel:Archive the files as myfiles.tar.gz-execute:rm file1.txt file2.txtlabel:Delete the original files-stage:name:Un-Archiverunner:dockerenvironment:ubuntu:latestcommands:-install:tar -xzf myfiles.tar.gzlabel:Extract the archive to the current directory-stage:name:Resultrunner:dockerenvironment:ubuntu:latestcommands:-test:cat file1.txt file2.txt-execute:rm file1.txt file2.txtlabel:Cleanup the files-execute:rm myfiles.tar.gzlabel:Cleanup the archiveNotice how optional labels can be added to each command that will be printed as part of the build-magic output instead of the corresponding command.You can execute Config Fileconfig.yamlwith:>build-magic--configconfig.yaml--verboseA single stage in a Config File can be run with:>build-magic--configconfig.yaml--targetSetupMultiple stages can be executed this way, even in a different order:>build-magic--configconfig.yaml--targetSetup--targetResultCode reuse with anchorsYAML anchors and aliases can be used to reuse repeated values. In the Config File above, the commandrm file1.txt file2.txtis used more than once, and the same runner and environment is used for each stage. These values can be replaced with anchors and called with aliases. YAML anchors can be defined in aprepare:section of the Config File:prepare:-&delete_filesrm file1.txt file2.txt-&runnerdocker-&environmentubuntu:latestbuild-magic:-stage:name:Setuprunner:*runnerenvironment:*environmentcommands:-execute:echo hello > file1.txt-execute:echo world > file2.txt-stage:name:Archiverunner:*runnerenvironment:*environmentcommands:-build:tar -czf myfiles.tar.gz file1.txt file2.txtlabel:Archive the files as myfiles.tar.gz-execute:*delete_fileslabel:Delete the original files-stage:name:Un-Archiverunner:*runnerenvironment:*environmentcommands:-install:tar -xzf myfiles.tar.gzlabel:Extract the archive to the current directory-stage:name:Resultrunner:*runnerenvironment:*environmentcommands:-test:cat file1.txt file2.txt-execute:*delete_fileslabel:Cleanup the files-execute:rm myfiles.tar.gzlabel:Cleanup the archiveThe execution of this Config File is identical to the previous one.Environment VariablesEnvironment variables cannot be set by build-magic commands. However, they can be used by build-magic commands by using the--envoption followed by the variable name and value:>build-magic--envFOOhello--envBARworldecho'$FOO''$BAR'Environment variables can also be used in Config Files:build-magic:-stage:environment variables:FOO:helloBAR:worldcommands:-execute:echo $FOO $BARAlternatively, a dotenv file can be passed to set multiple environment variables at once:>build-magic--dotenvhello.envecho'$FOO''$BAR'As well as in a Config File:build-magic:-stage:dotenv:hello.envcommands:-execute:echo $FOO $BARRuntime VariablesBuild-magic supports variable substitution in Config Files that works similar to a Jinja template.build-magic:-stage:commands:-execute:>curl -u {{ user }}:{{ password }}https://myrepo/myproject/{{ version }}In the Config File above, theuser,password, andversionruntime variables are declared and can be substituted at runtime using the following command:>build-magic-vuservision-vpasswordwanda-vversion12-Cconfig.yamlSince it isn't a good idea to provides secrets in clear text that can appear on screen or in command history, the--promptoption can be used to interactively prompt the user for the value of the variable without printing it to the screen.>build-magic-vuservision--promptpassword-vversion12-Cconfig.yaml
>password:Including FilesFiles can be copied from another directory into the current working directory by passing the directory to copy from with the--copyoption and providing the files to copy as arguments:>build-magic\--copymyproject/src\-cexecute"go build main.go"\audio.goequalizer.goeffects.goCopying files is handled similarly in a Config File:build-magic:-stage:copy from directory:myproject/srcartifacts:-audio.go-equalizer.go-effects.gocommands:-execute:go build main.go