Package: ebbs
Eons Basic Build SystemEBBS (or ebbs) is a framework for designing modular build pipelines for any language and system. Builders are python scripts that are downloaded and run on the fly with configuration provided by json config files, environment variables, and command line arguments!Here, at eons, we have found building and distributing code to be too hard and far too disparate between languages. Thus, we designed ebbs to make packaging and distributing code consistent. No matter what language you're working with and no matter what you want to do with your code.Want to compile C++ in a containerized environment then publish that code as an image back to Docker? How about publish your python code to PyPI or even just build a simple Wordpress plugin? No matter how intricate or big your project becomes, you'll be able to rely on ebbs to automate every step of your build process. It's just python. That's literally it.With ebbs, there will be no more:having to learn a new way to package code for every language.having to change your code to fit your build system.having to specify unnecessary configuration for every project.Instead, you write your code the way you want and your ebbs build system will put the pieces together for you.
Ebbs has been written in adherence to theeons naming conventionsandeons directory conventions. However, we do try to make overriding these conventions as easy as possible so that you don't have to change your existing code to use our systems.For example, if you use "include" instead of the eons-preferred "inc", you can tell ebbs:"copy":[{"../include":"inc"}]In the same fashion, you can bypass the eons directory scheme ("my_project.exe", "my_project.lib", "my_project.img", etc.) by specifying"name" : "my_project"and"type" : "exe"or whatever you'd like.If you find ebbs to be getting in the way or overly challenging, let us know! Seriously, building code should be easy and we're always happy to know how we can improve. Feel free to open issues or just email us atsupport@eons.llc.Installationpip install ebbsUsageEbbs must be invoked from the directory you wish to build from.
For example, a well-designed project should allow you compile it locally by:cdbuild
ebbsPer (eons)[https://github.com/eons-dev/eons.lib], ebbs supports:-vor--verbose(count, i.e-vv= 2) or--verbosity #, where # is some number, or theverbosityenvironment or config value: will show more information and increase the logging level, e.g. print debug messages (3 for debug; 2 for info).--configor-c(string): the path to a json config file from which other values may be retrieved.--no-repoor theno_repoenvironment or config value (bool, i.e. 'True', 'true', etc.): whether or not to enable reaching out to online servers for code (see Dynamic Functionality, below).--log-fileor thelog_fileenvironment or config value (string; supports formatting, e.g. '/var/log/eons/{this.name}.log'): optional value for logging to a file in addition to stderr.As always, useebbs --helpfor help ;)ConfigurationEbbs is intended to keep your build process separate from your code. With that said, it can be useful to specify some project-wide settings and build configurations.
Note that there isn't any real reason you can't move the build.json or even write an ebbs script to generate build.json and then call ebbs with it ;)When running ebbs, the builder you select will pull its configuration values from the following external sources:the command line (e.g. in case you want to override anything)a "build.json" in the provided build folder (which can be specified via--config)a json file provided toebbs.the system environment (e.g. for keeping passwords out of repo-files and commands)
Any existing member variables will override these external values.You can manually specify the builder you'd like in one of 2 ways:the-bargument to ebbs."build" : "SOMETHING"in the build.jsonLastly, you can specify a build folder (i.e. a folder to create within your project for all build output) with:-ion the cli; the default is "build" (e.g. "/path/to/my/project/build")"build_in" : "BUILD_FOLDER"in the build.jsonYou can also specify any number of other arguments in any of the command line, build.json, and system environments.
For example,export pypi_username="__token__"would makethis.Fetch('pypi_username)in the "py" Builder return__token__, assuming you don't set"pypi_username" : "something else"in the build.json nor specify--pypi-username "something else"on the command line.IMPORTANT NOTE: Most ebbs Builders will DELETE the build folder you pass to them.This is done so that previous builds cannot create stale data which influence future builds. However, if you mess up and call, say,ebbs -b cppfrom./srcinstead of from./build, you could lose your "src" folder. Please use this tool responsibly and read up on what each Builder does.
To make things easy, you can search forclearBuildPathin the python file andclear_build_pathin the config files. If you seethis.clearBuildPath = Falseit should be okay to use that Builder with any directory (such is the case for the Publish Builder, which zips the contents of any directory and uploads them to an online repository). Conversely, take note of where"clear_build_path": trueis set.Where Are These "Builders"?All Builders are searched for in the local file system from where ebbs was called within the following folders:"./eons"#per the eons.Executor.defaultRepoDirectoryIf the build you specified is not found within one of those directories, ebbs will try to download it from the remote repository with a name of{builder}.build. The downloaded build script will be saved to whatever directory you set in--repo-store(default "./eons/").Unfortunately, python class names cannot have dashes ("-") in them. Instead, a series of underscores (_) is often used instead. While this deviates from the eons naming schema, it should still be intelligible for short names. You are, of course, welcome to use whatever naming scheme you would like instead!The Build PathAs mentioned, ebbs depends on the directory it is invoked from. TherootPathprovided to each Builder will be this directory. ThebuildPathis specified relative to therootPath. If you would like to use a single folder for all Builders, please set therepo_storeenvironment variable with an absolute path.For example, if you have a "git" and a "workspace" folder in your home directory and you want to use your custom Builder, "my_build" on all the projects in the git folder, instead of copying my_build to every project's workspace, you could simplyexport repo_store="~/workspace"and call ebbs from the appropriate build directory for each project.
Something like:me@mine:~/git/my_cpp_project.exe/build$ ebbs -b my_build. NOTE: if the build.json file~/git/my_cpp_project.exe/build/build.jsonexists, it will affect the behavior ofmy_buildand potentially even effect other Builders. To ensure no side-effects are generated from project build configurations, you should create an empty folder to invoke your custom build process from (e.glocal/).Your home folder would then look something like:home/
├─ git/
│  ├─ my_cpp_project.exe/
├─ workspace/
│  ├─ my_build.pyRepositoryOnline repository settings can be specified with:--repo-store (default = ./eons/)
--repo-url (default = https://api.infrastructure.tech/v1/package)
--repo-username
--repo-passwordNOTE: you do not need to supply any repo credentials or other settings in order to download packages from the public repository.For more info on the repo integration, seethe eons libraryIt is also worth noting that the online repository system is handled upstream (and downstream, for Publish) of ebbs.By default, ebbs will use theinfrastructure.techpackage repository. See theInfrastructure web serverfor more info.IMPORTANT CAVEAT FOR ONLINE PACKAGES:the package name must be suffixed with the "build" extension to be found by ebbs.For example, if you want to use-b my_buildfrom the repository, ebbs will attempt to download "my_build.build". The package zip (my_build.build.zip) is then downloaded, extracted, registered, and instantiated.All packages are .zip files.Example Build Scripts:PythonC++DockerPublish<- this one makes other Builders available online.In Container<- this one moves the remaining build process into a docker container.Arbitrary<- this one just runs commands.Proxy<- this one loads another json config file.Test<- this one runs commands and validates the outputs.Cascading BuildsAs with any good build system, you aren't limited to just one step or even one file. With ebbs, you can specify "next" in your build.json (see below), which will execute a series of Builders after the initial.Here's an example build.json that builds a C++ project then pushes it to Dockerhub (taken from theInfrastructure web server):{"clear_build_path":true,"next":[{"build":"in_container","config":{"image":"eons/dev-webserver.img","copy_env":["docker_username","docker_password"],"next":[{"build":"cpp","build_in":"build","copy":[{"../../inc/":"inc/"},{"../../src/":"src/"}],"config":{"file_name":"entrypoint","cpp_version":17,"libs_shared":["restbed","cpr"],"next":[{"build":"docker","path":"infrastructure.srv","copy":[{"out/":"src/"}],"config":{"base_image":"eons/webserver.img","image_name":"eons/infrastructure.srv","image_os":"debian","entrypoint":"/usr/local/bin/entrypoint","also":["EXPOSE 80"]}}]}}]}}]}This script can be invoked with justebbs(assuming the appropriate docker credentials are stored in your environment, docker is installed, etc.).For other examples, check out thebuildfolder of this repo and any other mentioned above!DesignI Want One!Before diving too deep into EBBS, please also give a quick look at the parent library:eons.
TheUserFunctor Utilitieswill be of particular use in your Builders.Ebbs builds packages or whatever withebbs.Builders, which extend the self-registeringeons.UserFunctor. This means you can write your own build scripts and publish them, distribute them with your code, or store them locally in therepo_store(see above). A major driving force behind ebbs is to encourage you to share your automation tools with colleagues, friends, and enemies! For example, you could create "my_build.py", containing something like:importloggingfromebbsimportBuilderclassmy_build(Builder):def__init__(this,name="My Build"):super().__init__(name)# delete whatever dir was provided to this, so we can start fresh.this.clearBuildPath=Truethis.supportedProjectTypes=[]#all#or# this.supportedProjectTypes.append("lib")# this.supportedProjectTypes.append("exe")# this.supportedProjectTypes.append("test")#this.requiredKWArgs will cause an error to be thrown prior to execution (i.e. .*Build methods) iff they are not found in the system environment, build.json, nor command line.this.requiredKWArgs.append("my_required_arg")#this.my_optional_arg will be "some default value" unless the user overrides it from the command line or build.json file.this.optionalKWArgs["my_optional_arg"]="some default value"#Check if the output of all your this.RunCommand() and whatever other calls did what you expected.#The "next" step will only be executed if this step succeeded.defDidBuildSucceed(this):returnTrue;#yeah, why not?defPreBuild(this):logging.info(f"Got{this.my_required_arg}and{this.my_optional_arg}")#Required Builder method. See that class for details.defBuild(this):#DO STUFF!That file can then go in a "./ebbs/" or "./eons/" directory, perhaps within your project repository or oninfrastructure.tech!
ebbs can then be invoked with something like:ebbs -b my_build --my-required-arg my-value, which will run your Builder in the current path!Also note the "--" preceding "--my-required-arg", which evaluates to "my_required_arg" (without the "--" and with "_" in place of "-") once in the Builder. This is done for convenience of both command line syntax and python code.You could also do something like:cat<< EOF > ./build.json{"my_required_arg" : "my-value","my_optional_arg" : ["some","other","value","that","you","don't","want","to","type"]}EOFebbs-bmy_buildHere, the build.json file will be automatically read in, removing the need to specify the args for your build.If you'd like to take this a step further, you can remove the need for-b my_buildby specifying it under an empty builder in the build.json, like so:cat<< EOF > ./build.json{"next": [{"build" : "my_build","build_in" : "build","copy" : [{"../src/" : "src/"},{"../inc/" : "inc/"},{"../test/" : "test/"}],"config" : {"my_required_arg" : "my-value","my_optional_arg" : ["some","other","value","that","you","don't","want","to","type"]}}]}EOFebbs#no args needed!Regardingthis.clearBuildPath, as mentioned above, it is important to not call ebbs on the wrong directory. If your Builder does not need a fresh build path, setthis.clearBuildPath = False.
With that said, most compilation, packaging, etc. can be broken by stale data from past builds, so make sure to setthis.clearBuildPath = Trueif you need to.You may also have noticed the combination of camelCase and snake_case. This is used to specify builtInValues from user_provided_values. This convention may change with a future release (let us know what you think!).ForsupportedProjectTypes, theBuilderclass will split the folder containing the buildPath (i.e. therootPath) on underscores (_), storing the first value asthis.projectTypeand the second asthis.projectName. TheprojectTypeis checked against the used build'ssupportedProjectTypes. If no match is found, the build is aborted prior to executing the build. If you would like your Builder to work with all project types (and thus ignore that whole naming nonsense), setthis.supportedProjectTypes = [], where none (i.e.[], not actuallyNone) means "all".You'll also get the following paths variables populated by default:this.srcPath=f"{this.rootPath}/src"this.incPath=f"{this.rootPath}/inc"this.depPath=f"{this.rootPath}/dep"this.libPath=f"{this.rootPath}/lib"this.exePath=f"{this.rootPath}/exe"this.testPath=f"{this.rootPath}/test"When aBuilderis executed, the following are called in order:(kwargs is the same for all)this.ValidateArgs()# <- not recommended to override.this.BeforeFunction()# <- virtual (ok to override)#Builder sets the above mentioned variables herethis.PreBuild()# <- virtual (ok to override)#Supported project types are checked herethis.Build()# <- abstract method for you  (MUST override)this.PostBuild()# <- virtual (ok to override)if(this.DidBuildSucceed()):this.BuildNext()this.AfterFunction()# <- virtual (ok to override)