Package: hashkern
Hash kernHash kern is the core of the Hash platform, it allows you to define your resources in your Mono Repository and
it can be used to manage building, testing, publishing, and deploying those resources to your own environments,
Hash kern uses yaml files to define resources and their kinds along with the environments used in your project.
It allows you also to specify dependencies between resources and in many cases it can automatically detect
them. It uses a backend store for storing the state of your resources which include the hash of the resource's
code along with the results from running actions on the resource in an env.Hash kern achieves all of this using well designed kern package, Dependency Graph and a set of plugins for defining
resources, state storage backends and targets for executing some actions in enviornments (more about these later).Quick Architecture IntroductionThe following diagram shows the most important packages and modules in Hash kern and their interactionskern PackageIt contains modules for defining state, hash templates, actions, planning actions and executing them.
It is designed to be independent from resources and it interacts with resources and store packages to manage resources and store their state.Resources PackageIt contains the base class for all resources and it also defines the Environment resource, along with some built in resources and the targets used by resources to execute some actions in some environments.Store PackageThis package contains the base for all store plugins and some built in plugins, it is used to store the state which results from running actions on resources in its own store which can be local file, GCP buckets, Digital Ocean spaces, etc....DAG ModuleThis module defines the dependency graph as directed asyclic graph, it is used to create a plan for running action x on resource y in enviornment z, which includes running actions on all of the resource's dependencies in the right orderInstalling Hash kernInstall from PyPiHash kern is published to PyPihere, you can install it using this commandpipinstallhashkernThen you can use the CLI like this>haks
usage:hash[-h][--storageSTORAGE][--configCONFIG][--envENV][--planPLAN]{build,test,publish,deploy,hash,render,clear,version}...

Atooltobuildresourcesbasedontheirhashandtypepositionalarguments:{build,test,publish,deploy,hash,render,clear,version}optionalarguments:-h,--helpshowthishelpmessageandexit--storageSTORAGEThestoragesystemuseddefaultisLocalFile--configCONFIGTheconfigurationfiledefaultisconfig.ini--envENVAnenvironmenttoruntheactioninit--planPLANOnlydoaplanandsaveittothisfileInstall from sourceInstall the required packages to clone the code, create virtual env and run the tests with makesudoaptinstallgitmakepython3python3-venvpython3-aptNow clone the repository and cd into its directorygitclonehttps://gitlab.com/hash-platform/hashkern.gitcdhashkernCreate a virtual env and activate it with these commandspython3-mvenvvenvsourcevenv/bin/activateNow install hash kern in editable mode and all of its dependecnies with these commandspipinstall-e.
pipinstall-rrequirements.txtNow you can use the CLI located atsrc/client/main.pyYou can run tests with this commandmaketestUsing the clientHash kern comes with a CLI interface which exposes its main functionality, which is running an action on a resource
in an environment, you will see two directories in src calledcliandclient,cliis the client that used to interact
with old hash kern implemnation, it will be deleted soon after moving all of its features to new client inclientdirectory.To run the client first enable the virtual env with this command:sourcevenv/bin/activatePrint the help for the client with this command>pythonsrc/client/main.py

usage:hash[-h][--storageSTORAGE][--configCONFIG][--envENV][--planPLAN]{build,test,publish,deploy,hash,render,clear,version}...

Atooltobuildresourcesbasedontheirhashandtypepositionalarguments:{build,test,publish,deploy,hash,render,clear,version}optionalarguments:-h,--helpshowthishelpmessageandexit--storageSTORAGEThestoragesystemuseddefaultisLocalFile--configCONFIGTheconfigurationfiledefaultisconfig.ini--envENVAnenvironmenttoruntheactioninit--planPLANOnlydoaplanandsaveittothisfileThe client takes three options:--storagethis one is used to select a storage backend for the state, the options for storage backend are in the config file.--configthis option is used to select a config file for storage backends, it is an INI file, its default value isconfig.ini--envthis option is used to select the environment name where the action will be executed, its default value isNone, this
value is acceptabe for some actions on some resources and it is not acceptable for some other actions, the environment
must exist in the repository otherwise we get an error.--planthis option is used to only run the plan and save it to the file specified in the option, it defaults to false.The format for config file is as follows:[LocalFile]output=hash_test/storageorganization=hashioproject=hashHereLocalFileis the name of storage backend, and everything inside are options for the backend, these options are different
from one backend to the other except fororganizationandprojectwhich are required for all storage backends.For now they are used to select paths for the resources specific to one project in an organization, more about these will
be added later.The LocalFile backend requires one other option that isoutputdirectory for storing the state inside it.We currently have three sub-commands for the CLI, these arebuild,test,publishanddeployall of these
commands take one argument that is the path to the resource's file, they accept directory argument if the name
of the resource's file isresource.yaml, otherwise you need to use a path to the file itself.For example to build resource in pathservices/Ain environmentdevelopmentuse this commandpythonsrc/client/main.py--envdevelopmentbuildservices/AAssuming the name of resource's file isresource.yamlif it isresource.A.yamlthen use this commandpythonsrc/client/main.py--envdevelopmentbuildservices/A/resource.A.yamlMore features will be added later to the CLI according to our Road Map, and the old CLI code will be removed.DemosThe demos with their guides and code are availablehereHash kern principles and goalsAs with any successfull open source project, it must follow a strict set of rules and principles to keep it growing
and successfull, here at hash kern we have established a set of rules and principles to follow when developing, they
help us when discussing features and where to implement those features, in the kern, resources, store or dag etc...Simplicity is a tool to achieve something and not a target by itselfAt hash kern we use simplicity as a tool when implementing features, we DO NOT care about simplicity when planning
the features because we believe that if we refuse to have feature X because it is hard or complex then someone else
will do it in his own simple way and outsmart use, so we make sure that any needed feature for our goals and developers
is implemented in the best simple way no matter how hard/complex is the actual feature is.Trust your users and give them freedomHere we mean by users, the developers who will use hash kern to manage their own resources, these are the devs who
will write the resource yaml files and the code for their resources wether it is go code, terraform, yaml or Dockerfiles etc...
We SHOULD NOT judge the users or make decessions on their behalf. However, we have to help them to protect them selves from
any mistakes they may make, we will work to implement this later with project and organizational constraints.Hash kern should be independent from resourcesThe kern package should not contain anycode that is specific to one resource and all of its functionality must be tested
using a Fake resource. We should try to keep the kern as simple as possible and also well tested. However, it is sometimes
preferable to implement the complex features in the kern because this implementation is written only once and tested well
so resource plugins' developers and users can benefit from those trusted features in the kern and not have to implement
them on their own in their resources which might contain more bugs and be less tested.The kern is the best place to implement complex featuresAs we said in the previous point, when we have a complex feature we need to consider implemnting it in the kern, if this
feature could be usefull for many resources, the next is to implement the feature in the resources plugin itself, if this
feature cannot be useful for all resources, and lastly is to leave it for users to implement it in their own resources.Now we talked about our rules and principles, let's talk about our goalsHash kern should be able to manage your resources everywhere, wether in CI or on your local machineHash kern uses a state storage which can be shared among developers, this storage along with code hash
is used to determine which resources needs building, testing, publishing or deploying to run an action
on a single resource, so the input is the action, resource name and environment name, and the output is
a plan to run this action on this resoucre in this environment while respecting dependencies. Hash kern
doesn't need git data or a list of changed files to run just your resource code and the state storage,
determining what needs to be done given a list of changes in a Merge Request for example will be the
task of hash CI to be implemented later which will work along side with hash kern to give you the
best CI experience in a mono repository.You can have dynamic levels of abstractions using your resourcesPeople often ask about the right amount of abstraction, while we at hash kern would like to give you the freedom
to have dynamic levels of abstractions based on your resources and mabe later you can help us or help others
to get the right amount of abstraction. Your resources might be as simple as compiling a go code, generating
manifests and pushing a docker image all of your choice, or you can imbed the manifests and your Dockerfile
in the resources plugin's source code or even more implement creating the right storage buckets, databases
and granting permissions for your service in your plugin's source code, it is up to you to decide what your
developers need to worry about and manage them selves.DO NOT repeat the same action using the same inputs and state and expect a different resultsThat is the actual defenition of insanity, we try our best at hash kern not to re-run the actions on resources
if they are not needed, this is implemented by checking the hash of the resource, the hashes of deps and whether
this action was run before in this env or not if we spot any differences then we need to re-run the action. However,
sometimes the action must be re-run even if those conditions are not met such as changing the version of go compiler,
that's why we enable the resources to force re-running the action even if all of the previous conditions are False.DO NOT repeat the values that change at the same timeHash templates help us to re-use outputs from other resources in any resource, so we don't need to hard code this
output in our resources, this adds a dependency on the other resource.Road MapSo far Hash kern has the most basic features to manage your resources in a mono repository and it also contains
some built-in resources which help you to run your services and deploy the infrastructure needed for them, these
resources are still limited and very basic.Our Road Map is the fllowingImprove the built-in resources that we already have and add more of them.Improve the CLI output and options, add an option to do a plan only and save the plan to a file that can be executed later.Test the remote state storage backends for performance.Add logging, metrics and tracing to different hash kern components so we can debug better later.Test hash kern in real world use cases and add more features accordingly.Evaluate our graph implementation and add more tests to it.Contribution GuideWe highly welcome any contributions to Hash kern as it helps us to improve it and test it more before it is declared as production ready.You can check open issueshere, you can follow the installation guide
above to get sarted with development, hash kern is written in python so you can install it in development mode and get started
quickly, it is tested using python 3.7, 3.8, 3.9, 3.10 and 3.11.Make sure to install the latest version of python, create the virtual env and get started with one of the issues.You can run the test suite after you are finished implemneting the feature or fix using this commandmaketestRelease versionsWe use a tag based releases, the versions correspond to the tags.Semantic versioningis used to create tags, all
versions before the first stable release1.0.0might include backward incompatible changes.To create a new version you simply increase the version argument in setup.py, create a merge request to merge it with main
and then create a tag that corresponds to the new version number.TheCHANGLOG.mdcontains all changes between releases and their release dates.More detailed contribution guide will be shared later :)