Package: git-heimdall
Git-Heimdall is a guardian/gatekeeper tool for scanning sensitive data before
committing files to github. It also provides functionality for creating
configuration files from existing template files.ContentsTable of Contents:ContentsWhat is Git-Heimdall?Why Git-Heimdall?InstallationMore about Git-Heimdall secretfyDevelopmentHow to UseConfig template file samplesFAQSupportLicenseWhat is Git-Heimdall?Git-Heimdall is a tool for scanning sensitive data before the staged files are
added to a commit. So, if a developer makes some changes in his repository and
tries to commit those changes viagit commit, those files will be first
scanned byGit-Heimdallto check whether there are any sensitive data in those
changes and informs developer about it.This process is automated. You don’t have to run extra commands in order to scan those files. Infact those files are scanned automatically when git commit is called.Why Git-Heimdall?Committing sensitive data has been one of the most common vulnerabilities in
security world. Developers commit sensitive data and expose it to internet
without even realising about it.This is where Git-Heimdall comes in. Git-Heimdall provides a set of functionalities like :-Scanning files before commit to track secretsKeep configuration files in template format so that there are less chances
of everytime modifying the config file and committing of secrets.(More about
Git-Heimdall secretfy)InstallationThis section provides quick steps of how to setup this tool.Create a virtual Python environment and install Git-heimdall in it.python3-mvenvvheimdall.vheimdall/bin/activatepip3installgit-heimdallheimdall-iRun Sanity testheimdallsecretfy-mThe above command creates mock templates, secrets file at/tmp/git-heimdalldirectory. The-cor--configoption is for providing your config.yaml file.Register current project with git-heimdallheimdall-a<absolute-path-of-project>The above command registers your project with heimdall scans. Next time when
you try to commit staged files, it would start scanning those files before
it gets added to commit and pushed to github.More about Git-Heimdall secretfygit-heimdall providessecretfyoption for generating config files
dynamically from your template files. The templates are nothing but
configuration files, which holds your configuration in mustache format.secretfytool generates the required configuration file with help of secrets
file which would contain the real values required for actual config/properties
file.Let’s just say you have a set of configuration which you keep in a file
config.yaml, config.json, application.properties etc. These configuration might
have some highly sensitive information required to execute your project like
your user credentials, email, phone number, private key etc. Everytime, in your
development process, you need to add these sensitive values to the config file
and remove them before committing the code into github.This process is pretty painful and often you endup committing one or the other
sensitive information to git.So, instead of having a config file, you can have a template which resembles
your config file. Now before executing your project. All you need to do is
generate the desired config file with the help of this tool and then follow the
usual approach of running the project. The best part is that you don’t have to
worry about accidently commit the actual config file to the git repo. That file
won’t be shown in git status unless you forcibly add it.DevelopmentThis section describes how to set up a development environment for
Git-Heimdall. This section is useful for those who would like to contribute to
Git-Heimdall or run Git-Heimdall directly from its source.We use primarily three tools to perform development on this project: Python 3,
Git, and Make. Your system may already have these tools. But if not, here are
some brief instructions on how they can be installed.On macOS, if you haveHomebrewinstalled, then these tools can be be installed easily with the following command:brewinstallpythongitOn a Debian GNU/Linux system or in another Debian-based Linux distribution, they can be installed with the following commands:apt-getupdateapt-getinstallpython3python3-venvgitmakeOn any other system, we hope you can figure out how to install these tools yourself.Clone the project repository and enter its top-level directory:gitclonehttps://github.com/sunnysharmagts/git-heimdallcdgit-heimdallCreate a virtual Python environment for development purpose:makevheimdalldepsThis creates a virtual Python environment at~/.vheimdall/git-heimdall.
Additionally, it also creates a convenience script namedvheimdallin
the current directory to easily activate the virtual Python
environment which we will soon see in the next point.To undo this step at anytime in future, i.e., delete the virtual
Python environment directory, either enterrm-rfvheimdall~/.vheimdall/.Activate the virtual Python environment:../vheimdallIn the top-level directory of the project, enter this command:python3-mheimdall-iThis initializes git-heimdall tool. This is just aone time processand
need not be run everytime, unless if there is any change in the template
resources. This command just updates in the location of the git templateDir
in git configuration.python3-mheimdallsecretfy-mThis generates mock data at/tmp/git-heimdall. This step serves as a
sanity check that ensures that the development environment is correctly set
up. Also, it gives a brief idea of how to create a config in form of
template.Now to simulate the environment and test the tool. Do the following:-.~/.vheimdall/git-heimdall/bin/activatemkdir/tmp/git-heimdall-tool-testcd/tmp/git-heimdall-tool-testecho-n"print('<insert-some-sensitive-value>')">>sample.pygitinitgitadd.gitcommit-m"Sample commit"This will start scanning the sample.py file and will provide you with the sensitive data that you have in this file. Add more files and play with the tool to familiar yourself.How to UseThis section provides samples of how to use this tool.heimdallis initialized once after installation. That can be done for the following :-heimdall-iAfter initialization, any new repository created or cloned will be scanned by
heimdall once any staged files are committed.You can register existing repositories to heimdall to integrate scanning process.heimdall-reg<absolute-path-of-repo>heimdall secretfyconsist of 3 components :-Secrets file- This file can be in yaml, json and xml format.Template files- These files are configuration files in template format. For
eg:- If you have a fileconfig.jsonthen your template file will beconfig.json.mustache.Extension- This is the file extension of your configuration file. Following
are the example config files and their respective extension.a.config.yaml:yamlb.config.xml:xmlc.config.json:jsond.config.properties:propertiesThese parameters can be added to abaseconfig.yamlfile in the following wayheimdall:secret:res/secrets.yamltemplates:-file:res/example.yaml.mustacheextension:yaml-file:res/example.json.mustacheextension:json-file:res/example.xml.mustacheextension:xmlThebaseconfig.yamlfile starts withheimdalltag.1.secretis the absolute path of the secrets file containing sensitive
values.2.templatestag is an array of template files. All these files are in.mustacheformat whose sensitive values resides insecrets.yamlfile.fileis the absolute path of the template file.extensionis the extension of the configuration file which will be
generated from the template file.NOTE: Make sure that the template file are in<file_name>.<extension>.<mustache>format.Run the following command to generate the config files.heimdallsecretfy-cbaseconfig.yamlThis will create config files in the respective directories. Note that these
configurations won’t be seen in git history. You can check that by doinggit status.Config template file samplessecrets.yamlsecrets:item:val1:foo@bar.comval2:my_passworditem1:val3:usernameval4:my_private_keyexample.yaml.mustachesecrets:item:val1:{{secrets.item.val1}}val2:{{secrets.item.val2}}result:Thisisjustadummydescription.item1:val3:{{secrets.item1.val3}}val4:{{secrets.item1.val4}}result:Thisisanotherdummydescription.Thesecrets.yamlfile contains the sensitive information andexample.yaml.mustacheis the template file which contains the keys inmustacheformat. Hence the keysecrets.item.val2has valuemy_passwordwhich will be populated viaheimdall secretfytool.NOTE: You can run `heimdall secretfy-m`to get more sample baseconfig, templates, secret files. These files will get generated at`/tmp/git-heimdall`.FAQHow can i deploy my code in CICD pipeline or on remote server since it
doesn’t have config files and needs to be generated.You can generate all the config files required for your repository to compile
and run in CICD pipeline or at remote server by the following command.heimdallsecretfy-emustache-s<secrets_file_path>-r<repository_path>-eis the template extension,-sis the absolute path of the secrets file
and-ris absolute path of the repositorySupportTo report bugs, suggest improvements, or ask questions, please create a
new issue athttps://github.com/sunnysharmagts/git-heimdall/issues.LicenseThis is free software. You are permitted to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of it, under the
terms of the MIT License. SeeLICENSE.mdfor the complete license.This software is provided WITHOUT ANY WARRANTY; without even the implied
warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. SeeLICENSE.mdfor the complete disclaimer.