Package: deterministic-zip-go
deterministic-zipSimple (almost drop-in) replacement for zip that produces deterministic files.Featuresdropin for zipremove all metadata from files addedimmutable zip utilInstallationAutomatic installbash<(curl-sShttps://raw.githubusercontent.com/timo-reymann/deterministic-zip/main/installer)ManualLinux (64-bit)curl-LOhttps://github.com/timo-reymann/deterministic-zip/releases/download/$(curl-Lso/dev/null-w%{url_effective}https://github.com/timo-reymann/deterministic-zip/releases/latest|grep-o'[^/]*$')/deterministic-zip_linux-amd64&&\chmod+xdeterministic-zip_linux-amd64&&\sudomvdeterministic-zip_linux-amd64/usr/local/bin/deterministic-zipDarwin (Intel)brewbrewtaptimo-reymann/deterministic-zip
brewinstalldeterministic-zipmanualcurl-LOhttps://github.com/timo-reymann/deterministic-zip/releases/download/$(curl-Lso/dev/null-w%{url_effective}https://github.com/timo-reymann/deterministic-zip/releases/latest|grep-o'[^/]*$')/deterministic-zip_darwin-amd64&&\chmod+xdeterministic-zip_darwin-amd64&&\sudomvdeterministic-zip_darwin-amd64/usr/local/bin/deterministic-zipInstall with gogoget-ugithub.com/timo-reymann/deterministic-zipDockerPlease check theContainerized section in Usagefor
more details.Supported platformsThe following platforms are supported (and have prebuilt binaries /
ready to use integration):Linux32-bit64-bitARM 64-bitARM 32-bitDarwin64-bitARM (M1/M2)WindowsARM32-bit64-bitFreeBSD32-bit64-bitARM 64-bitARM 32-bitOpenBSD32-bit64-bitOCI compatible container engines (Docker, podman etc)ARM64-bitCircleCIGitHub ActionsWhere to find the latest release for your platformBinariesBinaries for all of these can be found on thelatest release page.DockerFor the docker image check thedocker hub.CI ProviderCircleCI orbGitHub ActionUsageCommand LineIf you installed the binary via Releases, Install-Script or using go you
can just run deterministic-zip as a command.deterministic-zip-hContainerizedPlease be aware that the image contains just the binary, no OS, libs or
anything else. It also runs as root to be able to zip files no matter
the ownership, feel free to build your own images based on that as well.Using the container directlyIf you want to use the tool on a platform not supported yet or dont want
to install the tool locally you can also mount your folder in/workspacewhich is the default working directory. Than you can just
execute commands as you want to.dockerrun-v$PWD:/workspacetimoreymann/deterministic-zip:latestIntegrating into your CI imageIf you want to integrate the tool directly into your build image, you
can also utilize the auto updates from tools like renovatebot or
dependabot. Using docker built in features you can just get the binary
directly from the image.FROMbase-image:tag# do your customizationsCOPY--from=timoreymann/deterministic-zip:latest/deterministic-zip/usr/bin/deterministic-zipMotivationWhy another zip-tool? What is this deterministic stuff?!When we are talking about deterministic it means that the hash of the zip file won't change unless the contents of the
zip file changes.This means only the content, no metadata. You can achieve this with zip, yes.The problem that still remains is that the order is almost unpredictable and zip is very platform specific, so you will
end up with a bunch of crazy shell pipelines. And I am not even talking about windows at this point.So this is where this tool comes in, it is intended to be a drop-in replacement for zip in your build process.The use cases for this are primary:Zipping serverless codeBackups or other files that get rsyncedWant to know more about the topic of deterministic/reproducible builds?I can recommend the following resources:reproducible-builds.orgDebian WikiDocumentationHow reliable is it?Of course, it is not as reliable as the battle-proven and billions of times executed zip.Even though I am heavily relying on the go stdlib this software can of course have bugs. And you are welcome to report
them and help make this even more stable. Of course there will be tests to cover most use cases but at the end this is
still starting from scratch, so if you need advanced features or just dont feel comfortable about using this tool don't
do it!Differences between zip and deterministic-zipPlease seedocs/differencesContributingI love your input! I want to make contributing to this project as easy and transparent as possible, whether it's:Reporting a bugDiscussing the current state of the configurationSubmitting a fixProposing new featuresBecoming a maintainerTo get started please read theContribution Guidelines.DevelopmentRequirementsGoGNU makeTestmaketest-coverage-reportBuildmakebuildAlternativesAs far as I know the following (GitHub) projects exist:bboe/deterministic_zip (Python)You must list files explicitlyChanged order -> changed zipYou will need to install Python (no problem on Linux/Mac) and the packagebitgenics/deterministic-zip (NodeJS/JavaScript)Support for globs and ignores orderYou need to install node.js, the package, and it has no cli interfaceorf/deterministic-zip (Rust)has prebuilt binaries for all relevant platforms (and other can be built easily)very basic, but you can customize compression (nice feature)All in all they are just simply not what I needed. My favourite is Rust, because its just simply dropping in a binary.
Something that's very convenient especially when it comes to Docker builds.The main problem that all these solutions share is that it in my opinion cool things like excluding patterns, that I
regularly use are simply not implemented, and i REALLY love glob patterns.CreditsThis whole project wouldnt be possible with the great work of the
following libraries:glob by gobwaspflag by spf13go stdlib