Package: apt-repoman
Repoman: a scalable apt server using Amazon SimpleDB and S3IntroductionBack in 2011, a scrappy young startup called "Weatherbill" found itself with a
conundrum: we were using Debian'sdpkgandapttools to package and serve
binary software within our own infrastructure, but we were also starting to use
Amazon's Elastic Mapreduce to create ephemeral Hadoop clusters for geospatial
and weather data processing, and it turned out that spinning up a 2,000-node
Hadoop cluster and downloading several dozen gigabytes of packages from our
standalone apt server onto each Hadoop compute node at once was an excellent
way to make the apt server vanish in a puff of smoke.So with the optimism that comes from youth and too much caffiene, we wrote our
own apt server, based on Amazon's Simple Storage Service (S3) and SimpleDB
offerings.  We called it "Repoman", because the life of a repo man, much like
that of an engineer at a small startup, is always intense.  And since then,
Repoman has successfully served millions of packages to various servers and
services here at The Climate Corporation (which Weatherbill became) and we're
happy to finally get a chance to share it with you.RequirementsIn order to operate an apt repository with Repoman, you will need at
a minimum:A working Python 2.7 or 3.5 installationAn Amazon Web Services accountUser credentials in that AWS account, either in your shell environment,
a~/.aws/Credentialsfile, or via an EC2 Instance Profile or ECS Task
Role.  In general if theAWS CLIruns
successfully in your environment, Repoman should run.AnAWS SimpleDB Domainwhich Repoman will use to store metadata about your packages.AnAWS S3 Bucketwhich Repoman will use to store your actual.debpackage files and
also the generated metadata files that form an Apt repository.Your user credentials must have sufficient permissions (viaAmazon IAM)
to manipulate both the SimpleDB domain and the S3 bucket: you will need at a
minimum to be able to create and delete keys in both.  A sample IAM security
policy is provided inthe docs folderSome basic familiarity with the concepts and nomenclature of running an apt
server is assumed: if you don't know what distributions, components and
architectures are, you may want to reviewthe apt
documentationOptionalRepoman can function with your own user credentials, or it can assume an IAM
role before querying AWS APIs; see theinstallationsection
for details on how to use roles.Repoman cansign your apt repository's metadata filesin order to provide strong assurances that the packages in the repo
come from their claimed source. You will need a working GPG keyring,
and key management is not provided by Repoman.Commands, flags and helpTherepoman-clicli tool offers the following commands:setup-- initial setup of the repositorycheckup-- check that SimpleDB and S3 are configured correctlyadd-- add packages to the repositoryrm-- delete packages from the repositorycp-- copy packages from one distribution or component in the
repository to another distribution or complnentquery-- list packages in the repository based on filterspublish-- publish the current SimpleDB repository to state to S3backup-- backup the current SimpleDB state to a JSON filerestore-- restore SimpleDB state from a JSON filerepo-- repository management sub-commands:repo add-distribution-- add a distribution for the repo to serverepo rm-distribution-- remove a distribution for the repo to serverepo add-component-- add a component for the repo to serverepo rm-component-- remove a component for the repo to serverepo add-architecture-- add a architecture for the repo to serverepo rm-architecture-- remove a architecture for the repo to serverepo add-topic-- add an SNS topic to log notifications torepo rm-topic-- remove any configured SNS topicrepo show-config-- show the current repository configurationrepo add-origin-- set an Origin string for the published repositoryrepo add-label-- set a Label string for the published repositoryTherepoman-cliutility itself and all of its commands and sub-commands will
take a-hor--helpflag to show help text and all locally relevant flags.Most commands that mutate the repository will prompt for confirmation;
this step can be bypassed by passing the-yor--confirmflag.Some particularly dangerous commands (e.g. deleting an entire distribution's
worth of packages) will prompt for anextraconfirmation step; this too
can be bypassed by passing in the--i-fear-no-evilflag but this is astrictlyat-your-own-risk proposition.To automatically publish the repository to s3 after adding, copying or
removing packages, pass the--publishflag.Further DocumentationBasic operationsTheory of OperationInstallation, Configuration and SetupAdding binary and source packages to RepomanQuerying the repositoryPublishing and signing the repositoryCopying and promoting packages within the repositoryRemoving and purging pacakges within the repositoryAdvanced topicsAdding and removing sections (distributions, components, architectures) from the repositorySubscribing apt clients to the repositoryServing public repositories directly from S3Backups and restoresLogging and notificationsRecovering deleted packages