Package: peru
peruMaybe sometimes better than copy-paste.Peru is a tool for including other people's code in your projects. It
fetches from anywhere -- git, hg, svn, tarballs -- and puts files
wherever you like. Peru helps you track exact versions of your
dependencies, so that your history is always reproducible. And it fits
inside your scripts andMakefiles, so your build
stays simple and foolproof.Why?If you build withmake, you don't have to do anything special when you
switch branches or pull new commits. Build tools notice those changes
without any help. But if you depend on other people's code, the tools
aren't so automatic anymore. You need to remember when togit submodule updateorgo get -uorpip install -r. If you forget a step you can
break your build, or worse, you might build something wrong without
noticing.Peru wants you to automate dependency management just like you automate
the rest of your build. It doesn't interfere with your source control or
install anything global, so you can just throw it in at the start of a
script and forget about it. It'll run every time, and your dependencies
will never be out of sync. Simple, and fast as heck.The name "peru", along with our love for reproducible builds, was inspired byAmazon's Brazil build system.
It also happens to be an anagram for "reup".InstallationPeru supports Linux, macOS, and Windows. It requirespython(3.5 or later)
andgit, and optionallyhgandsvnif you want fetch from those
types of repos.UsingpipUsepipto install it:pipinstallperuNote that depending on how Python is set up on your machine, you might
need to usesudowith that, and Python 3's pip might be calledpip3.
Also, if you have to use Python 3.3 or 3.4, those were supported up to
peru 1.1.4.Using OS package managersOn Arch Linux, you can installperufrom the
AUR.Homebrew has aPeru formulafor macOS
and Linux.brew install peruwill install it running on the latest Python
version that Homebrew supports.Getting StartedHere's the peru version of thefirst git submodules
examplefrom theGit Book. We're going to add the Rack
library to our project. First, create aperu.yamlfile like this:imports:rack_example:rack/# This is where we want peru to put the module.git module rack_example:url:git://github.com/chneukirchen/rack.gitNow runperu sync.What the heck just happened?Peru cloned Rack for you, and imported a copy of it under therackdirectory.
It also created a magical directory called.peruto hold that clone and some
other business. If you're using source control, now would be a good time to put
these directories in your ignore list (like.gitignore). You usually don't
want to check them in.Runningperu cleanwill make the imported directory disappear.  Runningperu syncagain will make it come back, and it'll be a lot faster this time,
because peru caches everything.Getting FancyFor a more involved example, let's use peru to manage some dotfiles. We're big
fans of theSolarized colorscheme, and
we want to get it working in bothlsandvim. Forlsall we need peru to
do is fetch a Solarized dircolors file. (That'll get loaded somewhere like.bashrc, not included in this example.) Forvimwe're going to need theSolarized vim plugin,
and we also wantPathogen, which makes
plugin installation much cleaner. Here's theperu.yaml:imports:# The dircolors file just goes at the root of our project.dircolors:./# We're going to merge Pathogen's autoload directory into our own.pathogen:.vim/autoload/# The Solarized plugin gets its own directory, where Pathogen expects it.vim-solarized:.vim/bundle/solarized/git module dircolors:url:https://github.com/seebi/dircolors-solarized# Only copy this file. Can be a list of files. Accepts * and ** globs.pick:dircolors.ansi-darkcurl module pathogen:url:https://codeload.github.com/tpope/vim-pathogen/tar.gz/v2.3# Untar the archive after fetching.unpack:tar# After the unpack, use this subdirectory as the root of the module.export:vim-pathogen-2.3/autoload/git module vim-solarized:url:https://github.com/altercation/vim-colors-solarized# Fetch this exact commit, instead of master or main.rev:7a7e5c8818d717084730133ed6b84a3ffc9d0447The contents of thedircolorsmodule are copied to the root of our repo. Thepickfield restricts this to just one file,dircolors.ansi-dark.Thepathogenmodule uses thecurltype instead ofgit, and its URL points
to a tarball. (This is for the sake of an example. In real life you'd probably
usegithere too.) Theunpackfield means that we get the contents of the
tarball rather than the tarball file itself. Because the module specifies anexportdirectory, it's that directory rather than the whole module that gets
copied to the import path,.vim/autoload. The result is that Pathogen'sautoloaddirectory gets merged with our own, which is the standard way to
install Pathogen.Thevim-solarizedmodule gets copied into its own directory underbundle,
which is where Pathogen will look for it. Note that it has an explicitrevfield, which tells peru to fetch that exact revision, rather than the default
branch (masterormainin git). That's aSuper Serious Best Practice™,
because it means your dependencies will always be consistent, even when you
look at commits from a long time ago.You really want all of your dependencies to have hashes, but editing
those by hand is painful. The next section is about making that easier.Magical UpdatesIf you runperu reup, peru will talk to each of your upstream repos, get
their latest versions, and then edit yourperu.yamlfile with any updates. If
you don't haveperu.yamlchecked into some kind of source control, you should
probably do that first, because the reup will modify it in place. When we reup
the example above, the changes look something like this:diff --git a/peru.yaml b/peru.yamlindex 15c758d..7f0e26b 100644--- a/peru.yaml+++ b/peru.yaml@@ -6,12 +6,14 @@ imports:git module dircolors:url: https://github.com/seebi/dircolors-solarizedpick: dircolors.ansi-dark+    rev: a5e130c642e45323a22226f331cb60fd37ce564fcurl module pathogen:url: https://codeload.github.com/tpope/vim-pathogen/tar.gz/v2.3unpack: tarexport: vim-pathogen-2.3/autoload/+    sha1: 9c3fd6d9891bfe2cd3ed3ddc9ffe5f3fccb72b6agit module vim-solarized:url: https://github.com/altercation/vim-colors-solarized-    rev: 7a7e5c8818d717084730133ed6b84a3ffc9d0447+    rev: 528a59f26d12278698bb946f8fb82a63711eec21Peru made three changes:Thedircolorsmodule, which didn't have arevbefore, just got one. By
default forgit, this is the currentmasterormain. To change that,
you can set thereupfield to the name of a different branch.Thepathogenmodule got asha1field. Unlikegit, acurlmodule is
plain old HTTP, so it's stuck downloading whatever file is at theurl. But
it will check this hash after the download is finished, and it will raise an
error if there's a mismatch.Thevim-solarizedmodule had a hash before, but it's been updated. Again,
the new value comes frommasterormainby default.At this point, you'll probably want to make a new commit ofperu.yamlto
record the version bumps. You can do this every so often to keep your plugins
up to date, and you'll always be able to reach old versions in your history.CommandssyncPull in your imports.syncyells at you instead of overwriting existing
or modified files. Use--force/-fto tell it you're serious.cleanRemove imported files. Same--force/-fflag assync.reupUpdate module fields with new revision information. Forgit,hg, andsvn, this updates therevfield. Forcurl, this sets thesha1field. You can optionally give specific module names as arguments.copyMake a copy of all the files in a module. Either specify a directory to put
them in, or peru will create a temp dir for you. You can use this to see
modules you don't normally import, or to play with different module/rule
combinations (see "Rules" below).overrideReplace the contents of a module with a local directory path, usually a
clone you've made of the same repo. This lets you test changes to imported
modules without needing to push your changes upstream or editperu.yaml.Module Typesgit, hg, svnFor cloning repos. These types all provide the same fields:url: required, any protocol supported by the underlying VCSrev: optional, the specific revision/branch/tag to fetchreup: optional, the branch/tag to get the latest rev from when runningperu reupThegittype also supports settingsubmodules: falseto skip
fetching git submodules. Otherwise they're included by default.curlFor downloading a file from a URL. This type is powered by Pythons's standard
library, rather than an external program.url: required, any kind supported byurllib(HTTP, FTP,file://)filename: optional, overrides the default filenamesha1: optional, checks that the downloaded file matches the checksumunpack: optional,tarorzipPeru includes a few other types mostly for testing purposes. Seersyncfor an
example implemented in Bash.Creating New Module TypesModule type plugins are as-dumb-as-possible scripts that only know how to
sync, and optionally reup. Peru shells out to them and then handles most of
the caching magic itself, though plugins can also do their own caching as
appropriate.  For example, the git and hg plugins keep track of repos they
clone. Peru itself doesn't need to know how to do that. For all the details,
seeArchitecture: Plugins.RulesSome fields (likerevandunpack) are specific to certain module
types. There are also fields you can use in any module, which modify the
tree of files after it's fetched. Some of these made an appearance in
the fancy example above:copy: A map or multimap of source and destination paths to copy.
Works likecpon the command line, so if the destination is a
directory, it'll preserve the source filename and copy into the
destination directory.move: A map or multimap of source and destination paths to move.
Similar tocopyabove, but removes the source.drop: A file or directory, or a list of files and directories, to
remove from the module. Paths can contain*or**globs.pick: A file or directory, or a list of files and directories, to
include in the module. Everything else is dropped. Paths can contain*or**globs.executable: A file or list of files to make executable, as if
callingchmod +x. Also accepts globs.export: A subdirectory that peru should treat as the root of the
module tree. Everything else is dropped, including parent directories.Note that these fields always take effect in the order listed above, regardless
of the order they're given inperu.yaml. For example, amoveis always
performed before apick. Also note that these fields can't be given twice.
For example, instead of using two separatemovefields (one of which would be
ignored), use a singlemovefield containing multiple moves. In practice,
things work this way because these fields are parsed as keys in a dictionary,
which don't preserve ordering and can't repeat.Besides using those fields in your modules, you can also use them in "named
rules", which let you transform one module in multiple ways. For example, say
you want theasynciosubdir from the Tulip project, but you also want the
license file somewhere else. Rather than defining the same module twice, you
can use one module and two named rules, like this:imports:tulip|asyncio:python/asyncio/tulip|license:licenses/git module tulip:url:https://github.com/python/asynciorule asyncio:export:asyncio/rule license:pick:COPYINGAs in this example, named rules are declared a lot like modules and then
used in theimportslist, with the syntaxmodule|rule.  The|operator there works kind of like a shell pipeline, so you can even do
twisted things likemodule|rule1|rule2, with each rule applying to the
output tree of the previous.RecursionIf you import a module that has a peru file of its own, peru can include
that module's imports along with it, similar to how git submodules
behave withgit clone --recursive. To enable this, addrecursive: truein a module's definition.It's also possible to directly import modules that are defined in theperu.yamlfile of another module. If your project defines a modulefoo, andfoohas a peru file in it that defines a modulebar, you
can usefoo.barin your own imports. This works even if you never
actually importfoo, and it does not require settingrecursive: true.ConfigurationThere are several flags and environment variables you can set, to
control where peru puts things. Flags always take precedence.--file=<file>: The path to your peru YAML file. By default peru
looks forperu.yamlin the current directory or one of its parents.
This setting tells peru to use a specific file. If set,--sync-dirmust also be set.--sync-dir=<dir>: The path that allimportsare interpreted
relative to. That is, if you import a module to./, the contents of
that module go directly in the sync dir. By default this is the
directory containing yourperu.yamlfile. If set,--filemust also
be set.--state-dir=<dir>: The directory where peru stashes all of its state
metadata, and also the parent of the cache dir. By default this is.peruinside the sync dir. You should not share this directory
between two projects, orperu syncwill get confused.--cache-dir=<dir>orPERU_CACHE_DIR: The directory where peru
keeps everything it's fetched. If you have many projects fetching the
same dependencies, you can use a shared cache dir to speed things up.--file-basename=<name>: Change the default peru file name (normallyperu.yaml). As usual, peru will search the current directory and its
parents for a file of that name, and it will use that file's parent
dir as the sync dir. Incompatible with--file.LinksDiscussion and announcements (Google
Group)Architecture docUsing peru with make