Package: git-test
git testgit-testis a command-line script for running automated tests against commits
in a Git repository. It is especially targeted at developers who like their
tests to pass oneverycommit in a branch, not just the branch tip.The best way to usegit testis to keep a window open in a second linked
worktree of your repository, and as often as you like rungit test run master..mybranchgit testwill test the commits in the specified range, reporting any
failures. The pass/fail results of running tests are also recorded permanently
in your repository as Git "notes" (seegit-notes(1)).If a commit in the range has already been tested, then by defaultgit testreports the old results rather than testing it again. This means that you can
run the above command over and over as you work, andgit testwon't repeat
tests whose results it already knows. (Of course there are options to allow you
to request explicitly that commits be retested.)The test results are recorded by thetreethat was tested, not the commit, so
old test results remain valid even across some kinds of commit rewriting:If commits are rewritten to change their log messages, authorship, dates,
etc., the test results remain valid.If consecutive commits are squashed, the results remain valid.If a commit is split into two, only the first (partial) commit needs to be
tested.If some commits deep in a branch are reordered, the test results for commits
built on top of the reordered commits often remain valid.Of course this means that your tests should not depend on things besides the
files in the tree. For example, whether your test passes/fails shouldnotdepend on the current branch name or commit message.UsageDefining testsFirst define the test that you would like to run; for example,git test add "make -j8 && make test"The string that you specify can be an arbitrary command; it is run withsh -c. Its exit code should be 0 if the test passes, or nonzero if it fails. The
test definition is stored in your Git config.Test one or more commitsBy default,git test runtestsHEAD:git test run(If the working copy is dirty, the test is run anyway but the results are not
recorded.)You can test a range of Git commits with a single command:git test run commit1..commit2The test is run against each commit in the range, in order from old to new. If
a commit fails the test,git testreports the error and stops with the broken
commit checked out. You can also specify individual commits to test:git test run commit1 commit2 commit3or test an arbitrary set of commits supplied via standard input:git rev-list feature1 feature2 ^master | git test run --stdinYou can adjust the verbosity of the output using the--verbosity/-vor--quiet/-qoptions. Either of these options can be specified multiple
times.Define multiple testsYou can define multiple tests in a single repository (e.g., cheap vs. expensive
tests). Their results are kept separate. By default, the test calleddefaultis run, but you can specify a different test to add/run using the--test=<name>/-t <name>option:git test add "make test"
git test run commit1..commit2
git test add --test=build "make"
git test run --test=build commit1..commit2Retrying tests and/or forgetting old test resultsIf you have flaky tests that occasionally fail for bogus reasons, you might
want to re-run the test against a commit even thoughgit testhas already
recorded a result for that commit. To do so, rungit test runwith the--force/-for--retestoptions.If you want to forget particular old test results without retesting, rungit test runwith the--forgetoption.If you want to permanently forgetallstored results for a particular test
(e.g., if something in your environment has changed), rungit test forget-results [--test=<name>]Continue on test failuresNormally,git test runstops at the first broken commit that it finds. If
you'd prefer for it to continue even after a failure, use the--keep-going/-koption.Removing testsTo permanently remove a test definition and all of its stored results, rungit test remove [--test=<name>]For helpGeneral help aboutgit testcan be obtained by runninggit test helpHelp about a particular subcommand can be obtained via eithergit test help runorgit test run --helpBest practice: usegit testin a linked worktreegit testworks really well together withgit worktree. Keep a second
worktree and use it for testing your current branch continuously as you work:git worktree add --detach ../test HEAD
cd ../test
git test run master..mybranchThe last command can be re-run any time; it only does significant work when
something changes on your branch. Plus, with this setup you can continue to
work in your main working tree while the tests run.Because linked worktrees share branches and the git configuration with the main
repository, test definitions and test results are visible across all worktrees.
So you could even run multiple tests at the same time in multiple linked
worktrees.InstallationRequirements:A recent Git command-line clientA Python interpreter.git testhas been tested with Python versions 2.7 and
3.4. It will probably work with any Python3 version starting with 3.2 (it
requiresargparse).Just putbin/git-testsomewhere in your$PATH, adjusting its first line if
necessary to invoke the desired Python interpreter properly in your
environment.Ideas for future enhancementsSome other features that would be nice:Be more consistent about restoringHEAD.git test runcurrently checks
out the branch that you started on when it is finished, but only if all of the
tests passed. We need some kind ofgit test resetcommand analogous togit bisect reset.git test bisect: rungit bisect runagainst a range of commits, using a
configured test as the command thatbisectuses to decide whether a commit is
good/bad.git test prune: delete notes for obsolete trees.Continuous testing mode, wheregit testwatches the repository for changes
and re-runs itself automatically whenever the commits it is watching change.Dependencies between tests; for example:Provide a way to say "if myfulltest passes, that implies that thebuildtest would also pass".Provide a way to run thebuildtest (and record thebuildtest's
results) as the first step of thefulltest.Allow trees to be markedskip, if they shouldn't be tested (e.g., due to a
known breakage). Perhaps allow the test script to emit a special return code to
ask that the commit be markedskip(probably following the convention ofgit bisect run).Remember return codes and give them back out if the old result is reused.Add agit test fix <range>, which starts an interactive rebase, changing
the command for the first broken commit from "pick" to "edit".Support tests that depend on thecommit, not thetree, that they are run
against.Licensegit testis released under the GPLv2+ license. Pull requests are welcome at
the project's GitHub page,https://github.com/ligurio/git-testCaveats and disclaimersgit testhas pretty good automated tests, but it undoubtedly still has bugs
and rough edges. Use it at your own risk.Please note that when you tellgit test runto test specified commits, it
checks those commits out in your working directory. If the tests fail, it
leaves the failing commit checked outin a detached HEAD state. This is
intentional, so that you can examine the cause of the failure. But it means
that if you had changes on your original HEAD that weren't part of any branch,
they will now be unreachable.If you don't know what a detached HEAD state is, please read up on it.
Additionally,it is recommended that you rungit testin a separate
worktree, which is more convenient anyway (see above for instructions). Note
that thegit worktreecommand was added in Git release 2.5, so make sure you
are using that version of Git or (preferably) newer.The above considerations don't apply to runninggit testagainst HEAD or your
current working tree. In other words,git test runandgit test run HEADdon't change the commit that is checked out, and they won't change your working
copy to a detached HEAD state.