Multiplex stdout and stderr over stderr for "bup on".

Previously, when running "bup on" (i.e. when in BUP_SERVER_REVERSE
mode), bup would redirect the remote command's stdout to stderr
because stderr is the only remaining avenue back to the local console,
given that in reverse mode, stdout (and stdin) are connected back to a
local "bup server" instance (hence the name).

Of course that makes it impossible to reliably capture the
non-diagnostic output from the remote commands, i.e.

  $ commit_id="$(bup on HOST save -t ...)"

To fix that, have all "bup on" subcommands encode their stdout and
stderr output via TaggedOutput instances whenever they detect that
they're running in reverse.

With the current trivial protocol, the end result is that all remote
commands will write lines to stderr that look like this:

  on This is a stdout line that had a newline terminator.
  o_ This is a stdout line that didn't.
  en This is a stderr line that had a newline terminator.
  o_ This is a stderr line that didn't.

The loop in on-cmd.py handles demultiplexing this output (from the
remote stderr) to the local stdout/stderr.

Questions:

Right now TaggedOutput does line-buffering, which makes things more
complicated -- is that worthwhile?

And the interaction with _hard_write() is messy -- I wonder if we
still need _hard_write() (i.e. has ssh still broken), and if so,
exactly what the expectations are, i.e. is *all* stderr output
expected to go through _hard_write(), or just log()?

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Allow per-instance specification of the repo_dir for CatPipe.

This allows the creation of CatPipes that refer to repositories other
than the default, which in turn allows copying between repositories,
etc.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Allow the specification of a repo_dir to various VFS and git operations.

Previously, these VFS and git operations would only operate on the
default repository (git.repo()).  Note that these changes may not be
comprehensive, but they should at least change the bits needed by "bup
get".

Have vfs.cp() handle more than one repository (via an internal cache).

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Use the author date, not committer date for the bup save date.

Since "bup get" is going to change the committer date for fetched
commits to match the current date, use the author date to represent
saves so that fetched saves will have the same names as the originals.

This should have no effect with respect to backward compatibility
because until now, bup always used the same date for the author and
committer.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add PackWriter write() for cases where we know the object doesn't exist().

This avoids having to check twice, and to compute the sha twice (via
maybe_write()), and will be used by "bup get".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Have PackWriter new_commit() accept author and committer information.

This will support "bup get", which needs to write a commit with
differing author and committer information.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
git.py: allow the specification of a repo_dir to update_ref().

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
git.py: generalize update_ref() to support tags in addition to heads.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add limit_to_heads, limit_to_tags, and include_type to list_refs().

And adjust git.py to use the limits when beneficial.  This will allow
improvements to the VFS, which currently crashes if the repository
contains a non-commit tag, and will support "bup get".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
vfs.py: make it easier to distinguish node types, and allow non-commit tags.

Use git.list_refs() in the VFS to handle non-commit tags.

Add CommitLink and BranchCommitLink node classes to make it easy to
distinguish commits that are saves (BranchCommitLink) and tagged
commits (CommitLink).

Present tagged trees and blobs, respectively, as Dir and File nodes in
.tag/.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add a VFS path_info() command and corresponding server 'path-info' command.

Provide further support for "bup get".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add bup get; see the documentation for further information.

WARNING: this is preliminary, don't trust it yet.

Thanks to Karl Kiniger <karl.kiniger@med.ge.com> for helping track
down various bugs in earlier versions, and for noting that we might
want --verbose to be a little more effusive.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Check subprocess status in readpipe() and use it instead of check_output().

Checking the exit status seems like a good idea, and we can't use
check_output() because it wasn't added until Python 2.7.

Thanks to Alexander Barton <alex@barton.de> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move git.list_refs() include_type functionality to git.object_info().

Break list_refs() up into two functions so that list_ref() just
returns (refname, hash) tuples (as it used to), and a new
object_info(objects) function returns (hash, type, size) tuples for
each object named.

The primary justification is that older git versions don't support a
custom --batch-check format, but this design's arguably better.

Use the default format, unconditionally, since retrieving the size
shouldn't be notably more expensive (it should be adjacent to the type
in the git object format).

Thanks to Alexander Barton <alex@barton.de> for reporting the
incompatibility.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Merge branch 'tmp/bup-get-v5' of github.com:williamstein/bup-1
Merge branch 'master' of github.com:williamstein/bup-1