Allow per-instance specification of the repo_dir for CatPipe.

This allows the creation of CatPipes that refer to repositories other
than the default, which in turn allows copying between repositories,
etc.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Multiplex stdout and stderr over stderr for "bup on".

Previously, when running "bup on" (i.e. when in BUP_SERVER_REVERSE
mode), bup would redirect the remote command's stdout to stderr
because stderr is the only remaining avenue back to the local console,
given that in reverse mode, stdout (and stdin) are connected back to a
local "bup server" instance (hence the name).

Of course that makes it impossible to reliably capture the
non-diagnostic output from the remote commands, i.e.

  $ commit_id="$(bup on HOST save -t ...)"

To fix that, have all "bup on" subcommands encode their stdout and
stderr output via TaggedOutput instances whenever they detect that
they're running in reverse.

With the current trivial protocol, the end result is that all remote
commands will write lines to stderr that look like this:

  on This is a stdout line that had a newline terminator.
  o_ This is a stdout line that didn't.
  en This is a stderr line that had a newline terminator.
  e_ This is a stderr line that didn't.

The loop in on-cmd.py handles demultiplexing this output (from the
remote stderr) to the local stdout/stderr.

Questions:

Right now TaggedOutput does line-buffering, which makes things more
complicated -- is that worthwhile?

And the interaction with _hard_write() is messy -- I wonder if we
still need _hard_write() (i.e. is ssh still broken), and if so,
exactly what the expectations are, i.e. is *all* stderr output
expected to go through _hard_write(), or just log()?

Thanks to Alexander Barton <alex@barton.de> for pointing out an error
in a previous version of this commit message.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Allow the specification of a repo_dir to various VFS and git operations.

Previously, these VFS and git operations would only operate on the
default repository (git.repo()).  Note that these changes may not be
comprehensive, but they should at least change the bits needed by "bup
get".

Have vfs.cp() handle more than one repository (via an internal cache).

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add PackWriter write() for cases where we know the object doesn't exist().

This avoids having to check twice, and to compute the sha twice (via
maybe_write()), and will be used by "bup get".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Have PackWriter new_commit() accept author and committer information.

This will support "bup get", which needs to write a commit with
differing author and committer information.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
git.py: allow the specification of a repo_dir to update_ref().

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
git.py: generalize update_ref() to support tags in addition to heads.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add limit_to_heads and limit_to_tags to list_refs().

And adjust git.py to use the limits when beneficial.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add git.object_info(objects), returning (hash, type, size) for each object.

This will allow improvements to the VFS, which currently crashes if
the repository contains a non-commit tag, and will support "bup get".

Even though we won't always want the size, use the default git
cat-file --batch-check format and always return the size, because
older versions of git don't support custom formats, and retrieving the
size shouldn't be notably more expensive (since it should be adjacent
to the type in the git object format).

Thanks to Alexander Barton <alex@barton.de> for reporting the git
incompatibility in an earlier version of this patch.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
vfs.py: make it easier to distinguish node types, and allow non-commit tags.

Use git.list_refs() in the VFS to handle non-commit tags.

Add CommitLink and BranchCommitLink node classes to make it easy to
distinguish commits that are saves (BranchCommitLink) and tagged
commits (CommitLink).

Present tagged trees and blobs in .tag/ as Dir and File nodes
respectively.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add a VFS path_info() command and corresponding server 'path-info' command.

Provide further support for "bup get".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add bup get; see the documentation for further information.

WARNING: this is preliminary, don't trust it yet.

Thanks to Karl Kiniger <karl.kiniger@med.ge.com> for helping track
down various bugs in earlier versions, and for noting that we might
want --verbose to be a little more effusive.  And thanks to Patryck
Rouleau <prouleau72@gmail.com> for suggesting improvements to the
documentation.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
add uid and gid options to bup fuse, so you can specify who the files will appear to be owned by.

[rlb@defaultvalue.org: cherry-pick onto newer master; don't print options
 in fuse-cmd.py; check m.uid and m.gid against None]
don't show metadata for the commits -- only for files/paths inside them; this makes getting list of commits massively faster.

[rlb@defaultvalue.org: cherry-pick onto newer master]
implement on-disk caching of rev_list

[rlb@defaultvalue.org: cherry-pick onto newer master; import cPickle
 as pickle]
Added command "bup log"

"bup log" can be used to inspect the backup history.
It is a wrapper for "git log" and "git diff-tree" and uses them in a
way that is convenient for the bup use case. Furthermore, "bup log"
knows about name mangling and will hide the name mangling from the
user, a feature that "git log" cannot provide directly.
The man-page contains detailed information and examples.

Signed-off-by: Holger Dell <github@holgerdell.com>
Include savename in default output
fuse-cmd.py: flush the cache occasionally
fuse-cmd.py: move VFS top inside BupFs

In preparation for some initial controls on heap growth.
fuse-cmd.py: update prefix when reconstructing cache parents

Previously, the cache reconstruction algorithm would fail whenever it
had to restore multiple parents when caching a given path.  Clearing
the cache revealed the problem.