Add overwrite control to bup-restore

Adds the --overwrite-files and --directory-merge options to bup restore.
These control whether restore will overwrite files, or modify directory
structures, of any files or directories which exist in the restore
location.

Outstanding issue:
	- raw_input headers don't print till after the selection is made.

Signed-off-by: Will Rouesnel <w.rouesnel@gmail.com>
Don't delete previously updated paths in the same index run.

This patch forces update_index to keep track of the root paths it has
previously written to. If a subsequent, root somehow excludes a previous
one, update_index is prevented from marking the previously updated files
as deleted.

By itself this change has no effect, but it is necessary for the
improved xdev shadowing and subsequent patches to work.
Do not reduce paths on separate devices when -x is enabled

This patch stops bup index -x/--xdev/--one-file-system from reducing
paths in the same hierarchy if they are on separate devices.

Previously bup index's reduce_paths function would look at a /usr and
/usr/local and remove /usr/local since it is included by /usr.

However if /usr and /usr/local are on different devices and -x is set,
then it would remove /usr/local but would fail to index it.

This patch causes the reduce_paths() function involved to check that a
path marked to be reduced is on the *same* device *if* -x is enabled.

This solves the problem and produces expected functionality.

This patch includes a relevant test in test-xdev.sh to confirm
functionality.
reduce_paths() check directories between folded paths are traversable

This patch causes reduce_paths() to check that all directories between a
candidate to be reduced and the path it will be reduced to can have
their contents listed.

This ensures drecurse is able to descend into these directories then
read permissions are not available to the current user/group that bup is
running as.

If a directory cannot be traversed, then it is not reduced and is backed
up explicitly.

For example:
	/home/user/root/www

Supposing /home/user/root does not have read permissions
(i.e. 0111 permissions) but the user specifies:

	bup index -u /home/user /home/user/root/www

Then bup will correctly retain the /home/user/root/www path that it
can access, and index it appropriately.
bup index: allow explicitly overriding --exclude'd directories

This patch extends the anti-shadowing feature of reduce_paths to also
respect subdirectories of paths specified with --exclude.

For example:
	/home/user/root/www

	bup index --exclude /home/user/root /home/user /home/user/root/www

bup index will respect the user's explicit specification, and index
/home/user and /home/user/root/www but not any other files or
directories in /home/user/root.

This can be used to pick one or two directories out of a complex file
structure, without needing to write many exclude rules.
Fix HEAD request handling in bup-web.

HEAD requests to files did not set self.finished(). This causes problems
with user-agents that expect some type of request since it never closes
(OpenOffice/LibreOffice will notably stall opening files from a URL).

This patch stands alone from WebDAV but also fixes the problem there.
WebDAV support for bup-web

Read-only WebDav implemented natively inside the bup-web server. Starting
bup-web now seamlessly handles WebDAV-protocol requests on the same address.
Dynamic VFS view of bup-repositories.

This patch adds support for efficiently updating the VFS's view of a bup
repository, enabling client applications like bup-fuse and bup-web to
show new commits and branches as they added to the backend repository.

It accomplishes this by checking the the repository for new
branches/commits when a root node is resolved, and patching the view
rather then discarding it. This retains cached metadata in memory and
avoids slowdowns due to file-size calculations.

Any VFS-using bup command will automatically take advantage of this
feature. This principally enables exporting a bup-fuse or bup-web view
as a persistent service.
Add -e,--sshcmd support to remote commands.

Added support for the rsync-style -e command line option to explicitly
specify the command line bup should use for the remote shell. This
allows the use of advanced SSH options by specifying them on the command
line ala:

	bup init -r user@remotehost -e 'ssh -oCompression=yes'

or the use of alternate shells which accept addressing and command
invocation in the SSH style.

Signed-off-by: Will Rouesnel <w.rouesnel@gmail.com>
Revised documentation to show the new -e,--sshcmd arguments.

Updates the documentation for init, join, save and split to be inline
with the -e,--sshcmd argument and their usage in relation to
-r,--remote.

Also updates the documentation for -r,--remote for init, join, save, split
and on to indicate that user@host notation can be used (no code changes
required it was already possible).

Signed-off-by: Will Rouesnel <w.rouesnel@gmail.com>
Merge branch 'improved-ssh-operation' into cutting-edge
Merge branch 'index-shadowing-v2' into cutting-edge
Merge branch 'remote-restore-server-protocols' into cutting-edge
Merge branch 'bup-web-head-fix' into cutting-edge
Merge branch 'bup-webdav' into cutting-edge

Conflicts:
	lib/bup/helpers.py
Merge branch 'restore-overwrite-control' into cutting-edge
Improve standards compliance in WebDAV support, support Windows 7.

Fix the handling of various WebDAV tags, and specifically support Windows 7.
Merge branch 'bup-webdav' into cutting-edge
Patch slashremove() to eatslash() - it was kind of a pointless change.
Assign minimal metadata to metadata objects in protocol.py

Some nodes will never have metadata, particularly in the case of grafted
paths. So a default blank metadata needs to be sent to keep the protocol
synchronized.