giseldo/neodataset huggingface dataset:
UnboundLocalError: local variable 'document' referenced before assignment (lp:#726823)

[Original report](https://bugs.launchpad.net/bugs/726823) created by **Tobias Wichtrey (mail-tobias-wichtrey)**
```
When I invoke duplicity with

duplicity -v9 localdir/
webdav://username:password@dav.server.com/remotedir/

(with details filled in, of course), then I always get the following error:

------

Linux dhcppc0 2.6.34.7-0.7-desktop #1 SMP PREEMPT 2010-12-13 11:13:53 +0100
x86_64 x86_64
/usr/bin/python 2.6.5 (r265:79063, Oct 28 2010, 20:56:23)
[GCC 4.5.0 20100604 [gcc-4_5-branch revision 160292]]
================================================================================
Using temporary directory /tmp/duplicity-P_wNM6-tempdir
Registering (mkstemp) temporary file /tmp/duplicity-P_wNM6-tempdir/mkstemp-
WlUnpu-1
Temp has 8678006784 available, backup will use approx 34078720.
Listing directory /remotedir/ on WebDAV server
WebDAV PROPFIND attempt #1 failed: 405 Method Not Allowed
Listing directory /remotedir/ on WebDAV server
WebDAV PROPFIND attempt #2 failed: 405 Method Not Allowed
Listing directory /remotedir/ on WebDAV server
WebDAV PROPFIND attempt #3 failed: 405 Method Not Allowed
Listing directory /remotedir/ on WebDAV server
WebDAV PROPFIND attempt #4 failed: 405 Method Not Allowed
Listing directory /remotedir/ on WebDAV server
WebDAV PROPFIND attempt #5 failed: 405 Method Not Allowed
Removing still remembered temporary file /tmp/duplicity-
P_wNM6-tempdir/mkstemp-WlUnpu-1
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1239, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1232, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1133, in main
    sync_archive()
  File "/usr/bin/duplicity", line 910, in sync_archive
    remlist = globals.backend.list()
  File "/usr/lib64/python2.6/site-
packages/duplicity/backends/webdavbackend.py", line 181, in list
    log.Info("%s" % (document,))
UnboundLocalError: local variable 'document' referenced before assignment

------

I am using duplicity 0.6.08b with python 2.6.5 on an OpenSuSE 11.3 system.

Maybe, it is because my username contains an '@'. I tried writing '\@' and
putting the username in "s, but the same error occcurs. It is also the same
with duplicity 0.6.11.

Cheers
Tobias
```

Story points: 22.0
dejadup crashes cant take backup displays "Failed with an unknown error." (lp:#725117)

[Original report](https://bugs.launchpad.net/bugs/725117) created by **Rakesh Jain (rakeshchhabra+launchpad)**
```
$ dpkg-query -W deja-dup duplicity
deja-dup        16.1.1-0ubuntu1
duplicity       0.6.10-0ubuntu1
$ lsb_release -d
Description:    Ubuntu 10.10
$
```

Story points: 6.0
Manifest needs to list all files in each difftar (lp:#721618)

[Original report](https://bugs.launchpad.net/bugs/721618) created by **nemoinis (nemoinis)**
```
I'm trying to restore a 0.5KB text file from my duplicity backup (stored on
the net in my website). I expected duplicity to only download the difftar
volume containing the file, but it looks like it's downloading everything,
all 34GB of it! In the attached log I've stopped it after volume 22, but by
that time it already had downloaded (and discarded) over 1GB of data. I
can't afford to waste bandwidth and time. Am I missing the "bandwidth
efficient" switch?
This is using duplicity 0.6.11 on Ubuntu 10.04. I've attached the log (up
to when I nuked it).
The command line I used is:
duplicity restore -t now --verbosity 5 --file-to-restore
Documents/MYFILE.txt --archive-dir /path/to/localarchive/
scp://albsync/sync/backup  /home/me/tmp/restored_file.txt
```

Story points: 8.0
Duplicity restore fails with ssl socket timeout (lp:#714880)

[Original report](https://bugs.launchpad.net/bugs/714880) created by **Rich Davis (rdavis)**
```
I'm attempting to restore a backup at Rackspace CloudFiles.  It consists of
153 volumes in Rackspace Cloudfiles and fails on different volumes with
timeouts with  "BackendException: Error downloading 'backup_db1/duplicity-
full.20110207T185831Z.vol13.difftar.gz'"  This example indicates failure on
volume 13, but I've had it fail on volumes 1, 4, etc.  Details below.

--------    START DT-CF-BACKUP SCRIPT    --------

Using archive dir: /root/.cache/duplicity/eb2b19f4b4b8e487e7e198a9559285e3
Using backup name: eb2b19f4b4b8e487e7e198a9559285e3
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.sshbackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.ftpbackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
Import of duplicity.backends.giobackend Failed: No module named gio
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.cloudfilesbackend Succeeded
Main action: restore
================================================================================
duplicity 0.6.11 (November 20, 2010)
Args: /usr/bin/duplicity restore --full-if-older-than 14D -v9 --no-
encryption cf+http://backup /var/foo/
Linux myserver.com 2.6.18-194.26.1.el5 #1 SMP Fri Oct 29 14:21:16 EDT 2010
x86_64 x86_64
/usr/bin/python 2.4.3 (#1, Nov  3 2010, 12:52:40)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-48)]
================================================================================
Using temporary directory /tmp/duplicity-c_nD6W-tempdir
Registering (mkstemp) temporary file /tmp/duplicity-
c_nD6W-tempdir/mkstemp-7c5Ao4-1
Temp has 1839988736 available, backup will use approx 34078720.
Listed container 'backup'
Local and Remote metadata are synchronized, no sync needed.
Listed container 'backup'
153 files exist on backend
2 files exist in cache


duplicity 0.6.11
Python 2.4.3
Red Hat Enterprise Linux Server release 5.5 (Tikanga)
```

Story points: 6.0
Reply to request for error correction ideas for new format (lp:#714278)

[Original report](https://bugs.launchpad.net/bugs/714278) created by **Stuart Gathman (stuart-gathman)**
```
The web page describing the tar replacement format asks for ideas to
improve error recovery.  Here is an important one that I learned from
designing a file system that has no critical blocks (can survive corruption
or loss anywhere with limited data loss).  Each file (including meta files
such as index data) should be assigned an "id" within the backup.  The
block header for each block should include the file id, and perhaps an
offset within the file.  This way, even with loss of meta data, blocks
belonging to the same file can be identified, and files still recovered in
a "lost+found" directory.   The pointers in the index are not sufficient,
because you often don't know how many bytes have been "skipped" when
recovering from errors.  (And you may not have the index.)
```

Story points: 6.0
Backup on a webdav server fails with "SSLError: The read operation timed out" (lp:#714175)

[Original report](https://bugs.launchpad.net/bugs/714175) created by **Lutz Niggl (lutz-niggl)**
```
Version 6.11
Python 2.7
Suse 11.3
webdavs backend

Backing up to webdav.mediencenter.t-online.de

Error "SSLError: The read operation timed out" repeatedly shows up after
some 10-30MB. (I reduced volsize to 5MB). Restarting works and continues
the backup. Once the full backup is on the server the incremental backups
run without observed problems.

Rgrds
Lutz
```

Story points: 22.0
RFE: ability to use -verify as argument in backup command (as well as a restore command) (lp:#704760)

[Original report](https://bugs.launchpad.net/bugs/704760) created by **Aaron Whitehouse (aaron-whitehouse)**
```
Verifying backups is a very important part of the backup process to me.

I have complicated duplicity backup commands set up to backup different
lists of files on a host to different locations. I have found it very
difficult to construct verify commands that can give me confidence that my
files have all backed up correctly.

It would be excellent if I could add a simple "-verify" option to my backup
command and have it:
(a) complete the backup as normal; and then
(b) verify each as if I had managed to construct the correct verify command
that mirrors my backup line (ie verifies each of the backed up files
contained in the list of the backup command).

I would imagine that such an option would be very popular.
```

Story points: 6.0
Provide better error message if gpg is not found (lp:#703345)

[Original report](https://bugs.launchpad.net/bugs/703345) created by **Tokuko (launchpad-net-tokuko)**
```
Currently duplicity complains about a temporary directory if gpg cannot be
executed:
x@yuna:~# duplicity --log-file duplicity.log /somepath
webdavs://someone@someserver/somepath
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
No signatures found, switching to full backup.
Cleanup of temporary directory /tmp/duplicity-3F1Ehm-tempdir failed - this
is probably a bug.
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1245, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1238, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1216, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 417, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 295, in write_multivol
    globals.gpg_profile, globals.volsize)
  File "/usr/lib/python2.6/site-packages/duplicity/gpg.py", line 275, in
GPGWriteFile
    bytes_to_go = data_size - get_current_size()
  File "/usr/lib/python2.6/site-packages/duplicity/gpg.py", line 267, in
get_current_size
    return os.stat(filename).st_size
OSError: [Errno 2] No such file or directory: '/tmp/duplicity-3F1Ehm-
tempdir/mktemp-p60v_2-2'

Running these two commands on Solaris 11 Express solved the issue for me:
x@yuna:~# cd /usr/bin/
x@yuna:/usr/bin# ln -s gpg2 gpg
```

Story points: 14.0
Try overwriting file with an 0-byte file on delete failure (workaround for Hetzner's backup backend) (lp:#700395)

[Original report](https://bugs.launchpad.net/bugs/700395) created by **Daniel Hahler (blueyed)**
```
I've run into an issue with the backup backend provided by Hetzner (a
German provider):
when the backup space is full, you cannot remove any files, but have to
overwrite the file with a 0-byte file of the same name. Only then you can
delete the original file.

In duplicity the failure looks like this:
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=2
u123@u123.your-backup.de' (attempt #1)
State = sftp, Before = 'Connected to u123.your-backup.de.'
sftp command: 'cd "moby/"'
State = sftp, Before = 'cd "moby/"'
sftp command: 'rm "duplicity-full.20110105T145659Z.vol1.difftar.gpg"'
State = sftp, Before = 'rm "duplicity-
full.20110105T145659Z.vol1.difftar.gpg"
Removing /moby/duplicity-full.20110105T145659Z.vol1.difftar.gpg'
Could not delete file in command='sftp  -oServerAliveInterval=15
-oServerAliveCountMax=2 u123@u123.your-backup.de'
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=2
u123@u123.your-backup.de' failed (attempt #1)


Replaying this via "sftp" looks like this:
sftp> rm "duplicity-full.20110105T145659Z.vol1.difftar.gpg"
Removing /moby/duplicity-full.20110105T145659Z.vol1.difftar.gpg
Couldn't delete file: Failure

There are two issues here:
1. The error "Couldn't delete file: Failure" should get logged. At least
with verbosity=9, but also with something like 4 (default?!).
2. At least in case of this exact error ("Couldn't delete file: Failure")
it should try the following procedure:
upload a 0-byte file, overwriting the file that should get deleted. If this
works, try the "rm" again.
```

Story points: 18.0
Nanosecond timestamp support (lp:#696614)

[Original report](https://bugs.launchpad.net/bugs/696614) created by **Mechanical snail (replicator-snail)**
```
Duplicity currently has a timestamp granularity of 1 second, whereas
current Linux filesystems support nanosecond resolution. Preserving high-
resolution timestamps is important for many purposes (e.g. GNU make relies
on the full nanosecond timestamps), besides the principle that a backup
should preserve everything).

Possibly relevant implementation details: GNU tar does support nanosecond
timestamps, but you have to explicitly tell it to create POSIX-format tar
files.

Since GNU tar automatically detects the POSIX format when extracting, it
should be possible to fix this without breaking backwards compatibility.
```

Story points: 6.0
Crash when destination volume has no free space (lp:#692305)

[Original report](https://bugs.launchpad.net/bugs/692305) created by **Milan Bouchet-Valat (nalimilan)**
```
I'm getting this trace when my destination directory is full (it's a NFS
volume mounted to /mnt). duplicity is 0.6.10-0ubuntu1

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1257, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1250, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1232, in main
    incremental_backup(sig_chain)
  File "/usr/bin/duplicity", line 488, in incremental_backup
    globals.backend)
  File "/usr/bin/duplicity", line 316, in write_multivol
    (tdp, dest_filename)))
  File "/usr/lib/python2.6/dist-packages/duplicity/asyncscheduler.py", line
145, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/usr/lib/python2.6/dist-packages/duplicity/asyncscheduler.py", line
171, in __run_synchronously
    ret = fn(*params)
  File "/usr/bin/duplicity", line 315, in <lambda>
    async_waiters.append(io_scheduler.schedule_task(lambda tdp,
dest_filename: put(tdp, dest_filename),
  File "/usr/bin/duplicity", line 241, in put
    backend.put(tdp, dest_filename)
  File "/usr/lib/python2.6/dist-
packages/duplicity/backends/localbackend.py", line 57, in put
    target_path.writefileobj(source_path.open("rb"))
  File "/usr/lib/python2.6/dist-packages/duplicity/path.py", line 595, in
writefileobj
    if fin.close() or fout.close():
IOError: [Errno 5] Input/output error
```

Story points: 12.0
Should warn when backup destination doesn't exist (lp:#692237)

[Original report](https://bugs.launchpad.net/bugs/692237) created by **Milan Bouchet-Valat (nalimilan)**
```
If you forget to add a third / to the backup path, like file://mnt/somedir,
duplicity doesn't warn, and later fails to find previous signatures. Same
if you happen to pass a non-existent directory by mistake.

Instead of saying:
Fatal Error: Unable to start incremental backup.  Old signatures not found
and incremental specified
and instead of starting a full backup when incremental was expected, it
would be nicer to precise that backup directory doesn't exist. It's much
clearer for users.

For the specific case of file:// lacking a /, it would be even better to
give the user a hint that the URI was likely mistyped if the directory
turns out not to exist.


(This is with duplicity 0.6.10-0ubuntu1.)
```

Story points: 6.0
backup via localbackend fails sometimes (lp:#691214)

[Original report](https://bugs.launchpad.net/bugs/691214) created by **Christian Ruppert (idl0r) (spooky85)**
```
backup via localbackend fails sometimes. I'm not sure if that is related
but I always do a "cleanup" first. (cleanup/remove-older-than/remove-all-
but-n-full/remove-all-inc-of-but-n-full)

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1245, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1238, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1220, in main
    incremental_backup(sig_chain)
  File "/usr/bin/duplicity", line 488, in incremental_backup
    globals.backend)
  File "/usr/bin/duplicity", line 316, in write_multivol
    (tdp, dest_filename)))
  File "/usr/lib64/python2.6/site-packages/duplicity/asyncscheduler.py",
line 145, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/usr/lib64/python2.6/site-packages/duplicity/asyncscheduler.py",
line 171, in __run_synchronously
    ret = fn(*params)
  File "/usr/bin/duplicity", line 315, in <lambda>
    async_waiters.append(io_scheduler.schedule_task(lambda tdp,
dest_filename: put(tdp, dest_filename),
  File "/usr/bin/duplicity", line 241, in put
    backend.put(tdp, dest_filename)
  File "/usr/lib64/python2.6/site-
packages/duplicity/backends/localbackend.py", line 57, in put
    target_path.writefileobj(source_path.open("rb"))
  File "/usr/lib64/python2.6/site-packages/duplicity/path.py", line 589, in
writefileobj
    fout = self.open("wb")
  File "/usr/lib64/python2.6/site-packages/duplicity/path.py", line 531, in
open
    result = open(self.name, mode)
IOError: [Errno 2] No such file or directory: '/mnt/backup/foo/duplicity-
inc.20101214T174307Z.to.20101216T020302Z.vol1.difftar.gpg'
```

Story points: 6.0
wishlist: option to ignore corrupt backup set (lp:#689177)

[Original report](https://bugs.launchpad.net/bugs/689177) created by **az (az-debian)**
```
this is a forward of debian bug #606182, which lives here:
http://bugs.debian.org/606182

the original reporter can't restore any of a number of full backups,
because duplicity croaks on a broken/empty
remote manifest file that belongs to one of the backup sets.
(i think the empty files were caused by some vfs/disk cache or unmount
issue, as the data is on a removable disk.)

as far as i can tell, duplicity can't be instructed to disregard such a
faulty backup set - which is what is needed in the long run.

regards,
az
```

Story points: 14.0
fix documentation: duplicity can't backup a remote folder (lp:#687291)

[Original report](https://bugs.launchpad.net/bugs/687291) created by **edso (ed.so)**
```
Manpage suggests:

SYNOPSIS
       duplicity [options] source_directory target_url
       duplicity [options] source_url target_directory

to restore I need --restore-too. In man it looks like I *can* backup remote
dir over scp.

I think it should be cleaner in man that duplicity can't backup remote dir.
```

Story points: 6.0
MemoryError occurs with large signature files. (lp:#686839)

[Original report](https://bugs.launchpad.net/bugs/686839) created by **Tom Eastman (tveastman)**
```
I just tried switching from using the SFTP backend to the WebDAV backend.
But am unable to do so because the WebDAV backend crashes when trying to
retrieve the signature file from my backup set.

================================================================================
duplicity 0.6.09 (July 25, 2010)
Args: /usr/bin/duplicity -v5 --name XXXX_full --volsize 500 --exclude-
globbing-filelist /etc/duplicity/ALL.exclude --include-globbing-filelist
/etc/duplicity/ALL.include  / webdavs://XXXXX@XXXX.XXXX/duplicity/XXXX.XXXX
Linux XXXXXXXXXXX 2.6.26-2-xen-amd64 #1 SMP Thu Sep 16 16:32:15 UTC 2010
x86_64
/usr/bin/python 2.5.2 (r252:60911, Jan 24 2010, 17:44:40)
[GCC 4.3.2]
================================================================================


Synchronizing remote metadata to local cache...
Deleting local /home/XXXXXX/.cache/duplicity/XXXXXXXX_full/duplicity-full-
signatures.20101110T214029Z.sigtar.gz (not authoritative at backend).
Deleting local /home/XXXX/.cache/duplicity/XXXXXXXXXX/duplicity-
full.20101110T214029Z.manifest (not authoritative at backend).
Copying duplicity-full-signatures.20100917T120401Z.sigtar to local cache.
Retrieving /duplicity/XXXXXXXXX/duplicity-full-
signatures.20100917T120401Z.sigtar.gpg from WebDAV server
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1251, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1244, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1145, in main
    sync_archive()
  File "/usr/bin/duplicity", line 959, in sync_archive
    copy_to_local(fn)
  File "/usr/bin/duplicity", line 911, in copy_to_local
    fileobj = globals.backend.get_fileobj_read(rem_name)
  File "/usr/lib/python2.5/site-packages/duplicity/backend.py", line 463,
in get_fileobj_read
    self.get(filename, tdp)
  File "/usr/lib/python2.5/site-
packages/duplicity/backends/webdavbackend.py", line 235, in get
    target_file.write(response.read())
  File "/usr/lib/python2.5/httplib.py", line 516, in read
    s = self._safe_read(self.length)
  File "/usr/lib/python2.5/httplib.py", line 607, in _safe_read
    return ''.join(s)
MemoryError

If I read this correctly, then 'target_file.write(response.read())' is
trying to read the entire response into a string in memory before writing
it to a file.  This particular signature file is 1.2 gigabytes and
duplicity thus exhausts all RAM, but in general I'll bet the signature file
will ALWAYS be big enough that you don't want to deal with it as a string
in RAM.

My workaround for the moment will just have to be to return to the SFTP
backend, which doesn't have this problem.
```

Story points: 24.0
corrupted backup, CRC check failed (lp:#682469)

[Original report](https://bugs.launchpad.net/bugs/682469) created by **TTimo (ttimo)**
```
duplicity 0.6.10 and 0.6.11

I have a 22G backup that is corrupted. That was being backed up from an OSX
MacBook Pro with duplicity 0.6.10 compiled against OSX 10.5's python
(Leopard). I can't check the exact version as this machine died now. The
data was being backed up to Amazon S3.

I am trying to recover the backup on a Linux machine with duplicity 0.6.11
after copying the data locally, but it fails with a CRC error after about
5G of data being processed. I was able to recover more of the data past the
initial CRC check failure by asking for specific folders and files, and I
eventually narrowed things down to a short list of files that could not be
recovered. I am attaching the error log for the first file that I can not
recover.

I doubt there is much way to recover all the individual files at this
point, but I thought I'd point out a few things:

- I highly recommend the --test-restore option getting done
(https://bugs.launchpad.net/bugs/643973)

- It took some serious script-fu to continue recovering past the initial
CRC error. There's still a significant amount of data that is lost, but
it's not that bad. Duplicity should have an option to continue working
through the backup and attempt to recover as much as it can.
```

Story points: 10.0
Feature Request: Make restore support include/exclude parameters (lp:#677177)

[Original report](https://bugs.launchpad.net/bugs/677177) created by **Daniel Hahler (blueyed)**
```
When you have to retrieve multiple files, you currently have to call
duplicity multiple times using "--file-to-restore" for every single
file/folder.

It would be useful to specify patterns like with the exclude/include lists.
```

Story points: 58.0