giseldo/neodataset huggingface dataset:
Spelling error in the error message when passing a bad time string (lp:#1599433)

[Original report](https://bugs.launchpad.net/bugs/1599433) created by **Dan (dannyhajj)**
```
When passing a bad time string in the duplicity command, the following
error message shows:

> The acceptible time strings are intervals (like "3D64s"), w3-datetime
> strings, like "2002-04-26T04:22:01-07:00" (strings like
> "2002-04-26T04:22:01" are also acceptable - duplicity will use the
> current time zone), or ordinary dates like 2/4/1997 or 2001-04-23
> (various combinations are acceptable, but the month always precedes
> the day).

The word "acceptible" is misspelled and should be "acceptable"

For example: duplicity remove-older-than 2016-0101 file:///home/user/deja-
dup/

$ duplicity --version
duplicity 0.7.06

$ python --version
Python 2.7.11+

OS: Ubuntu 16.04
```
Original tags: spelling

Story points: 6.0
IMAP corrupts chain and won't resume after stopping mid backup. (lp:#1597209)

[Original report](https://bugs.launchpad.net/bugs/1597209) created by **Richard Scott (2-launchpad-pointb-co-uk)**
```
When backing up a data set to an IMAP server that aborts part way through
(either via CTRL+C, killing the process, an accidental reboot or a press of
a reset button) the next time I try and use that target I am unable to do
ANYTHING with the IMAP data store.

*** No backup is usable, not even old full/inc chains that have previously
uploaded fine ***

I am using Duply to manage my backups, and the problem seems to be due to
the manifest files not being uploaded to the target server until late on in
the backup process... this causes the next run to abort as Duplicity gets
really confused as it can't find any manifest files either remote or
locally... even tho the local ones do exist and have not been deleted.

I get this error:

IMAP LIST: duplicity-
inc.20160628T132706Z.to.20160628T132842Z.vol114.difftar.gpg target
IMAP LIST: duplicity-
inc.20160628T132706Z.to.20160628T132842Z.vol115.difftar.gpg target
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1544, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1538, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1392, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1417, in do_backup
    globals.archive_dir).set_values()
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
710, in set_values
    self.get_backup_chains(partials + backend_filename_list)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
835, in get_backup_chains
    add_to_sets(f)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
823, in add_to_sets
    if set.add_filename(filename):
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
105, in add_filename
    (self.volume_name_dict, filename)
AssertionError: ({1: 'duplicity-
inc.20160628T132706Z.to.20160628T132842Z.vol1.difftar.gpg', 2: 'duplicity-
inc.20160628T132706Z.to.20160628T132842Z.vol2.difftar.gpg', 3: 'duplicity-
inc.20160628T132706Z.to.20160628T132842Z.vol3.difftar.gpg', 4: 'duplicity-
inc.20160628T132706Z.to.20160628T132842Z.vol4.difftar.gpg'}, 'duplicity-
inc.20160628T132706Z.to.20160628T132842Z.vol4.difftar.gpg')

18:59:42.624 Task 'BKP' failed with exit code '30'.

so far, the only fix I can find is to delete all the emails in that folder
on the IMAP server and start again.
```

Story points: 6.0
Duplicity doesn't want create incremental backup (lp:#1595857)

[Original report](https://bugs.launchpad.net/bugs/1595857) created by **Андрей Калинин (prize2step)**
```
duplicity 0.7.07.1 (April 19, 2016)

Python version:
Python 2.7.11

OS:
FreeBSD mercury.pkb.local 9.3-RELEASE FreeBSD 9.3-RELEASE #0 r268512: Fri
Jul 11 03:13:02 UTC 2014
root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC  i386

Running command:
/usr/local/bin/duplicity -v9 --full-if-older-than 1M --volsize 250
/var/mail/exim ftp://*****@192.168.0.8/ftp_shares/BACKUP/mail
```

Story points: 6.0
Swift upload performance (lp:#1594068)

[Original report](https://bugs.launchpad.net/bugs/1594068) created by **Dinacel (o-admin-2)**
```
Using Duplicity with the Swift backend gives bad performances (at least
with OVH PCS/Hubic), Swift client can send an object via multiple
connection to maximise bandwidth usage.
With 10 threads I can saturate a 100Mbps interface, but with the default
(1), I can only send at an average rate of 10Mbps.

http://docs.openstack.org/developer/python-
swiftclient/swiftclient.html#module-swiftclient.multithreading
```
Original tags: swift

Story points: 6.0
Log entries should have optional timestamps (lp:#1592799)

[Original report](https://bugs.launchpad.net/bugs/1592799) created by **Markus (mstoll-de)**
```
with long running backups warning and error log entries often are related
to network problems.
To be able to later on research these problems, it would be nice to have
timestamps with log entries (at least with errors and warnings)

Markus
```

Story points: 18.0
Backup fails with unknown error (assertion failure in duplicity) (lp:#1590926)

[Original report](https://bugs.launchpad.net/bugs/1590926) created by **Wim Lewis (wiml)**
```
Deja Dup and/or duplicity is failing for me now. This might have been
triggered by my being away from the machine when the SSH passphrase prompt
came up, and the session timing out. Or that may be a red herring.

The "Backup Failed / Failed with an unknown error:" window has this
traceback in it:

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1494, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1488, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1337, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1370, in do_backup
    globals.archive_dir).set_values()
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
697, in set_values
    self.get_backup_chains(partials + backend_filename_list)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
819, in get_backup_chains
    map(add_to_sets, filename_list)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
809, in add_to_sets
    if set.add_filename(filename):
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
96, in add_filename
    self.set_manifest(filename)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
127, in set_manifest
    remote_filename)
AssertionError: ('duplicity-full.20160527T063251Z.manifest.gpg',
'duplicity-full.20160527T063251Z.manifest')

$ lsb_release -d
Description:    Ubuntu 14.04.4 LTS
$ dpkg-query -W deja-dup duplicity
deja-dup        30.0-0ubuntu4
duplicity       0.6.23-1ubuntu4.1
```

Story points: 6.0
cleanup not working as expected (lp:#1590540)

[Original report](https://bugs.launchpad.net/bugs/1590540) created by **Markus (mstoll-de)**
```
Duplicity version 0.7.07.01
Python version 2.7
OS OSX 10.11.5


When using cleanup after an incomplete backup, from the documentation I
expected, that cleanup does list the files to be removed from backend (and
I would have to add option "—force" to actually delete the files). However,
I observed that the incomplete backup is always deleted though "—force"
option is not set.

If cleaning up gpg encrypted backups, cleanup is always asking for the
GnuGP passphrase at the end, though abviously the passphrase is not needed
and not used.
```

Story points: 6.0
Document glob behavior with trailing slash (lp:#1586032)

[Original report](https://bugs.launchpad.net/bugs/1586032) created by **René 'Necoro' Neumann (necoro)**
```
With bug #1479545, globbing was changed (or restored) to behave
differently, when the pattern has a trailing /. This fix silently broke my
backup, as I had given
  --include /var/vmail/ --exclude '**', thus removing all emails from the
backup.
This line had worked for quite some time...

While I understand the rationale behind the change in the aforementioned
bug, it breaks the general assumption that /some/dir and /some/dir/ are
equivalent (i.e. when no globbing is involved).

At the very least, it should be documented prominently! Currently it is not
mentioned at all in the man-page...
```

Story points: 10.0
Duplicity looking for wrong file (lp:#1581508)

[Original report](https://bugs.launchpad.net/bugs/1581508) created by **Florent B (florent-z)**
```
Hi,

I use Duplicity 0.7.07.1 on Debian Wheezy.

Backups (full & incremental) are running fine for a few days, then suddenly
Duplicity failed to backup returning this error :

Copying duplicity-new-
signatures.20160504T132402Z.to.20160504T142402Z.sigtar. to local cache.
Attempt 1 failed. BackendException: scp get duplicity-new-
signatures.20160504T132402Z.to.20160504T142402Z.sigtar. failed: incorrect
response '#001scp: cephfs/project-api3/api//duplicity-new-
signatures.20160504T132402Z.to.20160504T142402Z.sigtar.: No such file or
directory'
...
With 5 attempts.

As you can see, Duplicity is looking for "duplicity-new-
signatures.20160504T132402Z.to.20160504T142402Z.sigtar." file, it's missing
"gpg" at the end of filename !

What could be the origin of this "bug" ?

Thank you.
```

Story points: 6.0
Slow exclude pattern matching (lp:#1576389)

[Original report](https://bugs.launchpad.net/bugs/1576389) created by **Arthur Peters (amp)**
```
Large exclude files cause the duplicity to scan files very slowly. My
suspicion is that the time is spent checking if each file matches the
exclude list. The problem is particularly noticable if the exclude files
includes the full prefix of every file. For example:
/home/user/data/something/excluded.dat
would be slower than
**/something/excluded.dat

I collected some time data and it appears that the time spent is increasing
rapidly, but linearly in the number of exclude patterns.
For example, with 0 patterns a backup of ~1700 file took 14s, with 40
patterns it took 72s. Based on this and the rest of the data on this backup
set every pattern adds about 1.5s to the time taken.  This was tested with
prefixes of the attached "excludeFromBackup-HDD.lst".

Removing the long prefixes and replacing them with ** shows a significant
improvement, but still shows noticeable slowdown. In this case the slowdown
was 0.1s per pattern. However I suspect the difference would increase with
more input files. This was tested with the attached "excludeFromBackup-
HDD2.lst".

I had better performance in 0.7.03. This is a bit of a problem for me since
my actual backups use just over 1800 patterns (generated from makefile
clean rules among other things).

I have also attached a file "dupl.log" with the first and last 200 lines.
The command line for this run was:
time duplicity full -v9 --exclude-filelist excludeFromBackup-HDD.lst
$PWD/JunkFromKitteh/ file://$HOME/tmp/tempbackup/

$PWD/JunkFromKitteh/ is the ~1700 files I tested with above.

All testing was on:
Ubuntu 15.10
duplicity 0.7.07.1 (from PPA)
Python 2.7.10
Target FS: local btrfs
```
Original tags: performance regression

Story points: 28.0
Duplicity cannot handle sparse files efficiently (lp:#1576051)

[Original report](https://bugs.launchpad.net/bugs/1576051) created by **Sven Mueller (smu-u)**
```
When running backup with duplicity, it cannot handle sparse files
efficiently.

I have a virtual machine image on my home dir,

26G -rw-r--r-- 1 libvirt-qemu kvm 33G huhti 28 08:59 dev02.img

And duplicity is backing it up 33G, when it should only back up 26G.

Steps to reproduce

1. Use qemu-image to create a sparse file
2. Run duplicity
3. Observe duplicity back it up whole instead of very small size
```

Story points: 22.0
Duplicity 0.7.07 completely fails to upload to Backblaze B2 backend (lp:#1571514)

[Original report](https://bugs.launchpad.net/bugs/1571514) created by **Lee W (z-ubunguone-z)**
```
Duplicity version - 0.7.07
Python version - 2.7.6
OS Distro and version - Ubuntu 14.04.04 LTS
Type of target filesystem: Linux, backing up to b2://

Log output from -v9 option - /usr/local/bin/duplicity -v9 --full-if-older-
than 1M --encrypt-key <gpg-key> --gpg-options=--always-trust --volsize 512
--asynchronous-upload /var/backups/test b2://<acct-
id>:<key>@test1234-fl/test

The b2 acct-ID and key have been verified as working via the Backblaze 'b2'
CLI tool and 'b2 authorize_account'.

Per the -v9 output, the request to
https://api001.backblaze.com/b2api/v1/b2_list_file_names returns with
status 200, but a request to
https://api001.backblaze.com/b2api/v1/b2_get_upload_url returns an HTTP 403
Forbidden.

Full -v9 output is here: http://www.fpaste.org/356908/09617151/
```
Original tags: b2 backblaze

Story points: 10.0
encountering locked excluded files wrongly considered as an error (lp:#1571204)

[Original report](https://bugs.launchpad.net/bugs/1571204) created by **mkoniecz (matkoniecz)**
```
--exclude shell_pattern

is documented as "Exclude the file or files matched by shell_pattern. If a
directory is matched, then files under that directory will also be matched.
See the FILE SELECTION section for more information."

but

duplicity /  --include '/home/user1' --exclude '**'
file:///media/mateusz/Backup/user1_backup

behaves in unexpected way with two errors reported:

Error accessing possibly locked file /lost+found
Error accessing possibly locked file /root

Given that these folders are matched by shell_pattern it should not matter
whatever duplicity may potentially access them - testing whatever file is
inaccessible and complaining should be done only for files supposed to be
included in backup.

This weird --include --exclude pattern was suggested in
http://askubuntu.com/questions/757358/restore-duplicity-archive-with-
entire-path-remembered/757608#757608 as solution for a different problem

duplicity --version
duplicity 0.6.23

python --version
Python 2.7.6


lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.4 LTS
Release:        14.04
Codename:       trusty

ext4 to ext4, the same partition
```

Story points: 14.0
unclear documentation in section "Known Issues / Bugs" (lp:#1570069)

[Original report](https://bugs.launchpad.net/bugs/1570069) created by **mkoniecz (matkoniecz)**
```
"Bad signatures will be treated as empty instead of logging appropriate
error message." - what does it mean?

From other parts of documentation I am guessing that with incremental
backup in case where Duplicity produced metadata was corrupted user will
not be notified and Duplicity will silently produce full backup.

But I am not 100% sure that my understanding is correct.

I think that it would be preferable to make "Known Issues / Bugs" section
clear, also for newcomers.
```

Story points: 6.0
broken formatting at http://duplicity.nongnu.org/duplicity.1.html#toc (lp:#1570067)

[Original report](https://bugs.launchpad.net/bugs/1570067) created by **mkoniecz (matkoniecz)**
```
at http://duplicity.nongnu.org/duplicity.1.html#toc section after "Query
Parameters" is titled in such way that indicates broken formatting
("mode=stripeThis mode (the default) performs round-robin access to the
list ofbackends. In this mode, all backends must be reliable as a loss of
onemeans a loss of one of the archive files.mode=mirrorThis mode accesses
backends as a RAID1-store, storing every file inevery backend and reading
files from the first-successful backend.A loss of any backend should result
in no failure. Note that backendsadded later will only get new files and
may require a manual syncwith one of the other operating
ones.onfail=continueThis setting (the default) continues all write
operations in asbest-effort. Any failure results in the next backend tried.
Failureis reported only when all backends fail a given operation with
theerror result from the last failure.onfail=abortThis setting considers
any backend write failure as a terminatingcondition and reports the
error.Data reading and listing operations are independent of this andwill
try with the next backend on failure.JSON File Example")
```

Story points: 6.0
B2 backend crashes when logging bad request (lp:#1568678)

[Original report](https://bugs.launchpad.net/bugs/1568678) created by **Mikael Moutakis (mikaelmoutakis)**
```
I'm using duplicity 0.7.06 on FreeBSD

I was backuping a large (100 gig+) dataset to Backblaze B2. The program
crashed with the following message

Traceback (most recent call last):
  File "/usr/local/bin/duplicity", line 1532, in <module>
    with_tempdir(main)
  File "/usr/local/bin/duplicity", line 1526, in with_tempdir
    fn()
  File "/usr/local/bin/duplicity", line 1380, in main
    do_backup(action)
  File "/usr/local/bin/duplicity", line 1501, in do_backup
    full_backup(col_stats)
  File "/usr/local/bin/duplicity", line 567, in full_backup
    globals.backend)
  File "/usr/local/bin/duplicity", line 448, in write_multivol
    (tdp, dest_filename, vol_num)))
  File "/usr/local/lib/python2.7/site-
packages/duplicity/asyncscheduler.py", line 146, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/usr/local/lib/python2.7/site-
packages/duplicity/asyncscheduler.py", line 172, in __run_synchronously
    ret = fn(*params)
  File "/usr/local/bin/duplicity", line 447, in <lambda>
    async_waiters.append(io_scheduler.schedule_task(lambda tdp,
dest_filename, vol_num: put(tdp, dest_filename, vol_num),
  File "/usr/local/bin/duplicity", line 338, in put
    backend.put(tdp, dest_filename)
  File "/usr/local/lib/python2.7/site-packages/duplicity/backend.py", line
374, in inner_retry
    code = _get_code_from_exception(self.backend, operation, e)
  File "/usr/local/lib/python2.7/site-packages/duplicity/backend.py", line
346, in _get_code_from_exception
    return backend._error_code(operation, e) or log.ErrorCode.backend_error
  File "/usr/local/lib/python2.7/site-
packages/duplicity/backends/b2backend.py", line 180, in _error_code
    return log.ErrorCode.bad_request
NameError: global name 'log' is not defined
```
Original tags: b2

Story points: 10.0
OverflowError while writing signatures (lp:#1558093)

[Original report](https://bugs.launchpad.net/bugs/1558093) created by **Compizfox (compizfox)**
```
Hi,

This is my first time backing up with Duplicity. I'm trying to backup
around 450 GB total. The command I use is:

duplicity --volsize 1000 --asynchronous-upload --full-if-older-than 1M
--encrypt-key "$keyid" --exclude-filelist "$excludelist" "$local" "$remote"

My first full backup is almost finished, but on the end I get this error:

Writing duplicity-full-signatures.20160315T174511Z.sigtar.gpg
WebDAV PUT /remote.php/webdav/duplicity_ILOS/duplicity-full-
signatures.20160315T174511Z.sigtar.gpg request with headers: {'Connection':
'keep-alive', 'Authorization': 'Basic '}
WebDAV data length: 3944337508
Attempt 1 failed. OverflowError: string longer than 2147483647 bytes

Because of this, it can't finish the backup. What should I do?

Duplicity version 0.7.06
Python version 2.7.11
FreeBSD 10.2-RELEASE

Thanks in advance.
```

Story points: 10.0
Can't restore my backup (lp:#1554109)

[Original report](https://bugs.launchpad.net/bugs/1554109) created by **Antonio (antonio-herrera)**
```
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.4 LTS
Release:        14.04
Codename:       trusty

Deja Dup 34.1

Restore Failed
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1494, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1488, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1337, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1370, in do_backup
    globals.archive_dir).set_values()
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
697, in set_values
    self.get_backup_chains(partials + backend_filename_list)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
819, in get_backup_chains
    map(add_to_sets, filename_list)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
809, in add_to_sets
    if set.add_filename(filename):
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
96, in add_filename
    self.set_manifest(filename)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
127, in set_manifest
    remote_filename)
AssertionError: ('duplicity-full.20160221T020245Z.manifest.tar.gz',
'duplicity-full.20160221T020245Z.manifest')

Please any help would be very much appreciated.  I have tried everything
but nothing seems to work.
```

Story points: 8.0
Problem restoring sparse files (lp:#1548549)

[Original report](https://bugs.launchpad.net/bugs/1548549) created by **hpgisler (hpgisleropen)**
```
Making a backup which contains sparse files (e.g. a file 'data' of size
100GB, with 1.3GB real size) seems to work, i.e. the backup.gpg files seem
of ok size, i.e.  1..2GB.

However, if restoring the file requires a lot of 'real' disk space (and
seems quite slow); after approx. 8GB diskspace had been consumed, I aborted
the restoring process.. Note, the disks size is approx 50GB. So, the
creation of a real 100GB file would fail..

I've tried to add the following option to duplicity: --rsync-options
--sparse
However, restoring fails completely now after the temporary restore file in
/temp folder reaches approx. 1GB (note: I've set the --volsize 1000), it
seems setting the sparse switch make things worse rather than better.

Perhaps I am missing something important here..
What is the correct handling procedure with large sparse files?
```

Story points: 10.0
restore --force complains about existing symlinks  and doesn't overwrite them (lp:#1545666)

[Original report](https://bugs.launchpad.net/bugs/1545666) created by **Martin Häcker (spamfaenger)**
```
We use duplicity to restore backups from a main machine to a backup
machine, where we use them to run a failover service.

Doing this we noticed that we get a) complaints about existing symlinks
(even though we use --force for the restore) and b) that those symlinks are
not updated if they change on the master machine.

I think that this is a bug, as --force as per the documentation should mean
that files are overwritten.

The command goes something like this:

'/usr/bin/python /usr/bin/duplicity --name mirroring --encrypt-key $key
--verbosity 4  -t now --force sftp://backup@backup.host/some_path
/root/redumpster/mirrordir'

And the output generated looks similar to this

```
Fehler '[Errno 17] File exists' beim Verarbeiten von some/path/symlink
```

Please note that I have verified that changes in the existing symlinks are
not overwritten by a subsequent `restore --force`.
```

Story points: 8.0