giseldo/neodataset huggingface dataset:
Archive failed (TypeError: %d format: a number is required, not NoneType in restart_position_iterator) (lp:#1024388)

[Original report](https://bugs.launchpad.net/bugs/1024388) created by **Anthony O. (netangel+launchpad)**
```
When saving my files, I've got that error :

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1241, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1234, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1207, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 416, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 262, in write_multivol
    restart_position_iterator(tarblock_iter)
  File "/usr/bin/duplicity", line 192, in restart_position_iterator
    ("/".join(last_index), last_block,
"/".join(tarblock_iter.previous_index)),
TypeError: %d format: a number is required, not NoneType

I'm using deja-dup (14.0.3-0ubuntu1).

Duplicity version : 0.6.08b-0ubuntu2
Python version : 2.6.5-0ubuntu1
OS Distro and version : Ubuntu 10.04.4 LTS (Linux myhost 2.6.32-41-generic
#91-Ubuntu SMP Wed Jun 13 11:43:55 UTC 2012 x86_64 GNU/Linux)
```
Original tags: archive deja-dup

Story points: 6.0
WebDav does not work 0.6.19 (lp:#1019678)

[Original report](https://bugs.launchpad.net/bugs/1019678) created by **Dmitry (diia)**
```
Duplicity version 0.6.19
Python version 2.7
OS Distro and version: Ubuntu Server 12.04
Type of target filesystem: webdav

The command doesnt upload anything to my webdav storage, i get this error:

Traceback (most recent call last):
  File "/usr/local/bin/duplicity", line 1236, in <module>
    with_tempdir(main)
  File "/usr/local/bin/duplicity", line 1229, in with_tempdir
    fn()
  File "/usr/local/bin/duplicity", line 1130, in main
    sync_archive()
  File "/usr/local/bin/duplicity", line 910, in sync_archive
    remlist = globals.backend.list()
  File "/usr/local/lib/python2.7/dist-
packages/duplicity/backends/webdavbackend.py", line 168, in list
    response = self.request("PROPFIND", self.directory, self.listbody)
  File "/usr/local/lib/python2.7/dist-
packages/duplicity/backends/webdavbackend.py", line 110, in request
    response = self.conn.getresponse()
  File "/usr/lib/python2.7/httplib.py", line 1018, in getresponse
    raise ResponseNotReady()
ResponseNotReady
```

Story points: 12.0
ftps/lftp backend ftp:ssl-protect-data should not be enforced (lp:#1010393)

[Original report](https://bugs.launchpad.net/bugs/1010393) created by **Eugene Crosser (crosser)**
```
Given that the files stored on the backend are encrypted anyway, there is
no need to make data transfer SSL-protected. On the other hand, using SSL
slows down transfter (there is especially bad case when LFTP is compiled
against gnutls library). I think that unconditionally including "set
ftp:ssl-protect-data true" is ill-adviced and should rather be removed. If
the user really needs that he or she can include this directive into their
.lftp/rc.
```

Story points: 12.0
OverflowError: join() is too long for a Python string  (lp:#1008343)

[Original report](https://bugs.launchpad.net/bugs/1008343) created by **duplicity (duplicity)**
```
# duplicity --version
duplicity 0.6.19

# duplicity incremental --no-encryption /storage file:///ba
Local and Remote metadata are synchronized, no sync needed.
Last full backup left a partial set, restarting.
Last full backup date: Mon May 21 08:35:03 2012
RESTART: Volumes 62030 to 62030 failed to upload before termination.
         Restarting backup at volume 62030.
Restarting after volume 62029, file data/data_backup/blogI.tar, block
7463456
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1391, in ?
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1384, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1354, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 500, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 346, in write_multivol
    restart_position_iterator(tarblock_iter)
  File "/usr/bin/duplicity", line 222, in restart_position_iterator
    while tarblock_iter.next():
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line 504,
in n                            ext
    result = self.process_continued(size)
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line 676,
in p                            rocess_continued
    data, last_block = self.get_data_block(self.process_fp, size - 512)
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line 662,
in g                            et_data_block
    if fp.close():
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line 431,
in c                            lose
    self.callback(self.sig_gen.getsig(), *self.extra_args)
  File "/usr/lib64/python2.4/site-packages/duplicity/librsync.py", line
214, in                             getsig
    return ''.join(self.sigstring_list)
OverflowError: join() is too long for a Python string
```

Story points: 12.0
huge memory usage on big files (lp:#1005478)

[Original report](https://bugs.launchpad.net/bugs/1005478) created by **nocturo (nocturo)**
```
Hello,

I'm running the following:
duplicity 0.6.19
Python 2.4.3
CentOS 5.8 x64

this is all on local backend.

Synopsis:
When backing up LVM space with huge 140GB file (image file) duplicity is
using huge amount of memory.

root      2781  0.4 80.7 2500696 1269952 ?     D    May26  13:29
/usr/bin/python /usr/bin/duplicity --archive-dir /backup/node/ --name
vm2842 --no-encryption --verbosity 4 --full-if-older-than 4W --volsize 100
--allow-source-mismatch --exclude-globbing-filelist
/etc/duply/vm2842/exclude /mnt/lvm/vm2842 file:///backup/node/vm2842

free -m output:
             total       used       free     shared    buffers     cached
Mem:          1536       1526          9          0          0         14
-/+ buffers/cache:       1511         24
Swap:         6143       2462       3680

I've attached pmap reference as well. It's stuck at this file for a while:
-rw------- 1 x x 138G May 23 08:45 /mnt/lvm/vm2842/home/x/imgs/msm-org-
production-sparse-small.img
```

Story points: 10.0
Support for non-default S3 regions (lp:#1003159)

[Original report](https://bugs.launchpad.net/bugs/1003159) created by **Luke (lnbeamer)**
```
Support for Amazon's S3 eastern data centers would be much appreciated!

The connections are already supported in Boto:
http://docs.pythonboto.org/en/latest/ref/s3.html#module-boto.s3.connection

Please expose these S3 features of Boto so that those of us located in non-
US/EU locations can take advantage of the better bandwidth of using buckets
that are physically closer.

For example, an option like --s3-region with the following options:
ap-northeast-1
ap-southeast-1
EU
sa-east-1
us-west-1
us-west-2
```

Story points: 50.0
Verify should only operate on changed files in incremental backup (lp:#982902)

[Original report](https://bugs.launchpad.net/bugs/982902) created by **Padfoot (padfoot)**
```
This is more of a wishlist request to change or extend the verify function.

Currently, verify will check every file (as if I have done a full backup)
rather than just the newly backed up files in an incremental backup.

To me, this seems a waste of processor resources, as presumably, the
unchanged files would have been verified on the initial full backup (if
users are following a recommended backup regime).

Doing an incremental backup is very fast indeed, yet when I verify
afterwards, it takes a very long time as it is not only verifying the
increnental changes, but all the already verified files.

May I request the verify option to check the entire backup for a full
backup and only the incremental file changes in an incremental backup?

Cheers.
```

Story points: 12.0
Problem backing up to S3 (lp:#569826)

[Original report](https://bugs.launchpad.net/bugs/569826) created by **pf (poon-fung)**
```
I tried to backup to S3 and got an error. I am running duplicity on Ubuntu
8.0.4LTS. The same command backs up to local file system worked just fine:

duplicity full --encrypt-key=$GPG_KEY --sign-key=$GPG_KEY --volsize=1024 \
        --archive-dir=/duplicity/archive --tempdir=/duplicity/tmp \
        --include=/data1 --include=/data2 --exclude=/ / file:///duplicity

But an error occuried when backing up to S3. The S3 bucket specified
already exists and is empty.

duplicity full --encrypt-key=$GPG_KEY --sign-key=$GPG_KEY --volsize=1024 \
        --archive-dir=/duplicity/archive --tempdir=/duplicity/tmp \
    --include=/data1 --include=/data2 --exclude=/ / s3+http://mybucket-
test-backup

Here is the console output:

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 828, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 821, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 795, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 336, in full_backup
    bytes_written = write_multivol("full", tarblock_iter, globals.backend)
  File "/usr/bin/duplicity", line 205, in write_multivol
    mf = manifest.Manifest().set_dirinfo()
  File "/usr/lib/python2.5/site-packages/duplicity/manifest.py", line 54,
in set_dirinfo
    self.local_dirname = globals.local_path.name
AttributeError: 'NoneType' object has no attribute 'name'

Main action: inc
/usr/bin/python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)]
duplicity 0.5.16 (April 21, 2009)
Linux test1.testdomain.com 2.6.24-19-server #1 SMP Wed Aug 20 23:54:28 UTC
2008 i686
Using temporary directory /duplicity/tmp/duplicity-zMjQZU-tempdir
Registering (mkstemp) temporary file /duplicity/tmp/duplicity-zMjQZU-
tempdir/mkstemp-h-S85K-1
Temp has 22722093056 available, backup will use approx 2469606195.
0 files exist on backend
Extracting backup chains from list of files: []
Collection Status
-----------------
Connecting with backend: BotoBackend
Archive dir: (() /duplicity/archive dir)

Found 0 backup chains without signatures.
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
Last full backup date: none
Registering (mktemp) temporary file /duplicity/tmp/duplicity-zMjQZU-
tempdir/mktemp-27JOP7-2
Using temporary directory /duplicity/archive/duplicity-HjOA7b-tempdir
Registering (mktemp) temporary file /duplicity/archive/duplicity-
HjOA7b-tempdir/mktemp-e0tL_C-1
AsyncScheduler: instantiating at concurrency 0
Registering (mktemp) temporary file /duplicity/tmp/duplicity-zMjQZU-
tempdir/mktemp-FSGBBU-3
Selecting /
Comparing () and None
Getting delta of (() / dir) and None
Generating delta - new file: .
Error accessing possibly locked file /dbdata
Selecting /duplicity
Comparing ('duplicity',) and None
Getting delta of (('duplicity',) /duplicity dir) and None
Generating delta - new file: duplicity
```

Story points: 6.0
Should detect unencypted backups even if --no-encryption omitted (lp:#550368)

[Original report](https://bugs.launchpad.net/bugs/550368) created by **Olivier Berger (olivierberger)**
```
When a backup was made with --no-encryption, subsequent calls to
collection-status should auto-detect that it was not ecnrypted.

At the moment, issueing duplicity collection-status file:///whatever/ will
report an error like :
IOError: [Errno 2] No such file or directory: '/whatever/duplicity-full-
signatures.20100306T090815Z.sigtar.gpg'

It could detect that there's indeed a duplicity-full-
signatures.20100306T090815Z.sigtar.gz instead of the .gpg file.

Hope this helps.

Best regards,
```

Story points: 28.0
Should not auto-create missing target dirs (lp:#543226)

[Original report](https://bugs.launchpad.net/bugs/543226) created by **Olivier Berger (olivierberger)**
```
Hi.

It seems that duplicity desn't check and report missing destination dir,
but instead auto-creates it.

That may seem convenient to users, but can lead to dangerous situations.

Imagine that the (local) target dir is file:///mnt/sda1/backups for
instance, where /mnt/sda1 is a mounted volume.

If for whatever reason, the volume hasn't been mounted, then next duplicity
run will start writing inside the mount point directory.

It is quite common, I suppose, to use external disks of big capacity to
relieve the main system's disks for backups... but then it would start
filling-up the main system.

I think a check for existance of the target dir would be the first thing to
do at the begining of the backup.

Thanks in advance.
```

Story points: 8.0
Setting --tmpdir on Synology fails (lp:#532244)

[Original report](https://bugs.launchpad.net/bugs/532244) created by **hasko (hasko)**
```
Installed duplicity 0.6.05 on a Synology disk station. Using --tmpdir
option and alternatively TMPDIR environment var renders the following
messages:

GnuPG passphrase:
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
No signatures found, switching to full backup.
Retype passphrase to confirm:
Cleanup of temporary directory /volume1/public/tmp/duplicity-AfRpy0-tempdir
failed - this is probably a bug.
Traceback (most recent call last):
  File "/opt/bin/duplicity-2.6", line 1241, in <module>
    with_tempdir(main)
  File "/opt/bin/duplicity-2.6", line 1234, in with_tempdir
    fn()
  File "/opt/bin/duplicity-2.6", line 1212, in main
    full_backup(col_stats)
  File "/opt/bin/duplicity-2.6", line 417, in full_backup
    globals.backend)
  File "/opt/bin/duplicity-2.6", line 295, in write_multivol
    globals.gpg_profile, globals.volsize)
  File "/opt/lib/python2.6/site-packages/duplicity/gpg.py", line 278, in
GPGWriteFile
    bytes_to_go = data_size - get_current_size()
  File "/opt/lib/python2.6/site-packages/duplicity/gpg.py", line 270, in
get_current_size
    return os.stat(filename).st_size
OSError: [Errno 2] No such file or directory:
'/volume1/public/tmp/duplicity-AfRpy0-tempdir/mktemp-ZlwWIw-2'

Is this me being stupid? The Synology acting up weird? A bug??

Python 2.6, OS: Synology DSM 2.2-0959, target file system Amazon S3. I'll
happily send the longer log if necessary, i.e. if it's really an obscure
bug.
```

Story points: 28.0
Allow testing an exclude file (output list of files it matches) (lp:#505366)

[Original report](https://bugs.launchpad.net/bugs/505366) created by **Daniel Hahler (blueyed)**
```
I'd like to have an interface/command to list the files that matches a
given include/exclude pattern/file.

USE CASE:
I'm facing a problem, where duplicity (0.5.06, Ubuntu Hardy) backs up far
too many files, and it appears to be related to my refactoring of excludes
in the exclude file, from:
  + /var/lib/vz/private/111/var/lib/mysql/sqldump
  + /var/lib/vz/private/142/var/lib/mysql/sqldump
  /var/lib/vz/private/111/var/lib/mysql/*/
  /var/lib/vz/private/142/var/lib/mysql/*/
to
  + **/var/lib/mysql/sqldump
  **/var/lib/mysql/*/*
  (and adding)
  **/var/lib/mysql/ibdata1
  **/var/lib/mysql/ib_logfile*

It would be very helpful to easily test those patterns to track this down.


```

Story points: 20.0
unbalanced parenthesis in password breaks duplicity (lp:#504413)

[Original report](https://bugs.launchpad.net/bugs/504413) created by **edso (ed.so)**
```
text from:
http://lists.gnu.org/archive/html/duplicity-talk/2009-10/msg00050.html

In connection to a bug in duply i stumbled over this one in duplicity ... I
used a password containing an unclosed parenthesis eg. 'test)foo' ... I
also spend some time to figure out a fix. It could be the replacement of
line 359 in backend.py (version 0.6.05) .. here a modified version of the
function. The commented line is the original. The replacement above splits
the url in protocol, credentials, rest and creates
'prot://[user:address@hidden' string which should suffice for obfuscation .
The documentation has to be adapted of course.

...ede


--FIX-->


def munge_password(self, commandline):
[SNIP]
if self.parsed_url.password:
return re.sub(r'^([^:/]+)://(([^:/@]*):?([^:/@]*))@?(.*)$',
r'\1://\3:address@hidden', commandline)
#return re.sub(self.parsed_url.password, '<passwd>', commandline)
else:
return commandline


---ERROR-->


Traceback (most recent call last):
File "/srv/www/vhosts/jamoke.net/_apps/duplicity-0.6.05/bin/duplicity",
line 1242, in <module>
with_tempdir(main)
File "/srv/www/vhosts/jamoke.net/_apps/duplicity-0.6.05/bin/duplicity",
line 1235, in with_tempdir
fn()
File "/srv/www/vhosts/jamoke.net/_apps/duplicity-0.6.05/bin/duplicity",
line 1136, in main
sync_archive()
File "/srv/www/vhosts/jamoke.net/_apps/duplicity-0.6.05/bin/duplicity",
line 915, in sync_archive
remlist = globals.backend.list()
File
"/srv/www/vhosts/jamoke.net/_apps/duplicity-0.6.05/duplicity/backends/ftpbackend.py",
line 109, in list
l = self.popen_persist(commandline).split('
')
File
"/srv/www/vhosts/jamoke.net/_apps/duplicity-0.6.05/bin/../duplicity/backend.py",
line 416, in popen_persist
private = self.munge_password(commandline)
File
"/srv/www/vhosts/jamoke.net/_apps/duplicity-0.6.05/bin/../duplicity/backend.py",
line 360, in munge_password
return re.sub(self.parsed_url.password, '<passwd>', commandline)
File "/usr/lib/python2.6/re.py", line 150, in sub
return _compile(pattern, 0).sub(repl, string, count)
File "/usr/lib/python2.6/re.py", line 243, in _compile
raise error, v # invalid expression
error: unbalanced parenthesis
```

Story points: 10.0
Unknown error while uploading duplicity-full-signatures (lp:#502609)

[Original report](https://bugs.launchpad.net/bugs/502609) created by **Olivier Berger (olivierberger)**
```
Hi,
I'am using Déjà dup since few days. I found it really convenient BTW :)
So, i was uploading my files since days and it finally reach 100% this
morning. It was now uploading a duplicity-full-signatures file and i had to
stop at 1,3G to continue later on.

Now when i restart Déjà-Dup i have an 'unknown error'. I suppose it's a
BUG.. I attached the log and config file

Here my version :
deja-dup        11.1-0ubuntu0karmic1
duplicity       0.6.06-0ubuntu0karmic1
Description:    Ubuntu 9.10

thanks indeed !
```

Story points: 26.0
IMAP Backend Error in _prepareBody (with python 2.4) (lp:#500902)

[Original report](https://bugs.launchpad.net/bugs/500902) created by **Mickey Knox (mickey-knox)**
```
Hi,

on an older system i have the following error.

duplicity 0.6.06
python 2.4.4

Args: /usr/bin/duplicity --verbosity 9 --encrypt-key FFFFFFFF --sign-key
FFFFFFFF --gpg-opti
ons= --exclude-globbing-filelist /root/.duply/some/exclude /var/www
imaps://someone%40g
ooglemail.com:somepass@imap.googlemail.com

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1236, in ?
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1229, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1207, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 416, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 315, in write_multivol
    (tdp, dest_filename)))
  File "/usr/lib/python2.4/site-packages/duplicity/asyncscheduler.py", line
148, in schedule
_task
    return self.__run_synchronously(fn, params)
  File "/usr/lib/python2.4/site-packages/duplicity/asyncscheduler.py", line
175, in __run_sy
nchronously
    ret = fn(*params)
  File "/usr/bin/duplicity", line 314, in <lambda>
    async_waiters.append(io_scheduler.schedule_task(lambda tdp,
dest_filename: put(tdp, dest
_filename),
  File "/usr/bin/duplicity", line 240, in put
    backend.put(tdp, dest_filename)
  File "/usr/lib/python2.4/site-
packages/duplicity/backends/imapbackend.py", line 131, in pu
t
    body=self._prepareBody(f,remote_filename)
  File "/usr/lib/python2.4/site-
packages/duplicity/backends/imapbackend.py", line 103, in _p
repareBody
    mp = email.MIMEMultipart.MIMEMultipart()
AttributeError: 'module' object has no attribute 'MIMEMultipart'
```

Story points: 6.0
IMAP Backend Error in put() if manifest-file larger than --volsize (lp:#500901)

[Original report](https://bugs.launchpad.net/bugs/500901) created by **Mickey Knox (mickey-knox)**
```
Hi,

im still trying to get duplicity to work with imap backend with a
googlemail.com account.

Unfortunatly i receive the following error:

duplicity 0.6.06
python 2.5.2

Args: /usr/bin/duplicity --verbosity 9 --encrypt-key 7FFFAFFF --sign-key
FFFFFF9
C --gpg-options= --exclude-globbing-filelist /root/.duply/someconf/exclude
/ imaps:/
/someone%40googlemail.com:somepass@imap.gmail.com/somedir/

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1236, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1229, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1207, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 416, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 315, in write_multivol
    (tdp, dest_filename)))
  File "/usr/lib/python2.5/site-packages/duplicity/asyncscheduler.py", line
148, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/usr/lib/python2.5/site-packages/duplicity/asyncscheduler.py", line
175, in __run_synchronously
    ret = fn(*params)
  File "/usr/bin/duplicity", line 314, in <lambda>
    async_waiters.append(io_scheduler.schedule_task(lambda tdp,
dest_filename: put(tdp, dest_filename),
  File "/usr/bin/duplicity", line 240, in put
    backend.put(tdp, dest_filename)
  File "/usr/lib/python2.5/site-
packages/duplicity/backends/imapbackend.py", line 135, in put
    self._conn.append(globals.imap_mailbox, None, None, body)
  File "/usr/lib/python2.5/imaplib.py", line 318, in append
    return self._simple_command(name, mailbox, flags, date_time)
  File "/usr/lib/python2.5/imaplib.py", line 1055, in _simple_command
    return self._command_complete(name, self._command(name, *args))
  File "/usr/lib/python2.5/imaplib.py", line 892, in _command_complete
    raise self.error('%s command error: %s %s' % (name, typ, data))
error: APPEND command error: BAD ['Could not parse command']
```

Story points: 12.0