giseldo/neodataset huggingface dataset:
Automatically make a log file and store it next to the backup files (lp:#1165192)

[Original report](https://bugs.launchpad.net/bugs/1165192) created by **Otto Kekäläinen (otto)**
```
I'd like to be able to view log information at the backup target, not
source.

Current use of option --log-file and --log-fd are both related to logging
at the source, i.e. logging at the computer where the backup if made.

I'd also like to have logs at the target, i.e. logs saved at the second
computer (or folder) where the backups where sent.

Compare to rdiff-backup: in the target directory, there is a subdirectory
"rdiff-backup" that contains a few log files.

Feature request: please automatically write a small log in the backup
target directory. Looking at the log an administrator should easily be able
to see when the last run was made, if there where any errors and in case
some runs had errors, the log should enable the admin to easily check the
time of the last fully successful log.
```

Story points: 8.0
fails to backup (lp:#1158984)

[Original report](https://bugs.launchpad.net/bugs/1158984) created by **Timski (stuben-rein)**
```
deja-dup backups some files and then exits with following error message

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1411, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1404, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1379, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 509, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 386, in write_multivol
    globals.gpg_profile, globals.volsize)
  File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 327, in
GPGWriteFile
    bytes_to_go = data_size - get_current_size()
  File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 320, in
get_current_size
    return os.stat(filename).st_size
OSError: [Errno 2] No such file or directory: '/home/tim/.cache/deja-
dup/tmp/duplicity-eoJhZh-tempdir/mktemp-RpMdBq-24'

I'm using
Ubuntu Raring Ringtail (development branch)
deja-dup        25.5-0ubuntu1
duplicity       0.6.21-0ubuntu1

----------------------------------
/tmp/deja-dup.gsettings

org.gnome.DejaDup backend 'file'
org.gnome.DejaDup delete-after 0
org.gnome.DejaDup exclude-list ['/home/tim/.local/share/Trash']
org.gnome.DejaDup full-backup-period 90
org.gnome.DejaDup include-list ['/home/tim']
org.gnome.DejaDup last-backup '2013-03-22T12:09:33.361212Z'
org.gnome.DejaDup last-restore '2012-12-22T02:01:09.090993Z'
org.gnome.DejaDup last-run '2013-03-22T12:09:33.361212Z'
org.gnome.DejaDup nag-check '2013-01-24T05:03:32.879047Z'
org.gnome.DejaDup periodic true
org.gnome.DejaDup periodic-period 7
org.gnome.DejaDup prompt-check '2013-01-23T23:23:12.723530Z'
org.gnome.DejaDup root-prompt true
org.gnome.DejaDup welcomed true
org.gnome.DejaDup.File icon '. GThemedIcon drive-harddisk-usb drive-
harddisk drive'
org.gnome.DejaDup.File name 'WDC WD75 00BPKT-00PK4T0: Beate'
org.gnome.DejaDup.File path '/home/tim/deja-dup'
org.gnome.DejaDup.File relpath b'brussel'
org.gnome.DejaDup.File short-name 'Beate'
org.gnome.DejaDup.File type 'volume'
org.gnome.DejaDup.File uuid '1425bdc1-3529-443d-a72a-fa67704d02d7'
org.gnome.DejaDup.Rackspace container 'brussel'
org.gnome.DejaDup.Rackspace username ''
org.gnome.DejaDup.S3 bucket ''
org.gnome.DejaDup.S3 folder 'brussel'
org.gnome.DejaDup.S3 id ''
org.gnome.DejaDup.U1 folder '/deja-dup/brussel'

----------------------------------

can't find a /tmp/deja-dup.log
```

Story points: 10.0
RFE: Support v2.0 of CLOUDFILES_AUTHURL (lp:#1152145)

[Original report](https://bugs.launchpad.net/bugs/1152145) created by **nodata (ubuntu-nodata)**
```
duplicity only supports version 1.0 of the cloud files authentication
scheme, e.g.
     CLOUDFILES_AUTHURL=https://auth.api.rackspacecloud.com/v1.0

This is deprecated, as can be seen here:
 <version id="v1.0" status="DEPRECATED" updated="2012-03-19T22:30:00Z">

Would it be possible to get version 2.0 support?
```

Story points: 6.0
Online documentation and man page needs updating (lp:#1130819)

[Original report](https://bugs.launchpad.net/bugs/1130819) created by **duebbert (kai-i)**
```
I was tearing my hair out because S3 backups didn't work in latest 0.6.21
version only to find by accident that there are options which are not
documented online or in the man page, e.g. --s3-use-multiprocessing
(without this switch S3 backup doesn't work but I'll file a separate bug
for that.)

Please update the documentation to show all options that are available.
```

Story points: 6.0
cloudfiles backend slow (lp:#1130649)

[Original report](https://bugs.launchpad.net/bugs/1130649) created by **Soren Hansen (soren)**
```
I noticed that the cloudfiles backend was incredibly slow. After poking
around for a bit, I've realised that the culprit is the call to
socket.getdefaulttimeout() in backend.py.

I created a simple test script to upload a 100 MB file to Cloud Files. It
does pretty much exactly what the cloudfiles backend does:


import os
import socket
from cloudfiles import Connection
from cloudfiles.errors import ResponseError
from cloudfiles import consts

conn_kwargs = {}
conn_kwargs['username'] = os.environ['CLOUDFILES_USERNAME']
conn_kwargs['api_key'] = os.environ['CLOUDFILES_APIKEY']
conn_kwargs['authurl'] = consts.default_authurl

conn = Connection(**conn_kwargs)
container = conn.create_container('speedtest')
sobject = container.create_object('100Mtest2rnd')
sobject.load_from_filename('100Mtest.rnd')


If I run it like that, it takes around 15 seconds to upload 100 MB. If I
add a call to socket.setdefaulttimeout(30) before the  Connection call, it
takes 11 *minutes*.

If I strace the two runs, I see a call to poll() before each write(). This
gets added by Python's socketmodule.c due to the defaulttimeout.

I tried adding timestamps to the strace log and counted how many system
calls each of the two runs makes over the course of a single second while
transferring the data. With the default sockettimeout, I got just over 100
system calls (poll, write, read, poll, write, read, etc.). It's dealing
with a block size of 4kB, so that's 4kB*(100/3) = 133 kB/s. Without the
default socket timeout, I got  around 1120 system calls (read, write, read,
write, etc.). That translates to 4kB*(1120/2) = 2240 kB/s. That's a pretty
hefty difference.

What confuses me, though, is that cloudfiles.Connection.__init__() also
calls socket.setdefaulttimeout (passing it a value of 5 by default).
```

Story points: 6.0
SSH connection timeouts, duplicity does not respect --timeout option (lp:#1129188)

[Original report](https://bugs.launchpad.net/bugs/1129188) created by **Olivier - interfaSys (olivier-interfasys)**
```
Our backup server seems to now take a long time to go through the
authentication process or something is wrong with duplicity.

In order to determine if the long process is the problem, I've tried to
raise the timeout limit, but duplicity doesn't seem to respect that.

The backend used is scp (scp://hostname), but sftp doesn't work either.
The options I use include:
 --timeout=160 --ssh-options='-oConnectTimeout=160'

I even edited globals.py to see if it would change something. It didn't.

The error message, using the wrapper duply, is:
--- Start running command STATUS at 16:16:04.000 ---
BackendException: ssh connection to hostname:22 failed: [Errno 64] Host is
down
16:16:19.000 Task 'STATUS' failed with exit code '23'.
--- Finished state FAILED 'code 23' at 16:16:19.000 - Runtime 00:00:15.000
---

It says the runtime is 15 seconds, which is way shorter than the timeout.

Connecting to the server directly via scp or sftp works.
```

Story points: 6.0
error message:  Backup location ‘/tmp/duplicity-gbDXuc-tempdir/mktemp-LJFF1G-430’ does not exist. (lp:#1122066)

[Original report](https://bugs.launchpad.net/bugs/1122066) created by **John Kirby (john-kirbynet)**
```
Backup location ‘/tmp/duplicity-gbDXuc-tempdir/mktemp-LJFF1G-430’ does not
exist.

Message received after several hours running.

First time I have tried to use the backup supplied as part of the Ubuntu
12.04 distribution.

I created a directory on a stand alone HD, configured the storage and
folder tabs, and kicked off the backup for the first time.
```

Story points: 6.0
Duplicity performance issue with /etc/localtime (lp:#1112450)

[Original report](https://bugs.launchpad.net/bugs/1112450) created by **Tomáš Varga (tomas-varga)**
```
Every 10 minutes my own daemon on my server executes duplicity to backup
about 100 MB of data. When doing full backup, it's ok, but sometimes while
doing incremental backup it takes at least 20 minutes and whole time
consumes 100 % of one CPU's core.

Some info about the data:
It's a map from game named Minecraft. (Bug caused by data format?)
The game server is running while it's doing the backup. I may not stop it.
(Cause - file being sometimes overwritten during backup?)

Info about backup directory:
It's 2121464904 B large (2023,186 MB) and contains 3724 files. (Too much
files in one directory?)

Duplicity version (duplicity --version): duplicity 0.6.08b
Python version (python --version): Python 2.6.6
OS (cat /etc/issue): Debian GNU/Linux 6.0 \n \l

root@delorean:~/duplicity# nohup strace duplicity -v9 --no-encryption
/home/gate/world file:///opt/backup >dup 2>str &
[1] 3678
Every second there were about 4500 lines written into strace's output file.

root@delorean:~/duplicity# du -sh *
1,8M    dup
1,1G    str
root@delorean:~/duplicity# wc -l *
     15442 dup
  16614942 str
  16630384 celkem
root@delorean:~/duplicity# grep -vEc '^stat\("/etc/localtime"' str
35628

The duplicity output file is attached and so is strace output file without
stat("/etc/localtime", ...) syscalls (in .tar file).

So, is this a bug or am I to rotate backups (move old to other folder)
every few days?

[edit]
I'm not sure if the attachment got uploaded because I can't find it
anywhere, so I'm pasting a link: http://213.129.147.24/dupstr.tar
[/edit]
```
Original tags: bug cpu duplicity issue kernel localtime performance stat syscall

Story points: 6.0
Server connection dropped (lp:#1108284)

[Original report](https://bugs.launchpad.net/bugs/1108284) created by **Pivert (g-launchpad-pivert-org)**
```
Some transfer are made during every try, but
I'm permanently having this kind of error:
.....
Comparing ('herbert3.img',) and None
Getting delta of (('herbert3.img',) /mnt/bkp_fde/current/vmstorage/vm-
storage-pool/herbert3.img reg) and None
A herbert3.img
AsyncScheduler: instantiating at concurrency 0
Registering (mktemp) temporary file /tmp/duplicity-
NoVY5L-tempdir/mktemp-1pbtjP-3
AsyncScheduler: running task synchronously (asynchronicity disabled)
Deleting /tmp/duplicity-NoVY5L-tempdir/mktemp-1pbtjP-3
Forgetting temporary file /tmp/duplicity-NoVY5L-tempdir/mktemp-1pbtjP-3
AsyncScheduler: task completed successfully
Processed volume 859
No handlers could be found for logger "paramiko.transport"
Removing still remembered temporary file /tmp/duplicity-
NoVY5L-tempdir/mkstemp-z5fIGD-1
Backend error detail: Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1411, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1404, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1374, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 510, in full_backup
    sig_outfp.to_remote()
  File "/usr/lib/python2.7/dist-packages/duplicity/dup_temp.py", line 184,
in to_remote
    globals.backend.move(tgt) #@UndefinedVariable
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 363,
in move
    self.put(source_path, remote_filename)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/sshbackend.py",
line 191, in put
    raise BackendException("sftp put of %s (as %s) failed: %s" %
(source_path.name,remote_filename,e))
BackendException: sftp put of
/root/.cache/duplicity/86878d6befd060f5e96646260ca047da/duplicity-full-
signatures.20130120T21255
 duplicity-full-signatures.20130120T212556Z.sigtar.gpg) failed: Server
connection dropped:

BackendException: sftp put of
/root/.cache/duplicity/86878d6befd060f5e96646260ca047da/duplicity-full-
signatures.20130120T21255
 duplicity-full-signatures.20130120T212556Z.sigtar.gpg) failed: Server
connection dropped:
Backup Ended
```

Story points: 8.0
Impossible to backup several directories without reading whole disk (lp:#1096492)

[Original report](https://bugs.launchpad.net/bugs/1096492) created by **vsespb (vi1tsr)**
```
Hello.

I need to backup /etc and /home/vse/.rvm into single location. I assume
correct way to do this is:

duplicity --no-encryption --include /etc  --include /home/vse/.rvm/
--exclude "**" / file:///home/duplicity
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
No signatures found, switching to full backup.
Error accessing possibly locked file /home/vse/.gvfs

"Error accessing possibly locked file /home/vse/.gvfs" - I assume that
means that it tries read whole disk (which is a bug cause it's slow and i
can contain other mounted filesystems) ? Why it tries to access
/home/vse/.gvfs if I need just /etc and /home/vse/.rvm ?

(duplicity 0.6.08b Ubuntu 10.04 Python,  EXT4)
```

Story points: 10.0
Déjà Dup guesses the wrong hostname (lp:#1086068)

[Original report](https://bugs.launchpad.net/bugs/1086068) created by **Jeroen Hoek (mail-jeroenhoek)**
```
deja-dup 22.0-0ubuntu2
duplicity 0.6.18-0ubuntu3
Ubuntu 12.04.1 LTS

Ubuntu currently creates /etc/hosts with these two lines, with
"computername" being the name of my computer:

127.0.0.1       localhost
127.0.1.1       computername

This works fine most of the time, but can cause problems with some
software. As a work-around I have changed /etc/hosts to look like this:

127.0.0.1       localhost computername

In both cases /etc/hostname and `hostname` say "computername", and
computername is pingable.

Now, this is perfectly valid, and should not have an effect on Déjà Dup.
Unfortunately when I configure my /etc/hosts like this, Déjà Dup now thinks
my computer is called "localhost" instead "computername", and warns me when
I try to backup to a location containing previous backups filed under
"computername".

Why is Déjà Dup guessing my computer is called "localhost"? Shouldn't
`hostname` or /etc/hostname be authoritative in this case?
```

Story points: 6.0
selecting testcase for otherFilesystem makes bad assumption (lp:#1084121)

[Original report](https://bugs.launchpad.net/bugs/1084121) created by **Thomas Eriksson (scrizt)**
```
duplicity-0.6.18

In /testing/tests/selectiontest.py, lines 249 - 272, there is an assumption
that / and /usr/bin resides in the same filesystem. I'd say it's common
that they do not, and suggest changing the test to use /bin instead of
/usr/bin.

Patch suggestion:
--- duplicity-0.6.18.orig/testing/tests/selectiontest.py
+++ duplicity-0.6.18/testing/tests/selectiontest.py
@ @ -252,12 +252,12@ @
         select = Select(root)
         sf = select.other_filesystems_get_sf(0)
         assert sf(root) is None
-        if os.path.ismount("/usr/bin"):
+        if os.path.ismount("/bin"):
             sfval = 0
         else:
             sfval = None
-        assert sf(Path("/usr/bin")) == sfval, \
-               "Assumption: /usr/bin is on the same filesystem as /"
+        assert sf(Path("/bin")) == sfval, \
+               "Assumption: /bin is on the same filesystem as /"
         if os.path.ismount("/dev"):
             sfval = 0
         else:
```

Story points: 6.0
webdav backend: fails on 302/Redirect responses (lp:#1079475)

[Original report](https://bugs.launchpad.net/bugs/1079475) created by **az (az-debian)**
```
this is a forward of debian bug #693370 which lives over there:
http://bugs.debian.org/693370

the original reporter can't use webdav with box.net because duplicity
interprets
the 302 Found that PROPFIND returns as an indication of failure.
the debian bug report is for version 0.6.08, but the problem is also
present in the newer versions
incl. 0.6.20 as there have been no relevant changes to the webdav backend.

regards
az
```

Story points: 6.0
--ignore-errors not working, bad variable name in commandline.py (lp:#1079183)

[Original report](https://bugs.launchpad.net/bugs/1079183) created by **Mikko Ohtamaa (mikko-red-innovation)**
```
In commandline.py:

 parser.add_option("--ignore-errors", action="callback",
                      dest="ignore_errors",
                      callback=lambda o, s, v, p: (log.Warn(
                          _("Running in 'ignore errors' mode due to %s;
please "
                            "re-consider if this was not intended") % s),
                          setattr(p.values, "ignore errors", True)))


It is setting global variable "ignore errors" when it should be
"ignore_errors", making --ignore-errors command line switch not working.

Trunk version.
```

Story points: 6.0
GID wrong (0) for os.chown when copying or moving files (lp:#1077647)

[Original report](https://bugs.launchpad.net/bugs/1077647) created by **Ralf Herold (ralf-herold)**
```
Using duplicity 0.6.20, python 2.7.2, gpg 2.0.17 under Mac OS X 10.8, the
following error is reproducible when trying to list remote backup (WebDAV)
contents on a different computer from where the backup was made:

Traceback (most recent call last):
  File "/opt/local/bin/duplicity", line 1403, in <module>
    with_tempdir(main)
  File "/opt/local/bin/duplicity", line 1396, in with_tempdir
    fn()
  File "/opt/local/bin/duplicity", line 1272, in main
    sync_archive(decrypt)
  File "/opt/local/bin/duplicity", line 1072, in sync_archive
    copy_to_local(fn)
  File "/opt/local/bin/duplicity", line 1021, in copy_to_local
    tdp.move(globals.archive_dir.append(loc_name))
  File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/duplicity/path.py", line 617, in move
    self.copy(new_path)
  File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/duplicity/path.py", line 443, in copy
    self.copy_attribs(other)
  File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/duplicity/path.py", line 448, in copy_attribs
    util.maybe_ignore_errors(lambda: os.chown(other.name, self.stat.st_uid,
self.stat.st_gid))
  File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/duplicity/util.py", line 65, in maybe_ignore_errors
    return fn()
  File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/duplicity/path.py", line 448, in <lambda>
    util.maybe_ignore_errors(lambda: os.chown(other.name, self.stat.st_uid,
self.stat.st_gid))
OSError: [Errno 1] Operation not permitted:
'/Users/<username>/.cache/duplicity/duply_<backupname>/duplicity-full-
signatures.20120929T191652Z.sigtar.gz'

In "copy" calling "copy_attribs", the self.stat.st_uid is correctly set
(e.g., 504) but the self.stat.st_gid is 0, which would be root and this
throws an OSError message as duplicity is run as normal user. I could not
find where in duplicity/path.py the gid is determined; this may be an issue
with the ROPath object?

This is a temporary fix: In duplicity/path.py, line 448, change
self.stat.st_gid to -1:

-  util.maybe_ignore_errors(lambda: os.chown(other.name, self.stat.st_uid,
self.stat.st_gid))
+ util.maybe_ignore_errors(lambda: os.chown(other.name, self.stat.st_uid,
-1))

(--ignore-errors does not fix the OSError.)

I have seen an informal report for duply
(http://niebegeg.net/post/32010111657/duplicity-backups-wiederherstellen)
which mentions above issue as well but this may not have been submitted as
bug.

Many thanks -
```

Story points: 12.0
Import error after update to 0.6.20 (lp:#1075766)

[Original report](https://bugs.launchpad.net/bugs/1075766) created by **iceflatline (iceflatline)**
```
I am receiving the following error message on all duplicity commands after
an update to duplicity-0.6.20

"Import of duplicity.backends.u1backend Failed: No module named httplib2"

I am using FreeBSD 9.0-Release; duplicity-0.6.19_2 updated to
duplicity-0.6.20 using portmaster.
```

Story points: 8.0
backup failed after moving backup location (lp:#1049002)

[Original report](https://bugs.launchpad.net/bugs/1049002) created by **Api (aa-pp-ii+launchpad)**
```
How to reproduce:
- full backup to a remote SSH location A
- then move the remote backup location to B and update the local cache
files accordingly, so that dejadup continues with incremental backups wrt
the previous full backup
I followed this guide http://blog.linux2go.dk/2011/01/20/moving-duplicity-
and-hence-deja-dup-backups/
- then run dejadup again: you expect an incremental backup to location B,
instead you get the following error
- as a side effect, DD deletes all local cache files, even those of other
duplicity backups

Ubuntu 12.04.1 LTS 64 bit
deja-dup        22.0-0ubuntu2
duplicity       0.6.19-0ubuntu0ppa18~precise1

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1391, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1384, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1264, in main
    globals.archive_dir).set_values()
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
696, in set_values
    self.get_backup_chains(partials + backend_filename_list)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
819, in get_backup_chains
    map(add_to_sets, filename_list)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
809, in add_to_sets
    if set.add_filename(filename):
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line
97, in add_filename
    (self.volume_name_dict, filename)
AssertionError: ({1: 'duplicity-full.20120625T080651Z.vol1.difftar.gpg', 2:
'duplicity-full.20120625T080651Z.vol2.difftar.gpg', 3: 'duplicity-
...
...
...
full.20120625T080651Z.vol943.difftar.gpg', 944: 'duplicity-
full.20120625T080651Z.vol944.difftar.gpg', 945: 'duplicity-
full.20120625T080651Z.vol945.difftar.gpg', 946: 'duplicity-
full.20120625T080651Z.vol946.difftar.gpg'}, 'duplicity-
full.20120625T080651Z.vol606.difftar.gpg')
```

Story points: 18.0
Invalid Manifest after Restart (lp:#1035349)

[Original report](https://bugs.launchpad.net/bugs/1035349) created by **jocen Lwer (jocenlwer)**
```
Hallo,

Versions: duplicity version 0.6.19, python 2.7.3, Ubuntu 12.04 (duplicity
is from Ubuntu 12.10)
Same Bug with 0.6.18 which is in Ubuntu 12.04
All files are in the zip, that is attached
Used duply-profile 'conf'

First I make a New Backup (run.1.log) an kill them, when it copies data.
The Manifest 'duplicity-full.20120810T143459Z.manifest.part' look like:
...
Volume 2:
    StartingPath   060521-002.MPG 6486
    EndingPath     060521-002.MPG 12961
    Hash SHA1 bd9e4caa974186f3201f15c518dfee46056a6683
Second I restart (run.2.log) with returned ok.
But the Manifest look like:
Volume 2:
    StartingPath   060521-002.MPG 6486
    EndingPath     060521-002.MPG 12961
    Hash SHA1 bd9e4caa974186f3201f15c518dfee46056a6683
Volume 2:
    StartingPath   060521-002.MPG 6486
    EndingPath     060521-002.MPG 12961
    Hash SHA1 dd8cbafc81d6d53eb2cb5df7be37fd3c397ad2cc
And Third when I make a delta backup (run.3.log)  (nothing is changed), it
nit work correct!

Ciao, Jocen
```

Story points: 10.0