Repository: borgbackup/borg

Issue:
Slow startup
It takes a long time for borg to just start up. This doesn't seem correct, and I don't think it was like this when I first installed the program.

$ time borg --version
borg 1.1.3

real	0m5.243s
user	0m0.193s
sys	0m0.040s

A little bit of further research indicates that the problem is in 'from borg.archiver import main' in the main script. Simply running python3.6 (which takes almost no time) and then typing 'import borg.archiver' similarly takes several seconds.

System: OSX 10.13.1. Borg installed with pip.




Discussion:
Sounds like a name resolving / DNS problem.

Make sure local names and IPs resolve correctly, in both directions.

I.E. make sure /etc/hosts has the usual stuff for localhost and your host name.
Another thing to note is that borg does some self-test at startup.
But on a usual 4y old laptop with ssd, this is still sub-second, even if you use the fat binary that first uncompresses itself into /tmp.
Everything resolves correctly. Only borg has problems / is slow to start. 

> On 22 Dec 2017, at 13:54, TW <notifications@github.com> wrote:
> 
> Sounds like a name resolving / DNS problem.
> 
> Make sure local names and IPs resolve correctly, in both directions.
> 
> I.E. make sure /etc/hosts has the usual stuff for localhost and your host name.
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub <https://github.com/borgbackup/borg/issues/3471#issuecomment-353656068>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJeMmAv24EcqtMJcjNBi3Y4MIS5aa6bMks5tC_rhgaJpZM4RLKZC>.
> 


OK, then do an strace to find out what it is. But I'm quite sure it is a local problem.
After some debugging I’ve traced the hang to borg/platform/__init__.py:

The very weird part is that if I comment out the entire if-elif-elif-block at the end, ‘import borg.platform' it does not hang. But if I comment out only the darwin part (I’m on OSX so sys.platform==‘darwin’), it still hangs.

> On 22 Dec 2017, at 21:23, TW <notifications@github.com> wrote:
> 
> OK, then do an strace to find out what it is. But I'm quite sure it is a local problem.
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub <https://github.com/borgbackup/borg/issues/3471#issuecomment-353701547>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJeMmM_elMKYxemuzkJEZjn6yUCcXTUWks5tDGQYgaJpZM4RLKZC>.
> 


To be precise, I have to comment out everything that loads the darwin or posix modules for borg/platform/__init__.py to not hang.

> On 22 Dec 2017, at 22:18, Tuomo Valkonen <tuomov@iki.fi> wrote:
> 
> After some debugging I’ve traced the hang to borg/platform/__init__.py:
> 
> The very weird part is that if I comment out the entire if-elif-elif-block at the end, ‘import borg.platform' it does not hang. But if I comment out only the darwin part (I’m on OSX so sys.platform==‘darwin’), it still hangs.
> 
>> On 22 Dec 2017, at 21:23, TW <notifications@github.com <mailto:notifications@github.com>> wrote:
>> 
>> OK, then do an strace to find out what it is. But I'm quite sure it is a local problem.
>> 
>> —
>> You are receiving this because you authored the thread.
>> Reply to this email directly, view it on GitHub <https://github.com/borgbackup/borg/issues/3471#issuecomment-353701547>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJeMmM_elMKYxemuzkJEZjn6yUCcXTUWks5tDGQYgaJpZM4RLKZC>.
>> 
> 


It appears socket.getfqdn() is broken and should not be used

https://bugs.python.org/issue5004 <https://bugs.python.org/issue5004>


> On 22 Dec 2017, at 22:23, Tuomo Valkonen <tuomov@iki.fi> wrote:
> 
> To be precise, I have to comment out everything that loads the darwin or posix modules for borg/platform/__init__.py to not hang.
> 
>> On 22 Dec 2017, at 22:18, Tuomo Valkonen <tuomov@iki.fi <mailto:tuomov@iki.fi>> wrote:
>> 
>> After some debugging I’ve traced the hang to borg/platform/__init__.py:
>> 
>> The very weird part is that if I comment out the entire if-elif-elif-block at the end, ‘import borg.platform' it does not hang. But if I comment out only the darwin part (I’m on OSX so sys.platform==‘darwin’), it still hangs.
>> 
>>> On 22 Dec 2017, at 21:23, TW <notifications@github.com <mailto:notifications@github.com>> wrote:
>>> 
>>> OK, then do an strace to find out what it is. But I'm quite sure it is a local problem.
>>> 
>>> —
>>> You are receiving this because you authored the thread.
>>> Reply to this email directly, view it on GitHub <https://github.com/borgbackup/borg/issues/3471#issuecomment-353701547>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJeMmM_elMKYxemuzkJEZjn6yUCcXTUWks5tDGQYgaJpZM4RLKZC>.
>>> 
>> 
> 


wtf.

I believed somehow that having own hostname in /etc/hosts is best practice, to avoid issue when dns malfunctions and also to speed up things.

But looks like it is also needed to work around such breakage. And they are "discussing" since 2009.

I'll have a deeper look into this before 1.1.4 (no promises). Maybe just add the /etc/hosts entries until a better solution is available?
@enkore ^
@vomout my impression is that python ticket is about hostname (short) vs. fqdn (long/full). For the purpose of creating a hopefully unique lock filename, borg wants the fqdn, but can also live with the short hostname (if unique).

Also, that's not quite your problem, your problem is slowness. And that slowness likely comes from DNS not working correctly and your system needing to do a DNS lookup as it can't be resolved otherwise. You have to check forward and reverse lookups.
No, the problem is that fqdn expects /etc/hosts to contain information that should not need to contain, and does not contain by default (I have never touched it, and should not have to touch it). Moreover, somehow getfqdn is very slow when it doesn’t find that information. Just doing the same lookup with ‘host’ on the command line is not slow.

> On 23 Dec 2017, at 12:43, TW <notifications@github.com> wrote:
> 
> @vomout <https://github.com/vomout> my impression is that python ticket is about hostname (short) vs. fqdn (long/full). For the purpose of creating a hopefully unique lock filename, borg wants the fqdn, but can also live with the short hostname (if unique).
> 
> Also, that's not quite your problem, your problem is slowness. And that slowness likely comes from DNS not working correctly and you system needing to do a DNS lookup as it can't be resolved otherwise. You have to check forward and reverse lookups.
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub <https://github.com/borgbackup/borg/issues/3471#issuecomment-353738999>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJeMmBJ5a4S6FmhEFe_hv1PUkDNyunCQks5tDTuwgaJpZM4RLKZC>.
> 


Also, I don’t understand why borg would need a unique hostname? A laptop almost by definition doesn’t have a fixed hostname.
After some playing around with /etc/hosts such that borg won't hang:

>>> import socket
>>> socket.getfqdn()
'1.0.0.127.in-addr.arpa’

That’s not a very useful ‘unique’ hostname.

> On 23 Dec 2017, at 17:09, Tuomo Valkonen <tuomov@iki.fi> wrote:
> 
> No, the problem is that fqdn expects /etc/hosts to contain information that should not need to contain, and does not contain by default (I have never touched it, and should not have to touch it). Moreover, somehow getfqdn is very slow when it doesn’t find that information. Just doing the same lookup with ‘host’ on the command line is not slow.
> 
>> On 23 Dec 2017, at 12:43, TW <notifications@github.com <mailto:notifications@github.com>> wrote:
>> 
>> @vomout <https://github.com/vomout> my impression is that python ticket is about hostname (short) vs. fqdn (long/full). For the purpose of creating a hopefully unique lock filename, borg wants the fqdn, but can also live with the short hostname (if unique).
>> 
>> Also, that's not quite your problem, your problem is slowness. And that slowness likely comes from DNS not working correctly and you system needing to do a DNS lookup as it can't be resolved otherwise. You have to check forward and reverse lookups.
>> 
>> —
>> You are receiving this because you were mentioned.
>> Reply to this email directly, view it on GitHub <https://github.com/borgbackup/borg/issues/3471#issuecomment-353738999>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJeMmBJ5a4S6FmhEFe_hv1PUkDNyunCQks5tDTuwgaJpZM4RLKZC>.
>> 
> 


I could reproduce the slowness by removing the (usually present) entry from /etc/hosts:
```
127.0.1.1 tux.mydomain.org tux
```
Instead, I added it to DNS so that short hostname (from /etc/hostname) + "." + domain (from /etc/resolv.conf) gets resolved.

After that, I had slowness with `socket.getfqdn()` while `socket.getaddrinfo(host_short, None, 0, SOCK_DGRAM, 0, AI_CANONNAME)` worked quickly - as proposed in the patch in the python issue tracker: https://bugs.python.org/file29919/python2.7-socket-getfqdn.patch

Issue: https://bugs.python.org/issue5004

Resolving patch:
diff --git a/src/borg/helpers.py b/src/borg/helpers.py
--- a/src/borg/helpers.py
+++ b/src/borg/helpers.py
@@ -139,7 +139,7 @@ def check_extension_modules():
         raise ExtensionModuleError
     if borg.crypto.low_level.API_VERSION != '1.1_02':
         raise ExtensionModuleError
-    if platform.API_VERSION != platform.OS_API_VERSION != '1.1_02':
+    if platform.API_VERSION != platform.OS_API_VERSION != '1.1_03':
         raise ExtensionModuleError
     if item.API_VERSION != '1.1_02':
         raise ExtensionModuleError
@@ -664,8 +664,8 @@ def format_line(format, data):
 
 def replace_placeholders(text):
     """Replace placeholders in text with their values."""
+    from .platform import fqdn
     current_time = datetime.now()
-    fqdn = socket.getfqdn()
     data = {
         'pid': os.getpid(),
         'fqdn': fqdn,
diff --git a/src/borg/platform/__init__.py b/src/borg/platform/__init__.py
--- a/src/borg/platform/__init__.py
+++ b/src/borg/platform/__init__.py
@@ -10,12 +10,12 @@
 from .base import set_flags, get_flags
 from .base import SaveFile, SyncFile, sync_dir, fdatasync, safe_fadvise
 from .base import swidth, API_VERSION
-from .base import process_alive, get_process_id, local_pid_alive
+from .base import process_alive, get_process_id, local_pid_alive, fqdn, hostname, hostid
 
 OS_API_VERSION = API_VERSION
 
 if not sys.platform.startswith(('win32', )):
-    from .posix import process_alive, get_process_id, local_pid_alive
+    from .posix import process_alive, local_pid_alive
 
 if sys.platform.startswith('linux'):  # pragma: linux only
     from .linux import API_VERSION as OS_API_VERSION
diff --git a/src/borg/platform/base.py b/src/borg/platform/base.py
--- a/src/borg/platform/base.py
+++ b/src/borg/platform/base.py
@@ -1,5 +1,7 @@
 import errno
 import os
+import socket
+import uuid
 
 from borg.helpers import truncate_and_unlink
 
@@ -15,7 +17,7 @@
 are correctly composed into the base functionality.
 """
 
-API_VERSION = '1.1_02'
+API_VERSION = '1.1_03'
 
 fdatasync = getattr(os, 'fdatasync', os.fsync)
 
@@ -183,12 +185,44 @@ def swidth(s):
     return len(s)
 
 
+# patched socket.getfqdn() - see https://bugs.python.org/issue5004
+def getfqdn(name=''):
+    """Get fully qualified domain name from name.
+
+    An empty argument is interpreted as meaning the local host.
+    """
+    name = name.strip()
+    if not name or name == '0.0.0.0':
+        name = socket.gethostname()
+    try:
+        addrs = socket.getaddrinfo(name, None, 0, socket.SOCK_DGRAM, 0, socket.AI_CANONNAME)
+    except socket.error:
+        pass
+    else:
+        for addr in addrs:
+            if addr[3]:
+                name = addr[3]
+                break
+    return name
+
+
+# for performance reasons, only determine hostname / fqdn / hostid once.
+# XXX this sometimes requires live internet access for issuing a DNS query in the background.
+hostname = socket.gethostname()
+fqdn = getfqdn(hostname)
+hostid = '%s@%s' % (fqdn, uuid.getnode())
+
+
 def get_process_id():
     """
-    Return identification tuple (hostname, pid, thread_id) for 'us'. If this is a FUSE process, then the PID will be
-    that of the parent, not the forked FUSE child.
+    Return identification tuple (hostname, pid, thread_id) for 'us'.
+    This always returns the current pid, which might be different from before, e.g. if daemonize() was used.
+
+    Note: Currently thread_id is *always* zero.
     """
-    raise NotImplementedError
+    thread_id = 0
+    pid = os.getpid()
+    return hostid, pid, thread_id
 
 
 def process_alive(host, pid, thread):


Test patch: