Repository: borgbackup/borg

Issue:
1.2.0 Cache permission issues

## Have you checked borgbackup docs, FAQ, and open Github issues?
Yes

## Is this a BUG / ISSUE report or a QUESTION?
BUG/ISSUE

## System information. For client/server mode post info for both machines.
Debian 11 as a Gen2 Hyper-V VM.

#### Your borg version (borg -V).
1.2.0 (does not work as intended) & 1.117 (works as intended)

#### Operating system (distribution) and version.
Debian 11 as a Gen2 Hyper-V VM. - 5.10.0-11-amd64

#### Hardware / network configuration, and filesystems used.
Filesystem: ext4

## Describe the problem you're observing.
Borg 1.2.0 is setting permission of cache files to 700, no matter what --umask or "umask" in script is set to.
This causes problems when running borg on different computers using a single shared cache (network share), as only the owner will have permission, and not the group.

#### Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.
Yes,  script:
```
#!/bin/bash

export LC_ALL=C
export BORG_CONFIG_DIR="Borg_config_test"
export BORG_CACHE_DIR="Borg_cache_test"
export BORG_SECURITY_DIR="${BORG_CONFIG_DIR}/security"
export BORG_KEYS_DIR="${BORG_CONFIG_DIR}/keys"
export BORG_PASSPHRASE="SOMEPASSWORD"

repo=repoTest

umask 0007

borg init --encryption=keyfile-blake2 $repo &> /dev/null

mkdir folderToBackup &> /dev/null
echo textToBackup > folderToBackup/file.txt

chown root $repo -R
chgrp root $repo -R
chmod 770 $repo -R
chmod 770 $BORG_CONFIG_DIR -R
chmod 770 $BORG_CACHE_DIR -R

echo "Permissions before backup:"
ls -lha $BORG_CACHE_DIR/*/config

borg create \
        --umask 0007 \
        --show-version \
        "${repo}::archive1_$(date +%Y%m%d-%H%M%S)" "folderToBackup" &> /dev/null

echo "Permissions after borg create 1.2.0:"
ls -lha $BORG_CACHE_DIR/*/config


echo -----------------------------------------------------------------------------

#Resetting permissions
chown root $repo -R
chgrp root $repo -R
chmod 770 $repo -R
chmod 770 $BORG_CONFIG_DIR -R
chmod 770 $BORG_CACHE_DIR -R

echo "Permissions before backup:"
ls -lha $BORG_CACHE_DIR/*/config

borg1.1.17 create \
        --umask 0007 \
        --show-version \
        "${repo}::archive1_$(date +%Y%m%d-%H%M%S)" "folderToBackup" &> /dev/null

echo "Permissions after borg create 1.1.17:"
ls -lha $BORG_CACHE_DIR/*/config

echo -----------------------------------------------------------------------------
echo "Permissions of file created using the 'touch' at the beggining of script:"
ls -lha folderToBackup/file.txt
```

Output:
```
Permissions before backup:
-rwxrwx--- 1 root root 590 Mar  4 14:34 Borg_cache_test/7c07863b42def5d41bf0e64e057e1c774414d071ed743738cc925c806eec30dc/config
Permissions after borg create 1.2.0:
-rw------- 1 root root 590 Mar  4 14:36 Borg_cache_test/7c07863b42def5d41bf0e64e057e1c774414d071ed743738cc925c806eec30dc/config
-----------------------------------------------------------------------------
Permissions before backup:
-rwxrwx--- 1 root root 590 Mar  4 14:36 Borg_cache_test/7c07863b42def5d41bf0e64e057e1c774414d071ed743738cc925c806eec30dc/config
Permissions after borg create 1.1.17:
-rw-rw---- 1 root root 590 Mar  4 14:36 Borg_cache_test/7c07863b42def5d41bf0e64e057e1c774414d071ed743738cc925c806eec30dc/config
-----------------------------------------------------------------------------
Permissions of file created using the 'touch' at the beggining of script:
-rw-rw---- 1 root root 13 Mar  4 14:36 folderToBackup/file.txt
 ```


Discussion:
Are you using the borg "pyinstaller-made fat binary" from the github releases page?
> Are you using the borg "pyinstaller-made fat binary" from the github releases page?
Yes, tried using "borg-linux64" downloaded from github AND pip install "borgbackup==1.2.0".
The result is the same.
BTW, it is a bit unusual what you are doing.

The cache is usually a local cache used by one machine and one user.
Its storage location is on same machine, same user, because borg assumes that location to be fully trusted.
The cache contents are not encrypted and some stuff (e.g. in the security dir) is relevant for security.

So, if you move that to a network share on a server and use it by multiple machines, you must trust them all.

So, there are potential problems lurking in case something goes wrong and you trusted too much.
And the only advantage I see is saving some storage space for `chunks.archives.d` and avoiding a chunks cache sync IF you use the same repo for all clients.
Your reproducer script is doing way too much and requires root, how about this:
```
# reproduce issue 6400

export BORG_BASE_DIR=borg_base_dir
export BORG_REPO=repo

umask
umask 0007
umask
mkdir $BORG_BASE_DIR

borg init --umask 0007 -e none
borg create --umask 0007 ::{now} src
ls -ld $BORG_BASE_DIR
ls -laR $BORG_BASE_DIR
ls -ld $BORG_REPO
ls -laR $BORG_REPO

borg delete --umask 0007 --force
rm -rf $BORG_BASE_DIR
```

Updated: ls needs `-a` to show `.stuff`
**One potential issue in your usage is inconsistent usage of `--umask`.**

If you use that borg option, you always need to use it in a consistent way.
If you do not give that option, it will use the default of `--umask 0077`.
Output of the above script:
```
0022
0007
drwxrwx---  4 tw  staff  128 Mar  4 19:47 borg_base_dir
total 0
drwxrwx---   4 tw  staff   128 Mar  4 19:47 .
drwxr-xr-x  54 tw  staff  1728 Mar  4 19:47 ..
drwxrwx---   3 tw  staff    96 Mar  4 19:47 .cache
drwxrwx---   3 tw  staff    96 Mar  4 19:47 .config

borg_base_dir/.cache:
total 0
drwxrwx---  3 tw  staff   96 Mar  4 19:47 .
drwxrwx---  4 tw  staff  128 Mar  4 19:47 ..
drwx------  4 tw  staff  128 Mar  4 19:47 borg          <---- BUG

borg_base_dir/.cache/borg:
total 8
drwx------  4 tw  staff  128 Mar  4 19:47 .
drwxrwx---  3 tw  staff   96 Mar  4 19:47 ..
drwxrwx---  8 tw  staff  256 Mar  4 19:47 56b65780e9f22493f528f0498e3e6b000f68e7819cf7373dfd041cdced723101
-rw-------  1 tw  staff  198 Mar  4 19:47 CACHEDIR.TAG          <---- BUG

borg_base_dir/.cache/borg/56b65780e9f22493f528f0498e3e6b000f68e7819cf7373dfd041cdced723101:
total 176
drwxrwx---  8 tw  staff    256 Mar  4 19:47 .
drwx------  4 tw  staff    128 Mar  4 19:47 ..
-rw-rw----  1 tw  staff     68 Mar  4 19:47 README
-rw-rw----  1 tw  staff  45382 Mar  4 19:47 chunks
drwxrwx---  2 tw  staff     64 Mar  4 19:47 chunks.archive.d
-rw-------  1 tw  staff    581 Mar  4 19:47 config          <---- BUG
-rw-------  1 tw  staff  27065 Mar  4 19:47 files          <---- BUG
-rw-------  1 tw  staff      2 Mar  4 19:47 pre12-meta          <---- BUG

borg_base_dir/.cache/borg/56b65780e9f22493f528f0498e3e6b000f68e7819cf7373dfd041cdced723101/chunks.archive.d:
total 0
drwxrwx---  2 tw  staff   64 Mar  4 19:47 .
drwxrwx---  8 tw  staff  256 Mar  4 19:47 ..

borg_base_dir/.config:
total 0
drwxrwx---  3 tw  staff   96 Mar  4 19:47 .
drwxrwx---  4 tw  staff  128 Mar  4 19:47 ..
drwx------  3 tw  staff   96 Mar  4 19:47 borg          <---- BUG

borg_base_dir/.config/borg:
total 0
drwx------  3 tw  staff  96 Mar  4 19:47 .
drwxrwx---  3 tw  staff  96 Mar  4 19:47 ..
drwxrwx---  3 tw  staff  96 Mar  4 19:47 security

borg_base_dir/.config/borg/security:
total 0
drwxrwx---  3 tw  staff   96 Mar  4 19:47 .
drwx------  3 tw  staff   96 Mar  4 19:47 ..
drwx------  5 tw  staff  160 Mar  4 19:47 56b65780e9f22493f528f0498e3e6b000f68e7819cf7373dfd041cdced723101          <---- BUG

borg_base_dir/.config/borg/security/56b65780e9f22493f528f0498e3e6b000f68e7819cf7373dfd041cdced723101:
total 24
drwx------  5 tw  staff  160 Mar  4 19:47 .
drwxrwx---  3 tw  staff   96 Mar  4 19:47 ..
-rw-------  1 tw  staff    1 Mar  4 19:47 key-type          <---- BUG
-rw-------  1 tw  staff   21 Mar  4 19:47 location          <---- BUG
-rw-------  1 tw  staff   26 Mar  4 19:47 manifest-timestamp          <---- BUG
drwxrwx---  8 tw  staff  256 Mar  4 19:47 repo
total 120
drwxrwx---   8 tw  staff    256 Mar  4 19:47 .
drwxr-xr-x  54 tw  staff   1728 Mar  4 19:47 ..
-rw-rw----   1 tw  staff     73 Mar  4 19:47 README
-rw-------   1 tw  staff    209 Mar  4 19:47 config          <---- BUG
drwxrwx---   3 tw  staff     96 Mar  4 19:47 data
-rw-rw----   1 tw  staff     90 Mar  4 19:47 hints.5
-rw-rw----   1 tw  staff  41258 Mar  4 19:47 index.5
-rw-rw----   1 tw  staff    190 Mar  4 19:47 integrity.5

repo/data:
total 0
drwxrwx---  3 tw  staff   96 Mar  4 19:47 .
drwxrwx---  8 tw  staff  256 Mar  4 19:47 ..
drwxrwx---  8 tw  staff  256 Mar  4 19:47 0

repo/data/0:
total 9800
drwxrwx---  8 tw  staff      256 Mar  4 19:47 .
drwxrwx---  3 tw  staff       96 Mar  4 19:47 ..
-rw-rw----  1 tw  staff      476 Mar  4 19:47 0
-rw-rw----  1 tw  staff       17 Mar  4 19:47 1
-rw-rw----  1 tw  staff  4993652 Mar  4 19:47 2
-rw-rw----  1 tw  staff       49 Mar  4 19:47 3
-rw-rw----  1 tw  staff      539 Mar  4 19:47 4
-rw-rw----  1 tw  staff       17 Mar  4 19:47 5
```
Hmm, guess the unexpected modes of `config` (and some other files) is due to security reasons:
Python's tempfile code likely creates the temp file with access for the user only (intentionally ignoring the umask).

So guess we need a `chmod` after renaming the temp file to its final destination.

Not sure why some of the directory modes are also wrong.
The file mode issue was introduced by the race-condition fix that introduced `tempfile.mkstemp` usage, so it affects 1.1-maint, 1.2-maint and master. (see #6325 #6306)

The directory mode issue is likely present since longer, but most users did not notice, because they didn't need group or other permissions.
After applying PR #6403 it looks like this:
```
(borg-env) tw@mba2020 borg % ./reproduce-6400.sh
0022
0007
drwxrwx---  4 tw  staff  128 Mar  4 21:35 borg_base_dir
total 0
drwxrwx---   4 tw  staff   128 Mar  4 21:35 .
drwxr-xr-x  54 tw  staff  1728 Mar  4 21:35 ..
drwxrwx---   3 tw  staff    96 Mar  4 21:35 .cache
drwxrwx---   3 tw  staff    96 Mar  4 21:35 .config

borg_base_dir/.cache:
total 0
drwxrwx---  3 tw  staff   96 Mar  4 21:35 .
drwxrwx---  4 tw  staff  128 Mar  4 21:35 ..
drwxrwx---  4 tw  staff  128 Mar  4 21:35 borg

borg_base_dir/.cache/borg:
total 8
drwxrwx---  4 tw  staff  128 Mar  4 21:35 .
drwxrwx---  3 tw  staff   96 Mar  4 21:35 ..
drwxrwx---  8 tw  staff  256 Mar  4 21:35 8dbe69469c0c6f589f1f6ba1628a66f7eac38ec3699f0bb7634136c9a3c27bc5
-rw-rw----  1 tw  staff  198 Mar  4 21:35 CACHEDIR.TAG

borg_base_dir/.cache/borg/8dbe69469c0c6f589f1f6ba1628a66f7eac38ec3699f0bb7634136c9a3c27bc5:
total 176
drwxrwx---  8 tw  staff    256 Mar  4 21:35 .
drwxrwx---  4 tw  staff    128 Mar  4 21:35 ..
-rw-rw----  1 tw  staff     68 Mar  4 21:35 README
-rw-rw----  1 tw  staff  45382 Mar  4 21:35 chunks
drwxrwx---  2 tw  staff     64 Mar  4 21:35 chunks.archive.d
-rw-rw----  1 tw  staff    581 Mar  4 21:35 config
-rw-rw----  1 tw  staff  27066 Mar  4 21:35 files
-rw-rw----  1 tw  staff      2 Mar  4 21:35 pre12-meta

borg_base_dir/.cache/borg/8dbe69469c0c6f589f1f6ba1628a66f7eac38ec3699f0bb7634136c9a3c27bc5/chunks.archive.d:
total 0
drwxrwx---  2 tw  staff   64 Mar  4 21:35 .
drwxrwx---  8 tw  staff  256 Mar  4 21:35 ..

borg_base_dir/.config:
total 0
drwxrwx---  3 tw  staff   96 Mar  4 21:35 .
drwxrwx---  4 tw  staff  128 Mar  4 21:35 ..
drwxrwx---  3 tw  staff   96 Mar  4 21:35 borg

borg_base_dir/.config/borg:
total 0
drwxrwx---  3 tw  staff  96 Mar  4 21:35 .
drwxrwx---  3 tw  staff  96 Mar  4 21:35 ..
drwxrwx---  3 tw  staff  96 Mar  4 21:35 security

borg_base_dir/.config/borg/security:
total 0
drwxrwx---  3 tw  staff   96 Mar  4 21:35 .
drwxrwx---  3 tw  staff   96 Mar  4 21:35 ..
drwxrwx---  5 tw  staff  160 Mar  4 21:35 8dbe69469c0c6f589f1f6ba1628a66f7eac38ec3699f0bb7634136c9a3c27bc5

borg_base_dir/.config/borg/security/8dbe69469c0c6f589f1f6ba1628a66f7eac38ec3699f0bb7634136c9a3c27bc5:
total 24
drwxrwx---  5 tw  staff  160 Mar  4 21:35 .
drwxrwx---  3 tw  staff   96 Mar  4 21:35 ..
-rw-rw----  1 tw  staff    1 Mar  4 21:35 key-type
-rw-rw----  1 tw  staff   21 Mar  4 21:35 location
-rw-rw----  1 tw  staff   26 Mar  4 21:35 manifest-timestamp
drwxrwx---  8 tw  staff  256 Mar  4 21:35 repo
total 120
drwxrwx---   8 tw  staff    256 Mar  4 21:35 .
drwxr-xr-x  54 tw  staff   1728 Mar  4 21:35 ..
-rw-rw----   1 tw  staff     73 Mar  4 21:35 README
-rw-rw----   1 tw  staff    209 Mar  4 21:35 config
drwxrwx---   3 tw  staff     96 Mar  4 21:35 data
-rw-rw----   1 tw  staff     90 Mar  4 21:35 hints.5
-rw-rw----   1 tw  staff  41258 Mar  4 21:35 index.5
-rw-rw----   1 tw  staff    190 Mar  4 21:35 integrity.5

repo/data:
total 0
drwxrwx---  3 tw  staff   96 Mar  4 21:35 .
drwxrwx---  8 tw  staff  256 Mar  4 21:35 ..
drwxrwx---  8 tw  staff  256 Mar  4 21:35 0

repo/data/0:
total 9800
drwxrwx---  8 tw  staff      256 Mar  4 21:35 .
drwxrwx---  3 tw  staff       96 Mar  4 21:35 ..
-rw-rw----  1 tw  staff      476 Mar  4 21:35 0
-rw-rw----  1 tw  staff       17 Mar  4 21:35 1
-rw-rw----  1 tw  staff  4994118 Mar  4 21:35 2
-rw-rw----  1 tw  staff       49 Mar  4 21:35 3
-rw-rw----  1 tw  staff      539 Mar  4 21:35 4
-rw-rw----  1 tw  staff       17 Mar  4 21:35 5
```
Also, for the usual `0o077` umask, it still looks good (everything **only** accessible for the user).

Resolving patch:
diff --git a/src/borg/helpers.py b/src/borg/helpers.py
--- a/src/borg/helpers.py
+++ b/src/borg/helpers.py
@@ -2533,3 +2533,71 @@ def umount(mountpoint):
         return subprocess.call(['fusermount', '-u', mountpoint], env=env)
     except FileNotFoundError:
         return subprocess.call(['umount', mountpoint], env=env)
+
+# below is a slightly modified tempfile.mkstemp that has an additional mode parameter.
+# see https://github.com/borgbackup/borg/issues/6933 and https://github.com/borgbackup/borg/issues/6400
+
+import os as _os  # NOQA
+import sys as _sys  # NOQA
+import errno as _errno  # NOQA
+from tempfile import _sanitize_params, _get_candidate_names, TMP_MAX, _text_openflags, _bin_openflags  # NOQA
+
+
+def _mkstemp_inner(dir, pre, suf, flags, output_type, mode=0o600):
+    """Code common to mkstemp, TemporaryFile, and NamedTemporaryFile."""
+
+    dir = _os.path.abspath(dir)
+    names = _get_candidate_names()
+    if output_type is bytes:
+        names = map(_os.fsencode, names)
+
+    for seq in range(TMP_MAX):
+        name = next(names)
+        file = _os.path.join(dir, pre + name + suf)
+        # _sys.audit("tempfile.mkstemp", file)  # no .audit in sys module of python 3.5
+        try:
+            fd = _os.open(file, flags, mode)
+        except FileExistsError:
+            continue  # try again
+        except PermissionError:
+            # This exception is thrown when a directory with the chosen name
+            # already exists on windows.
+            if _os.name == "nt" and _os.path.isdir(dir) and _os.access(dir, _os.W_OK):
+                continue
+            else:
+                raise
+        return fd, file
+
+    raise FileExistsError(_errno.EEXIST, "No usable temporary file name found")
+
+
+def mkstemp_mode(suffix=None, prefix=None, dir=None, text=False, mode=0o600):
+    """User-callable function to create and return a unique temporary
+    file.  The return value is a pair (fd, name) where fd is the
+    file descriptor returned by os.open, and name is the filename.
+    If 'suffix' is not None, the file name will end with that suffix,
+    otherwise there will be no suffix.
+    If 'prefix' is not None, the file name will begin with that prefix,
+    otherwise a default prefix is used.
+    If 'dir' is not None, the file will be created in that directory,
+    otherwise a default directory is used.
+    If 'text' is specified and true, the file is opened in text
+    mode.  Else (the default) the file is opened in binary mode.
+    If any of 'suffix', 'prefix' and 'dir' are not None, they must be the
+    same type.  If they are bytes, the returned name will be bytes; str
+    otherwise.
+    The file is readable and writable only by the creating user ID.
+    If the operating system uses permission bits to indicate whether a
+    file is executable, the file is executable by no one. The file
+    descriptor is not inherited by children of this process.
+    Caller is responsible for deleting the file when done with it.
+    """
+
+    prefix, suffix, dir, output_type = _sanitize_params(prefix, suffix, dir)
+
+    if text:
+        flags = _text_openflags
+    else:
+        flags = _bin_openflags
+
+    return _mkstemp_inner(dir, prefix, suffix, flags, output_type, mode)
diff --git a/src/borg/platform/base.py b/src/borg/platform/base.py
--- a/src/borg/platform/base.py
+++ b/src/borg/platform/base.py
@@ -1,10 +1,8 @@
 import errno
 import os
 import socket
-import tempfile
 import uuid
 
-from borg.constants import UMASK_DEFAULT
 from borg.helpers import safe_unlink
 
 """
@@ -183,7 +181,9 @@ def __init__(self, path, binary=False):
 
     def __enter__(self):
         from .. import platform
-        self.tmp_fd, self.tmp_fname = tempfile.mkstemp(prefix=self.tmp_prefix, suffix='.tmp', dir=self.dir)
+        from ..helpers import mkstemp_mode
+
+        self.tmp_fd, self.tmp_fname = mkstemp_mode(prefix=self.tmp_prefix, suffix='.tmp', dir=self.dir, mode=0o666)
         self.f = platform.SyncFile(self.tmp_fname, fd=self.tmp_fd, binary=self.binary)
         return self.f
 
@@ -195,14 +195,6 @@ def __exit__(self, exc_type, exc_val, exc_tb):
             safe_unlink(self.tmp_fname)  # with-body has failed, clean up tmp file
             return  # continue processing the exception normally
 
-        # tempfile.mkstemp always uses owner-only file permissions for the temp file,
-        # but as we'll rename it to the non-temp permanent file now, we need to respect
-        # the umask and change the file mode to what a normally created file would have.
-        # thanks to the crappy os.umask api, we can't query the umask without setting it. :-(
-        umask = os.umask(UMASK_DEFAULT)
-        os.umask(umask)
-        os.chmod(self.tmp_fname, mode=0o666 & ~ umask)
-
         try:
             os.replace(self.tmp_fname, self.path)  # POSIX: atomic rename
         except OSError:


Test patch: