Repository: borgbackup/borg

Issue:
1.2.0 Cache permission issues

## Have you checked borgbackup docs, FAQ, and open Github issues?
Yes

## Is this a BUG / ISSUE report or a QUESTION?
BUG/ISSUE

## System information. For client/server mode post info for both machines.
Debian 11 as a Gen2 Hyper-V VM.

#### Your borg version (borg -V).
1.2.0 (does not work as intended) & 1.117 (works as intended)

#### Operating system (distribution) and version.
Debian 11 as a Gen2 Hyper-V VM. - 5.10.0-11-amd64

#### Hardware / network configuration, and filesystems used.
Filesystem: ext4

## Describe the problem you're observing.
Borg 1.2.0 is setting permission of cache files to 700, no matter what --umask or "umask" in script is set to.
This causes problems when running borg on different computers using a single shared cache (network share), as only the owner will have permission, and not the group.

#### Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.
Yes,  script:
```
#!/bin/bash

export LC_ALL=C
export BORG_CONFIG_DIR="Borg_config_test"
export BORG_CACHE_DIR="Borg_cache_test"
export BORG_SECURITY_DIR="${BORG_CONFIG_DIR}/security"
export BORG_KEYS_DIR="${BORG_CONFIG_DIR}/keys"
export BORG_PASSPHRASE="SOMEPASSWORD"

repo=repoTest

umask 0007

borg init --encryption=keyfile-blake2 $repo &> /dev/null

mkdir folderToBackup &> /dev/null
echo textToBackup > folderToBackup/file.txt

chown root $repo -R
chgrp root $repo -R
chmod 770 $repo -R
chmod 770 $BORG_CONFIG_DIR -R
chmod 770 $BORG_CACHE_DIR -R

echo "Permissions before backup:"
ls -lha $BORG_CACHE_DIR/*/config

borg create \
        --umask 0007 \
        --show-version \
        "${repo}::archive1_$(date +%Y%m%d-%H%M%S)" "folderToBackup" &> /dev/null

echo "Permissions after borg create 1.2.0:"
ls -lha $BORG_CACHE_DIR/*/config


echo -----------------------------------------------------------------------------

#Resetting permissions
chown root $repo -R
chgrp root $repo -R
chmod 770 $repo -R
chmod 770 $BORG_CONFIG_DIR -R
chmod 770 $BORG_CACHE_DIR -R

echo "Permissions before backup:"
ls -lha $BORG_CACHE_DIR/*/config

borg1.1.17 create \
        --umask 0007 \
        --show-version \
        "${repo}::archive1_$(date +%Y%m%d-%H%M%S)" "folderToBackup" &> /dev/null

echo "Permissions after borg create 1.1.17:"
ls -lha $BORG_CACHE_DIR/*/config

echo -----------------------------------------------------------------------------
echo "Permissions of file created using the 'touch' at the beggining of script:"
ls -lha folderToBackup/file.txt
```

Output:
```
Permissions before backup:
-rwxrwx--- 1 root root 590 Mar  4 14:34 Borg_cache_test/7c07863b42def5d41bf0e64e057e1c774414d071ed743738cc925c806eec30dc/config
Permissions after borg create 1.2.0:
-rw------- 1 root root 590 Mar  4 14:36 Borg_cache_test/7c07863b42def5d41bf0e64e057e1c774414d071ed743738cc925c806eec30dc/config
-----------------------------------------------------------------------------
Permissions before backup:
-rwxrwx--- 1 root root 590 Mar  4 14:36 Borg_cache_test/7c07863b42def5d41bf0e64e057e1c774414d071ed743738cc925c806eec30dc/config
Permissions after borg create 1.1.17:
-rw-rw---- 1 root root 590 Mar  4 14:36 Borg_cache_test/7c07863b42def5d41bf0e64e057e1c774414d071ed743738cc925c806eec30dc/config
-----------------------------------------------------------------------------
Permissions of file created using the 'touch' at the beggining of script:
-rw-rw---- 1 root root 13 Mar  4 14:36 folderToBackup/file.txt
 ```


Discussion:
Are you using the borg "pyinstaller-made fat binary" from the github releases page?
> Are you using the borg "pyinstaller-made fat binary" from the github releases page?
Yes, tried using "borg-linux64" downloaded from github AND pip install "borgbackup==1.2.0".
The result is the same.
BTW, it is a bit unusual what you are doing.

The cache is usually a local cache used by one machine and one user.
Its storage location is on same machine, same user, because borg assumes that location to be fully trusted.
The cache contents are not encrypted and some stuff (e.g. in the security dir) is relevant for security.

So, if you move that to a network share on a server and use it by multiple machines, you must trust them all.

So, there are potential problems lurking in case something goes wrong and you trusted too much.
And the only advantage I see is saving some storage space for `chunks.archives.d` and avoiding a chunks cache sync IF you use the same repo for all clients.
Your reproducer script is doing way too much and requires root, how about this:
```
# reproduce issue 6400

export BORG_BASE_DIR=borg_base_dir
export BORG_REPO=repo

umask
umask 0007
umask
mkdir $BORG_BASE_DIR

borg init --umask 0007 -e none
borg create --umask 0007 ::{now} src
ls -ld $BORG_BASE_DIR
ls -laR $BORG_BASE_DIR
ls -ld $BORG_REPO
ls -laR $BORG_REPO

borg delete --umask 0007 --force
rm -rf $BORG_BASE_DIR
```

Updated: ls needs `-a` to show `.stuff`
**One potential issue in your usage is inconsistent usage of `--umask`.**

If you use that borg option, you always need to use it in a consistent way.
If you do not give that option, it will use the default of `--umask 0077`.
Output of the above script:
```
0022
0007
drwxrwx---  4 tw  staff  128 Mar  4 19:47 borg_base_dir
total 0
drwxrwx---   4 tw  staff   128 Mar  4 19:47 .
drwxr-xr-x  54 tw  staff  1728 Mar  4 19:47 ..
drwxrwx---   3 tw  staff    96 Mar  4 19:47 .cache
drwxrwx---   3 tw  staff    96 Mar  4 19:47 .config

borg_base_dir/.cache:
total 0
drwxrwx---  3 tw  staff   96 Mar  4 19:47 .
drwxrwx---  4 tw  staff  128 Mar  4 19:47 ..
drwx------  4 tw  staff  128 Mar  4 19:47 borg          <---- BUG

borg_base_dir/.cache/borg:
total 8
drwx------  4 tw  staff  128 Mar  4 19:47 .
drwxrwx---  3 tw  staff   96 Mar  4 19:47 ..
drwxrwx---  8 tw  staff  256 Mar  4 19:47 56b65780e9f22493f528f0498e3e6b000f68e7819cf7373dfd041cdced723101
-rw-------  1 tw  staff  198 Mar  4 19:47 CACHEDIR.TAG          <---- BUG

borg_base_dir/.cache/borg/56b65780e9f22493f528f0498e3e6b000f68e7819cf7373dfd041cdced723101:
total 176
drwxrwx---  8 tw  staff    256 Mar  4 19:47 .
drwx------  4 tw  staff    128 Mar  4 19:47 ..
-rw-rw----  1 tw  staff     68 Mar  4 19:47 README
-rw-rw----  1 tw  staff  45382 Mar  4 19:47 chunks
drwxrwx---  2 tw  staff     64 Mar  4 19:47 chunks.archive.d
-rw-------  1 tw  staff    581 Mar  4 19:47 config          <---- BUG
-rw-------  1 tw  staff  27065 Mar  4 19:47 files          <---- BUG
-rw-------  1 tw  staff      2 Mar  4 19:47 pre12-meta          <---- BUG

borg_base_dir/.cache/borg/56b65780e9f22493f528f0498e3e6b000f68e7819cf7373dfd041cdced723101/chunks.archive.d:
total 0
drwxrwx---  2 tw  staff   64 Mar  4 19:47 .
drwxrwx---  8 tw  staff  256 Mar  4 19:47 ..

borg_base_dir/.config:
total 0
drwxrwx---  3 tw  staff   96 Mar  4 19:47 .
drwxrwx---  4 tw  staff  128 Mar  4 19:47 ..
drwx------  3 tw  staff   96 Mar  4 19:47 borg          <---- BUG

borg_base_dir/.config/borg:
total 0
drwx------  3 tw  staff  96 Mar  4 19:47 .
drwxrwx---  3 tw  staff  96 Mar  4 19:47 ..
drwxrwx---  3 tw  staff  96 Mar  4 19:47 security

borg_base_dir/.config/borg/security:
total 0
drwxrwx---  3 tw  staff   96 Mar  4 19:47 .
drwx------  3 tw  staff   96 Mar  4 19:47 ..
drwx------  5 tw  staff  160 Mar  4 19:47 56b65780e9f22493f528f0498e3e6b000f68e7819cf7373dfd041cdced723101          <---- BUG

borg_base_dir/.config/borg/security/56b65780e9f22493f528f0498e3e6b000f68e7819cf7373dfd041cdced723101:
total 24
drwx------  5 tw  staff  160 Mar  4 19:47 .
drwxrwx---  3 tw  staff   96 Mar  4 19:47 ..
-rw-------  1 tw  staff    1 Mar  4 19:47 key-type          <---- BUG
-rw-------  1 tw  staff   21 Mar  4 19:47 location          <---- BUG
-rw-------  1 tw  staff   26 Mar  4 19:47 manifest-timestamp          <---- BUG
drwxrwx---  8 tw  staff  256 Mar  4 19:47 repo
total 120
drwxrwx---   8 tw  staff    256 Mar  4 19:47 .
drwxr-xr-x  54 tw  staff   1728 Mar  4 19:47 ..
-rw-rw----   1 tw  staff     73 Mar  4 19:47 README
-rw-------   1 tw  staff    209 Mar  4 19:47 config          <---- BUG
drwxrwx---   3 tw  staff     96 Mar  4 19:47 data
-rw-rw----   1 tw  staff     90 Mar  4 19:47 hints.5
-rw-rw----   1 tw  staff  41258 Mar  4 19:47 index.5
-rw-rw----   1 tw  staff    190 Mar  4 19:47 integrity.5

repo/data:
total 0
drwxrwx---  3 tw  staff   96 Mar  4 19:47 .
drwxrwx---  8 tw  staff  256 Mar  4 19:47 ..
drwxrwx---  8 tw  staff  256 Mar  4 19:47 0

repo/data/0:
total 9800
drwxrwx---  8 tw  staff      256 Mar  4 19:47 .
drwxrwx---  3 tw  staff       96 Mar  4 19:47 ..
-rw-rw----  1 tw  staff      476 Mar  4 19:47 0
-rw-rw----  1 tw  staff       17 Mar  4 19:47 1
-rw-rw----  1 tw  staff  4993652 Mar  4 19:47 2
-rw-rw----  1 tw  staff       49 Mar  4 19:47 3
-rw-rw----  1 tw  staff      539 Mar  4 19:47 4
-rw-rw----  1 tw  staff       17 Mar  4 19:47 5
```
Hmm, guess the unexpected modes of `config` (and some other files) is due to security reasons:
Python's tempfile code likely creates the temp file with access for the user only (intentionally ignoring the umask).

So guess we need a `chmod` after renaming the temp file to its final destination.

Not sure why some of the directory modes are also wrong.

Resolving patch:
diff --git a/src/borg/helpers/fs.py b/src/borg/helpers/fs.py
--- a/src/borg/helpers/fs.py
+++ b/src/borg/helpers/fs.py
@@ -21,12 +21,12 @@
 py_37_plus = sys.version_info >= (3, 7)
 
 
-def ensure_dir(path, mode=stat.S_IRWXU, pretty_deadly=True):
+def ensure_dir(path, mode=stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO, pretty_deadly=True):
     """
     Ensures that the dir exists with the right permissions.
     1) Make sure the directory exists in a race-free operation
     2) If mode is not None and the directory has been created, give the right
-    permissions to the leaf directory
+    permissions to the leaf directory. The current umask value is masked out first.
     3) If pretty_deadly is True, catch exceptions, reraise them with a pretty
     message.
     Returns if the directory has been created and has the right permissions,
diff --git a/src/borg/platform/base.py b/src/borg/platform/base.py
--- a/src/borg/platform/base.py
+++ b/src/borg/platform/base.py
@@ -4,6 +4,7 @@
 import tempfile
 import uuid
 
+from borg.constants import UMASK_DEFAULT
 from borg.helpers import safe_unlink
 from borg.platformflags import is_win32
 
@@ -237,6 +238,15 @@ def __exit__(self, exc_type, exc_val, exc_tb):
         if exc_type is not None:
             safe_unlink(self.tmp_fname)  # with-body has failed, clean up tmp file
             return  # continue processing the exception normally
+
+        # tempfile.mkstemp always uses owner-only file permissions for the temp file,
+        # but as we'll rename it to the non-temp permanent file now, we need to respect
+        # the umask and change the file mode to what a normally created file would have.
+        # thanks to the crappy os.umask api, we can't query the umask without setting it. :-(
+        umask = os.umask(UMASK_DEFAULT)
+        os.umask(umask)
+        os.chmod(self.tmp_fname, mode=0o666 & ~ umask)
+
         try:
             os.replace(self.tmp_fname, self.path)  # POSIX: atomic rename
         except OSError:


Test patch: