Repository: borgbackup/borg

Issue:
TypeError: hexdigest() missing required argument 'length' in helpers.py:in hash_item()
## Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

## Is this a BUG / ISSUE report or a QUESTION?

BUG

## System information. For client/server mode post info for both machines.

#### Your borg version (borg -V).

borg 1.1.17

#### Operating system (distribution) and version.

Fedora release 33 (Thirty Three)

#### Hardware / network configuration, and filesystems used.

network: irrelevant, filesystem: ext4

#### How much data is handled by borg?

1024 bytes (this is a toy example)

#### Full borg commandline that lead to the problem (leave away excludes and passwords)

```sh
borg list --format "{shake_256}{NL}" /path/to/repo::ArchiveFoo
```

## Describe the problem you're observing.
When I ask for a shake_256 or shake_128 hash, I get a TypeError instead.

#### Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.
1. Create a borg archive /path/to/repo::ArchiveFoo with one file
2. Run the above command

#### Include any warning/errors/backtraces from the system logs
```
Local Exception
Traceback (most recent call last):
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/archiver.py", line 4703, in main
    exit_code = archiver.run(args)
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/archiver.py", line 4635, in run
    return set_ec(func(args))
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/archiver.py", line 177, in wrapper
    return method(self, args, repository=repository, **kwargs)
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/archiver.py", line 1460, in do_list
    return self._list_archive(args, repository, manifest, key)
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/archiver.py", line 1489, in _list_archive
    _list_inner(cache=None)
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/archiver.py", line 1482, in _list_inner
    sys.stdout.write(formatter.format_item(item))
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/helpers.py", line 1763, in format_item
    return self.format.format_map(self.get_item_data(item))
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/helpers.py", line 2023, in get_item_data
    item_data[key] = self.call_keys[key](item)
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/helpers.py", line 2057, in hash_item
    return hash.hexdigest()
TypeError: hexdigest() missing required argument 'length' (pos 1)

Platform: Linux borg 5.4.156-1.fc25.qubes.x86_64 #1 SMP Fri Oct 29 02:51:34 CEST 2021 x86_64
Linux: Unknown Linux  
Borg: 1.1.17  Python: CPython 3.9.9 msgpack: 0.5.6.+borg1
PID: 4546  CWD: /home/user
sys.argv: ['/home/user/.local/bin/borg', 'list', '--format', '{shake_256}{NL}', '/path/to/repo::ArchiveFoo']
SSH_ORIGINAL_COMMAND: None
```


Discussion:
There's no way go give the desired length in the placeholder, so I guess we should just not support these.

That API is a bit crappy, I would have expected that there is some default length (== max length) at least, so that it does not require giving the length explicitly.

Alternatively, we could maybe work around the crappy api and give 128 or 256 for these.
@hexagonrecursion just out of curiosity: why did you need the shake hashes? Do they have some practical relevance?
> Do they have some practical relevance?

Not to me. I simply noticed that the feature exists and that it is broken.
OK, so I'll "fix" this by just removing support for shake_*. :-)

Resolving patch:
diff --git a/src/borg/helpers.py b/src/borg/helpers.py
--- a/src/borg/helpers.py
+++ b/src/borg/helpers.py
@@ -1898,6 +1898,9 @@ def format_time(self, ts):
 
 
 class ItemFormatter(BaseFormatter):
+    # we provide the hash algos from python stdlib (except shake_*).
+    # shake_* is not provided because it uses an incompatible .digest() method to support variable length.
+    hash_algorithms = hashlib.algorithms_guaranteed.difference({'shake_128', 'shake_256'})
     KEY_DESCRIPTIONS = {
         'bpath': 'verbatim POSIX path, can contain any character except NUL',
         'path': 'path interpreted as text (might be missing non-text characters, see bpath)',
@@ -1914,7 +1917,7 @@ class ItemFormatter(BaseFormatter):
         ('type', 'mode', 'uid', 'gid', 'user', 'group', 'path', 'bpath', 'source', 'linktarget', 'flags'),
         ('size', 'csize', 'dsize', 'dcsize', 'num_chunks', 'unique_chunks'),
         ('mtime', 'ctime', 'atime', 'isomtime', 'isoctime', 'isoatime'),
-        tuple(sorted(hashlib.algorithms_guaranteed)),
+        tuple(sorted(hash_algorithms)),
         ('archiveid', 'archivename', 'extra'),
         ('health', )
     )
@@ -1988,7 +1991,7 @@ def __init__(self, archive, format, *, json_lines=False):
             'ctime': partial(self.format_time, 'ctime'),
             'atime': partial(self.format_time, 'atime'),
         }
-        for hash_function in hashlib.algorithms_guaranteed:
+        for hash_function in self.hash_algorithms:
             self.add_key(hash_function, partial(self.hash_item, hash_function))
         self.used_call_keys = set(self.call_keys) & self.format_keys
 
@@ -2060,6 +2063,7 @@ def calculate_csize(self, item):
         return item.get_size(compressed=True)
 
     def hash_item(self, hash_function, item):
+        assert hash_function in self.hash_algorithms
         if 'chunks' not in item:
             return ""
         hash = hashlib.new(hash_function)


Test patch: