Repository: borgbackup/borg

Issue:
TypeError: hexdigest() missing required argument 'length' in helpers.py:in hash_item()
## Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

## Is this a BUG / ISSUE report or a QUESTION?

BUG

## System information. For client/server mode post info for both machines.

#### Your borg version (borg -V).

borg 1.1.17

#### Operating system (distribution) and version.

Fedora release 33 (Thirty Three)

#### Hardware / network configuration, and filesystems used.

network: irrelevant, filesystem: ext4

#### How much data is handled by borg?

1024 bytes (this is a toy example)

#### Full borg commandline that lead to the problem (leave away excludes and passwords)

```sh
borg list --format "{shake_256}{NL}" /path/to/repo::ArchiveFoo
```

## Describe the problem you're observing.
When I ask for a shake_256 or shake_128 hash, I get a TypeError instead.

#### Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.
1. Create a borg archive /path/to/repo::ArchiveFoo with one file
2. Run the above command

#### Include any warning/errors/backtraces from the system logs
```
Local Exception
Traceback (most recent call last):
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/archiver.py", line 4703, in main
    exit_code = archiver.run(args)
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/archiver.py", line 4635, in run
    return set_ec(func(args))
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/archiver.py", line 177, in wrapper
    return method(self, args, repository=repository, **kwargs)
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/archiver.py", line 1460, in do_list
    return self._list_archive(args, repository, manifest, key)
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/archiver.py", line 1489, in _list_archive
    _list_inner(cache=None)
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/archiver.py", line 1482, in _list_inner
    sys.stdout.write(formatter.format_item(item))
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/helpers.py", line 1763, in format_item
    return self.format.format_map(self.get_item_data(item))
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/helpers.py", line 2023, in get_item_data
    item_data[key] = self.call_keys[key](item)
  File "/home/user/.local/pipx/venvs/borgbackup/lib64/python3.9/site-packages/borg/helpers.py", line 2057, in hash_item
    return hash.hexdigest()
TypeError: hexdigest() missing required argument 'length' (pos 1)

Platform: Linux borg 5.4.156-1.fc25.qubes.x86_64 #1 SMP Fri Oct 29 02:51:34 CEST 2021 x86_64
Linux: Unknown Linux  
Borg: 1.1.17  Python: CPython 3.9.9 msgpack: 0.5.6.+borg1
PID: 4546  CWD: /home/user
sys.argv: ['/home/user/.local/bin/borg', 'list', '--format', '{shake_256}{NL}', '/path/to/repo::ArchiveFoo']
SSH_ORIGINAL_COMMAND: None
```


Discussion:
There's no way go give the desired length in the placeholder, so I guess we should just not support these.

That API is a bit crappy, I would have expected that there is some default length (== max length) at least, so that it does not require giving the length explicitly.

Alternatively, we could maybe work around the crappy api and give 128 or 256 for these.
@hexagonrecursion just out of curiosity: why did you need the shake hashes? Do they have some practical relevance?
> Do they have some practical relevance?

Not to me. I simply noticed that the feature exists and that it is broken.

Resolving patch:
diff --git a/src/borg/helpers/parseformat.py b/src/borg/helpers/parseformat.py
--- a/src/borg/helpers/parseformat.py
+++ b/src/borg/helpers/parseformat.py
@@ -672,7 +672,9 @@ def format_time(self, ts):
 
 
 class ItemFormatter(BaseFormatter):
-    hash_algorithms = hashlib.algorithms_guaranteed.union({'xxh64'})
+    # we provide the hash algos from python stdlib (except shake_*) and additionally xxh64.
+    # shake_* is not provided because it uses an incompatible .digest() method to support variable length.
+    hash_algorithms = hashlib.algorithms_guaranteed.union({'xxh64'}).difference({'shake_128', 'shake_256'})
     KEY_DESCRIPTIONS = {
         'bpath': 'verbatim POSIX path, can contain any character except NUL',
         'path': 'path interpreted as text (might be missing non-text characters, see bpath)',
@@ -836,10 +838,10 @@ def calculate_csize(self, item):
     def hash_item(self, hash_function, item):
         if 'chunks' not in item:
             return ""
-        if hash_function in hashlib.algorithms_guaranteed:
-            hash = hashlib.new(hash_function)
-        elif hash_function == 'xxh64':
+        if hash_function == 'xxh64':
             hash = self.xxh64()
+        elif hash_function in self.hash_algorithms:
+            hash = hashlib.new(hash_function)
         for data in self.archive.pipeline.fetch_many([c.id for c in item.chunks]):
             hash.update(data)
         return hash.hexdigest()


Test patch: