Repository: borgbackup/borg

Issue:
[Question] borg create from stdin and caching
I was not able to found clarification or any description of this scenarion in documentation:

How metadata caching is working in case of creating snapshot from stdin pipe? Example:

`mysqldump ... | borg create $REPOSITORY::mysql-$now -`

Is there any sense to add option `--files-cache=ctime,size` in this case? It's not file, so may be better to add `--no-cache-sync` or `--no-files-cache`? Will deduplication be working in this case?

btw, does `--files-cache=ctime,size` doing the same as `--ignore-inode` ?



Discussion:
as stdin is only a stream of data, there is no such metadata associated with it.

so, the files cache should not be used for stdin at all. maybe we should check whether borg loads the files cache from disk in such a case (or stores it back to disk).

iirc, --ignore-inode is deprecated (do you see a warning?) and just removes `inode` from the `--files-cache` default of `ctime,size,inode`.
> cache should not be used for stdin at all

According to the documentation, "in order for the deduplication used by Borg to work, it needs to keep a local cache containing checksums of all file chunks already stored in the repository".

As I see by stats after `borg create` (and from repo size) - deduplication is working for backups from stdin.

Example for my random MySQL db (`mysqlump ... | borg create ... -`):

```
Number of files: 1
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:               96.43 MB             11.32 MB            605.73 kB
```

So, how it is working then?
In my previous comment, I meant the `files cache`, not the `chunks cache`.
So, chunk cache is used, files cache is not used, it's ok to set `--no-files-cache` (or do not specify at all), but not good idea to set `--no-cache-sync`, am I correct?
Guess it should not do harm to use --no-files-cache, but maybe that is not even needed because borg does not load the files cache anyway when just processing stdin.

Don't use --no-cache-sync without good reason.
I checked the code:

`LocalCache._do_open` first reads the chunks cache and then the files cache (except if it is **disabled**).

If the files cache is loaded, it is also saved back to disk at the end. You can see that by giving --progress.

So, one could save a bit of files cache loading/saving time by disabling the files cache for doing stdin-only archiving.

Thanks! 
Maybe adding this to the docs would be useful, PRs welcome.
I'll do a PR for the docs.

Resolving patch:
diff --git a/src/borg/archiver.py b/src/borg/archiver.py
--- a/src/borg/archiver.py
+++ b/src/borg/archiver.py
@@ -3175,6 +3175,11 @@ def define_borg_mount(parser):
         creating an archive should the command fail. The command is considered failed
         when it returned a non-zero exit code.
 
+        Reading from stdin yields just a stream of data without file metadata
+        associated with it, and the files cache is not needed at all. So it is
+        safe to disable it via ``--files-cache disabled`` and speed up backup
+        creation a bit.
+
         By default, the content read from stdin is stored in a file called 'stdin'.
         Use ``--stdin-name`` to change the name.
         """)


Test patch: