Repository: borgbackup/borg
Pull request: Add function to clear empty directories at end of compact process.
Compact moves data to new segments, and then removes the old segments. When enough segments are moved, directories holding the now cleared segments may thus become empty.

With this commit any empty directories are cleared after segments compacting. Fixes #6823


Problem statement:
empty folders in repo_dir/data/ are not automatically removed
## Have you checked borgbackup docs, FAQ, and open Github issues?
Yes

## Is this a BUG / ISSUE report or a QUESTION?
QUESTION

## System information. For client/server mode post info for both machines.

#### Your borg version (borg -V).
1.2.0

#### Operating system (distribution) and version.
Distribution: Manjaro Linux
Release:	21.3.1
Codename: Ruah

#### Hardware / network configuration, and filesystems used.
Running a borg Server into a Docker Container with Version 1.2.0

#### How much data is handled by borg?
```
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
All archives:               94.82 GB             71.92 GB             36.81 GB

                       Unique chunks         Total chunks
Chunk index:                  208674               530953
```

#### Full borg commandline that lead to the problem (leave away excludes and passwords)
borg compact --cleanup-commits 

## Describe the problem you're observing.
Having into the Borg Repo Data Folder many empty Folders, overall 11 Folders and 7 Folders are empty

#### Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.
created many Backups over the time and prune the old ones, use Command above and have 7 empty Folders

## Question
Can i delete the empty Folders or do borg neat them in any way?
Can borg delete this empty folders automatically?

Hints from issue discussion:
i don't think the empty folders are needed, so if they bother you, you can delete them.

maybe avoid running borg in parallel while you manually delete stuff in the repo - and be careful.

BTW:
- 1.2.1 is out
- only run `borg compact --cleanup-commits` once after upgrading from 1.1, you should not always run it with `--cleanup-commits` as the bug was only present in 1.1.x and was fixed in 1.2.

yeah, deleting the empty dirs automatically is a good idea guess would be a good additional job for `borg compact`.
What issue am I to resolve here
by compact the repository the old files will be remove and then i had some empty folders
Taking a look at this.
Have looked into the repository module code that deals with compacting. While writing a new segment, when segment_num % self.segments_per_dir is 0, `LoggedIO.get_write_fd` creates a new directory to place the segment. But these directories are never cleared during the prune/compact process. 

I've written a patch that adds a function in `LoggedIO` to additionally clear empty directories at the end of the compact process.

Commit message: Add function to clear empty directories at end of compact process.

Compact moves data to new segments, and then removes the old segments.
When enough segments are moved, directories holding the now cleared segments
may thus become empty.

With this commit any empty directories are cleared after segments compacting.
Fixes #6823

Patch to review:
diff --git a/src/borg/repository.py b/src/borg/repository.py
index 94df223a9b..3cf4fd1af2 100644
--- a/src/borg/repository.py
+++ b/src/borg/repository.py
@@ -907,6 +907,7 @@ def complete_xfer(intermediate=True):
             pi.show()
         pi.finish()
         complete_xfer(intermediate=False)
+        self.io.clear_empty_dirs()
         quota_use_after = self.storage_quota_use
         logger.info("Compaction freed about %s repository space.", format_file_size(quota_use_before - quota_use_after))
         logger.debug("Compaction completed.")
@@ -1554,6 +1555,21 @@ def delete_segment(self, segment):
         except FileNotFoundError:
             pass
 
+    def clear_empty_dirs(self):
+        """Delete empty segment dirs, i.e those with no segment files."""
+        data_dir = os.path.join(self.path, "data")
+        segment_dirs = self.get_segment_dirs(data_dir)
+        for segment_dir in segment_dirs:
+            try:
+                # os.rmdir will only delete the directory if it is empty
+                # so we don't need to explicitly check for emptiness first.
+                os.rmdir(segment_dir)
+            except OSError:
+                # OSError is raised by os.rmdir if directory is not empty. This is expected.
+                # It's subclass FileNotFoundError may be raised if the directory already does not exist. Ignorable.
+                pass
+        sync_dir(data_dir)
+
     def segment_exists(self, segment):
         filename = self.segment_filename(segment)
         # When deleting segments, they are first truncated. If truncate(2) and unlink(2) are split


Code review comments:
diff_hunk: @@ -1554,6 +1555,21 @@ def delete_segment(self, segment):
         except FileNotFoundError:
             pass
 
+    def clear_empty_dirs(self):
+        """Delete empty segment dirs, i.e those with no segment files."""
+        data_dir = os.path.join(self.path, "data")
+        segment_dirs = self.get_segment_dirs(data_dir)
+        for segment_dir in segment_dirs:
+            try:
+                # os.rmdir will only delete the directory if it is empty
+                # so we don't need to explicitly check for emptiness first.
+                os.rmdir(segment_dir)
+            except OSError:
+                # OSError is raised by os.rmdir if directory is not empty. This is expected.
+                # It's subclass FileNotFoundError may be raised if the directory already does not exist. Ignorable.
path: src/borg/repository.py
text: @user1:
Typo: "It's"  (should be: "Its").

@user1:
fixed it.