Repository: SourceHorizon/logger

Issue:
Allow to limit the number of files created by AdvancedFileOutput
Log rotation is not only about splitting logs between multiple files but also about limiting the number of those files.
What do you think about a setting in the AdvancedFileOutput class to limit the count of archived files?

Suggested logic:
- the active file `_file` should not be considered for the count
- at every rotation:
  - count files different to `_file` in the destination folder
  - sort files by creation date asc
  - remove the first file in the list until the list length is not equal to maximum number of files accepted
- the default value for `maximumArchivedLogFiles` should be null, so that it keeps the current behaviour without cancellations


I will be happy to provide a PR, let me know if it could be reasonable



Discussion: Hi!
This sounds like a good idea, but I am not sure if we can really rely on the creation date of the files.
AFAIK, some file systems and programs tend to be funny with it and either set it to `0` (1970-01-01) or update it along with the modification date.
Additionally, it could be re-set to the current date/time when moved between disks (old file deleted -> new file created).

I would like to have these things checked first.
However, as this would be opt-in anyway, I don't see a reason against providing this option. :+1: 
Doing a fast search, also the Dart documentation doesn't recommend to rely on `changed` property of FileStat: https://api.dart.dev/stable/3.3.3/dart-io/FileStat/changed.html

What do you think about rely on filenames sorted by natural order? The main problem could be the `fileNameFormatter` property but this function takes a DateTime as input so it's implicit that every filename will have different names.
If this could be acceptable, the documentation must specify this logic to prevent wrong usages
TBH, not really a fan of it :/

> this function takes a DateTime as input so it's implicit that every filename will have different names

I don't think so, the time is basically optional and nothing guarantees that it is the right order (e.g. could be random UUIDs).

Leaning on e.g. [Log4j](https://logging.apache.org/log4j/2.x/manual/appenders.html#log-archive-retention-policy-delete-on-rollover), the default `PathSorter` (which is invoked prior to deletion) sorts after "most recently modified".
Maybe even make it customizable for the user, this could range from a simple boolean flag (use `FileStat.modified` vs. `changed`), over a metadata callback for a single file to a whole `PathSorter`-like class.

As it's the default in a fairly popular logger library, I guess it's reasonable to go this or a similar route (it's opt-in after all).
What do you think?
It should be ok.

I will try to open a PR, ok?
Yeah, sure, go ahead!

Fix patch:
diff --git a/lib/src/outputs/advanced_file_output.dart b/lib/src/outputs/advanced_file_output.dart
index 8b562e8..e70b15d 100644
--- a/lib/src/outputs/advanced_file_output.dart
+++ b/lib/src/outputs/advanced_file_output.dart
@@ -48,6 +48,15 @@ class AdvancedFileOutput extends LogOutput {
   ///
   /// [path] is either treated as directory for rotating or as target file name,
   /// depending on [maxFileSizeKB].
+  ///
+  /// [maxRotatedFilesCount] controls the number of rotated files to keep. By default
+  /// is null, which means no limit.
+  /// If set to a positive number, the output will keep the last
+  /// [maxRotatedFilesCount] files. The deletion step will be executed by sorting
+  /// files following the [fileSorter] ascending strategy and keeping the last files.
+  /// The [latestFileName] will not be counted. The default [fileSorter] strategy is
+  /// sorting by last modified date, beware that could be not reliable in some
+  /// platforms and/or filesystems.
   AdvancedFileOutput({
     required String path,
     bool overrideExisting = false,
@@ -58,6 +67,8 @@ class AdvancedFileOutput extends LogOutput {
     int maxFileSizeKB = 1024,
     String latestFileName = 'latest.log',
     String Function(DateTime timestamp)? fileNameFormatter,
+    int? maxRotatedFilesCount,
+    Comparator<File>? fileSorter,
   })  : _path = path,
         _overrideExisting = overrideExisting,
         _encoding = encoding,
@@ -73,6 +84,8 @@ class AdvancedFileOutput extends LogOutput {
               // ignore: deprecated_member_use_from_same_package
               Level.wtf,
             ],
+        _maxRotatedFilesCount = maxRotatedFilesCount,
+        _fileSorter = fileSorter ?? _defaultFileSorter,
         _file = maxFileSizeKB > 0 ? File('$path/$latestFileName') : File(path);
 
   /// Logs directory path by default, particular log file path if [_maxFileSizeKB] is 0.
@@ -86,6 +99,8 @@ class AdvancedFileOutput extends LogOutput {
   final int _maxFileSizeKB;
   final int _maxBufferSize;
   final String Function(DateTime timestamp) _fileNameFormatter;
+  final int? _maxRotatedFilesCount;
+  final Comparator<File> _fileSorter;
 
   final File _file;
   IOSink? _sink;
@@ -106,6 +121,14 @@ class AdvancedFileOutput extends LogOutput {
         '-${t.millisecond.toDigits(3)}.log';
   }
 
+  /// Sort files by their last modified date.
+  /// This behaviour is inspired by the Log4j PathSorter.
+  ///
+  /// This method fulfills the requirements of the [Comparator] interface.
+  static int _defaultFileSorter(File a, File b) {
+    return a.lastModifiedSync().compareTo(b.lastModifiedSync());
+  }
+
   @override
   Future<void> init() async {
     if (_rotatingFilesMode) {
@@ -157,6 +180,7 @@ class AdvancedFileOutput extends LogOutput {
         // Rotate the log file
         await _closeSink();
         await _file.rename('$_path/${_fileNameFormatter(DateTime.now())}');
+        await _deleteRotatedFiles();
         await _openSink();
       }
     } catch (e, s) {
@@ -181,6 +205,35 @@ class AdvancedFileOutput extends LogOutput {
     _sink = null; // Explicitly set null until assigned again
   }
 
+  Future<void> _deleteRotatedFiles() async {
+    // If maxRotatedFilesCount is not set, keep all files
+    if (_maxRotatedFilesCount == null) return;
+
+    final dir = Directory(_path);
+    final files = dir
+        .listSync()
+        .whereType<File>()
+        // Filter out the latest file
+        .where((f) => f.path != _file.path)
+        .toList();
+
+    // If the number of files is less than the limit, don't delete anything
+    if (files.length <= _maxRotatedFilesCount!) return;
+
+    files.sort(_fileSorter);
+
+    final filesToDelete =
+        files.sublist(0, files.length - _maxRotatedFilesCount!);
+    for (final file in filesToDelete) {
+      try {
+        await file.delete();
+      } catch (e, s) {
+        print('Failed to delete file: $e');
+        print(s);
+      }
+    }
+  }
+
   @override
   Future<void> destroy() async {
     _bufferFlushTimer?.cancel();
diff --git a/lib/src/outputs/advanced_file_output_stub.dart b/lib/src/outputs/advanced_file_output_stub.dart
index 6848327..03f3c68 100644
--- a/lib/src/outputs/advanced_file_output_stub.dart
+++ b/lib/src/outputs/advanced_file_output_stub.dart
@@ -1,4 +1,5 @@
 import 'dart:convert';
+import 'dart:io';
 
 import '../log_level.dart';
 import '../log_output.dart';
@@ -42,6 +43,15 @@ class AdvancedFileOutput extends LogOutput {
   ///
   /// [path] is either treated as directory for rotating or as target file name,
   /// depending on [maxFileSizeKB].
+  ///
+  /// [maxRotatedFilesCount] controls the number of rotated files to keep. By default
+  /// is null, which means no limit.
+  /// If set to a positive number, the output will keep the last
+  /// [maxRotatedFilesCount] files. The deletion step will be executed by sorting
+  /// files following the [fileSorter] ascending strategy and keeping the last files.
+  /// The [latestFileName] will not be counted. The default [fileSorter] strategy is
+  /// sorting by last modified date, beware that could be not reliable in some
+  /// platforms and/or filesystems.
   AdvancedFileOutput({
     required String path,
     bool overrideExisting = false,
@@ -52,6 +62,8 @@ class AdvancedFileOutput extends LogOutput {
     int maxFileSizeKB = 1024,
     String latestFileName = 'latest.log',
     String Function(DateTime timestamp)? fileNameFormatter,
+    int? maxRotatedFilesCount,
+    Comparator<File>? fileSorter,
   }) {
     throw UnsupportedError("Not supported on this platform.");
   }


Test patch:
diff --git a/test/outputs/advanced_file_output_test.dart b/test/outputs/advanced_file_output_test.dart
index ced7f29..3a75cdb 100644
--- a/test/outputs/advanced_file_output_test.dart
+++ b/test/outputs/advanced_file_output_test.dart
@@ -108,6 +108,87 @@ void main() {
     );
   });
 
+  test('Rolling files with rotated files deletion', () async {
+    var output = AdvancedFileOutput(
+      path: dir.path,
+      maxFileSizeKB: 1,
+      maxRotatedFilesCount: 1,
+    );
+
+    await output.init();
+    final event0 = OutputEvent(LogEvent(Level.fatal, ""), ["1" * 1500]);
+    output.output(event0);
+    await output.destroy();
+
+    // Start again to roll files on init without waiting for timer tick
+    await output.init();
+    final event1 = OutputEvent(LogEvent(Level.fatal, ""), ["2" * 1500]);
+    output.output(event1);
+    await output.destroy();
+
+    // And again for another roll
+    await output.init();
+    final event2 = OutputEvent(LogEvent(Level.fatal, ""), ["3" * 1500]);
+    output.output(event2);
+    await output.destroy();
+
+    final files = dir.listSync();
+
+    // Expect only 2 files: the "latest" that is the current log file
+    // and only one rotated file. The first created file should be deleted.
+    expect(files, hasLength(2));
+    final latestFile = File('${dir.path}/latest.log');
+    final rotatedFile = dir
+        .listSync()
+        .whereType<File>()
+        .firstWhere((file) => file.path != latestFile.path);
+    expect(await latestFile.readAsString(), contains("3"));
+    expect(await rotatedFile.readAsString(), contains("2"));
+  });
+
+  test('Rolling files with custom file sorter', () async {
+    var output = AdvancedFileOutput(
+      path: dir.path,
+      maxFileSizeKB: 1,
+      maxRotatedFilesCount: 1,
+      // Define a custom file sorter that sorts files by their length
+      // (strange behavior for testing purposes) from the longest to
+      // the shortest: the longest file should be deleted first.
+      fileSorter: (a, b) => b.lengthSync().compareTo(a.lengthSync()),
+    );
+
+    await output.init();
+    final event0 = OutputEvent(LogEvent(Level.fatal, ""), ["1" * 1500]);
+    output.output(event0);
+    await output.destroy();
+
+    // Start again to roll files on init without waiting for timer tick
+    await output.init();
+    // Create a second file with a greater length (it should be deleted first)
+    final event1 = OutputEvent(LogEvent(Level.fatal, ""), ["2" * 3000]);
+    output.output(event1);
+    await output.destroy();
+
+    // And again for another roll
+    await output.init();
+    final event2 = OutputEvent(LogEvent(Level.fatal, ""), ["3" * 1500]);
+    output.output(event2);
+    await output.destroy();
+
+    final files = dir.listSync();
+
+    // Expect only 2 files: the "latest" that is the current log file
+    // and only one rotated file (the shortest one).
+    expect(files, hasLength(2));
+    final latestFile = File('${dir.path}/latest.log');
+    final rotatedFile = dir
+        .listSync()
+        .whereType<File>()
+        .firstWhere((file) => file.path != latestFile.path);
+    expect(await latestFile.readAsString(), contains("3"));
+    expect(await rotatedFile.readAsString(), contains("1"));
+  });
+
   test('Flush temporary buffer on destroy', () async {
     var output = AdvancedFileOutput(path: dir.path);
     await output.init();