Repository: joblib/joblib

Issue:
Wishlist: more flexible control (not just by size) for clean up
Follow up to #1170 . Thank you for the clarifications on logic -- great to know that both `last_access` attribute it stored!

In our use case (https://github.com/con/fscacher/) size though is not an issue -- it is just a sheer number of files, and thus inodes consumed. I made a mistake to use "good old" EXT4 for HOME partition instead of BTRFS as for the main bulk storage and now I am paying the price of "running out of space" while still having GBs of actual space available. There is virtually no rule of thumb for what size (in bytes) we should aim for in our use case.  But it might be feasible to figure out how many cached items to keep (also the information which is available since you know the number of cached items) and/or even better -- for how long (known since `last_access` is stored). I would have then set in our case cache to not exceed 10k entries and be no older than 1 week or so.

Do you think such modes of cleanup could be added to `reduce_size` call?  (due to 
(FWIW: unless `reduce_size` is to be triggered automagically, I see no reason for `bytes_limit` or any of those settings actually to be listed in the constructor, and not just to become arguments for `reduce_size` call).

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/joblib/joblib/pull/1200?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) Report
> Merging [#1200](https://codecov.io/gh/joblib/joblib/pull/1200?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) (7287741) into [master](https://codecov.io/gh/joblib/joblib/commit/0fa2cb91f920a8c1733689e0af5da4e6879d689c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) (0fa2cb9) will **decrease** coverage by `22.92%`.
> The diff coverage is `88.88%`.

[![Impacted file tree graph](https://codecov.io/gh/joblib/joblib/pull/1200/graphs/tree.svg?width=650&height=150&src=pr&token=gA6LF5DGTW&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib)](https://codecov.io/gh/joblib/joblib/pull/1200?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib)

```diff
@@             Coverage Diff             @@
##           master    #1200       +/-   ##
===========================================
- Coverage   94.51%   71.58%   -22.93%     
===========================================
  Files          47       47               
  Lines        7034     6866      -168     
===========================================
- Hits         6648     4915     -1733     
- Misses        386     1951     +1565     
```


| [Impacted Files](https://codecov.io/gh/joblib/joblib/pull/1200?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) | Coverage Δ | |
|---|---|---|
| [joblib/\_store\_backends.py](https://codecov.io/gh/joblib/joblib/pull/1200/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19zdG9yZV9iYWNrZW5kcy5weQ==) | `89.89% <86.66%> (-0.88%)` | :arrow_down: |
| [joblib/memory.py](https://codecov.io/gh/joblib/joblib/pull/1200/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL21lbW9yeS5weQ==) | `94.75% <100.00%> (-0.82%)` | :arrow_down: |
| [joblib/test/test\_dask.py](https://codecov.io/gh/joblib/joblib/pull/1200/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Rlc3QvdGVzdF9kYXNrLnB5) | `3.77% <0.00%> (-95.11%)` | :arrow_down: |
| [joblib/\_dask.py](https://codecov.io/gh/joblib/joblib/pull/1200/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19kYXNrLnB5) | `23.76% <0.00%> (-70.33%)` | :arrow_down: |
| [joblib/executor.py](https://codecov.io/gh/joblib/joblib/pull/1200/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL2V4ZWN1dG9yLnB5) | `30.95% <0.00%> (-69.05%)` | :arrow_down: |
| [joblib/test/test\_memmapping.py](https://codecov.io/gh/joblib/joblib/pull/1200/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Rlc3QvdGVzdF9tZW1tYXBwaW5nLnB5) | `37.07% <0.00%> (-62.17%)` | :arrow_down: |
| [joblib/pool.py](https://codecov.io/gh/joblib/joblib/pull/1200/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Bvb2wucHk=) | `27.04% <0.00%> (-60.76%)` | :arrow_down: |
| [joblib/\_memmapping\_reducer.py](https://codecov.io/gh/joblib/joblib/pull/1200/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19tZW1tYXBwaW5nX3JlZHVjZXIucHk=) | `38.63% <0.00%> (-57.97%)` | :arrow_down: |
| [joblib/\_multiprocessing\_helpers.py](https://codecov.io/gh/joblib/joblib/pull/1200/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19tdWx0aXByb2Nlc3NpbmdfaGVscGVycy5weQ==) | `28.00% <0.00%> (-52.00%)` | :arrow_down: |
| [joblib/backports.py](https://codecov.io/gh/joblib/joblib/pull/1200/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL2JhY2twb3J0cy5weQ==) | `39.47% <0.00%> (-44.74%)` | :arrow_down: |
| ... and [27 more](https://codecov.io/gh/joblib/joblib/pull/1200/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/joblib/joblib/pull/1200?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/joblib/joblib/pull/1200?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib). Last update [0fa2cb9...7287741](https://codecov.io/gh/joblib/joblib/pull/1200?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).

jwodder: @tomMoral I've made your requested changes (including changing `age_limit` to a timedelta; not sure why I can't reply to that comment).

I agree that tests should be added; however, I'm not sure how exactly to write them and would appreciate advice.
tomMoral: > @tomMoral I've made your requested changes (including changing `age_limit` to a timedelta; not sure why I can't reply to that comment).

Thanks a lot for the modifications.

> I agree that tests should be added; however, I'm not sure how exactly to write them and would appreciate advice.

For the tests, I would put some results in cache, then check that they are indeed cached, `reduce_size` with some harsh limit (`age_limits/item_limits=0`...) and check that the results are not in cache anymore.
You can use the `check_call_in_cache` helper to check if a specific call is in cache:

```python
cached_foo = mem.cache(foo)
cached_foo.check_call_in_cache(x)
>>> False
cached_foo(x)
cached_foo.check_call_in_cache(x)
>>> True
```

Patch:
diff --git a/CHANGES.rst b/CHANGES.rst
index 3decb56..b6c3610 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -4,15 +4,15 @@ Latest changes
 In development
 --------------
 
-- Ensure native byte order for memmap arrays in `joblib.load`.
+- Ensure native byte order for memmap arrays in ``joblib.load``.
   https://github.com/joblib/joblib/issues/1353
 
 - Add ability to change default Parallel backend in tests by setting the
-  JOBLIB_TESTS_DEFAULT_PARALLEL_BACKEND environment variable.
+  ``JOBLIB_TESTS_DEFAULT_PARALLEL_BACKEND`` environment variable.
   https://github.com/joblib/joblib/pull/1356
 
 - Fix temporary folder creation in `joblib.Parallel` on Linux subsystems on Windows
-  which do have `/dev/shm` but don't have the `os.statvfs` function 
+  which do have `/dev/shm` but don't have the `os.statvfs` function
   https://github.com/joblib/joblib/issues/1353
 
 - Drop runtime dependency on ``distutils``. ``distutils`` is going away
@@ -24,7 +24,7 @@ In development
 
 - A warning is raised when a pickling error occurs during caching operations.
   In version 1.5, this warning will be turned into an error. For all other
-  errors, a new warning has been introduced: `joblib.memory.CacheWarning`.
+  errors, a new warning has been introduced: ``joblib.memory.CacheWarning``.
   https://github.com/joblib/joblib/pull/1359
 
 - Avoid (module, name) collisions when caching nested functions. This fix
@@ -40,12 +40,18 @@ In development
   tracebacks and more efficient running time.
   https://github.com/joblib/joblib/pull/1393
 
-- Add the `parallel_config` context manager to allow for more fine-grained
+- Add the ``parallel_config`` context manager to allow for more fine-grained
   control over the backend configuration. It should be used in place of the
-  `parallel_backend` context manager. In particular, it has the advantage
+  ``parallel_backend`` context manager. In particular, it has the advantage
   of not requiring to set a specific backend in the context manager.
   https://github.com/joblib/joblib/pull/1392
 
+- Add ``items_limit`` and ``age_limit`` in :meth:`joblib.Memory.reduce_size`
+  to make it easy to limit the number of items and remove items that have
+  not been accessed for a long time in the cache.
+  https://github.com/joblib/joblib/pull/1200
+
+
 Release 1.2.0
 -------------
 
@@ -59,7 +65,7 @@ Release 1.2.0
 
 - Avoid unnecessary warnings when workers and main process delete
   the temporary memmap folder contents concurrently.
-  https://github.com/joblib/joblib/pull/1263 
+  https://github.com/joblib/joblib/pull/1263
 
 - Fix memory alignment bug for pickles containing numpy arrays.
   This is especially important when loading the pickle with
@@ -118,7 +124,7 @@ Release 1.0.1
 
 - Add check_call_in_cache method to check cache without calling function.
   https://github.com/joblib/joblib/pull/820
- 
+
 - dask: avoid redundant scattering of large arguments to make a more
   efficient use of the network resources and avoid crashing dask with
   "OSError: [Errno 55] No buffer space available"
@@ -134,7 +140,7 @@ Release 1.0.0
   or a third party library involved in the cached values definition is
   upgraded.  In particular, users updating `joblib` to a release that includes
   this fix will see their previous cache invalidated if they contained
-  reference to `numpy` objects. 
+  reference to `numpy` objects.
   https://github.com/joblib/joblib/pull/1136
 
 - Remove deprecated `check_pickle` argument in `delayed`.
diff --git a/joblib/_store_backends.py b/joblib/_store_backends.py
index 21a4958..de1e939 100644
--- a/joblib/_store_backends.py
+++ b/joblib/_store_backends.py
@@ -294,9 +294,15 @@ class StoreBackendMixin(object):
         """Clear the whole store content."""
         self.clear_location(self.location)
 
-    def reduce_store_size(self, bytes_limit):
-        """Reduce store size to keep it under the given bytes limit."""
-        items_to_delete = self._get_items_to_delete(bytes_limit)
+    def enforce_store_limits(
+            self, bytes_limit, items_limit=None, age_limit=None
+    ):
+        """
+        Remove the store's oldest files to enforce item, byte, and age limits.
+        """
+        items_to_delete = self._get_items_to_delete(
+            bytes_limit, items_limit, age_limit
+        )
 
         for item in items_to_delete:
             if self.verbose > 10:
@@ -310,16 +316,38 @@ class StoreBackendMixin(object):
                 # the folder already.
                 pass
 
-    def _get_items_to_delete(self, bytes_limit):
-        """Get items to delete to keep the store under a size limit."""
+    def _get_items_to_delete(
+            self, bytes_limit, items_limit=None, age_limit=None
+    ):
+        """
+        Get items to delete to keep the store under size, file, & age limits.
+        """
         if isinstance(bytes_limit, str):
             bytes_limit = memstr_to_bytes(bytes_limit)
 
         items = self.get_items()
         size = sum(item.size for item in items)
 
-        to_delete_size = size - bytes_limit
-        if to_delete_size < 0:
+        if bytes_limit is not None:
+            to_delete_size = size - bytes_limit
+        else:
+            to_delete_size = 0
+
+        if items_limit is not None:
+            to_delete_items = len(items) - items_limit
+        else:
+            to_delete_items = 0
+
+        if age_limit is not None:
+            older_item = min(item.last_access for item in items)
+            deadline = datetime.datetime.now() - age_limit
+        else:
+            deadline = None
+
+        if (
+            to_delete_size <= 0 and to_delete_items <= 0
+            and (deadline is None or older_item > deadline)
+        ):
             return []
 
         # We want to delete first the cache items that were accessed a
@@ -328,13 +356,19 @@ class StoreBackendMixin(object):
 
         items_to_delete = []
         size_so_far = 0
+        items_so_far = 0
 
         for item in items:
-            if size_so_far > to_delete_size:
+            if (
+                (size_so_far >= to_delete_size)
+                and items_so_far >= to_delete_items
+                and (deadline is None or deadline < item.last_access)
+            ):
                 break
 
             items_to_delete.append(item)
             size_so_far += item.size
+            items_so_far += 1
 
         return items_to_delete
 
diff --git a/joblib/memory.py b/joblib/memory.py
index 90fe41e..70d0f60 100644
--- a/joblib/memory.py
+++ b/joblib/memory.py
@@ -926,10 +926,12 @@ class Memory(Logger):
             Verbosity flag, controls the debug messages that are issued
             as functions are evaluated.
 
-        bytes_limit: int, optional
+        bytes_limit: int | str, optional
             Limit in bytes of the size of the cache. By default, the size of
             the cache is unlimited. When reducing the size of the cache,
-            ``joblib`` keeps the most recently accessed items first.
+            ``joblib`` keeps the most recently accessed items first. If a
+            str is passed, it is converted to a number of bytes using units
+            { K | M | G} for kilo, mega, giga.
 
             **Note:** You need to call :meth:`joblib.Memory.reduce_size` to
             actually reduce the cache size to be less than ``bytes_limit``.
@@ -1022,16 +1024,53 @@ class Memory(Logger):
         if self.store_backend is not None:
             self.store_backend.clear()
 
-            # As the cache in completely clear, make sure the _FUNCTION_HASHES
+            # As the cache is completely clear, make sure the _FUNCTION_HASHES
             # cache is also reset. Else, for a function that is present in this
             # table, results cached after this clear will be have cache miss
             # as the function code is not re-written.
             _FUNCTION_HASHES.clear()
 
-    def reduce_size(self):
-        """Remove cache elements to make cache size fit in ``bytes_limit``."""
-        if self.bytes_limit is not None and self.store_backend is not None:
-            self.store_backend.reduce_store_size(self.bytes_limit)
+    def reduce_size(self, bytes_limit=None, items_limit=None, age_limit=None):
+        """Remove cache elements to make the cache fit its limits.
+
+        The limitation can impose that the cache size fits in ``bytes_limit``,
+        that the number of cache items is no more than ``items_limit``, and
+        that all files in cache are not older than ``age_limit``.
+
+        Parameters
+        ----------
+        bytes_limit: int | str, optional
+            Limit in bytes of the size of the cache. By default, the size of
+            the cache is unlimited. When reducing the size of the cache,
+            ``joblib`` keeps the most recently accessed items first. If a
+            str is passed, it is converted to a number of bytes using units
+            { K | M | G} for kilo, mega, giga.
+
+        items_limit: int, optional
+            Number of items to limit the cache to.  By default, the number of
+            items in the cache is unlimited.  When reducing the size of the
+            cache, ``joblib`` keeps the most recently accessed items first.
+
+        age_limit: datetime.timedelta, optional
+            Maximum age of items to limit the cache to.  When reducing the size
+            of the cache, any items last accessed more than the given length of
+            time ago are deleted.
+        """
+        if bytes_limit is None:
+            bytes_limit = self.bytes_limit
+
+        if self.store_backend is None:
+            # No cached results, this function does nothing.
+            return
+
+        if bytes_limit is None and items_limit is None and age_limit is None:
+            # No limitation to impose, returning
+            return
+
+        # Defers the actual limits enforcing to the store backend.
+        self.store_backend.enforce_store_limits(
+            bytes_limit, items_limit, age_limit
+        )
 
     def eval(self, func, *args, **kwargs):
         """ Eval function func with arguments `*args` and `**kwargs`,


Test patch:
diff --git a/joblib/test/test_memory.py b/joblib/test/test_memory.py
index 035e9a1..5fa9d53 100644
--- a/joblib/test/test_memory.py
+++ b/joblib/test/test_memory.py
@@ -942,7 +942,7 @@ def test__get_items_to_delete(tmpdir):
             min(ci.last_access for ci in surviving_items))
 
 
-def test_memory_reduce_size(tmpdir):
+def test_memory_reduce_size_bytes_limit(tmpdir):
     memory, _, _ = _setup_toy_cache(tmpdir)
     ref_cache_items = memory.store_backend.get_items()
 
@@ -973,6 +973,64 @@ def test_memory_reduce_size(tmpdir):
     assert cache_items == []
 
 
+def test_memory_reduce_size_items_limit(tmpdir):
+    memory, _, _ = _setup_toy_cache(tmpdir)
+    ref_cache_items = memory.store_backend.get_items()
+
+    # By default reduce_size is a noop
+    memory.reduce_size()
+    cache_items = memory.store_backend.get_items()
+    assert sorted(ref_cache_items) == sorted(cache_items)
+
+    # No cache items deleted if items_limit greater than the size of
+    # the cache
+    memory.reduce_size(items_limit=10)
+    cache_items = memory.store_backend.get_items()
+    assert sorted(ref_cache_items) == sorted(cache_items)
+
+    # items_limit is set so that only two cache items are kept
+    memory.reduce_size(items_limit=2)
+    cache_items = memory.store_backend.get_items()
+    assert set.issubset(set(cache_items), set(ref_cache_items))
+    assert len(cache_items) == 2
+
+    # bytes_limit set so that no cache item is kept
+    memory.reduce_size(items_limit=0)
+    cache_items = memory.store_backend.get_items()
+    assert cache_items == []
+
+
+def test_memory_reduce_size_age_limit(tmpdir):
+    import time
+    import datetime
+    memory, _, put_cache = _setup_toy_cache(tmpdir)
+    ref_cache_items = memory.store_backend.get_items()
+
+    # By default reduce_size is a noop
+    memory.reduce_size()
+    cache_items = memory.store_backend.get_items()
+    assert sorted(ref_cache_items) == sorted(cache_items)
+
+    # No cache items deleted if age_limit big.
+    memory.reduce_size(age_limit=datetime.timedelta(days=1))
+    cache_items = memory.store_backend.get_items()
+    assert sorted(ref_cache_items) == sorted(cache_items)
+
+    # age_limit is set so that only two cache items are kept
+    time.sleep(1)
+    put_cache(-1)
+    put_cache(-2)
+    memory.reduce_size(age_limit=datetime.timedelta(seconds=1))
+    cache_items = memory.store_backend.get_items()
+    assert not set.issubset(set(cache_items), set(ref_cache_items))
+    assert len(cache_items) == 2
+
+    # age_limit set so that no cache item is kept
+    memory.reduce_size(age_limit=datetime.timedelta(seconds=0))
+    cache_items = memory.store_backend.get_items()
+    assert cache_items == []
+
+
 def test_memory_clear(tmpdir):
     memory, _, g = _setup_toy_cache(tmpdir)
     memory.clear()