Repository: joblib/joblib
Pull request #1674: ENH add auto .gitignore in the Memory cache folder
Author: login: StefanieSenger
Created: 2025-02-27T12:04:19Z | Merged: 2025-03-05T09:43:13Z

Closes #1643

This PR adds an automatically created `.gitignore` file to the cache folder created with `Memory().__init__()`.

<strike>`Memory().clear()` and `rm_subdirs()` are modified to delete joblib's `.gitignore` file while removing the other joblib folders.</strike>` Now it stays.

PR also add a corresponding test.

Linked issue #1643: Auto-added `.gitignore` in cache directories
I am quite sure that nobody wants to add cache to their version control. Therefore, many relevant tools like Mypy, Ruff, or, Pytest automatically adds (within the cache directory) a `.gitignore` file, as simple as:

```gitignore
# .cache/.gitignore
*
```

I guess this could also a feature proposal for the current project.

Anyway, thank you all for creating and maintaining it 👏🏻 ❤️
Linked issue comments: user: login: lesteve
created_at: 2025-01-17T08:05:37Z
body: I was not aware of this but indeed from my scikit-learn checkout I see that these tools automatically create .gitignore files:
```
❯ head *_cache/.gitignore 
==> .mypy_cache/.gitignore <==
# Automatically created by mypy
*

==> .pytest_cache/.gitignore <==
# Created by pytest automatically.
*

==> .ruff_cache/.gitignore <==
*%                                                                                                                                                             
```

I am not planning to look at this in the near future, but a PR would be more than welcome!
author_association: MEMBER
user: login: tomMoral
created_at: 2025-01-17T10:45:13Z
body: Yes that would be a great feature! Happy to help review a PR on this one too.
author_association: CONTRIBUTOR

Diff:
diff --git a/CHANGES.rst b/CHANGES.rst
index efe85043a..77f45bfea 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -40,6 +40,9 @@ In development
   the warnings that are emitted in this case.
   https://github.com/joblib/joblib/pull/1681
 
+- The Memory object now automatically creates a .gitignore file in its cache directory,
+  instructing git to ignore the entire folder.
+  https://github.com/joblib/joblib/pull/1674
 
 Release 1.4.2 -- 2024/05/02
 ---------------------------
diff --git a/doc/memory.rst b/doc/memory.rst
index a692f2fa4..9a4760c17 100644
--- a/doc/memory.rst
+++ b/doc/memory.rst
@@ -15,7 +15,7 @@ On demand recomputing: the `Memory` class
 Use case
 --------
 
-The :class:`~joblib.Memory` class defines a context for lazy evaluation of
+The :class:`~joblib.Memory` class defines a context for lazy evaluation of a
 function, by putting the results in a store, by default using a disk, and not
 re-running the function twice for the same arguments.
 
@@ -28,12 +28,12 @@ A simple example:
 
   First, define the cache directory::
 
-    >>> cachedir = 'your_cache_location_directory'
+    >>> location = 'your_cache_location_directory'
 
   Then, instantiate a memory context that uses this cache directory::
 
     >>> from joblib import Memory
-    >>> memory = Memory(cachedir, verbose=0)
+    >>> memory = Memory(location, verbose=0)
 
   After these initial steps, just decorate a function to cache its output in
   this context::
@@ -131,8 +131,8 @@ Using memmapping
 Memmapping (memory mapping) speeds up cache looking when reloading large numpy
 arrays::
 
-    >>> cachedir2 = 'your_cachedir2_location'
-    >>> memory2 = Memory(cachedir2, mmap_mode='r')
+    >>> location2 = 'your_2nd_cache_location_directory'
+    >>> memory2 = Memory(location2, mmap_mode='r')
     >>> square = memory2.cache(np.square)
     >>> a = np.vander(np.arange(3)).astype(float)
     >>> square(a)
@@ -521,8 +521,8 @@ without actually needing to call the function itself::
 
     >>> import shutil
     >>> try:
-    ...     shutil.rmtree(cachedir)
-    ...     shutil.rmtree(cachedir2)
+    ...     shutil.rmtree(location)
+    ...     shutil.rmtree(location2)
     ... except OSError:
     ...     pass  # this can sometimes fail under Windows
 
diff --git a/joblib/__init__.py b/joblib/__init__.py
index 3ffe33a1d..93d280070 100644
--- a/joblib/__init__.py
+++ b/joblib/__init__.py
@@ -56,8 +56,8 @@
    computation to disk and rerun it only if necessary::
 
       >>> from joblib import Memory
-      >>> cachedir = 'your_cache_dir_goes_here'
-      >>> mem = Memory(cachedir)
+      >>> location = 'your_cache_dir_goes_here'
+      >>> mem = Memory(location, verbose=1)
       >>> import numpy as np
       >>> a = np.vander(np.arange(3)).astype(float)
       >>> square = mem.cache(np.square)
@@ -86,7 +86,7 @@
    *joblib.dump* & *joblib.load* ).
 
 ..
-    >>> import shutil ; shutil.rmtree(cachedir)
+    >>> import shutil ; shutil.rmtree(location)
 
 """
 
diff --git a/joblib/_store_backends.py b/joblib/_store_backends.py
index c94e67caf..c822f0f93 100644
--- a/joblib/_store_backends.py
+++ b/joblib/_store_backends.py
@@ -456,6 +456,21 @@ def configure(self, location, verbose=1, backend_options=None):
         if not os.path.exists(self.location):
             mkdirp(self.location)
 
+        # Automatically add `.gitignore` file to the cache folder.
+        # XXX: the condition is necessary because in `Memory.__init__`, the user
+        # passed `location` param is modified to be either `{location}` or
+        # `{location}/joblib` depending on input type (`pathlib.Path` vs `str`).
+        # The proper resolution of this inconsistency is tracked in:
+        # https://github.com/joblib/joblib/issues/1684
+        cache_directory = (
+            os.path.dirname(location)
+            if os.path.dirname(location) and os.path.basename(location) == "joblib"
+            else location
+        )
+        with open(os.path.join(cache_directory, ".gitignore"), "w") as file:
+            file.write("# Created by joblib automatically.\n")
+            file.write("*\n")
+
         # item can be stored compressed for faster I/O
         self.compress = backend_options.get("compress", False)
 
diff --git a/joblib/disk.py b/joblib/disk.py
index f48713ac4..61222e2bb 100644
--- a/joblib/disk.py
+++ b/joblib/disk.py
@@ -78,7 +78,7 @@ def rm_subdirs(path, onerror=None):
     If onerror is set, it is called to handle the error with arguments (func,
     path, exc_info) where func is os.listdir, os.remove, or os.rmdir;
     path is the argument to that function that caused it to fail; and
-    exc_info is a tuple returned by sys.exc_info().  If onerror is None,
+    exc_info is a tuple returned by sys.exc_info(). If onerror is None,
     an exception is raised.
     """
 
diff --git a/joblib/memory.py b/joblib/memory.py
index 97656f56d..aff91c975 100644
--- a/joblib/memory.py
+++ b/joblib/memory.py
@@ -977,9 +977,8 @@ class Memory(Logger):
         the Memory object is completely transparent. This option
         replaces cachedir since version 0.12.
 
-    backend: str, optional
+    backend: str, optional, default='local'
         Type of store backend for reading/writing cache files.
-        Default: 'local'.
         The 'local' backend is using regular filesystem operations to
         manipulate data (open, mv, etc) in the backend.
 
diff --git a/joblib/test/test_memory.py b/joblib/test/test_memory.py
index b8ceb9402..985593bf0 100644
--- a/joblib/test/test_memory.py
+++ b/joblib/test/test_memory.py
@@ -1262,7 +1262,10 @@ def test_instanciate_store_backend_with_pathlib_path():
     # Instantiate a FileSystemStoreBackend using a pathlib.Path object
     path = pathlib.Path("some_folder")
     backend_obj = _store_backend_factory("local", path)
-    assert backend_obj.location == "some_folder"
+    try:
+        assert backend_obj.location == "some_folder"
+    finally:  # remove cache folder after test
+        shutil.rmtree("some_folder", ignore_errors=True)
 
 
 def test_filesystem_store_backend_repr(tmpdir):
@@ -1538,3 +1541,33 @@ def test_call_method_not_memorized(self, memory):
         assert isinstance(meta, dict), (
             "Metadata are not returned by MemorizedFunc.call."
         )
+
+
+@with_numpy
+@parametrize(
+    "location",
+    [
+        "test_cache_dir",
+        pathlib.Path("test_cache_dir"),
+        pathlib.Path("test_cache_dir").resolve(),
+    ],
+)
+def test_memory_creates_gitignore(location):
+    """Test that using the memory object automatically creates a `.gitignore` file
+    within the new cache directory."""
+
+    mem = Memory(location)
+    arr = np.asarray([[1, 2, 3], [4, 5, 6]])
+    costly_operation = mem.cache(np.square)
+    costly_operation(arr)
+
+    location = pathlib.Path(location)
+
+    try:
+        path_to_gitignore_file = os.path.join(location, ".gitignore")
+        gitignore_file_content = "# Created by joblib automatically.\n*\n"
+        with open(path_to_gitignore_file) as f:
+            assert gitignore_file_content == f.read()
+
+    finally:  # remove cache folder after test
+        shutil.rmtree(location, ignore_errors=True)


Reviews:
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-27T12:05:35Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-27T12:27:46Z
user: login: tomMoral
body: For directory that we do not create, I would not add the `.gitignore`.
I also added a few nipicks.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-27T12:54:18Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-27T13:15:04Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-27T13:21:27Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-27T13:25:33Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-27T13:30:27Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-27T13:59:43Z
user: login: lesteve
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-27T14:35:54Z
user: login: lesteve
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-27T14:37:00Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-27T14:50:17Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-28T08:17:28Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-28T09:44:50Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-28T10:17:48Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-28T10:19:14Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-28T11:00:15Z
user: login: lesteve
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-28T11:02:19Z
user: login: lesteve
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-28T11:04:06Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-28T13:04:15Z
user: login: StefanieSenger
body: Hello @lesteve and @tomMoral,

thank you for your comments, I have fixed the doctest failure by adding a verbosity level to the memory object in this example (still not sure why this didn't raise before, since it was unrelated to the PR) and have implemented your requested changes.

This PR is ready for a review.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-03T09:35:30Z
user: login: ogrisel
body: I reopened a tracking issue for the `str` vs `pathlib.Path` inconsistency.

+1 for merging this PR without resolving that bug and instead fix it in a dedicated PR.
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-03T15:31:05Z
user: login: StefanieSenger
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T08:03:12Z
user: login: tomMoral
body: A few more comments but I think we are close to the end
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T13:24:25Z

Review comments:
diff_hunk: @@ -56,8 +56,8 @@
    computation to disk and rerun it only if necessary::
 
       >>> from joblib import Memory
-      >>> cachedir = 'your_cache_dir_goes_here'
-      >>> mem = Memory(cachedir)
+      >>> location = 'your_cache_dir_goes_here'
+      >>> mem = Memory(location)
path: joblib/__init__.py
user: login: StefanieSenger
body: `chachedir` is the old param name, that is not used anymore.
created_at: 2025-02-27T12:04:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -461,6 +461,14 @@ def configure(self, location, verbose=1, backend_options=None):
         if not os.path.exists(self.location):
             mkdirp(self.location)
 
+        # automatically add `.gitignore` file to the parent cache folder
+        cache_directory = (
+            os.path.dirname(location) if os.path.dirname(location) else location
+        )
+        with open(os.path.join(cache_directory, '.gitignore'), 'w') as file:
+            file.write('# Created by joblib automatically.\n')
+            file.write('*\n')
+
path: joblib/_store_backends.py
user: login: tomMoral
body: We should only add the `.gitignore` in the folder if we create it.
Otherwise, we might have some weird behavior when we pass a folder we did not expect.

Also, I would rather put the `.gitignore` directly in the `location` folder. Most of the time, the cache dir is directly in the repo, so adding the file to the parent folder would ignore everything.
But I might be mistaken on what `location` is exactly?
created_at: 2025-02-27T12:25:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -100,6 +100,8 @@ def rm_subdirs(path, onerror=None):
     for name in names:
         fullname = os.path.join(path, name)
         delete_folder(fullname, onerror=onerror)
+        if os.path.exists(os.path.join(path, '.gitignore')):
+            os.remove(os.path.join(path, '.gitignore')) # delete .gitignore if it exists
path: joblib/disk.py
user: login: tomMoral
body: Only delete it if it is the one created automatically by joblib?
But this would disappear if we put it directly in the cache folder no?
created_at: 2025-02-27T12:26:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -461,6 +461,14 @@ def configure(self, location, verbose=1, backend_options=None):
         if not os.path.exists(self.location):
             mkdirp(self.location)
 
+        # automatically add `.gitignore` file to the parent cache folder
+        cache_directory = (
+            os.path.dirname(location) if os.path.dirname(location) else location
+        )
path: joblib/_store_backends.py
user: login: StefanieSenger
body: The condition is necessary because location can be whatever path the user defined in `location` if it is a pathlib.Path or it can be user_defined_path/joblib if the user has passed a str.

see issue #1671
created_at: 2025-02-27T12:27:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1545,3 +1546,19 @@ def test_call_method_not_memorized(self, memory):
         assert isinstance(meta, dict), (
             "Metadata are not returned by MemorizedFunc.call."
         )
+
+@parametrize("location", ["my_cache_dir", pathlib.Path("my_cache_dir")])
+def test_memory_creates_gitignore(location):
+    """Test that using the memory object automatically creates a `.gitignore` file
+    within the new cache directory."""
+
+    mem = Memory(location)
+    arr = np.asarray([[1,2,3],[4,5,6]])
+    costly_operation = mem.cache(np.square)
+    costly_operation(arr)
+
+    if isinstance(location, pathlib.Path):
+        location = location.as_posix()
+
+    assert os.path.exists(os.path.join(location, '.gitignore'))
+    shutil.rmtree(location, ignore_errors=True) # remove cache folder after test
path: joblib/test/test_memory.py
user: login: tomMoral
body: ```suggestion
    try:
        assert os.path.exists(os.path.join(location, '.gitignore'))
    finally:
        shutil.rmtree(location, ignore_errors=True) # remove cache folder after test
```
created_at: 2025-02-27T12:51:54Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1545,3 +1546,19 @@ def test_call_method_not_memorized(self, memory):
         assert isinstance(meta, dict), (
             "Metadata are not returned by MemorizedFunc.call."
         )
+
+@parametrize("location", ["my_cache_dir", pathlib.Path("my_cache_dir")])
+def test_memory_creates_gitignore(location):
+    """Test that using the memory object automatically creates a `.gitignore` file
+    within the new cache directory."""
+
+    mem = Memory(location)
+    arr = np.asarray([[1,2,3],[4,5,6]])
path: joblib/test/test_memory.py
user: login: tomMoral
body: ```suggestion
    arr = np.asarray([[1,2,3], [4,5,6]])
```
created_at: 2025-02-27T12:52:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1545,3 +1546,19 @@ def test_call_method_not_memorized(self, memory):
         assert isinstance(meta, dict), (
             "Metadata are not returned by MemorizedFunc.call."
         )
+
+@parametrize("location", ["my_cache_dir", pathlib.Path("my_cache_dir")])
+def test_memory_creates_gitignore(location):
+    """Test that using the memory object automatically creates a `.gitignore` file
+    within the new cache directory."""
+
+    mem = Memory(location)
+    arr = np.asarray([[1,2,3],[4,5,6]])
+    costly_operation = mem.cache(np.square)
+    costly_operation(arr)
+
+    if isinstance(location, pathlib.Path):
+        location = location.as_posix()
path: joblib/test/test_memory.py
user: login: tomMoral
body: Maybe easier to use the pathlib and do `location = pathlib.Path(location)`?
created_at: 2025-02-27T12:53:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -56,8 +56,8 @@
    computation to disk and rerun it only if necessary::
 
       >>> from joblib import Memory
-      >>> cachedir = 'your_cache_dir_goes_here'
-      >>> mem = Memory(cachedir)
+      >>> location = 'your_cache_dir_goes_here'
+      >>> mem = Memory(location)
path: joblib/__init__.py
user: login: StefanieSenger
body: The doctest error related to this file seems unrelated to this change? :shrug: 
created_at: 2025-02-27T13:15:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -461,6 +461,14 @@ def configure(self, location, verbose=1, backend_options=None):
         if not os.path.exists(self.location):
             mkdirp(self.location)
 
+        # automatically add `.gitignore` file to the parent cache folder
+        cache_directory = (
+            os.path.dirname(location) if os.path.dirname(location) else location
+        )
+        with open(os.path.join(cache_directory, '.gitignore'), 'w') as file:
+            file.write('# Created by joblib automatically.\n')
+            file.write('*\n')
+
path: joblib/_store_backends.py
user: login: StefanieSenger
body: > Also, I would rather put the .gitignore directly in the location folder.

Yes, that is exactly what it is doing. `.gitignore` here is created directly in the folder that the user has passed as `location`. The problem is that the `location` is modified [here](https://github.com/joblib/joblib/blob/2a54a1306c6ccc4ee9c77cdebc65adab0144c670/joblib/memory.py#L964-L966) to be 'location/joblib' if a string was passed.

With "parent" I meant the parent of 'location' and location/joblib' would both be 'location'. I could change the wording of the comment so it is not confusing?

Edit: I have now added a comment in the code explaining this better.

> We should only add the .gitignore in the folder if we create it.

Do you mean the user could pass the path to a directory they had already created before, and in this case we shouldn't gitignore it?
created_at: 2025-02-27T13:21:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -100,6 +100,8 @@ def rm_subdirs(path, onerror=None):
     for name in names:
         fullname = os.path.join(path, name)
         delete_folder(fullname, onerror=onerror)
+        if os.path.exists(os.path.join(path, '.gitignore')):
+            os.remove(os.path.join(path, '.gitignore')) # delete .gitignore if it exists
path: joblib/disk.py
user: login: StefanieSenger
body: > Only delete it if it is the one created automatically by joblib?

Hm, we could check if it starts with "# Created by joblib automatically" as a first line?

> But this would disappear if we put it directly in the cache folder no?

I don't think so, since `delete_folder()` only deletes folders and would keep any file that had been created on the top level.
created_at: 2025-02-27T13:25:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1545,3 +1546,19 @@ def test_call_method_not_memorized(self, memory):
         assert isinstance(meta, dict), (
             "Metadata are not returned by MemorizedFunc.call."
         )
+
+@parametrize("location", ["my_cache_dir", pathlib.Path("my_cache_dir")])
+def test_memory_creates_gitignore(location):
+    """Test that using the memory object automatically creates a `.gitignore` file
+    within the new cache directory."""
+
+    mem = Memory(location)
+    arr = np.asarray([[1,2,3],[4,5,6]])
+    costly_operation = mem.cache(np.square)
+    costly_operation(arr)
+
+    if isinstance(location, pathlib.Path):
+        location = location.as_posix()
+
+    assert os.path.exists(os.path.join(location, '.gitignore'))
+    shutil.rmtree(location, ignore_errors=True) # remove cache folder after test
path: joblib/test/test_memory.py
user: login: StefanieSenger
body: Good idea, thanks!
created_at: 2025-02-27T13:30:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -100,6 +100,8 @@ def rm_subdirs(path, onerror=None):
     for name in names:
         fullname = os.path.join(path, name)
         delete_folder(fullname, onerror=onerror)
+        if os.path.exists(os.path.join(path, '.gitignore')):
+            os.remove(os.path.join(path, '.gitignore')) # delete .gitignore if it exists
path: joblib/disk.py
user: login: StefanieSenger
body: I have added a condition on only deleting the `.gitignore` if joblib had created it.
created_at: 2025-02-27T13:59:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -56,8 +56,8 @@
    computation to disk and rerun it only if necessary::
 
       >>> from joblib import Memory
-      >>> cachedir = 'your_cache_dir_goes_here'
-      >>> mem = Memory(cachedir)
+      >>> location = 'your_cache_dir_goes_here'
+      >>> mem = Memory(location)
path: joblib/__init__.py
user: login: lesteve
body: It may actually be related somehow and you need to use the `location` variable instead of `cachedir` in the failing doctest?
created_at: 2025-02-27T14:35:54Z
author_association: MEMBER
diff_hunk: @@ -56,8 +56,8 @@
    computation to disk and rerun it only if necessary::
 
       >>> from joblib import Memory
-      >>> cachedir = 'your_cache_dir_goes_here'
-      >>> mem = Memory(cachedir)
+      >>> location = 'your_cache_dir_goes_here'
+      >>> mem = Memory(location)
path: joblib/__init__.py
user: login: lesteve
body: Or rather define the `location` variable in the failing doctest + use `location` in `Memory` creation
created_at: 2025-02-27T14:37:00Z
author_association: MEMBER
diff_hunk: @@ -56,8 +56,8 @@
    computation to disk and rerun it only if necessary::
 
       >>> from joblib import Memory
-      >>> cachedir = 'your_cache_dir_goes_here'
-      >>> mem = Memory(cachedir)
+      >>> location = 'your_cache_dir_goes_here'
+      >>> mem = Memory(location)
path: joblib/__init__.py
user: login: StefanieSenger
body: I am not sure. As a test I had changed it back to how it was before, and ran pytest locally: I had the same failure. I will investigate further tomorrow.
created_at: 2025-02-27T14:50:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -56,8 +56,8 @@
    computation to disk and rerun it only if necessary::
 
       >>> from joblib import Memory
-      >>> cachedir = 'your_cache_dir_goes_here'
-      >>> mem = Memory(cachedir)
+      >>> location = 'your_cache_dir_goes_here'
+      >>> mem = Memory(location)
path: joblib/__init__.py
user: login: StefanieSenger
body: I have now run the test on main (both with python 3.12.2 and with 3.13.1) and it failed as well. I can only deduce that the failure is unrelated to this PR, isn't it?

However, 
1. I cannot see any recent merge that could have caused this. Could someone else please check if they pull from main their `joblib/__init__.py` also fails pytest?
2. I cannot see this failure in the CI artefacts anymore, but it seems to me that in joblib, not the whole CI runs automatically and part of it needs to be triggered by maintainers?
created_at: 2025-02-28T08:17:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -74,13 +74,13 @@ def mkdirp(d):
 def rm_subdirs(path, onerror=None):
     """Remove all subdirectories in this path.
 
-    The directory indicated by `path` is left in place, and its subdirectories
-    are erased.
+    The directory indicated by `path` is left in place, and its subdirectories and the
+    .gitignore file are erased.
path: joblib/disk.py
user: login: StefanieSenger
body: An empty directory is not tracked by git anyways, but I assume there must be some benefit of re-using this directory somehow (so the .gitignore file should also be removed), otherwise I cannot see how users would benefit from a function like this one - compared to simply deleting the whole folder.

Let's say users have added other files into the cache folder - they would be wiped as well. This is what puzzles me the most about this...
created_at: 2025-02-28T09:44:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -56,8 +56,8 @@
    computation to disk and rerun it only if necessary::
 
       >>> from joblib import Memory
-      >>> cachedir = 'your_cache_dir_goes_here'
-      >>> mem = Memory(cachedir)
+      >>> location = 'your_cache_dir_goes_here'
+      >>> mem = Memory(location)
path: joblib/__init__.py
user: login: StefanieSenger
body: After talking with @adrinjalali, we have decided to fix the example in joblib.__init__, so doctest can pass. Maybe it was simply not tested before.

This is resolved now.
created_at: 2025-02-28T10:17:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -461,6 +461,14 @@ def configure(self, location, verbose=1, backend_options=None):
         if not os.path.exists(self.location):
             mkdirp(self.location)
 
+        # automatically add `.gitignore` file to the parent cache folder
+        cache_directory = (
+            os.path.dirname(location) if os.path.dirname(location) else location
+        )
+        with open(os.path.join(cache_directory, '.gitignore'), 'w') as file:
+            file.write('# Created by joblib automatically.\n')
+            file.write('*\n')
+
path: joblib/_store_backends.py
user: login: StefanieSenger
body: I now understand that you probably meant the `.gitignore` should be added in the created folder (not IF it was created by joblib). This is already the case.

So I think the issue is resolved.
created_at: 2025-02-28T10:19:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -56,8 +56,8 @@
    computation to disk and rerun it only if necessary::
 
       >>> from joblib import Memory
-      >>> cachedir = 'your_cache_dir_goes_here'
-      >>> mem = Memory(cachedir)
+      >>> location = 'your_cache_dir_goes_here'
+      >>> mem = Memory(location)
path: joblib/__init__.py
user: login: StefanieSenger
body: Okay, it is not resolved. I think doctest runs in parallel and sometimes it expects the logging output of Memory() and sometimes it doesn't ...
created_at: 2025-02-28T11:00:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -74,13 +74,13 @@ def mkdirp(d):
 def rm_subdirs(path, onerror=None):
     """Remove all subdirectories in this path.
 
-    The directory indicated by `path` is left in place, and its subdirectories
-    are erased.
+    The directory indicated by `path` is left in place, and its subdirectories and the
+    .gitignore file are erased.
path: joblib/disk.py
user: login: lesteve
body: Good catch but I would not add `.gitignore` logic here, mostly for simplicity reasons.

`rm_subdirs` seems to imply that it deletes `subdirectories` so it was not deleting any file before and I think we can decide to keep it like this, at least not changing anything doesn't seem to have any downside
created_at: 2025-02-28T11:02:18Z
author_association: MEMBER
diff_hunk: @@ -1538,3 +1541,21 @@ def test_call_method_not_memorized(self, memory):
         assert isinstance(meta, dict), (
             "Metadata are not returned by MemorizedFunc.call."
         )
+
+
+@parametrize("location", ["my_cache_dir", pathlib.Path("my_cache_dir")])
+def test_memory_creates_gitignore(location):
+    """Test that using the memory object automatically creates a `.gitignore` file
+    within the new cache directory."""
+
+    mem = Memory(location)
+    arr = np.asarray([[1, 2, 3], [4, 5, 6]])
+    costly_operation = mem.cache(np.square)
+    costly_operation(arr)
+
+    location = pathlib.Path(location)
+
+    try:
+        assert os.path.exists(os.path.join(location, ".gitignore"))
path: joblib/test/test_memory.py
user: login: lesteve
body: Maybe check the full content of the `.gitignore`?
created_at: 2025-02-28T11:04:06Z
author_association: MEMBER
diff_hunk: @@ -56,8 +56,8 @@
    computation to disk and rerun it only if necessary::
 
       >>> from joblib import Memory
-      >>> cachedir = 'your_cache_dir_goes_here'
-      >>> mem = Memory(cachedir)
+      >>> location = 'your_cache_dir_goes_here'
+      >>> mem = Memory(location)
path: joblib/__init__.py
user: login: StefanieSenger
body: This is now fixed by setting the verbosity of Memory object in this example.
created_at: 2025-02-28T13:04:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -74,13 +74,13 @@ def mkdirp(d):
 def rm_subdirs(path, onerror=None):
     """Remove all subdirectories in this path.
 
-    The directory indicated by `path` is left in place, and its subdirectories
-    are erased.
+    The directory indicated by `path` is left in place, and its subdirectories and the
+    .gitignore file are erased.
path: joblib/disk.py
user: login: StefanieSenger
body: Thank you, @lesteve, I have removed this. So the file stays in the new folder also after running `rm_subdirs()` or clearing cache.
created_at: 2025-03-03T08:50:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1538,3 +1541,21 @@ def test_call_method_not_memorized(self, memory):
         assert isinstance(meta, dict), (
             "Metadata are not returned by MemorizedFunc.call."
         )
+
+
+@parametrize("location", ["my_cache_dir", pathlib.Path("my_cache_dir")])
+def test_memory_creates_gitignore(location):
+    """Test that using the memory object automatically creates a `.gitignore` file
+    within the new cache directory."""
+
+    mem = Memory(location)
+    arr = np.asarray([[1, 2, 3], [4, 5, 6]])
+    costly_operation = mem.cache(np.square)
+    costly_operation(arr)
+
+    location = pathlib.Path(location)
+
+    try:
+        assert os.path.exists(os.path.join(location, ".gitignore"))
path: joblib/test/test_memory.py
user: login: StefanieSenger
body: Done.
created_at: 2025-03-03T09:05:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1538,3 +1541,26 @@ def test_call_method_not_memorized(self, memory):
         assert isinstance(meta, dict), (
             "Metadata are not returned by MemorizedFunc.call."
         )
+
+
+@with_numpy
+@parametrize("location", ["my_cache_dir", pathlib.Path("my_cache_dir")])
path: joblib/test/test_memory.py
user: login: ogrisel
body: ```suggestion
@parametrize("location", ["test_cache_dir", pathlib.Path("test_cache_dir")])
```

to make it more explicit that this folder was created when running the tests in case it survives the `shutil.rmtree(location, ignore_errors=True)` line for whatever reason.
created_at: 2025-03-03T15:25:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -456,6 +456,17 @@ def configure(self, location, verbose=1, backend_options=None):
         if not os.path.exists(self.location):
             mkdirp(self.location)
 
+        # automatically add `.gitignore` file to the cache folder; the condition is
+        # necessary because in Memory.__init__, the user passed `location` param is
+        # modified to be either `{location}` or `{location}/joblib `depending on input
+        # type
path: joblib/_store_backends.py
user: login: ogrisel
body: ```suggestion
        # Automatically add `.gitignore` file to the cache folder.
        # XXX: the condition is necessary because in `Memory.__init__`, the user 
        # passed `location` param is modified to be either `{location}` or
        # `{location}/joblib `depending on input type (`pathlib.Path` vs `str`).
        # The proper resolution of this inconsistency is tracked in:
        # https://github.com/joblib/joblib/issues/1684
```
created_at: 2025-03-03T15:29:54Z
author_association: CONTRIBUTOR
diff_hunk: @@ -456,6 +456,17 @@ def configure(self, location, verbose=1, backend_options=None):
         if not os.path.exists(self.location):
             mkdirp(self.location)
 
+        # automatically add `.gitignore` file to the cache folder; the condition is
+        # necessary because in Memory.__init__, the user passed `location` param is
+        # modified to be either `{location}` or `{location}/joblib `depending on input
+        # type
path: joblib/_store_backends.py
user: login: StefanieSenger
body: I commited this, but what is `XXX` supposed to mean or stand for?
created_at: 2025-03-04T08:03:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -456,6 +456,19 @@ def configure(self, location, verbose=1, backend_options=None):
         if not os.path.exists(self.location):
             mkdirp(self.location)
 
+        # Automatically add `.gitignore` file to the cache folder.
+        # XXX: the condition is necessary because in `Memory.__init__`, the user
+        # passed `location` param is modified to be either `{location}` or
+        # `{location}/joblib `depending on input type (`pathlib.Path` vs `str`).
path: joblib/_store_backends.py
user: login: tomMoral
body: ```suggestion
        # `{location}/joblib` depending on input type (`pathlib.Path` vs `str`).
```
created_at: 2025-03-04T08:57:22Z
author_association: CONTRIBUTOR
diff_hunk: @@ -461,6 +461,14 @@ def configure(self, location, verbose=1, backend_options=None):
         if not os.path.exists(self.location):
             mkdirp(self.location)
 
+        # automatically add `.gitignore` file to the parent cache folder
+        cache_directory = (
+            os.path.dirname(location) if os.path.dirname(location) else location
+        )
path: joblib/_store_backends.py
user: login: tomMoral
body: Ok thanks a lot for explaining.

However, this solution does not work if you pass an absolute path.
Passing `location = Path("__cache__")` will create `.gitignore` in `__cache__` while passing the absolute path as `location.resolve()` will create the file in the parent folder.

I think we should create the `.gitignore` in the parent directory only if the name is `joblib`:
```suggestion
        cache_directory = (
            os.path.dirname(location)
            if os.path.dirname(location) and os.path.basename(location) == "joblib"
            else location
        )
```


created_at: 2025-03-04T12:29:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1538,3 +1541,26 @@ def test_call_method_not_memorized(self, memory):
         assert isinstance(meta, dict), (
             "Metadata are not returned by MemorizedFunc.call."
         )
+
+
+@with_numpy
+@parametrize("location", ["test_cache_dir", pathlib.Path("test_cache_dir")])
path: joblib/test/test_memory.py
user: login: tomMoral
body: ```suggestion
@parametrize("location", [
    "test_cache_dir",
    pathlib.Path("test_cache_dir"),
    pathlib.Path("test_cache_dir").resolve()
])
```
created_at: 2025-03-04T13:24:01Z
author_association: CONTRIBUTOR

PR comments:
user: login: codecov[bot]
created_at: 2025-02-27T13:08:38Z
body: ## [Codecov](https://app.codecov.io/gh/joblib/joblib/pull/1674?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 95.43%. Comparing base [(`389daf2`)](https://app.codecov.io/gh/joblib/joblib/commit/389daf20d1bc874e371b5f95e41a3cc7a7c044ed?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) to head [(`5699336`)](https://app.codecov.io/gh/joblib/joblib/commit/5699336564b83968ed33fa42898421474505015a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).
> Report is 25 commits behind head on main.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #1674      +/-   ##
==========================================
- Coverage   95.51%   95.43%   -0.08%     
==========================================
  Files          46       46              
  Lines        7711     7731      +20     
==========================================
+ Hits         7365     7378      +13     
- Misses        346      353       +7     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/joblib/joblib/pull/1674?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).

<details><summary> :rocket: New features to boost your workflow: </summary>

- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
author_association: NONE
user: login: ogrisel
created_at: 2025-03-03T15:33:28Z
body: @StefanieSenger can you please add an entry in the changelog to document this enhancement?
author_association: CONTRIBUTOR
user: login: StefanieSenger
created_at: 2025-03-05T08:47:18Z
body: Thank you for reviewing and taking care of absolute paths, @tomMoral. I had not been aware that you could pass them.

I have committed your suggestions.
author_association: CONTRIBUTOR
user: login: tomMoral
created_at: 2025-03-05T09:44:03Z
body: Merging!
Thanks a lot @StefanieSenger for pushing this through! :tada: 
It will avoid a lot of cache to end up in github repo ^^
author_association: CONTRIBUTOR