Repository: ipwnponies/pytest-antilru (language: python)

Issue:
pytest-django loads earlier than pytest-antilru preventing lru_cache from being busted when AppConfig.ready is used
pytest-django uses [pytest_load_initial_conftests](https://github.com/pytest-dev/pytest-django/blob/fba51531f067a985ec6b6be4aec9a2ed5766d69c/pytest_django/plugin.py#L260-L261) and runs `_setup_django()` this occurs very early to parse command line options

pytest-antilru uses [pytest_collection(](https://github.com/ipwnponies/pytest-antilru/blob/7bd64965554a159c1c41e38fcf8c09d42ab9fe14/pytest_antilru/main.py#L31-L32) which runs at the beginning of each session and importantly runs _after_ `pytest_load_initial_conftests`

This means if you have an [AppConfig](https://docs.djangoproject.com/en/4.2/ref/applications/#django.apps.AppConfig.ready) that import code that uses `lru_cache`, then that function will be imported before `python-antilru` runs preventing the cache from being busted.

Minimal example
`exampleapp/models.py`

```python
from functools import lru_cache

def expensive_network_call() -> int:
    return 1

@lru_cache()
def cache_me() -> int:
    return expensive_network_call()
```

`exampleapp/tests.py`

```python
import sys
from unittest import mock
from . import models
def test_a_run_first() -> None:
    assert models.cache_me() == 1

def test_b_run_second() -> None:
    # We want to mock the network call for this test case
    with mock.patch.object(models, 'expensive_network_call', return_value=2) as mock_network_call:
        assert models.cache_me() == 2
        assert mock_network_call.called
```

in our app config we import a function that uses `lru_cache` *this is called super early by pytest-django in _setup_django preventing python-antilru from monkey patching lru_cache*
```python
from django.apps import AppConfig


class ExampleappConfig(AppConfig):
    default_auto_field = 'django.db.models.BigAutoField'
    name = 'exampleapp'

    def ready(self):
        from .models import cache_me  # this import is loaded before `pytest_collection` runs
```

in our project settings `example/settings.py` we add our `exampleapp`
```python
INSTALLED_APPS = [
    'django.contrib.admin',
    ...
    'django.contrib.staticfiles',
    'exampleapp',
]
```

Now we run with `python-antilru` installed, but without `pytest-django` [settings configured](https://pytest-django.readthedocs.io/en/latest/configuring_django.html#command-line-option-ds-settings) followed by with settings configured.

```python
$ pip install pytest-antilru
Collecting pytest-antilru
  Using cached pytest_antilru-1.1.1-py2.py3-none-any.whl (5.3 kB)
...
Successfully installed pytest-antilru-1.1.1

$ pytest exampleapp/tests.py::test_a_run_first exampleapp/tests.py::test_b_run_second
=============================================================== test session starts ===============================================================
platform linux -- Python 3.10.11, pytest-7.3.1, pluggy-1.0.0
plugins: django-4.5.2, antilru-1.1.1
collected 2 items

exampleapp/tests.py ..                                                                                                                      [100%]

================================================================ 2 passed in 0.02s ================================================================
$ pytest exampleapp/tests.py::test_b_run_second exampleapp/tests.py::test_a_run_first
=============================================================== test session starts ===============================================================
platform linux -- Python 3.10.11, pytest-7.3.1, pluggy-1.0.0
plugins: django-4.5.2, antilru-1.1.1
collected 2 items

exampleapp/tests.py ..                                                                                                                      [100%]

================================================================ 2 passed in 0.02s ================================================================
$ pytest -s --ds=example.settings exampleapp/tests.py
=============================================================== test session starts ===============================================================
platform linux -- Python 3.10.11, pytest-7.3.1, pluggy-1.0.0
django: settings: example.settings (from option)
rootdir: /home/example
plugins: django-4.5.2, antilru-1.1.1
collected 2 items

exampleapp/tests.py .F

==================================================================== FAILURES =====================================================================
________________________________________________________________ test_b_run_second ________________________________________________________________

    def test_b_run_second() -> None:
        # We want to mock the network call for this test case
        with mock.patch.object(models, 'expensive_network_call', return_value=2) as mock_network_call:
>           assert models.cache_me() == 2
E           assert 1 == 2
E            +  where 1 = <functools._lru_cache_wrapper object at 0x7f8ab13ae140>()
E            +    where <functools._lru_cache_wrapper object at 0x7f8ab13ae140> = models.cache_me

exampleapp/tests.py:12: AssertionError
============================================================= short test summary info =============================================================
FAILED exampleapp/tests.py::test_b_run_second - assert 1 == 2
=========================================================== 1 failed, 1 passed in 0.08s ===========================================================
```

I think a solution to this might be to make `pytest-antilru` use `pytest_load_initial_conftest` instead of `pytest_collection`, but thought I'd ask here first. I'm also aware that it would dependent on the order that plugins would be loaded, which I think [(currently) can't be specifed](https://github.com/pytest-dev/pytest/issues/935). Is there a better way to solve this? I'm happy to open a pr


Discussion / hints:
Thanks for the detailed report! I appreciate the minimal repo to help illustrate the situation. 

`pytest_collection` was chosen arbitrarily, as a early enough place to monkey patch before application code imported `lru_cache`. I have no reservations using `pytest_load_initial_conftest`, to get ahead of `pytest-django`.

I'd very much appreciate a PR, as I have not used django and verifying correctness would take time.

> Is there a better way to solve this?

I don't think there's much more we can do. The [bootstrap hooks](https://docs.pytest.org/en/7.1.x/reference/reference.html?highlight=pytest_collection#bootstrapping-hooks) are super early and without the ability to specify ordering, will be tricky. I think a doc section for gotchas will be all we can do, for pytest-django and other plugins that import early. The open pytest issue doesn't seem to provide much of a solution.

Fix patch:
diff --git a/README.md b/README.md
index 4cbc862..5734bd1 100644
--- a/README.md
+++ b/README.md
@@ -70,6 +70,31 @@ Simply install this in the same python environment that `pytest` uses and the re
 pip install pytest-antilru
 ```
 
+## Configuration
+
+Add this where ever [your pytest configurations](https://docs.pytest.org/en/stable/reference/customize.html) live.
+
+### `lru_cache_disabled`
+
+`lru_cache_disabled` is an allowlist of module paths to disable `lru_cache` for.
+This allows you to target disable caching for specific modules that are causing test pollution.
+
+The default behaviour of `pytest-antilru` is to disable `lru_cache` everywhere.
+However, this can interfere with other dependencies that are reliant on `lru_cache` behaviour to behave correctly
+within the same test run.
+
+```ini
+[tool.pytest.ini_options]
+lru_cache_disabled = '''
+    my_module.util
+    my_module.client_lib.database
+    '''
+```
+
+In this example, any usage of `lru_cache` in a file inside `my_module.util` or `my_module.client_lib.database`
+will be disabled.
+All other instances will continue to be cached within a test run.
+
 ## How to test the software
 
 ```sh


Test patch:
diff --git a/pytest_antilru/main.py b/pytest_antilru/main.py
index bc8e5ed..04dea57 100644
--- a/pytest_antilru/main.py
+++ b/pytest_antilru/main.py
@@ -7,13 +7,17 @@
 import pytest
 
 CACHED_FUNCTIONS = []
+old_lru_cache = None
 
 
-@pytest.hookimpl(hookwrapper=True)
-def pytest_collection(session):  # pylint: disable=unused-argument
+@pytest.hookimpl(tryfirst=True, hookwrapper=True)
+def pytest_load_initial_conftests(early_config, parser, args):  # pylint: disable=unused-argument
     """Monkey patch lru_cache, before any module imports occur."""
+    parser.addini('lru_cache_disabled', 'Allowlist of module prefixes to apply disable lru_cache on', type='linelist')
+    lru_cache_disabled_modules = early_config.getini('lru_cache_disabled')
 
     # Gotta hold on to this before we patch it away
+    global old_lru_cache
     old_lru_cache = functools.lru_cache
 
     @wraps(functools.lru_cache)
@@ -43,7 +47,13 @@ def lru_cache_wrapper(maxsize=Ellipsis, typed=Ellipsis, **kwargs):
         def decorating_function(user_function):
             """Wraps the user function, which is what everyone is actually using. Including us."""
             _wrapper = wrapper(user_function)
-            CACHED_FUNCTIONS.append(_wrapper)
+            if lru_cache_disabled_modules:  # pragma: no cover
+                for module_path in lru_cache_disabled_modules:
+                    if user_function.__module__.startswith(module_path):
+                        CACHED_FUNCTIONS.append(_wrapper)
+                        break
+            else:
+                CACHED_FUNCTIONS.append(_wrapper)
             return _wrapper
 
         return decorating_function
@@ -53,6 +63,10 @@ def decorating_function(user_function):
 
     yield
 
+
+@pytest.hookimpl(hookwrapper=True)
+def pytest_collection(session):
+    yield
     # Be a good citizen and undo our monkeying
     functools.lru_cache = old_lru_cache