Repository: jazzband/django-debug-toolbar

Issue:
DJDT not showing low level caching correctly in Docker
Hi,
I working with DJDT for a few months.
I figure out the "not showing" issue in docker, but the data inside the cache panel is not showing properly.
The inner data, doesnt match the outer data **and** in the arguments column it shows addresses or something alike.

A proper data: (taken from an online guide)
![image](https://user-images.githubusercontent.com/24847258/130961206-d2bcc51f-218d-44fe-8e58-a075ef24ec0a.png)
(From: https://testdriven.io/blog/django-low-level-cache/#cache-backend)

What it shows inside a Docker:
![ch](https://user-images.githubusercontent.com/24847258/130967922-046612c4-92f6-4a10-839f-11231f3e07e7.png)

BTW: It also shows it like a view cache, but the view is not cached (Its a dynamic view)
My Configuration:

```

DEBUG_TOOLBAR_ENABLED = True
if DEBUG and DEBUG_TOOLBAR_ENABLED:
    INSTALLED_APPS.append('debug_toolbar')
    MIDDLEWARE.insert(0, 'debug_toolbar.middleware.DebugToolbarMiddleware')
    MIDDLEWARE.insert(1, 'django.middleware.cache.UpdateCacheMiddleware')
    MIDDLEWARE.append('django.middleware.cache.FetchFromCacheMiddleware')
    #INTERNAL_IPS.append('127.0.0.1')

    DEBUG_TOOLBAR_CONFIG = {
        'SHOW_TOOLBAR_CALLBACK': lambda request: False if request.is_ajax() else True,
    }

```

Does anyone have idea how to solve it ?
Where to start ?


Discussion:
Hi @LiorA1, can you share the view or code that you're trying to use this with? Looking at the arguments, it seems like you're using some decorators to handle the caching and since none are `set`, I would assume any low level caching logic is being skipped over because the entire view's response is being cached.

Please also share any caching related settings, such as [middleware](https://docs.djangoproject.com/en/3.2/topics/cache/#the-per-site-cache), decorators, [caching applied via the urls](https://docs.djangoproject.com/en/3.2/topics/cache/#specifying-per-view-cache-in-the-urlconf) and whether or not you're using template caching for this view.
The view was view-cached in the past, but I remove it.
I have another static view that is view-cached.
I tried to restart Docker, and to rebuild the image.

My Base Settings:
```
MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
```
My Development Settings:
```
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.PyMemcacheCache',
        'LOCATION': 'cache:11211',
    }
}
```
urls:
```
path('list/', views.ResumeListView.as_view(), name="resume_list"),
```

The view:
```
class ResumeListView(OwnerListView):
    ""Display all the resumes""
    model = Resume
    ordering = ['-created_at']
    # template_name = "resumes/<modelName>_list.html"
    queryset = Resume.objects.prefetch_related('tags', 'author', 'author__profile')

    def get_queryset(self):
        base_queryset = super(ResumeListView, self).get_queryset()

        # Check for searchTerm existence
        searchTerm = self.request.GET.get("search", False)

        if searchTerm:
            tags_required = Tag.objects.tags_id_from_str(searchTerm)

            # ordering attributes kept
            ordering = self.get_ordering()
            if not ordering:
                ordering = []
            else:
                ordering = self.ordering[:]
            ordering.insert(0, '-score')

            # Query which resumes have the wanted tags, order by the match score.
            q_Query = Q(tags__in=tags_required)
            # res_queryset = base_queryset.filter(tags__isnull=False).distinct().annotate(score=Count('tags', filter=q_Query)).filter(score__gt=0).order_by(*ordering)

            res_queryset = base_queryset.filter(tags__isnull=False).distinct().annotate(score=Count('tags')).filter(q_Query, score__gt=0).order_by(*ordering)

        else:
            res_queryset = base_queryset.fetch_store_resume_list()

        return res_queryset
```

(I decided to do all the caching in the models file)
The specific Custom Manager method:
```
def fetch_store_resume_list(self):
        ""
        Used for Fetch and Store the query of all the resumes with the related fields.
        Used primarily by the 'ResumeListView' View.
        ""
        resumes_cache_key = "resumes"
        resumes_queryset = cache.get(resumes_cache_key)
        if resumes_queryset is None:
            resumes_queryset = self
            cache.set(resumes_cache_key, resumes_queryset, timeout=300)

        return resumes_queryset
```

If I forget smoething the project is on github:
https://github.com/LiorA1/resume_reviews
> The view was view-cached in the past, but I remove it.
> I have another static view that is view-cached.

Sounds good. Please retry the request without the view level caching and let us know what the toolbar presents in the cache panel.
Tried it.
I stopped using any caching in the following App, but now nothing shows.. (After erase all the docker containers/images/volumes)

This is a sequential request, in the first one SQL queries appeared.

![ch](https://user-images.githubusercontent.com/24847258/131127013-b63fe242-f34d-41d5-bc3e-446dfeedf70c.png)

It doesnt show nothing.
Is there any place that I check with debugger (in the code) ?

P.S-
I trying to cache entries with file storage at AWS S3. Maybe this cause the issue ?
Did you try without the view level decorators and [without the middleware](https://github.com/LiorA1/resume_reviews/blob/main/resume_project/settings/development.py#L41-L42)?

It does seem like something is broken, but I can't tell what. I'd really appreciate it if you could create a small reproducible example for us to more thoroughly investigate.
In the particular App I even dont import the cache_page decorator.
I didnt try without the Django middleware. It doesnt required to allow caching ?

I'm working with VSC and docker-compose, it hard to debug, but I will try and update here.

I will try to reproduce it and update here when finish.

Whoops. I didn't mean without all middleware. Only without the middleware highlighted in the link above which shows your project's settings:

```
    MIDDLEWARE.insert(1, 'django.middleware.cache.UpdateCacheMiddleware')
    MIDDLEWARE.append('django.middleware.cache.FetchFromCacheMiddleware')
```
Ok, Four hours later - reproduce it.
pls see: https://github.com/LiorA1/djdt_project

I will try to comment out these middleware.
I have there a json file for a superuser (name:root/pass:qwertyuiop) and few entries.

Edit: 
I tried to comment these middleware - The caching doesnt get called. (Only SQL queries happens)
Thanks for the help! There was indeed a bug, but it was unrelated to docker.

Resolving patch:
diff --git a/debug_toolbar/panels/cache.py b/debug_toolbar/panels/cache.py
--- a/debug_toolbar/panels/cache.py
+++ b/debug_toolbar/panels/cache.py
@@ -8,6 +8,7 @@
 except ImportError:
     ConnectionProxy = None
 
+import django
 from django.conf import settings
 from django.core import cache
 from django.core.cache import (
@@ -140,10 +141,27 @@ def decr_version(self, *args, **kwargs):
         return self.cache.decr_version(*args, **kwargs)
 
 
-class CacheHandlerPatch(CacheHandler):
-    def __getitem__(self, alias):
-        actual_cache = super().__getitem__(alias)
-        return CacheStatTracker(actual_cache)
+if django.VERSION < (3, 2):
+
+    class CacheHandlerPatch(CacheHandler):
+        def __getitem__(self, alias):
+            actual_cache = super().__getitem__(alias)
+            return CacheStatTracker(actual_cache)
+
+
+else:
+
+    class CacheHandlerPatch(CacheHandler):
+        def __init__(self, settings=None):
+            self._djdt_wrap = True
+            super().__init__(settings=settings)
+
+        def create_connection(self, alias):
+            actual_cache = super().create_connection(alias)
+            if self._djdt_wrap:
+                return CacheStatTracker(actual_cache)
+            else:
+                return actual_cache
 
 
 middleware_cache.caches = CacheHandlerPatch()
@@ -251,22 +269,40 @@ def title(self):
         )
 
     def enable_instrumentation(self):
-        if isinstance(middleware_cache.caches, CacheHandlerPatch):
-            cache.caches = middleware_cache.caches
+        if django.VERSION < (3, 2):
+            if isinstance(middleware_cache.caches, CacheHandlerPatch):
+                cache.caches = middleware_cache.caches
+            else:
+                cache.caches = CacheHandlerPatch()
         else:
-            cache.caches = CacheHandlerPatch()
+            for alias in cache.caches:
+                if not isinstance(cache.caches[alias], CacheStatTracker):
+                    cache.caches[alias] = CacheStatTracker(cache.caches[alias])
+
+            if not isinstance(middleware_cache.caches, CacheHandlerPatch):
+                middleware_cache.caches = cache.caches
 
         # Wrap the patched cache inside Django's ConnectionProxy
         if ConnectionProxy:
             cache.cache = ConnectionProxy(cache.caches, DEFAULT_CACHE_ALIAS)
 
     def disable_instrumentation(self):
-        cache.caches = original_caches
-        cache.cache = original_cache
-        # While it can be restored to the original, any views that were
-        # wrapped with the cache_page decorator will continue to use a
-        # monkey patched cache.
-        middleware_cache.caches = original_caches
+        if django.VERSION < (3, 2):
+            cache.caches = original_caches
+            cache.cache = original_cache
+            # While it can be restored to the original, any views that were
+            # wrapped with the cache_page decorator will continue to use a
+            # monkey patched cache.
+            middleware_cache.caches = original_caches
+        else:
+            for alias in cache.caches:
+                if isinstance(cache.caches[alias], CacheStatTracker):
+                    cache.caches[alias] = cache.caches[alias].cache
+            if ConnectionProxy:
+                cache.cache = ConnectionProxy(cache.caches, DEFAULT_CACHE_ALIAS)
+            # While it can be restored to the original, any views that were
+            # wrapped with the cache_page decorator will continue to use a
+            # monkey patched cache.
 
     def generate_stats(self, request, response):
         self.record_stats(


Test patch:
diff --git a/tests/middleware.py b/tests/middleware.py
new file mode 100644
--- /dev/null
+++ b/tests/middleware.py
@@ -0,0 +1,17 @@
+from django.core.cache import cache
+
+
+class UseCacheAfterToolbar:
+    """
+    This middleware exists to use the cache before and after
+    the toolbar is setup.
+    """
+
+    def __init__(self, get_response):
+        self.get_response = get_response
+
+    def __call__(self, request):
+        cache.set("UseCacheAfterToolbar.before", 1)
+        response = self.get_response(request)
+        cache.set("UseCacheAfterToolbar.after", 1)
+        return response
diff --git a/tests/settings.py b/tests/settings.py
--- a/tests/settings.py
+++ b/tests/settings.py
@@ -30,6 +30,7 @@
 MEDIA_URL = "/media/"  # Avoids https://code.djangoproject.com/ticket/21451
 
 MIDDLEWARE = [
+    "tests.middleware.UseCacheAfterToolbar",
     "debug_toolbar.middleware.DebugToolbarMiddleware",
     "django.middleware.security.SecurityMiddleware",
     "django.contrib.sessions.middleware.SessionMiddleware",
diff --git a/tests/test_integration.py b/tests/test_integration.py
--- a/tests/test_integration.py
+++ b/tests/test_integration.py
@@ -6,6 +6,7 @@
 import html5lib
 from django.contrib.staticfiles.testing import StaticLiveServerTestCase
 from django.core import signing
+from django.core.cache import cache
 from django.db import connection
 from django.http import HttpResponse
 from django.template.loader import get_template
@@ -102,6 +103,25 @@ def test_cache_page(self):
         self.client.get("/cached_view/")
         self.assertEqual(len(self.toolbar.get_panel_by_id("CachePanel").calls), 5)
 
+    def test_low_level_cache_view(self):
+        """Test cases when low level caching API is used within a request."""
+        self.client.get("/cached_low_level_view/")
+        self.assertEqual(len(self.toolbar.get_panel_by_id("CachePanel").calls), 2)
+        self.client.get("/cached_low_level_view/")
+        self.assertEqual(len(self.toolbar.get_panel_by_id("CachePanel").calls), 3)
+
+    def test_cache_disable_instrumentation(self):
+        """
+        Verify that middleware cache usages before and after
+        DebugToolbarMiddleware are not counted.
+        """
+        self.assertIsNone(cache.set("UseCacheAfterToolbar.before", None))
+        self.assertIsNone(cache.set("UseCacheAfterToolbar.after", None))
+        self.client.get("/execute_sql/")
+        self.assertEqual(cache.get("UseCacheAfterToolbar.before"), 1)
+        self.assertEqual(cache.get("UseCacheAfterToolbar.after"), 1)
+        self.assertEqual(len(self.toolbar.get_panel_by_id("CachePanel").calls), 0)
+
     def test_is_toolbar_request(self):
         self.request.path = "/__debug__/render_panel/"
         self.assertTrue(self.toolbar.is_toolbar_request(self.request))
@@ -376,7 +396,7 @@ def test_view_returns_template_response(self):
         self.assertEqual(response.status_code, 200)
 
     @override_settings(DEBUG_TOOLBAR_CONFIG={"DISABLE_PANELS": set()})
-    def test_incercept_redirects(self):
+    def test_intcercept_redirects(self):
         response = self.client.get("/redirect/")
         self.assertEqual(response.status_code, 200)
         # Link to LOCATION header.
diff --git a/tests/urls.py b/tests/urls.py
--- a/tests/urls.py
+++ b/tests/urls.py
@@ -20,6 +20,7 @@
     path("new_user/", views.new_user),
     path("execute_sql/", views.execute_sql),
     path("cached_view/", views.cached_view),
+    path("cached_low_level_view/", views.cached_low_level_view),
     path("json_view/", views.json_view),
     path("redirect/", views.redirect_view),
     path("login_without_redirect/", LoginView.as_view(redirect_field_name=None)),
diff --git a/tests/views.py b/tests/views.py
--- a/tests/views.py
+++ b/tests/views.py
@@ -1,4 +1,5 @@
 from django.contrib.auth.models import User
+from django.core.cache import cache
 from django.http import HttpResponseRedirect, JsonResponse
 from django.shortcuts import render
 from django.template.response import TemplateResponse
@@ -33,6 +34,15 @@ def cached_view(request):
     return render(request, "base.html")
 
 
+def cached_low_level_view(request):
+    key = "spam"
+    value = cache.get(key)
+    if not value:
+        value = "eggs"
+        cache.set(key, value, 60)
+    return render(request, "base.html")
+
+
 def json_view(request):
     return JsonResponse({"foo": "bar"})