Repository: mdn/kuma

Issue:
Let's CDN cache most site-search queries
I downloaded the 1,000 most frequent URLs when people do a site-search in `en-US`. 
In en-US alone we have 7,586 searches per day. And that's based only on the top 1,000 search terms. Remember, this only counts people who have GA tracking not blocked. 

And I estimate that roughly 59% of all our `/$locale/docs/` traffic is `en-US`. That means that **we get 13.5k site-searches per day**. And remember, this is just for the top 1,000 search terms. And since site-searches today is never cached by the CDN, that's how many times Kuma has to process 13.5k Elasticsearch queries per day (and equally all the Django code that wraps Elasticsearch into a request/(JSON)response). Perhaps roughly 20k site-searches per when you include all search terms and the people who don't let GA track.
Also, each request that goes into Django also triggers the rate limiter which means it has to use Redis too. 

As of today, site-search triggers a full Django+React SSR render and this accounts for 18% of the most time-consuming work our EC2 has to do:
<img width="443" alt="Screen Shot 2021-02-25 at 4 41 05 PM" src="https://user-images.githubusercontent.com/26739/109223288-647b6c80-7788-11eb-911c-d44582735c5e.png">
That number will shrink by itself once Yari takes over the search and it's just a `/api/v1/search` XHR request.

I speculate that we can leverage the CDN to:

- **Make users get faster search results**
- **Reduce the stress on our Kubernetes cluster and be able to use a lot less EC2 power and save money and electricity**

Many searches are a single word (i.e. 0 spaces)
```
0 spaces: 91.3%
1 spaces: 8.5%
2 spaces: 0.2%
```

Most searches are short single words. A complete list of the [top 1,000 most common searches are here](https://gist.github.com/peterbe/7cc2c8fa1cc72e976b9ac0caa0852055)

If you analyze the length of the search terms you get:
```
0-10       85.1%
11-20      14.6%
21+        0.3%
```

Since all traffic always goes through the CDN anyway, we can set a `Cache-Control` on these. Say, 24h. I don't think it's expected that a search for `splice` should get documents that have recently changed within the last day. 

We could potentially make the setting of the `Cache-Control` only on short and simple searches without any advanced filtering (e.g `?locale=ja&locale=en-us&slug_prefix=Web/CSS`). That way our CDN won't have to store too many things that aren't worth caching.


Discussion:
I'm wondering … are you able to invalidate Cache on the server-side?
I mean, you could have a diff of the changes since the last deployment (`--since=yesterday` or something).
That would give you a list of files that change.
If you would happen to know, for which queries those documents were shown, these query search terms would be in need of invalidation.

In turn, you could bump the caching period (CDN <-> ES) for longer …

Resolving patch:
diff --git a/kuma/api/v1/search/__init__.py b/kuma/api/v1/search/__init__.py
--- a/kuma/api/v1/search/__init__.py
+++ b/kuma/api/v1/search/__init__.py
@@ -1,5 +1,6 @@
 from django import http
 from django.conf import settings
+from django.utils.cache import patch_cache_control
 from elasticsearch import exceptions
 from elasticsearch_dsl import Q, query, Search
 from redo import retrying
@@ -8,6 +9,12 @@
 
 from .forms import SearchForm
 
+# This is the number of seconds to be put into the Cache-Control max-age header
+# if the search is successful.
+# We can increase the number as we feel more and more comfortable with how
+# the `/api/v1/search` works.
+SEARCH_CACHE_CONTROL_MAX_AGE = 60 * 60 * 12
+
 
 class JsonResponse(http.JsonResponse):
     """The only reason this exists is so that other Django views can call
@@ -67,7 +74,25 @@ def search(request, locale=None):
         params,
         make_suggestions=make_suggestions,
     )
-    return JsonResponse(results)
+    response = JsonResponse(results)
+
+    # The reason for caching is that most of the time, the searches people make
+    # are short and often stand a high chance of being reused by other users
+    # in the CDN.
+    # The worst that can happen is that we fill up the CDN with cached responses
+    # that end up being stored there and never reused by another user.
+    # We could consider only bothering with this based on looking at the parameters.
+    # For example, if someone made a search with "complex parameters" we could skip
+    # cache-control because it'll just be a waste to store it (CDN and client).
+    # The reason for not using a "shared" cache-control, i.e. `s-max-age` is
+    # because the cache-control seconds we intend to set are appropriate for both
+    # the client and the CDN. If the value set is 3600 seconds, that means that
+    # clients might potentially re-use their own browser cache if they trigger
+    # a repeated search. And it's an appropriate number for the CDN too.
+    # For more info about how our search patterns behave,
+    # see https://github.com/mdn/kuma/issues/7799
+    patch_cache_control(response, public=True, max_age=SEARCH_CACHE_CONTROL_MAX_AGE)
+    return response
 
 
 def _find(params, total_only=False, make_suggestions=False, min_suggestion_score=0.8):


Test patch:
diff --git a/kuma/api/v1/tests/test_search.py b/kuma/api/v1/tests/test_search.py
--- a/kuma/api/v1/tests/test_search.py
+++ b/kuma/api/v1/tests/test_search.py
@@ -13,6 +13,8 @@ def test_search_validation_problems(user_client, settings):
     response = user_client.get(url, {"q": "x", "locale": "xxx"})
     assert response.status_code == 400
     assert response.json()["errors"]["locale"][0]["code"] == "invalid_choice"
+    # all non-200 OK responses should NOT have a Cache-Control header set.
+    assert "cache-control" not in response
 
     # 'q' exceeds max allowed characters
     response = user_client.get(url, {"q": "x" * (settings.ES_Q_MAXLENGTH + 1)})
@@ -87,6 +89,10 @@ def test_search_basic_match(user_client, settings, mock_elasticsearch):
     url = reverse("api.v1.search")
     response = user_client.get(url, {"q": "x"})
     assert response.status_code == 200
+    assert "public" in response["Cache-Control"]
+    assert "max-age=" in response["Cache-Control"]
+    assert "max-age=0" not in response["Cache-Control"]
+
     assert response["content-type"] == "application/json"
     assert response["Access-Control-Allow-Origin"] == "*"
     data = response.json()