Repository: mdn/kuma

Issue:
Empty SERP
While I was researching something on MDN I noticed that some search results come up empty. Take https://developer.mozilla.org/en-US/search?q=media+overview for example:

![empty-serp](https://user-images.githubusercontent.com/3097194/106031824-03e90900-60d0-11eb-8ce5-fff31cc6092e.png)

What's going on here?
Can the situation be improved?


Discussion:
This happens because the document there got included because it contained a match on the `title` **only**. There were no matches at all in the `body` so there can not be any `excerpts`. 
You can see it better here: https://developer.mozilla.org/api/v1/search?q=media+overview (look for the `highlight.body` key on the 6th and 7th result. 

I think what we should do is that if there are no excerpts, we fall back to the document's summary. What do you think about that?

Hm, I think, I would rather downrank the result instead (if possible).
A match on the `title` is boosted over a match on the `body`. So there might be a doc that has matches on the body but much worse matches in the title. 
There's also the complexity of the fact that "downranking" is actually influenced by the popularity of the document. So I don't think it's feasible to try to push these down more. They have to appear because the search logic is "INCLUDE IF (MATCH IN TITLE OR MATCH IN BODY)"
One thing I need to consider is to search by the `summary`! If a match matches the summary, I think that should rank a bit higher than a match inside the body. 
Sounds like a complicated (as in: small tweaks have major impacts) topic.

Can you point me to the current implementation? Or is it for staff only?
> Can you point me to the current implementation? Or is it for staff only?

Implementation? Do you mean the code that executes the search? It's all in Kuma:
https://github.com/mdn/kuma/blob/3474f01b723e10009f61678d2ee67e0696c42868/kuma/api/v1/search/__init__.py#L62-L243


Resolving patch:
diff --git a/kuma/api/v1/search/__init__.py b/kuma/api/v1/search/__init__.py
--- a/kuma/api/v1/search/__init__.py
+++ b/kuma/api/v1/search/__init__.py
@@ -197,6 +197,7 @@ def _find(params, total_only=False, make_suggestions=False, min_suggestion_score
             "slug": hit.slug,
             "popularity": hit.popularity,
             "archived": hit.archived,
+            "summary": hit.summary,
             "highlight": {
                 "body": body_highlight,
                 "title": title_highlight,
diff --git a/kuma/search/views.py b/kuma/search/views.py
--- a/kuma/search/views.py
+++ b/kuma/search/views.py
@@ -101,6 +101,7 @@ def search(request, *args, **kwargs):
                         "title": x["title"],
                         "slug": x["slug"],
                         "locale": x["locale"],
+                        "summary": x["summary"],
                         "excerpt": "<br>".join(x["highlight"].get("body", [])),
                     }
                     for x in results["documents"]


Test patch:
diff --git a/kuma/api/v1/tests/test_search.py b/kuma/api/v1/tests/test_search.py
--- a/kuma/api/v1/tests/test_search.py
+++ b/kuma/api/v1/tests/test_search.py
@@ -71,6 +71,7 @@ def test_search_basic_match(user_client, settings, mock_elasticsearch):
         {
             "id": "/en-us/docs/Foo",
             "title": "Foo Title",
+            "summary": "Foo summary",
             "locale": "en-us",
             "archived": False,
             "slug": "Foo",
@@ -100,6 +101,7 @@ def test_search_basic_match(user_client, settings, mock_elasticsearch):
             "score": 1.0,
             "slug": "Foo",
             "title": "Foo Title",
+            "summary": "Foo summary",
         }
     ]
 
@@ -110,6 +112,7 @@ def test_search_case_insensitive_locale(user_client, settings, mock_elasticsearc
         {
             "id": "/en-us/docs/Foo",
             "title": "Foo Title",
+            "summary": "Foo summary",
             "locale": "en-us",
             "archived": False,
             "slug": "Foo",
diff --git a/kuma/javascript/src/search-results-page.test.js b/kuma/javascript/src/search-results-page.test.js
--- a/kuma/javascript/src/search-results-page.test.js
+++ b/kuma/javascript/src/search-results-page.test.js
@@ -15,12 +15,14 @@ const fakeResults = {
             title: 'title1',
             locale: 'en-US',
             excerpt: 'Test <mark>result</mark> Excerpt <mark>2</mark>',
+            summary: 'Some summary',
         },
         {
             slug: 'slug2',
             title: 'title2',
             locale: 'en-US',
             excerpt: 'empty',
+            summary: 'Different summary',
         },
     ],
     start: 0,