# Query Analytics Part 1: Know Your Queries

Do you know what your users are searching for? What they’re finding? Or not finding?

The quality of search results drives users toward or away from using a service. If you can’t find it, it doesn’t exist… or it may as well not exist. A lack of discoverability leads to a lost customer. A library patron can’t borrow a book they can’t find. The bio-medical researcher won’t glean insights from research papers or genetic information that is not in the search results. If users aren’t finding what they need, expect, or what delights them, they’ll go elsewhere.

As developers, we’ve successfully deployed full-text search into our application. We can clearly see that our test queries are able to match what we expect, and the relevancy of those test queries looks good. But as we know, our users immediately try things we didn’t think to try and account for and will be presented with results that may or may not be useful to them. If you’re selling items from your search results page and “Sorry, no results match your query” comes up, how much money have you _not_ made? Even more insidious are results for common queries that aren’t providing the best results you have to offer; while users get results, there might not be the desired product within quick and easy reach to click and buy now.

Having Atlas Search enabled and in production is really the beginning of your search journey and also the beginning of the value you’ll get out of a well-tuned, and monitored, search engine.  Atlas Search provides Query Analytics, giving us actionable insights into the `$search` activity of our Atlas Search indexes. 

Note: Query Analytics is available in public preview for all MongoDB Atlas clusters on an M10 or higher running MongoDB v5.0 or higher to view the analytics information for the tracked search terms in the Atlas UI. Atlas Search doesn't track search terms or display analytics for queries on free and shared-tier clusters.

Callout section: Atlas Search Query Analytics focuses entirely on the frequency and number of results returned from each $search call. There are also several search metrics available for operational monitoring of CPU, memory, index size, and other useful data points.

## Factors that influence search results quality

You might be thinking, “Hey, I thought this Atlas Search thing would magically make my search results work well — why aren’t the results as my users expect? Why do some seemingly reasonable queries return no results or not quite the best results?”

Consider these various types of queries of concern:

| Query challenge    | Example |
| :-------- | :------- |
| Common name typos/variations | Jacky Chan, Hairy Potter, Dotcor Suess |
| Relevancy challenged | the purple rain, the the yes, there’s a band called that], to be or not to be |
| Part numbers, dimensions, measurements |  ⅝” driver bit, 1/2" wrench, size nine dress, Q-36, Q36, Q 36 |
| Requests for assistance | Help!, support, want to return a product, how to redeem a gift card, fax number |
| Because you know better | cheap sushi [the user really wants “good” sushi, don’t recommend the cheap stuff], blue shoes [boost the brands you have in stock that make you the most money], best guitar for a beginner |
| Word stems | Find nemo, finds nemo, finding nemo |
| Various languages, character sets, romanization | Flughafen, integraçao,中文, ko’nichiwa |
| Context, such as location, recency, and preferences | MDB [boost most recent news of this company symbol], pizza [show me nearby and open restaurants] |

Consider the choices we made, or were dynamically made for us, when we built our Atlas Search index — specifically, the analyzer choices we make per string field. What we indexed determines what is searchable and in what ways it is searchable. A default `lucene.standard` analyzed field gives us pretty decent, language-agnostic “words” as searchable terms in the index. That’s the default and not a bad one. However, if your content is in a particular language, it may have some structural and syntactic rules that can be incorporated into the index and queries too. If you have part numbers, item codes, license plates, or other types of data that are precisely specified in your domain, users will enter them without the exact special characters, spacing, or case. Often, as developers or domain experts of a system, we don’t try the wrong or _almost_ correct syntax or format when testing our implementation, but our users do.

With the number of ways that search results can go astray, we need to be keeping a close eye on what our users are experiencing and carefully tuning and improving.

## Virtuous search query management cycle

Maintaining a healthy search-based system deserves attention to the kinds of challenges just mentioned. A healthy search system management cycle includes these steps:

1. (Re-)deploy search
2. Measure and test
3. Make adjustments
4. Go to 1, repeat

### (Re-)deploying search

How you go about re-deploying the adjustments will depend on the nature of the changes being made, which could involve index configuration and/or application or query adjustments. 

Here’s where the [local development environment for Atlas could be useful, as a way to make configuration and app changes in a comfortable local environment, push the changes to a broader staging environment, and then push further into production when ready.

### Measure and test

You’ll want to have a process for analyzing the search usage of your system, by tracking queries and their results over time. Tracking queries simply requires the addition of `searchTerms` tracking information to your search queries, as in this template:

```
{
  $search: {
    "index": "",
    "": {
      
    },
    "tracking": {
      "searchTerms": ""
    }
  }
}
```

### Make adjustments

You’ve measured, twice even, and you’ve spotted a query or class of queries that need some fine-tuning. It’s part art and part science to tune queries, and with a virtuous search query management cycle in place to measure and adjust, you can have confidence that changes are improving the search results for you and your customers.

Now, apply these adjustments, test, repeat, adjust, re-deploy, test... repeat.

So far, we’ve laid the general rationale and framework for this virtuous cycle of query analysis and tuning feedback loop. Let’s now see what actionable insights can be gleaned from Atlas Search Query Analytics.

## Actionable insights

The Atlas Search Query Analytics feature provides two reports of search activity: __All Tracked Search Queries__ and __Tracked Search Queries with No Results__. Each report provides the top tracked “search terms” for a selected time period, from the last day up to the last 90 days.

Let’s talk about the significance of each report.

### All Tracked Search Queries

What are the most popular search terms coming through your system over the last month? This report rolls that up for you.

. 

  [1]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt66de710be3567815/6597dec2dc76629c3b7ebbf0/last_30_all_search_queries_chart.png
  [2]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt859876a30d03a803/6597dff21c5d7c16060f3a34/last_30_top_search_terms.png
  [3]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt70d62c57c413a8b6/6597e06ab05b9eccd9d73b49/search_terms_agg_pipeline.png
  [4]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt6f6c0668307d3bb2/6597e0dc1c5d7ca8bc0f3a38/last_30_no_results.png