An AI Assistant in the life of a
Search Engine Administrator

Lucian PrecupMaëlly Dubois
2024-06-12
MIX-CAMP E-COMMERCE SEARCH #MICES2024@a2lean1

Who are we?

Experts in search technologies
Integrators of Elasticsearch,
OpenSearch and Solr
Consulting and Training providers
Developers of a2 E-Commerce andEnterprise Search solution
Developers of all.site - yourCollaborative Search Engine

2@M_I_C_E_S

2024-06-10#MICES2024@a2leanBerlin Buzzwords 2022 - The life of a search engine administrator

https://pretalx.com/bbuzz22/talk/
P9ZFJL/3@M_I_C_E_S

2024-06-10#MICES2024@a2leanAgendaA day in the life of a search engine
administrator
Improving the life of a search engine
administrator
NLU and LLMs to automate actions
Analytics and Machine Learning to bringinsights

4@M_I_C_E_S

2024-06-10#MICES2024@a2leanA day in the life of asearch engine
administrator
5@M_I_C_E_S

2024-06-10#MICES2024@a2lean

A search engine is a living entity
Evolution

Seasonality
Levels of stocks
Lifecycle of documents
Marketing events
News
User’s expectations
and position

Regression…

6@M_I_C_E_S

2024-06-10#MICES2024@a2leanDiﬀerent use cases, diﬀerent stakes
E-commerce-

Business relevance (stocks, personalization,marketing, agenda, etc.)

Enterprise Search-User productivity
Information access security

Hybrid-E.g. E-commerce + Reference website

Specialized
7@M_I_C_E_S

2024-06-10#MICES2024@a2leanSearch is about…
Leveraging (moving) signals in order to fulﬁll a user’s need8

@M_I_C_E_S

2024-06-10#MICES2024@a2leanObjectives and KPIs?
AverageRank tocart

SRP-to-cart

0 result

Sales% sessions
use facets
MarketshareSuccessProﬁt

0 click

CTR
Searches persession

Customer
Satisfaction

Dwell time
Rephraserate9@M_I_C_E_S

reﬁnements

What to monitor?
Business needs (relevance)
Data infrastructure-Crawling errors
Datasource availability
Data errors

User satisfaction…

10 @M_I_C_E_S

2024-06-10#MICES2024@a2leanHypothesis: Challenges are addressed
How to deﬁne the KPIs?
What are the domain speciﬁc issues and how to deal with them?
What to monitor?
How to deﬁne the roadmap?
…

11 @M_I_C_E_S

2024-06-10#MICES2024@a2leanThe search administrator’s roles

12 @M_I_C_E_S

2024-06-10#MICES2024@a2leanDay to day tools
Business people (merchandising)-

Business Console

Reporting teams-Analytics, dashboards

Data teams-

Automatic exports

13 @M_I_C_E_S

2024-06-10#MICES2024@a2leanBusiness Console - the ultimate tool

Connectors andIntegrations

Adelean a2Crawl
IngestTransform
Aggregate
Index

SearchServicesCore searchtechnology

Business Console
14 @M_I_C_E_S

2024-06-10#MICES2024@a2lean

UIBusiness Console the details

15 @M_I_C_E_S2024-06-10#MICES2024@a2leanBusiness Console / Debug
Simulate a user search, visualize the data, compare the relevance scores, ensure the non-regression

16 @M_I_C_E_S

2024-06-10#MICES2024@a2leanBusiness Console / Conﬁg
E.g. facets

17 @M_I_C_E_S

2024-06-10#MICES2024@a2leanTools - ensure non regression

18 @M_I_C_E_S

2024-06-10#MICES2024@a2leanImproving the lifeof a search engine
administrator
19 @M_I_C_E_S

2024-06-10#MICES2024@a2leanAutomatization with AI → no more Administrators?
Fine grained relevance tuning-VS.

BoostsFunction scores
Synonyms
Redirections
Query rewriting
User experience--

Takes into account Business needs(marketing, stock, partnerships, …)
Easily understandable andmaintainable by the Business

20 @M_I_C_E_S

Ranking model
Learning algorithm
Training data

Conclusion :Conclusion :-Learning to Rank
(automated relevance tuning)2024-06-10-Very expensive to implement properly
Needs a lot of maintenance
Does worse if not properly trained andmaintainedBlack box to the Business#MICES2024@a2leanhttps://2021.berlinbuzzwords.de/session/beyond-artificial-intellig
ence-search21Search Engine Administrator’s wish …
I need my Search Engine to be like Google

22

And my Business Console like ChatGPTExamples of API calls to automate
Please add a synonym for car = automobile for ﬁeld title
Can you boost by 10 the Münchner Paulaner brand from September 21 toOctober 6, 2024 when my users are searching for beer?
Please test if removing the description ﬁeld from the list of searchable ﬁelds
increases the relevance of my search engine.

23 @M_I_C_E_S

2024-06-10#MICES2024@a2lean

Examples of alerts and suggestions
It looks like the vast majority of your users that are searching for sugar, add
only products from the Baking Ingredients category to cart. Would you like tocreate a ﬁltered redirection?
Based on your product catalog and user activity, here is a synonym
suggestion that would reduce zero results queries: salad=letuce. Would you
like me to add it to your conﬁguration?

24 @M_I_C_E_S

2024-06-10#MICES2024@a2leanNatural Language
Understanding andLarge Language
Models toautomate actions
25 @M_I_C_E_S

2024-06-10#MICES2024@a2leanBerlin Buzzwords 2019 - Integrate your Search Engine witha Voice Assistant - an AI Assistant on top of APIs

https://2019.berlinbuzzwords.de/1
9/session/integrate-your-search-e
ngine-voice-assistant.html

26 @M_I_C_E_S

2024-06-10#MICES2024@a2lean🧠What can the Large Language Models (LLMs) do?
++ Understand human language (NLU, NLP)

→ generic machine learning models++ Perform a synthesis→ generic machine learning models++ Hold a conversation and understand a context→ the very principle of LLMs, remuneration for
word generation+ Have an exhaustive knowledge base→ if model trained on this basis+ Adapt and learn→ use the context (immediate), re-training ofthe model (long term)(+) Allow multimodal interactivity→ interpretation and generation of images(+) Understand the voice→ external module(+) Automate tasks→ code generation~ Respond eﬃciently and relevantly→ hallucinations

27 @M_I_C_E_S

2024-06-10#MICES2024@a2leanFrom voice to API calls

INTENTVOICESPEECH-TO-TEXTUTTERANCESLOTS

The user speaks to
his voice assistant

28 @M_I_C_E_S

Voice istransformed intotext

2024-06-10

The system recognizes
an utterance related
to an intention andextracts keywords
from the slots#MICES2024@a2lean

APICall the API
corresponding to theintent with theparameters derived
from the slots

AI Assistant in OpenSearch Dashboards
for Observability and Data Analytics use cases

29 @M_I_C_E_S

2024-06-10#MICES2024@a2lean

AI Assistant in OpenSearch Dashboards
for Observability and Data Analytics use cases

30 @M_I_C_E_S

2024-06-10#MICES2024@a2lean

AI Assistant in OpenSearch Dashboards
for Observability and Data Analytics use cases

31 @M_I_C_E_S

2024-06-10#MICES2024@a2lean

AI Assistant in OpenSearch Dashboards
for Observability and Data Analytics use cases

https://www.youtube.com/
watch?v=9r0RyB_oHKk&
ab_channel=OpenSearch

32 @M_I_C_E_S

AI Assistant in Kibana
for Observability and Data
Analytics use cases

33 @M_I_C_E_S

NLU and LLMs to automate actions → implementation

PATCH /catalogs/config/${catalog}{"op": "add","path":"${getFieldPath({field})}.valueBoosts${ index ?
('.' + index) : ''}","value": "index ? value : [ value ]"}34 @M_I_C_E_S

2024-06-10#MICES2024@a2leanNLU and LLMs to automate actions → implementation

PATCH /catalogs/config/${catalog}{"op": "add","path": "brand_name.valueBoosts.0","value": {"value": "Munchner PAULANER","approximate": null,"boost": "10","trigger": {"text": "beer","textMatchingMode": "TEST_SYNONYMS","beginDate": "20240920 000000","endDate": "20241006 000000"}}}

35 @M_I_C_E_S

2024-06-10#MICES2024@a2leanNLU and LLMs to automate actions → implementation
Step 1/

Step 2/

Prompt(Optionally) Validate the syntax and validityof the API call-

User query
Example of API call and parameters
Instructions on how to recognize anutterance related to an intention andextracts parameters from the slots

Launch the API call
(Optionally) Refresh the screen

Result-

The code of an API call with the rightparameters

36 @M_I_C_E_S

2024-06-10#MICES2024@a2leanNLU and LLMs to automate actions → the ﬁnal result

37 @M_I_C_E_S

2024-06-10#MICES2024@a2leanNLU and LLMs to automate actions → the ﬁnal result

38 @M_I_C_E_S

2024-06-10#MICES2024@a2leanNLU and LLMs to automate actions → the ﬁnal result

39 @M_I_C_E_S

2024-06-10#MICES2024@a2leanNLU and LLMs to automate actions → the ﬁnal result

40 @M_I_C_E_S

2024-06-10#MICES2024@a2leanTesting the
non-regression@M_I_C_E_S

41

Quepid

42 @M_I_C_E_STesting the non-regression

43Haystack EU 2022 - A practical approach to measuring therelevance and preventing regressions

https://haystackconf.com/eu2022/
talk-3/44 @M_I_C_E_S

2024-06-10#MICES2024@a2leanAnalytics andMachine Learning
to bring insights
45 @M_I_C_E_S

2024-06-10#MICES2024@a2leanReminder: Examples of alerts and suggestions
It looks like the vast majority of your users that are searching for sugar, add
only products from the Baking Ingredients category to cart. Would you like tocreate a ﬁltered redirection?
Based on your product catalog and user activity, here is a synonym
suggestion that would reduce zero results queries: salad=letuce. Would you
like me to add it to your conﬁguration?

46 @M_I_C_E_S

2024-06-10#MICES2024@a2leanMonitor zero
results queries
and suggest
synonyms
https://github.com/adelean/elasticsearch-word2vec@M_I_C_E_S

47

Improveprecision andrelevancy of
facet values
Issue: how to make facet values
relevant in the context of many
"less relevant" results ?
https://github.com/adelean/elasticsearch-word2vec@M_I_C_E_S

48

49 @M_I_C_E_S

2024-06-10#MICES2024@a2lean50 @M_I_C_E_S

2024-06-10#MICES2024@a2leanImprove precisionand relevancy of facet
values: the solution

Solutions: work on your search precision

Analytics and data science have clues: for instance, when clients
type “tomato”, is there a category which regroup most of theclicks ?
All you must do is prefilter some facets (or even all the results)
with this category : 80% of the result set will disappear and yourfilters will look good !

Examples of prefiltering at Carrefour:•11% of results for “tomatos” are in the “Fresh vegetables” category but they
represent 86% of products added to basket•24% of results for “rice” are in the “Pasta and Rice” category and represent 90%of purchases•8% of results for “sugar” are in the “Sugar and sweeteners” category andrepresent 90% of purchases

51 @M_I_C_E_S

2024-06-10#MICES2024@a2leanNLU and LLMs to automate actions → implementation
Step 1/

Step 2/

Monitor the clicks on search results

Construct conﬁguration API call

Identify long tails of less relevant resultsLaunch alert when necessary-Parameter values
Template

Result-

The code of an API call with the rightparameters

Launch the API call

52 @M_I_C_E_S

2024-06-10#MICES2024@a2leanScreenshot and API of ﬁltered redirection
PUT.a2_data_actions_redirections/_doc/r1{"keywords": "tomate,tomates","url": "/tomate","type": "recherche","active": true,"searchKey": "tomate","searchFilter": "product.category","searchFilterValues": ["Sauces, condiments","Légumes"],"id": "r1"}

53 @M_I_C_E_S

2024-06-10#MICES2024@a2leanThank you!
@M_I_C_E_S#MICES2024