Search Relevance
Organizational Maturity
Model

MICES 2019 Berlin | Eric Pugh | @dep4b

Search Relevance
Organizational Maturity
Model“The Missing Ingredient”
MICES 2019 Berlin | Eric Pugh | @dep4b

People are the
Magic Ingredient!

Sebastian leading the conversation atyesterday’s E-commerce Search for
Product Managers classMICES 2019 Berlin | Eric Pugh | @dep4b3

45

Business

UnderstandUser Needs

Search Tech

Experiment
DrivenAdvanced

Businessstakeholders
use real-time
KPIs

Producingquality data
fromanalytics

Develops
customplugins

Ops supports
A/B testing &offline tests

PractitionerOccasional
reportingSome user
testing /basicanalytics

Complexrelevance
config; uses
plugins

Baseline

Business
impact notmeasuredNo querylogs or user
testing

Stock ormoderately
tweak config

OSC’s Maturity Model for Search Teams

Content
Enrichment

Data
Inventory

Innovative
Discovery(chatbots,etc)

NLP & Data
science team

Varied,complex,large-scale
data

Available, but
complexexperiments

UI supports
findability

Taxonomies /Ontologies

Moderate
data
complexity

Test searchmanually,
deployedrarely10 searchlinks on page

Minorenrichment(synonyms)

Very simple
data model

UXMICES 2019 Berlin | Eric Pugh | @dep4b6

Business

UnderstandUser Needs

Search Tech

Experiment
DrivenAdvanced

Businessstakeholders
use real-time
KPIs

Producingquality data
fromanalytics

Develops
customplugins

Ops supports
A/B testing &offline tests

Practitioner

Occasional
reportingSome user
testing /basicanalytics

Complexrelevance
config; uses
plugins

Baseline

Business
impact notmeasuredNo querylogs or user
testing

Stock ormoderately
tweak config

OSC’s Maturity Model for Search Teams

ContentEnrichment

Data
Inventory

Innovative
Discovery(chatbots,etc)

NLP & Data
science team

Varied,complex,
large-scale
dataAvailable, butcomplex
experiments

UI supports
findability

Taxonomies /Ontologies

Moderate
datacomplexity

Test searchmanually,deployedrarely10 searchlinks on page

Minorenrichment(synonyms)

Very simple
data modelUX

MICES 2019 Berlin | Eric Pugh | @dep4b7

Business

UnderstandUser Needs

Search Tech

Experiment
DrivenAdvanced

Businessstakeholders
use real-time
KPIs

Producingquality datafrom
analytics

Develops
customplugins

Ops supports
A/B testing &offline tests

Practitioner

Occasional
reportingSome usertesting /basic
analyticsComplex
relevance
config; uses
plugins

Baseline

Business
impact notmeasuredNo querylogs or user
testing

Stock ormoderately
tweak config

OSC’s Maturity Model for Search Teams

ContentEnrichment

Data
Inventory

Innovative
Discovery(chatbots,etc)

NLP & Data
science team

Varied,complex,large-scale
data

Available, but
complexexperiments

UI supports
findability

Taxonomies /Ontologies

Moderate
datacomplexity

Test searchmanually,
deployedrarely10 searchlinks on page

Minorenrichment(synonyms)

Very simple
data modelUXMICES 2019 Berlin | Eric Pugh | @dep4b8

BusinessUnderstandUser Needs

Search Tech

Experiment
DrivenAdvanced

Businessstakeholders
use real-time
KPIs

Producingquality data
fromanalytics

Developscustom
plugins

Ops supports
A/B testing &offline tests

Practitioner

Occasional
reportingSome user
testing /basic
analytics

Complexrelevance
config; uses
plugins

Baseline

Business
impact notmeasuredNo querylogs or user
testing

Stock ormoderately
tweak config

OSC’s Maturity Model for Search Teams

Content
Enrichment

Data
Inventory

Innovative
Discovery(chatbots,etc)

NLP & Data
science team

Varied,complex,large-scale
data

Available, butcomplexexperiments

UI supports
findability

Taxonomies /Ontologies

Moderate
datacomplexity

Test searchmanually,
deployedrarely10 searchlinks on page

Minorenrichment(synonyms)

Very simple
data modelUX

MICES 2019 Berlin | Eric Pugh | @dep4b9

Maturity Model Laysout where you are,now do we have theteam to get there?
MICES 2019 Berlin | Eric Pugh | @dep4b10

Step 1: Brainstorming “The Work” of the Search TeamMICES 2019 Berlin | Eric Pugh | @dep4b11

Step 2: Cluster theTasks

MICES 2019 Berlin | Eric Pugh | @dep4b12

Step 2: Clustering Tasks Together

MICES 2019 Berlin | Eric Pugh | @dep4b13

Step 3: Naming the RolesMICES 2019 Berlin | Eric Pugh | @dep4b14

Trying to Learn to Rank before they canWalk!

MICES 2019 Berlin | Eric Pugh | @dep4b15

“App and Ops Focused” Team Starting toThink about Relevance

MICES 2019 Berlin | Eric Pugh | @dep4b16

“Search is a Team Sport”

MICES 2019 Berlin | Eric Pugh | @dep4b17

Thanks!
Lets talk:epugh@o19s.com
www.o19s.com@dep4bMICES 2019 Berlin | Eric Pugh | @dep4b18