Search Platform/Goals/OKR 2022-2023 Q3

In Q3, the Search Platform team will continue to primarily focus on fixing the Search Update Pipeline, to address architectural limitations that have been causing failures in edits being accurately reflected in search indexes for users.
We will be continuing the work to unpack language analyzers for emerging languages, to work towards making our content more discoverable for people who speak underserved languages.
We will address the Spark 3 migration as driven by the Data Engineering team.
While the technical work on the WDQS SLO should be complete in Q2, we will need to communicate it to our communities, with potentially some follow up to manage expectations and misunderstandings.
= Q3 OKRs =
== Objective 1: Users can reliably query information on Wikidata, and search on wikis ==
* KR: WDQS uptime and update lag >= 95%
* KR: Search Update pipeline errors TBD
* KR: Communicate WDQS strategy to community
== Objective 2: Searchers of emerging languages can search in their own language ==
KR 2.1: Unpack 3 language analyzers from emerging languages
== Objective 3: Paying down technical debt in infrastructure ==
KR 3.1: All of the Search Platform Spark jobs are running on Spark 3
= Project plan (Q3)= 
# Search Update Pipeline
# Communication of WDQS uptime SLO
# Spark 3 migration
# Unpack language analyzers
= Things we ship =
== Fix update pipeline (2+ quarters of work) ==
Higher level Objective: Users can reliably query and search for content, so that they can depend on Wikimedia Foundation projects as knowledge infrastructure (ERF: Technical Infrastructure, Product Platform)
KR1: End to end Search Update Pipeline for simple edits on one wiki
Description: The search update pipeline is currently broken, resulting in updates being sporadically lost often enough that users are reporting bugs. The Sanetizer is supposed to resolve this, but was shut off for a while, and restarted, but is running with ~4 weeks of lag. We want to resolve issues in the update pipeline so that our indexes are not out of sync by more than XX days
Phab: 
Milestones (note that there are no dependencies between those milestones):
* SLI and SLO are created for the Search update lag
* Search Update Pipeline works with all edit types and optimizations (re-renders, deletes, redirects, event re-ordering)
* Search Update Pipeline supports Weighted Tags
* SLI and SLO created for Search update lag and error rates
* Search Update pipeline is deployed on k8s with Flink operators
== WDQS uptime SLO ==
Higher level Objective: Users can reliably query information on Wikidata and search on wikis
KR: Community is updated on expectations for WDQS maintenance-only mode
Description: Due to the WDQS backend scaling pause, the Search team will be moving to maintenance only mode for WDQS. In addition to the update lag SLO, we will create an uptime SLO for WDQS. This will allow us to set expectations for users for service availability, while also setting expectations for the Search team to know how much time to spend on working on WDQS. The community will need to be updated with the current state of WDQS scaling.
Phab: 
Milestones:
* Community comms for WDQS (and WCQS)
* Communication and documentation for the larger SRE teams
== Spark 3 migration ==
TBD: checking with Data Engineering team if they have an OKR to which we can attach.
Higher level Objective: Paying down technical debt in infrastructure
KR1: All of the Search Platform Spark jobs are running on Spark 3
Phab: 
Milestones:
* Convert python dependencies/environments to conda
* Convert all spark jobs to Spark 3
* Specific jobs are tracked in List of Spark2 jobs running in Hadoop
* Affected teams are messaged
* Outputs are validated to be same (within margin of error)
== Unpack language analyzers ==
Higher level Objective: Searchers of languages can search for WMF in their preferred language
KR1: Unpack all language analyzers: Japanese (spillover), Armenian, Latvian, Hungarian
Description: Enable users of non mainstream languages to search for WMF content in their preferred languages. Reduce technical debt of having monolithic language analyzers, which make search efficacy opaque for those languages, and prevents general multilingual improvements to be deployed.
Phab:
* 
* 
*