nyuuzyou/wikis huggingface dataset:
Metrics Platform/Contextual attributes

Contextual attributes are fields in the event data that provide information about the performer who triggered the event and the wiki where the event occurred. The values of contextual attributes that are included in the stream configuration are populated automatically by Metrics Platform when the event is generated. This page documents the contextual attributes that are supported by each Metrics Platform client.
For attribute descriptions, types and other constraints (like their maximum length), see the base schema definitions.
== JavaScript ==
== PHP ==
== Java ==
== Swift ==
== Enabling attributes ==
All attributes must be enabled in the instrument's event stream configuration. To enable a contextual attribute, list the attribute name in the provide_values array. For more information, see the stream configuration guide.
For example, this stream configuration enables two contextual attributes: page_id and page_title.
== References ==
Metrics Platform/Contextual attributes/Java

This page documents the contextual attributes supported by the Metrics Platform Java client. For property descriptions and types, see the app base schema definition.
The Java client supports these contextual attributes:gitlab:repos/data-engineering/metrics-platform/-/blob/main/java/src/main/java/org/wikimedia/metrics_platform/context/ContextValue.java
* agent_app_flavor
* agent_app_install_id
* agent_app_theme
* agent_app_version
* agent_app_version_name
* agent_client_platform
* agent_client_platform_family
* agent_device_family
* agent_device_language
* agent_release_status
* mediawiki_database
* page_id
* page_title
* page_namespace_id
* page_namespace_name
* page_revision_id
* page_wikidata_qid
* page_content_language
* performer_id
* performer_name
* performer_is_logged_in
* performer_is_temp
* performer_session_id
* performer_pageview_id
* performer_groups
* performer_language_groups
* performer_language_primary
* performer_registration_dt
Metrics Platform/Contextual attributes/JavaScript

This page documents the contextual attributes supported by the Metrics Platform JavaScript client. For property descriptions and types, see the web base schema definition.
The JavaScript client supports these contextual attributes:gitlab:repos/data-engineering/metrics-platform/-/blob/main/js/src/ContextUtils.js
* agent_client_platform
* agent_client_platform_family
* page_id
* page_title
* page_namespace
* page_namespace_name
* page_revision_id
* page_wikidata_id
* page_wikidata_qid
* page_content_language
* page_is_redirect
* page_user_groups_allowed_to_move
* page_user_groups_allowed_to_edit
* mediawiki_skin
* mediawiki_version
* mediawiki_is_production
* mediawiki_is_debug_mode
* mediawiki_database
* mediawiki_site_content_language
* mediawiki_site_content_language_variant
* performer_is_logged_in
* performer_id
* performer_name
* performer_session_id
* performer_active_browsing_session_token
* performer_pageview_id
* performer_groups
* performer_is_bot
* performer_is_temp
* performer_language
* performer_language_variant
* performer_can_probably_edit_page
* performer_edit_count
* performer_edit_count_bucket
* performer_registration_dt
Metrics Platform/Contextual attributes/PHP

This page documents the contextual attributes supported by the Metrics Platform PHP client. For property descriptions and types, see the web base schema definition.
The PHP client supports these contextual attributes:gitlab:repos/data-engineering/metrics-platform/-/blob/main/php/src/StreamConfig/StreamConfig.php
* agent_client_platform
* agent_client_platform_family
* page_id
* page_title
* page_namespace
* page_namespace_name
* page_revision_id
* page_wikidata_id
* page_wikidata_qid
* page_content_language
* page_is_redirect
* page_user_groups_allowed_to_move
* page_user_groups_allowed_to_edit
* mediawiki_skin
* mediawiki_version
* mediawiki_is_production
* mediawiki_is_debug_mode
* mediawiki_database
* mediawiki_site_content_language
* mediawiki_site_content_language_variant
* performer_is_logged_in
* performer_id
* performer_name
* performer_groups
* performer_is_bot
* performer_is_temp
* performer_language
* performer_language_variant
* performer_can_probably_edit_page
* performer_edit_count
* performer_edit_count_bucket
* performer_registration_dt
Metrics Platform/Contextual attributes/Swift

This page documents the contextual attributes supported by the Metrics Platform Swift client. For property descriptions and types, see the app base schema definition.
The Swift client supports these contextual attributes:gitlab:repos/data-engineering/metrics-platform/-/blob/main/swift/Sources/WikimediaMetricsPlatform/StreamConfig/ContextAttribute.swift
* agent_app_install_id
* agent_client_platform
* agent_client_platform_family
* page_id
* page_title
* page_namespace
* page_namespace_name
* page_revision_id
* page_wikidata_id
* page_content_language
* page_is_redirect
* page_user_groups_allowed_to_edit
* page_user_groups_allowed_to_move
* mediawiki_skin
* mediawiki_version
* mediawiki_is_production
* mediawiki_is_debug_mode
* mediawiki_database
* mediawiki_site_content_language
* mediawiki_site_content_language_variant
* performer_is_logged_in
* performer_id
* performer_name
* performer_session_id
* performer_pageview_id
* performer_groups
* performer_is_bot
* performer_language
* performer_language_variant
* performer_can_probably_edit_page
* performer_edit_count
* performer_edit_count_bucket
* performer_registration_dt
Metrics Platform/Contribute

This page documents the process for contributing to Metrics Platform tools. The Metrics Platform libraries are open-source and hosted on GitLab at data-engineering/metrics-platform. Metrics Platform is owned by the Data Products team.
== Volunteer contributions welcome! ==
Contributions to the Metrics Platform client libraries are most welcome and appreciated.
Follow the steps below to get started, and feel free to ping any member of the WMF Data Products team (cjming, jennifer_ebe, mforns, phuedx, sfaci, wdoran, xcollazo) on for assistance.
== Making good merge requests ==
Ideally, merge requests from volunteers should be:
* Small and concise: Your merge request should fix a single specific issue, or implement a single specific feature. Be very clear about what your pull request does, and make sure the code is clean and readable.
* Testable: Whenever possible, include a unit test or two in your merge request.
Please avoid making these types of contributions:
* Complete overhauls of our code: A merge request that involves hundreds to thousands of lines of changes and touches many files is usually unfeasible because of the amount of reviewing, testing, and adjusting that we would need to do to accept it. Huge fundamental changes to our code are done gradually, and are decided by us internally as a team.
* Adopt shiny new library X throughout a particular client library: Similarly, adopting a new library (especially one that impacts our code in a fundamental way) is a decision that we make internally as a team. Please ask us first before undertaking such a task.
== Open bugs and feature requests ==
Work items are tracked on our work board in Phabricator. First time Metrics Platform contributors are encouraged to consider a good first task. Unassigned tasks in our backlog could be suitable for development and are roughly organized by descending priority though it is advised to reach out to one of the engineers or managers prior to taking on a given task.
Metrics Platform/Custom Data Monitor

This page documents the process for measuring metrics related to the adoption of Metrics Platform schemas.
== Quickstart ==
== Background ==
schemas/event/secondary stores the JSON Schema schemas for validating events submitted by analytics instrumentation. It also stores the Metrics Platform per-platform base schemas (herein "the MP base schemas") and the Legacy EventLogging base schema.
Data Products wants to track the number of high-level metrics about the schemas used for analytics instrumentation over time. Those high-level metrics are:
# The number of schemas
# The number of schemas that include the MP base schemas
# The average number of so-called "custom data" per schema
Here Data Products defines custom data as any property defined in the schema that isn't in either the MP base schemas or the Legacy EventLogging base schema.
Data Products runs the script and updates the spreadsheet roughly once per quarter. The work is tracked in Phabricator. Since the work is routine, they have a template task, which can be duplicated whenever it needs to be done. Clicking the button at the top of the page will create such a duplicate task.
== Links ==
* Custom Data Monitor Report spreadsheet
* https://gitlab.wikimedia.org/repos/data-engineering/custom-data-monitor
* T354965: [Epic] SDS 2.5 Establish baselines for Metrics Platform & Experimentation success indicators
* T356610: [SPIKE] Determine how to capture number of instruments developed total and via Metrics Platform over time
* T356610: Write a script to capture custom data properties counts in secondary schemas
Metrics Platform/Decision Records/Deprioritize Custom Data

Status: Accepted
Author: Sam Smith
Deciders: Sam Smith, Clare Ming, Marcel Forns, Morten Warncke-Wang
Consulted: Product Analytics
Informed:
Date authored: 2023-08-01
Date decided: 2023-08-01
Technical story: T343311: [EPIC] Create Core Interactions schemas/schema fragments
Keywords: monoschema, multischema, schemas, schema fragments, fragments, core interactions, interactions
Context and Problem Statement
Per T343311: [EPIC] Create Core Interactions schemas/schema fragments:The goals of eliminating schema creation during the creation of an analytics instrument and having a data contract between a data producer through to a Data Scientist are irreconcilable.
The first attempt at eliminating schema creation was to create the analytics/mediawiki/metrics_platform schema (AKA the monoschema). The monoschema contains many optional "Core Fields" – properties that are used relatively frequently in analytics instruments – and a custom_data property, which can contain arbitrary amounts of data that don't fit into those Core Fields. Using the monoschema in your instrument means that you don't have to create a new schema but also that there's no data contract between your instrument (the data producer) and the Data Scientist.Decision Drivers
Per the above, Data Engineering (represented by Marcel Forns) and Product Analytics (represented by Morten Warncke-Wang).
Considered Options
# Continue using the monoschema and onboard more feature teams to the client libraries
# Create a schema and/or a set of schema fragments that represent most interactions, while maintaining the existing monoschema and API in the client libraries
# Create a schema and/or a set of schema fragments that represent most interactions and immediately sunset the monoschema
Decision Outcome
We (see above) chose to explore option 2.
Positive Consequences
# The requirement from Data Engineering and Product Analytics that there is a contract between data producer and data consumer(s) is satisfied
# At least 50% of in-production instruments can be covered by a simple schema
# We can continue to use those parts of the monoschema that aren't the custom_data property
Negative Consequences
# The work that went into implementing the custom_data property is lost
# There is a loss of trust between the Metrics Platform team and the teams that they are already working with
Links
-
Additional Comments
-
Metrics Platform/Decommission an instrument

This page documents the process for decommissioning a Metrics Platform instrument.
== When to decommission an instrument ==
While instruments that measure product health may be long lived, instruments that capture metrics related to an experiment should be disabled once the experience is complete.
== Decommissioning an instrument ==
To disable an instrument:
# Remove the instrument code that calls a Metrics Platform client API.
# Remove the stream configuration. This will completely disable the instrument. 
# Remove the instrument from the instrument list.
== Schemas ==
At present, there is no need to remove schemas from the existing system in order to decommission an instrument. If you have a concern about an existing schema please don't hesitate to contact Data Products directly.
Metrics Platform/Feature availability

Note that, at this time, not all the Metrics Platform features are available for all the supported platforms (Javascript, PHP, Java and Swift).
The following table shows which features are available depending on the platform on which the instrument/experiment is built
 Feature availability
Feature 	Javascript 	PHP 	Java 	Swift
Client library 	 	 	 	
Analytics sampling 	 	 	 	
Custom sampling per wiki 	 	 	 	
Contextual attributes 	 	 	 	
Instrument class 	Metrics_Platform/JavaScript_API#Reference	 	 	
Experiment enrolment 	 	 	 	
Experiment enrolment overriding	 	 	 	
Base schema version 	web 1.3.0 	web 1.2.0	app 1.2.2	app 1.1.0
Metrics Platform/Interaction data

This page documents the interaction data supported by the Metrics Platform base schemas. For property descriptions, types and other constraints (like their maximum length), see the base schema definitions.
== Unmanaged data ==
Metrics Platform supports interaction data that you can customize to fit the needs of your instrument. You can set these properties to any meaningful value to provide the data required for your event. See the API docs for information about how to set these properties when submitting events.
;action (required)
:Examples: "click", "type", "scroll"
;action_subtype
:Examples: "submit" or "cancel" for click actions; "up" or "down" for scroll actions
;action_context
:A short (maximum of 64 characters) human-readable phrase to provide meaningful context for the event. Given the length limit, it is not recommend to stuff data via JSON-encoded strings into this field.
;action_source
:A short (maximum of 64 characters) human-readable set of words to indicate the source of the event. For example: "sidebar" or "visualeditor uploadwizard modal"
;element_friendly_name
:Analyst-friendly name of the UI element interacted with. For example: "Donate"
;element_id
:ID of the UI element interacted with
;funnel_entry_token
:A randomly generated identifier for linking multiple events belonging to a (work)flow. A single session (identified by performer.session_id may have multiple pageviews (identified by performer.pageview_id) which may have multiple instances of funnel_name that are separately identified by this token.
;funnel_name
:Analyst-friendly name such as "emergency-flow" or "Add a Link suggestion"
== Managed data ==
These properties are not manually set by your instrument but are instead managed by the client libraries (as long as you use the Instrument classMetrics_Platform/JavaScript_API#Reference).
;funnel_event_sequence_position
:0 for the first event submitted by instrument, incremented by 1 for each subsequent event
Metrics Platform/MPIC

Metrics Platform Instrument Configuration (MPIC) is an instrument and experiment management system. The application enables product teams to set up, manage, and configure various instruments for data collection and experimentation. This includes scheduling, designating metadata, setting sample rates, and targeting deployment environments. 
This project is still in the exploratory phase and active work is in progress towards an alpha release in the second half of 2024. The alpha release will include core functionalities and serve as a foundation for further development and refinement. 
We are targeting usage on this release by at least 1 product team. We are collaborating with the Growth team to configure their basic split A/B test for the Community Updates module. Our goal is to help product teams reduce their time to data collection by 50%. This work is helping to fulfill one of the Product + Technology departments Annual Plan Objectives: Signals & Data Services Objective 2 Product managers can quickly, easily, and confidently evaluate the impacts of product features.
== Base stream configuration ==
MPIC assigns instruments that use the base schemas to a dedicated base stream configuration. By default, these streams are configured with no additional contextual attributes.
; Base schema for web
: Event stream: product_metrics.web_baseSee product_metrics.web_base in ext-EventStreamConfig.php
; Base schema for apps
: Event stream: product_metrics.app_baseSee product_metrics.app_base in ext-EventStreamConfig.php
== References ==
Metrics Platform/MPIC/Administration

Metrics Platform Instrument Configuration (MPIC) is an instrument and experiment management system. The application enables product teams to set up, manage, and configure various instruments for data collection and experimentation. This includes scheduling, designating metadata, setting sample rates, and targeting deployment environments.
== Service details ==
== Deployment ==
Note that:
* mpic-next is the staging environment
* mpic is the production environment
# Do the following once you have prepared the change and the change has been reviewed and merged
ssh deployment.[eqiad|codfw].wmnet
cd /srv/deployment-charts
git log -n 1
# Check that the change has been pulled
cd helmfile/dse-k8s-services/<environment>
# Check changes and deploy
helmfile -e dse-k8s-eqiad diff
helmfile -e dse-k8s-eqiad -i apply
== Troubleshooting ==
Note that:
* mpic-next is the staging environment
* mpic is the production environment
* mpic-staging is the container name for staging environment (mpic-next)
* mpic-production is the container name for production environment (mpic)
=== Check pods ===
$ ssh deployment.[eqiad|codfw].wmnet
$ kube_env <environment> dse-k8s-eqiad
$ kubectl get pods -w
NAME READY STATUS RESTARTS AGE
mpic-staging-57d4bbb6bf-z2p5k 1/2 Error 0 10s
mpic-staging-57d4bbb6bf-z2p5k 1/2 Error 1 (1s ago) 10s
mpic-staging-57d4bbb6bf-z2p5k 1/2 CrashLoopBackOff 1 (1s ago) 11s
=== See logs ===
$ ssh deployment.[eqiad|codfw].wmnet
$ kube_env <environment> dse-k8s-eqiad
$ kubectl get pods -w
$ kubectl logs -f <pod_name> -c <container_name> --tail=10
{"level":50,"time":1714483575453,"pid":1,"hostname":"mpic-staging-66554859c4-p9m28","msg":"Error while setting an env variable. There is no value for SAL_PASSWORD"}
{"level":30,"time":1714483575592,"pid":1,"hostname":"mpic-staging-66554859c4-p9m28","msg":"Connected to the database at an-mariadb1001.eqiad.wmnet:mpic_staging"}
{"level":30,"time":1714483575600,"pid":1,"hostname":"mpic-staging-66554859c4-p9m28","msg":"mpic has started listening on port 8080"}
Metrics Platform/Measure product health

Product health metrics are key performance indicators for measuring and monitoring the impact of a product.mw:Product_Analytics/Glossary#Product_health_metric This page describes how to measure product health using the Metrics Platform.
== Plan ==
=== Measurement plan ===
=== Instrumentation spec ===
 For more information about designing an instrumentation spec, see the instrument guide.
=== Data collection guidelines ===
== Code ==
== Launch ==
Once your instrument code has been deployed to production, complete the launch-your-baseline-instrument form to configure your instrument and start collecting data.
== Monitor ==
You can use the experimentation directory to monitor the progress of your instrument. Your event data will be available in a Hive table with the same name as the event stream, with dots and dashes replaced with underscores.
== Manage ==
The experimentation directory provides options to edit and turn off your instrument using the Actions menu.
== References ==
Metrics Platform/Overriding experiment enrollment

For QA purposes, there are two ways to override the experiment enrollment assigned by the bucketing algorithm that MetricsPlatform extension runs. That way you can assign any specific value to any variant for the current user.
Note that this functionality is disabled by default. To enable it you will have to set $wgMetricsPlatformEnableExperimentOverrides to true.
=== Using a Javascript function ===
You can set any value for a given variant using the mw.metricsPlatform.overrideExperimentEnrollment function. 
For example, if you have an experiment called test-mediawiki-article-toc-exp with a homepage_module variant, you can set to true the value for that variant as follows:
mw.metricsPlatform.overrideExperimentEnrollment('test-mediawiki-article-toc-exp', 'homepage_module', true);
To revert all changes done by the previous function in one or several variants, after your QA process, you can just run:
mw.metricsPlatform.clearExperimentEnrollmentOverrides()
=== Passing a query parameter ===
You can also pass a query parameter in the URL to set any specific value for a variant of a given experiment.
The name of the query parameter must be mpo and the format to override a variant using this way is mpo=name_of_the_experiment:name_of_the_variant:the_value. For example, if we want to set to true the homepage_module variant of our test-mediawiki-article-toc-exp experiment when visiting https://es.wikipedia.org/wiki/Bicicleta, we must go to the following URL:
https://es.wikipedia.org/wiki/Bicicleta?mpo=test-mediawiki-article-toc-exp:homepage_module:true
To undo the overriden value, just visit the wiki without passing any mpo query parameter and the values assigned to any variant will be the ones that the MetricsPlatform extension has assigned while running the bucketing algorithm as usual.
Metrics Platform/Sampling

Sampling in Metrics Platform may refer to:
== Analytics sampling ==
Analytics sampling is the logic that determines whether an event is included in data collection for an event stream, including the sampling unit and rate. See Analytics sampling.
== Experiment enrolment sampling ==
Experiment enrolment sampling is the act of enrolling users into experiments (also called an A/B tests) and consistently assigning an enrolled user a variant of the feature that is being experimented on. Experiment enrolment sampling is also known as bucketing. See Experiment enrolment sampling.
Category:Disambiguation pages
Metrics Platform/Schemas

Schemas in Metrics Platform may refer to:
== Base schemas ==
Metrics Platform provides base schemas that are used to validate event data. These schemas are designed to fit the needs of most instruments:
* Base schema for web
* Base schema for mobile apps
== Custom schemas for specific product areas ==
Some product areas require event data fields that are not supported by the base schemas. These schemas are designed to be reused by multiple instruments in a given product area.
* Translation schema for web
== New custom schemas ==
In rare cases, your instrument may require data that is not supported by any of the schemas listed above. Learn more about creating a custom schema.
Category:Disambiguation pages
Microcode

All our servers (with the exceptions of some very old ones which are not supported by Intel anymore) are expected to use mitigations for the following three Intel CPU vulnerability classes requiring microcode updates (some issues were fixed without needing additional microcode):
* SSBD (cpu flag 'ssbd')
* MDS/TAA (cpu flag 'md_clear')
* L1TF (cpu flag 'flush_l1d')
There are two potential sources of error which might make microcode loading fail:
* The server might have been freshly installed, but wasn't finally rebooted (maybe the reimage script, which does that by default, didn't complete or wasn't used). This may lead to a situation where Puppet pulled in the microcode package, but it only gets effective with the next reboot. Simply reboot and see if it recovers.
* If a reboot doesn't help, have a look at dmesg. We had a few cases where servers were upgraded from Linux 4.9 to 4.19 and the new kernel refused to load microcode due to some outdated system firmware (T235250), so the firmware/BIOS needs to be updated.