nyuuzyou/wikis huggingface dataset:
Wikimediastatus.net/User instructions

Normally, the wikimediastatus.net page displays "All Systems Operational".
If something is known to be wrong, then instead there will be a brief description of the issue and who it affects (for example: "Wikipedia and other sites inaccessible for users especially in North America").
Updates are posted manually and will not be instant: they are written by the same engineers who are working on troubleshooting and resolving the problem.
The two high-level components on the page, "Reading" and "Editing", also provide information about who is affected: if there is an outage declared for only "Editing", then users who are logged out and only reading popular articles are unlikely to be affected.
== Metrics ==
A key feature of the new status page is a set of five high-level metrics that give a look into the overall health and performance of the wiki environment. We chose a set of indicators that would show widespread issues as obvious deviations from normal, so that non-technical people could look at the graphs and say “ah, yes, something is wrong”. Automatically publishing these metrics means that users can have some idea that something is wrong for everyone, not just themselves, even before our engineers have a chance to post an update.
Each metric is described below.
=== Total request volume ===
The number of HTTP requests per second our content delivery network is servicing. A normal range for this metric is between 100,000 to 200,000, depending on the time of day. Either sharp increases or decreases from the usual value may indicate an issue (but also may not).
=== User-reported connectivity errors ===
The number of Network Error Logging reports we're receiving of connectivity trouble from the web browsers of users. A value of under 1 report/second generally does not indicate any trouble; higher numbers or sustained values above 2 may indicate disruptions with either Wikimedia's network connectivity or issues on the Internet at large.
=== Wiki error responses ===
Out of the total incoming requests, the amount per second that produced error responses. Error responses have many causes, from invalid user input to known bugs; the presence of errors does not necessarily indicate any issue. Up to a few hundred per second is not necessarily a cause for concern.
=== Wiki response time ===
The average duration (in seconds) that our application servers took to reply to uncached requests. Note that most users receive much faster responses due to caching in our content delivery network. Values of 5 seconds or less are normal.
=== Successful edits ===
The number of edits made per second. Includes both human and bot edits. Values below 5 per second may indicate performance issues.
Wikitech

You are probably reading Wikitech right now. 
== What ==
The MediaWiki install is updated automatically as part of standard WMF deployment. 
== Test version ==
Wikitech has a test/staging version, the unfortunately named labtestwikitech.wikimedia.org with DB name labtestwiki (note the DB names are labswiki vs. labtestwiki).
== History ==
The current Wikitech is a merger of two original wikis:
* wikitech.wikimedia.org, wiki for internal technical documentation for the Wikimedia Foundation between 2004 and 2013. The wiki no longer existed and many (though not all pages) were imported to the second wiki.
* labsconsole.wikimedia.org, wiki that used OpenStackManager to served as a "console" to manage machines in Wikimedia Labs (now called Cloud VPS) since 2012; most of functions of OpenStackManager, historically provided by special pages on this wiki, are now moved to Horizon. In 2013, the wiki is moved to wikitech.wikimedia.org with many pages of old Wikitech imported here.
Before 2024 Wikitech was also used for developer account management. That functionality has been migrated to Bitu.
Category:Services
Wikitech documentation templates

This is a list of templates that exist on Wikitech and which may be useful for technical documentation. (Compare with mw:Documentation/Style guide/templates.)
==Codesample==
{{Codesample|name=metadata|lang=yaml|scheme=light|line=1|start=5|highlight=1,4|code=
 name: foo-redirect
 namespace: tool-foo
 labels:
 name: foo-redirect
 annotations:
 kubernetes.io/ingress.class: nginx
 nginx.ingress.kubernetes.io/permanent-redirect: https://bar.toolforge.org/$1$is_args$args
}}
==Collapse top and Collapse bottom==
 and 
{{Collapse top|Title}}
===Collapsed content===
The collapsed content can include multiple paragraphs.
{{Notice|And even include other templates}}
{{Collapse bottom}}
===Collapsed content===
The collapsed content can include multiple paragraphs.
==Caution==
{{Caution|Here is my note.}}
==Doing, Not Done, Done==
{{Doing}}
{{Not done}}
{{Done}}
== Draft ==
{{Draft}}
==Fixme==
{{Fixme|foo}}
==Fixme inline==
 {{Fixme inline|foo}}
 {{Fixme inline|where}}
 {{Fixme inline|when}}
== Hidden ==
Expand and collapse section contents with a stylized header.
{{Hidden|Title|Hidden content that will be shown after clicking Expand}}
== Historical ==
 aka. 
{{Historical}}
{{Historical|year=2005}}
== Just a note ==
{{Justanote|foo}}
==Merge==
{{Merge|foo}}
==Navigation sidebar==
 makes it easier for you to create navigation templates for a set of pages. For examples that use the template, see:
* 
* 
==Note==
{{Note|text=Foo}}
{{Note|type=info|text=Foo}}
{{Note|type=reminder|text=Foo}}
{{Note|type=reminder|text=Multiplelinesoftext}}
{{Note|type=warn|text=Foo}}
{{Note|type=error|text=Foo}}
{{Note}} Loose text
* Inline note {{Note|inline=1|text=Foo}}
 Loose text
* Inline note 
==Notice==
{{Notice|foo bar bang}}
== Outdated ==
==Remark==
This is a wrapper around {{Note}}. Using that template instead is recommended.
==See also==
{{See also|test|sandbox|w:foo}}
== Staging ==
{{Staging}}
== Status ==
{{status}} 	→	
{{status|done}} 	→	
{{status|on hold}} 	→	
{{status|not done}}	→	
:See template page for more color options, and wording change options.
==Warn==
{{Warn|Here is my note.}}
==Warning==
Alias for {{Warn}}
:Different version than on mediawikiwiki. Just uses ambox here.
== Linking to Gerrit, Phabricator, etc ==
=== Link to Phabricator ===
 
{{tracked|T327319}} will display a "Tracked in Phabricator" box like the one on the right.
You can also use [[phab:T327319]] without using a template, it will display like this: phab:T327319
=== Link to a repository ===
{{git repo|wikimedia/developer-portal}}
=== Link to a change ===
{{Gerrit|4203}}
=== Link to a file ===
{{Gitweb|project=mediawiki/extensions/examples|file=extension.json}}
Wlm.wikimedia.org

wlm.wikimedia.org (server name: yttrium) hosts the API for the Wiki Loves Monuments mobile application. It's a combined Apache/MySQL server. The only point of interest ATM is http://wlm.wikimedia.org/api/api.php
== Puppet ==
manifests/misc/wlm.pp
== Sources ==
wikimedia/wlm-api in Gerrit.
== MySQL ==
Passwords for maintenance are at /var/wlm/passwords.txt. The database is called wlm.
Category:Services
Wmf-laptop

wmf-laptop is a Debian package which simplifies SSH access to Wikimedia production servers:
* It ships README files and example configs for the initial setup of git/Gerrit and SSH with separate agents for production and Cloud VPS
* It includes wmf-update-known-hosts-production, a tool which syncs SSH known_host data for production
* it includes tunnelencabulator (https://github.com/cdanis/tunnelencabulator)
* It includes pwstore (https://wikitech.wikimedia.org/wiki/Pwstore, only relevant for Wikimedia SRE members)
* It also pulls in packages that you'll need as dependencies. 
wmf-laptop can be used on any Debian version (and derived distros).
It's available on our internal repository. The source is in the operations/debs/wmf-sre-laptop repository. The package will generally work on all Debian-based distros (so also Ubuntu or Mint e.g.).
* First of all you need to install the repository key for apt.wikimedia.org, it's documented here: APT repository#Security
* Then create a file under /etc/apt/sources.list.d/ called something like wikimedia.list containing the following:
 deb http://apt.wikimedia.org/wikimedia bookworm-wikimedia component/wmf-laptop
* Then apt install wmf-laptop it will pull in a few packages you'll need as well.
* /usr/share/doc/wmf-laptop/README contains documentation on how to setup git, SSH and pwstore access (the latter only relevant for Wikimedia SRE staff).
# quick install steps
sudo wget -O /etc/apt/trusted.gpg.d/wikimedia-apt-key.asc "https://wikitech.wikimedia.org/w/index.php?title=APT_repository/Stretch-Key&action=raw"
echo "deb http://apt.wikimedia.org/wikimedia bookworm-wikimedia component/wmf-laptop" | sudo tee /etc/apt/sources.list.d/wmf-laptop.list
sudo apt update
sudo apt install wmf-laptop
X-Analytics

X-Analytics is an HTTP header used for measurement purposes, including in cache log format and the webrequest data stream.
Generally, values are added to the header on the server side; the only keys accepted from the client side are preview and pageview.
== Format ==
The X-Analytics header is formatted as a list of key=value pairs separated by semicolons, like mf-m=b or zero=123-45;mf-m=b. If a key occurs more than once, it is undefined which one takes precedence.
The special value - must be interpreted as the empty string.
== Keys ==
Looks like X-Analytics extension might be adding these values if used: 
Key 	Value 	Origin 	Since	Until 	Team 	Contact	Use case
mf-m 	b, amc, b,amc, or not set 	appserver	? 	Current	Readers Web 	Phuedx 	If set, then the value b indicates that the user is opted into the beta mode (of the mobile site) (mf-m=b), the value amc indicates Advanced Mobile Contributions (mf-m=amc), and b,amc indicates both (mf-m=b,amc). See MobileContext.php.
proxy 	Proxy name, e.g. Opera 	varnish 	? 	Current	Wikipedia Zero	Yurik 	If set, indicates that this request has been received via one of the trusted proxies such as Opera Mini servers.
Currently, the following proxies can be expected:
{| class="wikitable"
Value 	Description 	 	 	 	 	 	
Opera 	Opera mini proxy 	 	 	 	 	 	
Nokiaprod	Nokia Xpress Production Proxy Servers 	 	 	 	 	 	
Nokiaqa 	Nokia Xpress QA Lab Proxy Servers 	 	 	 	 	 	
IORG 	Internet.org (set by analytics.inc.vcl in puppet)	 	 	 	 	 	
=== Former keys ===
Key 	Value 	Origin 	Since 	Until 	Team 	Contact	Use case
php 	zend, or hhvm 	appserver 	? 	Jan 2015 	SRE 	_joe_ 	If set, marks the used PHP implementation.
This tag was only set between September 2014 and January 2015 during the migration from Zend to HHVM. (See , and )
zero 	MCC-MNC of a zero carrier, e.g. 404‑01.	varnish 	? 	July 2019 	Wikipedia Zero	Yurik 	If set, indicates that this request has been associated with the given carrier. It does not mean that the request qualifies as page view.
* zero.inc.vcl.erb.
* mobile-frontend.inc.vcl.erb.
Removal in T213769.
zeronet 	Subdivision of a carrier, e.g. b 	varnish 	? 	July 2019	Wikipedia Zero	Yurik 	Used of disambiguate between different parts/configurations of a single carrier. Like broadband vs. special access points.
Removal in T213769.
max-snippet	1, 0, or not set 	appserver (WikimediaEvents)	Mar 2022	Current 	Readers Web 	cjming 	If set, the value 1 indicates the page's robots meta tag contains the max-snippet directive. The value 0 indicates the page's robots meta tag does not contain the max-snippet directive. If set, both 1 and 0 indicate that the page is part of an A/B test in the treatment and control groups respectively. Added in T301584.
Removed in I65ce99b04acc as part of T310267.
== Code ==
https://github.com/wikimedia/mediawiki-extensions-XAnalytics
== Extension documentation ==
mw:Extension:XAnalytics
== Tasks and Bugs ==
https://phabricator.wikimedia.org/tag/xanalytics/
== See also ==
* The wprov URL parameter documented under Provenance is likewise logged in the x_analytics_map field of the webrequest table.
XHGui

XHGui is a web application providing a graphical interface for PHP profiling data. It displays the call tree for a profiled web request with aggregations of memory usage, call count, and approximate time duration for all called functions.
The service is open for anyone to use via the WikimediaDebug browser extension, which captures data from a mwdebug server using php-tideways (formerly known as XHProf), and submits it from there to the XHGui database. To learn how to use XHGui, refer to WikimediaDebug#XHGui profiling.
Note that we strip most request metadata for security and privacy reasons. XHGui profiles do not record cookies, client IPs, post data (e.g. HTML form submissions), or query parameters (except "action").
== Runbook ==
See XHGui/Runbook.
== External link ==
* https://performance.wikimedia.org/xhgui/
* Upstream perftools/xhgui on GitHub
Category:Services
Category:SRE Observability
XHGui/Runbook

This is the internal runbook for XHGui service.
== Hosts ==
Hosts as of July 2023:
* webperf
* deployment-webperf21 (Beta cluster)
Provisioned by the Puppet role::webperf which installs the XHGui web app at https://performance.wikimedia.org/xhgui/ from the operations/software/xhgui repository (this is a mirror of the upstream perftools/xhgui repo).
Public web access is provided via an Apache proxy from the webperf host that serves performance.wikimedia.org.
==Meta==
*Public URL:
** Production: https://performance.wikimedia.org/xhgui/
** Beta cluster: https://performance.wikimedia.beta.wmflabs.org/xhgui/
*User documentation: XHGui
*Database: m2
== Upgrade XHGui ==
You will need:
*a local checkout of operations/software/xhgui.git, which is our mirror.
**including a remote for upstream. git remote add upstream git@github.com:perftools/xhgui.git
**including submodules checked out. git submodule update --init --recursive
*a local checkout of performance/docroot.git, which stores the https://performance.wikimedia.org/ website
**including submodules checked out. git submodule update --init --recursive
*commit access to operations/software/xhgui.git and performance/* repositories (granted to Gerrit group "performance", per access).
=== Update our mirror ===
# Check out our mirror.
# Merge in the new release tag from upstream.
#Push to Gerrit (fast-forward only).
 $ cd xhgui/
 xhgui$ git checkout wmf_deploy && git remote update && git reset --hard origin/wmf_deploy
 xhgui$ git merge 0.12.0
 xhgui$ git push origin HEAD:wmf_deploy
=== Update dependencies ===
To update our dependencies in performance/xhgui-vendor, follow steps in README.
Then, in your local xhgui checkout:
#Fast-forward the vendor submodule.
#Commit the submodule update.
#Push to Gerrit (fast-forward only).
 $ cd xhgui/
 
 xhgui$ cd vendor/
 xhgui/vendor$ git pull
 xhgui/vendor$ cd ..
 
 xhgui$ git add vendor && git commit
 xhgui$ git push origin HEAD:wmf_deploy
=== Upgrade our install ===
To deploy an XHGui upgrade, update the submodule pointer in performance/docroot.git
# Fast-forward the xhgui submodule.
# Commit the submodule update.
#Push to Gerrit for review.
 $ cd performance.wikimedia.org
 performance.wikimedia.org$ cd xhgui/
 performance.wikimedia.org/xhgui$ git pull
 performance.wikimedia.org/xhgui$ cd ..
 
 performance.wikimedia.org$ git add vendor && git commit
 performance.wikimedia.org$ git review
Test on Beta Cluster:
# ssh deployment-webperf21.deployment-prep.eqiad1.wikimedia.cloud
# Copy the "Download > Anonymous HTTP > Cherry pick" command from Gerrit for your docroot patch and run it as www-data on the beta server:
# cd /srv/org/wikimedia/performance$ sudo -u www-data sh -c 'git fetch … && git cherry-pick FETCH_HEAD'
Deploy to production:
Once the performance/docroot patch is reviewed and merged, Puppet will automatically apply it to production webperf* hosts within 30 minutes.
XHGui
Category:SRE Observability
XKey

The xkey Varnish module adds secondary hashes to objects, allowing fast purges of objects with the same key.
=== Setting Object Keys ===
Keys should preferably be set by the application in the xkey response
header. Multiple keys can be specified per header line with a space separator:
 HTTP/1.1 OK
 Server: Apache/2.2.15
 xkey: 8155054 166412 234323
or in multiple xkey response headers:
 HTTP/1.1 OK
 Server: Apache/2.2.15
 xkey: 8155054
 xkey: 166412
 xkey: 234323k
Alternatively, keys can also be set in VCL:
 sub vcl_backend_response {
 set beresp.http.xkey = "8155054 166412 234323";
 }
=== Scaling ===
We currently do not support setting the same XKey on very large numbers of objects. In practice something on the order of 1-100 objects attached to a given XKey is reasonable. It is not reasonable to use XKeys which would be relevant to hundreds of objects (or more), such as using XKeys to indicate broad references from templates, transclusions, backlinks, etc. Primarily we plan to have XKeys indicating unique objects such as wiki articles or image filenames, and shared between e.g. all of their direct representations (article: desktop HTML, mobile HTML, RESTBase HTML; various thumbnailed sizes of an image, etc) and metadata (?action=history of an article, etc).
=== Namespacing and Encoding ===
Because (slight) over-purging on namespacing and encoding collisions aren't really a pragmatic issue, there's no current plan to define a rigorous meta-schema like /v1/mw/enwiki/title/Foo. For wiki articles, the XKey should be the wiki's official database name prefixed and suffixed with forward slashes (e.g. /enwiki/), and then the article title. For images and thumbnails, the XKey should be the original image pathname on the upload servers. There is no special prefix when sent as an `XKey:` header in an HTTP response. When used for purging, XKey values should be prefixed with xkey: to differentiate them from raw https URL purges.
XKey values should be percent-encoded according to normal HTTP URI rules. XKey values should not encode characters in the Unreserved Set, may encode characters in the Reserved Set, and must encode all other characters. This encoding will be normalized on reception for both XKey header output and PURGE requests as best as possible, but not all application encoding errors can be corrected (especially under- or over- encoding of literal %). It would be best to ensure the application enforces sane encoding to avoid issues.
=== Examples ===
==== MediaWiki article ====
* Typical Request URI: https://en.wikipedia.org/wiki/Barack_Obama
* XKey: /enwiki/Barack_Obama
* PURGE: xkey:/enwiki/Barack_Obama
==== cache_upload images ====
* Typical Request URI: https://upload.wikimedia.org/wikipedia/en/thumb/b/bd/Civic_Arena_8.jpg/220px-Civic_Arena_8.jpg
* XKey: /wikipedia/en/b/bd/Civic_Arena_8.jpg
* PURGE: xkey:/wikipedia/en/b/bd/Civic_Arena_8.jpg
Category:Caching
Yongle

Yongle is a 3GB apache server. It's had hardware problems before, iirc. 
Solar: I get a "losing some ticks" error, and everyting seems buggy. Must be hardware, RMA'd.
Solar: Solution= notsc as a kernel argument according to SM.
2007-02-21 brion: was idle, tried to take over to replace downed isidore for experimental services. Died when rebooting it with its new IP, 66.230.200.66, so probably not configured right. Either fix config or replace with a fresh Ubuntu install if machine is still working.
* Services: search.wikimedia.org
** temporarily holding mobile.wikipedia.org due to anthony failure 2008-07-30
Yubikey4 and gpg-agent

Using the Yubikey4 for ssh key generation and storage as well as one-time-password generation. There are a ton of guides to a ton of subtly different methods of doing basically this, I mean only to provide a simple explanation of what worked for me.
== Use gpg-agent instead of ssh-agent ==
Disable ssh-agent if it is running. Use the script provided, which is essentially keychain but for gpg-agent. I put it in .bashrc rather than .xsession:
 if test -f $XDG_RUNTIME_DIR/gpg-agent-info && kill -0 $(head -n 1 $XDG_RUNTIME_DIR/gpg-agent-info | cut -d: -f2) 2>/dev/null ; then
 eval $(< $XDG_RUNTIME_DIR/gpg-agent-info)
 else
 eval $(gpg-agent --daemon --enable-ssh-support --write-env-file $XDG_RUNTIME_DIR/gpg-agent-info)
 fi
 export GPG_AGENT_INFO
 export SSH_AUTH_SOCK
== Generate an authentication key ==
I used the method described in this blog post to generate the authentication subkey on the Yubikey itself:
 $ gpg2 --edit-key YOURKEY
 gpg> addcardkey
Select authentication, provide expiry, create the key, and save. gpg2 -K should show the new key, and gpgkey2ssh AUTHKEY should provide a version for suitable authorized_keys. First access of the key will present a box for the PIN. After that PIN-less access and OTP generation should work until it is unplugged. I did not have to install any smart card related utilities for this to work as expected.
I found that despite being generated on the Yubikey the authentication private key remained on my local keychain. Removing it did not affect ssh behavior.
Category:How-To
ZTP Network Device Provision

=ZTP Provisioning=
Juniper's Zero-Touch Provisioning can be leveraged to configure new network devices without having to manually configure them. A ZTP cookbook has been created by DC-Ops and Infra-Foundations which can be used to upgrade device software to our preferred version, and push the minimum base config for network connectivity and user auth.
The cookbook currently supports QFX5120 series devices, but with a little effort this can be expanded to other platforms.
==ZTP Provisioning Steps==
* Add to Netbox (dc-ops)
** Include serial number and asset tag
** Set Netbox status to 'planned'
** Ensure there is only one interface 'enabled' in Netbox set to 'Management Only'
*** For QFX5120 this normally means disabling em1
* Connect power/console/mgmt
* Update console server's port (& test it)
* Run the ZTP cookbook for the device:
** e.g.: sudo cookbook sre.network.provision ssw1-a1-codfw
* Run the TLS certificate cookbook for (this step will soon be merged into the provision cookbook)
** e.g.: sudo cookbook sre.network.tls ssw1-a1-codfw
===Notes===
* The cookbook currently takes approx 1 hour to complete for a device. This is due to the time it takes to download the new JunOS image file, install it locally on the switch and reboot.
* Multiple cookbooks can be executed in parallel to configure more than one device. But step 1 of the cookbook invokes the sre.dns.netbox script, which we need to avoid running simultaneously as it is very intensive on the Netbox server. So the execution of the provision cookbooks for multiple devices should be staggered, wait until the first cookbook is at the "waiting for device" stage before starting the second.
==Remaining Configuration==
At this point the device should be reachable on our management network and we should be able to configure with Homer.
The remaining steps, depending on the use-case, are as follows:
* Add to Homer
** For L3 switches this requires the "l3_switch" var to be set in devices.yaml
** EVPN devices additionally need the 'evpn' var set, an IRB interface defined in Netbox and device added to BGP cluster in sites.yaml
* Create license key using activation-code on Juniper's portal, and add as a new 'license' inventory item for the device in Netbox
* Push initial configuration with Homer
** e.g.: homer ssw1-a1-codfw* commit "apply initial configuration"
* Save rescue configuration locally on the device
** request system configuration rescue save
* Set Netbox status to 'active'
* Run the sre.puppet.sync-netbox-hiera cookbook to import location to hiera
==Monitoring==
Add device to monitoring using Setup a new network device#Monitoring.
Category:Juniper
==Software Image==
Right now only QFX5120 devices are supported, and the DHCP script just pushes the image file stored in /srv/private/junos/qfx5120.tgz to the device. To adjust what version of JunOS the cookbook will install you can download the required version to the local apt server in a DC and rename it to that.