Incidents/2018-12-18 train-1.33.0-wmf.9

== Summary ==
There were several problems with the 2018-12-18 1.33.0-wmf.9 train (phab:T206663) resulting in 0 rollbacks.
=== Stats ===
* changes: 314
* repos: 99
* committers: 65
* finished on-time: yes
* rollbacks: 0
* issues: 4
* :mw:MediaWiki_1.33/wmf.9/Changelog
=== Issues ===
* phab:T211885 ErrorException from line 47 of /srv/mediawiki/php-1.33.0-wmf.8/extensions/Kartographer/includes/ApiQueryMapData.php: PHP Warning: data error
* phab:T212217 ErrorException from line 317 of /srv/mediawiki/php-1.33.0-wmf.9/extensions/ExtensionDistributor/includes/specials/SpecialBaseDistributor.php: PHP Notice: Undefined variable: downloadImg
* phab:T212218 Fatal error: Call to undefined function Kartographer\Wikimedia\suppressWarnings() in /srv/mediawiki/php-1.33.0-wmf.9/extensions/Kartographer/includes/ApiQueryMapData.php on line 49
* phab:T212427 No namespace configured for entity type `form`
== Timeline ==
This is a step by step outline of what happened to cause the incident and how it was remedied.
=== 2018-12-13 ===
* 14:46 phab:T211885 ErrorException from line 47 of /srv/mediawiki/php-1.33.0-wmf.8/extensions/Kartographer/includes/ApiQueryMapData.php: PHP Warning: data error
=== 2018-12-18 ===
* 14:40 <zfilipin@deploy1001> rebuilt and synchronized wikiversions files: group0 to 1.33.0-wmf.9
* 14:47 phab:T212217 ErrorException from line 317 of /srv/mediawiki/php-1.33.0-wmf.9/extensions/ExtensionDistributor/includes/specials/SpecialBaseDistributor.php: PHP Notice: Undefined variable: downloadImg
* 15:01 phab:T212218 Fatal error: Call to undefined function Kartographer\Wikimedia\suppressWarnings() in /srv/mediawiki/php-1.33.0-wmf.9/extensions/Kartographer/includes/ApiQueryMapData.php on line 49
==== End of Day Summary ====
* Group0: 1.33.0-wmf.9
* Group1: 1.33.0-wmf.8
* Group2: 1.33.0-wmf.8
* Status: 
=== 2018-12-19 ===
* 12:33 * phab:T212218 Fatal error: Call to undefined function Kartographer\Wikimedia\suppressWarnings() in /srv/mediawiki/php-1.33.0-wmf.9/extensions/Kartographer/includes/ApiQueryMapData.php on line 49
* 12:43 phab:T211885 ErrorException from line 47 of /srv/mediawiki/php-1.33.0-wmf.8/extensions/Kartographer/includes/ApiQueryMapData.php: PHP Warning: data error
* 16:59 phab:T212217 ErrorException from line 317 of /srv/mediawiki/php-1.33.0-wmf.9/extensions/ExtensionDistributor/includes/specials/SpecialBaseDistributor.php: PHP Notice: Undefined variable: downloadImg
* 20:03 <hashar@deploy1001> Synchronized php: group1 wikis to 1.33.0-wmf.9 (duration: 00m 51s)
==== End of Day Summary ====
* Group0: 1.33.0-wmf.9
* Group1: 1.33.0-wmf.9
* Group2: 1.33.0-wmf.8
* Status: 
=== 2018-12-20 ===
* 14:28 phab:T212427 No namespace configured for entity type `form`
* 16:59 <zfilipin@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.9
* 17:02 phab:T212427 No namespace configured for entity type `form`
==== End of Day Summary ====
* Group0: 1.33.0-wmf.9
* Group1: 1.33.0-wmf.9
* Group2: 1.33.0-wmf.9
* Status: 
== Conclusions ==
What weakness did we learn about and how can we address them?
== Links to relevant documentation ==
* Heterogeneous deployment/Train deploys
== Actionables ==
Explicit next steps to prevent this from happening again as much as possible, with Phabricator tasks linked for every step.
NOTE: Please add the #wikimedia-incident Phabricator project to these follow-up tasks and move them to the "follow-up/actionable" column.
* To do #1 (TODO: Create task)
* To do #2 (TODO: Create task)
* …