Release Engineering/SAL/Archive 5

== 2019-12-29 ==
* 20:34 hauskatze: github: Mirror deleted: "wikimedia/analytics-limn-edit-data" (T228982)
* 20:19 hauskatze: github: Mirror deleted: "wikimedia/analytics-limn-flow-data" (T228981)
* 20:18 hauskatze: github: Mirror deleted: "wikimedia/analytics-limn-flow-data" (T229981)
== 2019-12-28 ==
* 14:58 hauskatze: maurelio@deployment-deploy01:~$ mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=deploymentwiki --logwiki=deploymentwiki 'HoseaSeverson6' 'Random spammer username' T241294
* 14:23 Urbanecm: urbanecm@deployment-deploy01:~$ mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=arwiki --logwiki=deploymentwiki 'Riley Huntley' '~riley' (T241294)
* 14:21 Urbanecm: urbanecm@deployment-deploy01:~$ mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=enwiki --logwiki=deploymentwiki 'DannyS712 tes' 'DannyS712 test' (T241294)
* 13:13 hauskatze: maurelio@deployment-deploy01:~$ foreachwiki maintenance/emptyUserGroup.php petitiondata # T241504
== 2019-12-27 ==
* 12:23 hauskatze: Rebooting deployment-cpjobqueue
* 11:13 andrewbogott: migrating deployment-aqs03 to cloudvirt1009 in response to T241313
== 2019-12-26 ==
* 12:08 Krenair: projectadmin on deployment-prep for MarcoAurelio see discussion at https://gerrit.wikimedia.org/r/560607
== 2019-12-21 ==
* 21:59 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/560128
== 2019-12-20 ==
* 23:03 Amir1: ladsgroup@deployment-deploy01:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildItemTerms.php --wiki=wikidatawiki --sleep 2 --batch-size=10 (T241210)
* 22:00 marxarelli: created trigger-helloworldoid-pipeline-test and helloworldoid-pipeline-test jobs from jjb
* 11:43 hashar: integration/zuul.git : pushed upstream tags
* 11:41 hashar: integration/zuul push upstream 'master' branch to our 'upstream' branch
== 2019-12-19 ==
* 13:20 James_F: [BETA CLUSTER] mwscript maintenance/cleanupUsersWithNoId.php --wiki=testwiki --prefix=imported T235971
* 11:59 James_F: Docker: Publishing java8-sonar-scanner:0.5.1
* 09:41 hashar: Deploying Zuul change "Codehealth pipeline: Don't report back to gerrit" https://gerrit.wikimedia.org/r/559419 for T217008
== 2019-12-18 ==
* 19:13 halfak: deploying ores 
* 14:34 James_F: Zuul: Add ol ui li to i18n security allow list
* 14:30 James_F: Docker: Push java8-sonar-scanner:0.5.0
== 2019-12-17 ==
* 18:04 Krinkle: Create Jenkins Console section for "setup diffConfig", example at https://integration.wikimedia.org/ci/job/operations-mw-config-php72-composer-diffConfig-docker/181/console
* 17:39 James_F: Zuul: Revert Kartographer as dependency for WBMI per T240971
* 16:06 James_F: Zuul: Add Kartographer as dependency for WBMI
* 13:49 James_F: Zuul: [labs/tools/sonarqubebot] Add CI to run composer-test T240937
* 11:54 James_F: Starting the train cut for 1.35.0-wmf.11
== 2019-12-16 ==
* 09:35 hashar: doc1001: sudo -u doc-uploader rm -fR /srv/docroot/org/wikimedia/doc/DOCKER-mediawiki-core
== 2019-12-15 ==
* 17:42 James_F: Zuul: [mediawiki/extensions/CreateAPage] Add WikiEditor as a dependency
* 17:13 James_F: Zuul: [mediawiki/extensions/CreateAPage] Enable CI tests via quibble
== 2019-12-14 ==
* 19:49 James_F: Zuul: Deploying https://gerrit.wikimedia.org/r/557275
* 19:49 James_F: Zuul: Deploying https://gerrit.wikimedia.org/r/557138
== 2019-12-13 ==
* 20:19 hashar: Regenerating MediaWiki core doxygen (and jsduck doc) # T218233
* 20:18 hashar: Reloaded Zuul for "Use Zuul mutex for mediawiki-core-doxygen-docker-publish" https://gerrit.wikimedia.org/r/#/c/integration/config/+/557102/
* 18:04 hashar: Successfully tagged docker-registry.discovery.wmnet/releng/doxygen:0.6.1
* 16:39 twentyafterfour: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/557001
* 16:28 twentyafterfour: deployed codehealth jenkins jobs
* 12:54 addshore: reload zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/556998/
* 11:08 hashar: deployment-mediawiki07 : removing faulty entry mwdeploy:x:497:498::/var/lib/mwdeploy:/bin/bash in /etc/passwd # T73480
* 02:57 Krinkle: Restarting deployment-mediawiki-07. - T180761
* 02:32 Krinkle: It appears puppet-agent has been locally disabled on deployment-mediawiki-07 for at least three days with "no reason given". Re-enabling now to unbreak https://gerrit.wikimedia.org/r/556854 for T180761
* 02:09 Krinkle: Create 'mongo' security group and apply to deployment-xhgui01 (ingress tcp/27017). T180761
* 01:26 Krinkle: Set `profile::webperf::site::xhgui_host: deployment-xhgui01.deployment-prep.eqiad.wmflabs` in Hiera for deployment-webperf11. T180761
* 00:22 Krinkle: Apply Puppet role class "xhgui::app" to deployment-xhgui01. T238788, T180761
== 2019-12-12 ==
* 14:56 hashar: Removed jsduck from integration-slave-jessie-1002 and integration-slave-jessie-1004
* 14:09 hashar: Removed saucelabs-01 and saucelabs-02 , the last jobs using them got removed. # T240575
* 13:33 hashar: Remove label contintLabsSlave from integration-slave-jessie-1002 and integration-slave-jessie-1004 # T225031
* 11:16 hashar: Manually rebuilding last build of https://integration.wikimedia.org/ci/job/scap-beta-deb/ to generate a scap package without relying on zuul-cloner https://gerrit.wikimedia.org/r/556643 T240551
== 2019-12-11 ==
* 20:26 James_F: Docker: Pushing node10-test:0.6.0 and friends
* 15:20 addshore: reloaded zuul for https://gerrit.wikimedia.org/r/556359 T240445 (also pulled in 2 more changes?)
== 2019-12-10 ==
* 21:27 James_F: Zuul: Add Wugapodes to CI whitelist.
* 13:20 hashar: Removing m4executor labels from Jenkins agents # T225031
* 13:16 hashar: Update Jenkins jobs to drop the m4executor label https://gerrit.wikimedia.org/r/#/c/integration/config/+/556039/ # T225031
* 13:11 hashar: integration: rearmed keyholder on integration-cumin
== 2019-12-09 ==
* 19:58 James_F: Adding wikibase termbox docs publishing post-merge job
* 19:49 hashar: Removed Jenkins label "hasSlaveScripts" from contint1001 # T225031
* 18:08 Amir1: soft rebooting mwmaint01
* 17:51 Amir1: ladsgroup@deployment-mwmaint01:/srv/mediawiki-staging$ sudo apt-get install php-memcached (T239894)
* 17:41 James_F: Zuul: Add JJMC89 and TechneSiyam to CI whitelist
* 10:22 hashar: castor: nuked cache /srv/jenkins-workspace/caches/castor-mw-ext-and-skins/master/mwgate-node10-docker/ # T240174
== 2019-12-06 ==
* 10:34 hashar: Purged pbuilder / cowbuilder etc from integration-slave-jessie-1002 and integration-slave-jessie-1004 # T224943
== 2019-12-05 ==
* 20:52 hashar: Reloaded Zuul for https://gerrit.wikimedia.org/r/#/c/554944/
* 19:22 Reedy: killed stuck beta CI jobs, stuff is flowing again
* 19:15 James_F: Zuul: [labs/toollabs] Drop trusty, add jessie to gate as well as test T224943
* 18:36 James_F: Zuul: Add Majavah to CI whitelist
* 10:14 hashar: Deleted integration-slave-jessie-1001 , created integration-agent-pkgbuilder-1002 # T224943
* 10:14 hashar: Deleted integration-slave-jessie-1001 , created integration-agent-pkgbuilder-1002 # TT224943
== 2019-12-04 ==
* 22:35 hashar: integration: added Stretch instance integration-agent-pkgbuilder-1001 for Debian glue jobs # T224943
* 21:48 hashar: Updated debian glue non voting jobs to use git instead of zuul-cloner https://gerrit.wikimedia.org/r/#/c/integration/config/+/554588/ T224943
* 19:22 James_F: Zuul: Add CI for wikibase/vuejs-components
* 17:40 Lucas_WMDE: lucaswerkmeister-wmde@deployment-deploy01:~$ mwscript createAndPromote.php dewiki 'Lucas Werkmeister (WMDE)' --force --custom-groups=bureaucrat # make myself bureaucrat to confirm users
* 16:56 Amir1: MariaDB [wikidatawiki]> delete * from wb_changes_dispatch; (T237551)
* 16:49 James_F: Zuul: Add CI for operations/software/netbox-extra T233183
* 15:57 James_F: Zuul: Add CI for EventStreamConfig MW extension T233634
* 15:19 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/554530
== 2019-12-03 ==
* 21:24 James_F: Enabling PHP74 tests as voting for MW skins T233012
* 17:33 kevinbazira: deploying ores 
* 16:33 hashar: Restarted Jenkins
* 16:10 hashar: CI Jenkins: updating collapsing console sections plugin with upstream fixes (PR #16 and #18) # T236222
* 15:56 James_F: Zuul: Add IAmNetx to CI whitelist
* 10:45 hashar: Building releng/operations-puppet to upgrade tox and ship python3.4 3.6 and 3.7 # T239334
* 01:43 James_F: Docker: Pushing operations-dnslist:0.0.6
== 2019-12-02 ==
* 20:18 Krenair: deactivated and cleaned cert for deployment-eventgate-2 instance that appears to have been deleted at some point
* 20:15 brennen: Updating dev-images docker-pkg files on contint1001 for T238605
* 17:46 James_F: Zuul: Parsoid now has PHP74 as voting.
* 17:26 James_F: Zuul: Adding experimental PHP74 jobs to anything that moves T236132
* 17:13 James_F: Docker: Pushing initial php74 images forT236132
* 09:51 hashar: jenkins: removing Command Agent Launcher plugin https://plugins.jenkins.io/command-launcher
== 2019-11-30 ==
* 13:27 Krenair: found new instance deployment-xhgui01 with broken puppet (not trusting our puppetmaster CA), ran the usual commands to get it working
== 2019-11-29 ==
* 15:14 hashar: deployment-prep: rearming keyholder on deployment-deploy01
== 2019-11-28 ==
* 13:59 hashar: gerrit: changed labs.git HEAD to point to refs/meta/config , master did not exist and that makes it a PERMISSIONS repository
* 13:57 hashar: gerrit: changed jsdoc.git HEAD to point to refs/meta/config , master did not exist and that makes it a PERMISSIONS repository
* 11:40 hashar: deployment-prep: rebased puppet. Notably to fix wmf-beta-update-databases ( https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/553200/ ) # T239399
== 2019-11-27 ==
* 18:30 hashar: Fixed up the Jenkins job for beta cluster
== 2019-11-26 ==
* 20:00 Krinkle: Click "Disable Publishing" for extension-CollaborationKit, T143162, T219604
* 15:50 James_F: Docker: Pushing composer-php72:0.2.2 and composer-php73:0.1.7 for T235221
* 10:42 hashar: Building docker-registry.discovery.wmnet/releng/operations-puppet:0.5.5 for jbond42
* 01:02 Krenair: Shut down deployment-logstash2 T238707
== 2019-11-25 ==
* 19:12 James_F: Docker: node10-test-browser-php72-composer:0.1.2
* 18:20 James_F: Following the https://www.mediawiki.org/wiki/Continuous_integration/Jenkins#Hung_beta_code/db_update runbook for T239145
== 2019-11-24 ==
* 03:25 Krinkle: Create deployment-xhgui01 as Beta version of xhgui1001/xhgui2001 (Debian 9 Stretch, m1.small). – T238788, T238098, T180761
* 03:14 Krinkle: Delete deployment-webperf01 instance (to be re-created as xhgui01).
== 2019-11-22 ==
* 19:21 James_F: Zuul: Fix name of BlueSpiceUEModulePDFSubpages repo in config; add Flow to phan dependencies for CollaborationKit
* 19:19 James_F: Zuul: Drop the eventgate-ci repo
* 19:19 James_F: Zuul: Add Nullzero and Lens0021 to the whitelist
== 2019-11-20 ==
* 23:33 paladox: ssh -p 29418 paladox@gerrit.wikimedia.org replication start mediawiki/extensions/LDAPGroups --wait (per hauskater)
* 21:28 Krinkle: Create deployment-xhgui01 in Beta Cluster as m1.small/Debian 10 Buster. – T238098, T180761
* 18:47 hauskater: Manual replication cmd issued for new repo.
* 18:23 hauskater: gerrit: Created `operations/software/netbox-extras.git`; request on mediawiki
* 17:35 James_F: JJB: Pushed wikimedia-production-pipeline-publish and trigger-wikimedia-production-pipeline-publish
* 16:37 James_F: Zuul: Add CI for wikimedia/production
* 16:37 James_F: Zuul: Add CI for mediawiki/services/eventstreams T238658
* 00:53 Krenair: T238707 changed dateFormat (under management -> advanced settings) from 'MMMM Do YYYY, HH:mm:ss.SSS' to 'YYYY-MM-DDTHH:mm:ss', and dateFormat:tz from Browser to UTC to match old instance
* 00:51 Krenair: T238707 created old-logstash-beta proxy to point at old instance, created default Index Pattern on new logstash-beta
* 00:47 Krenair: T238707 moved kibana4/logstash-beta proxies to deployment-logstash03, copied /etc/logstash/htpasswd file
== 2019-11-18 ==
* 14:04 James_F: Zuul: Add BlueSpiceUEModulePDFRecursive / drop BlueSpiceRecursive, and fix BlueSpiceUEModuleSubpages naming
== 2019-11-17 ==
* 15:23 James_F: Zuul: Add dependency to BlueSpiceUEModuleBookPDF
* 15:23 James_F: Zuul: Archive the OAuth2 extension T238429
* 12:24 hauskater: Deleted GitHub mirror for extension-OAuth2 refs. T238429
* 00:57 James_F: Zuul: Enable seccheck on Wikisource; add BlueSpiceUEModuleBookPDF, BlueSpiceRecursive, BlueSpiceSubpages
* 00:27 James_F: Zuul: Add Wikibase as dependency for Wikisource
== 2019-11-15 ==
* 21:56 James_F: Docker: Adding new images refreshed for PHP 7.2.24 T237275
* 16:45 James_F: Gearman merge queue clear thanks to hashar, all fixed.
* 16:19 James_F: Gearman having fun. Stuck with 1233 merge changes.
== 2019-11-13 ==
* 15:07 James_F: Zuul: Add BlueSpiceWikiExplorer
== 2019-11-12 ==
* 15:30 James_F: Zuul: Enable phan for Wikisource extension T237680
* 15:16 James_F: Zuul: Add BlueSpicePageFormsConnector and BlueSpiceArticlePreviewCapture
* 11:35 hashar: Reloading Zuul for https://gerrit.wikimedia.org/r/550048 " Archive TimezoneSelector extension" T237894
* 11:18 hashar: Reloading Zuul for https://gerrit.wikimedia.org/r/550308 Add Jeblad in CI whitelist [
== 2019-11-11 ==
* 18:28 MaxSem: Nuked HHVM and php7 tags on all beta wikis T75181
* 04:41 Reedy: apt-get clean/autoclean and purge some 3.16 kernels from deployment-sca01
== 2019-11-08 ==
* 17:46 MaxSem: Upgraded php7.2 on deployment-mwmaint01, was too old for MW
* 15:50 James_F: Zuul: Add BlueSpiceBookshelf and BlueSpiceBookshelfUI
== 2019-11-07 ==
* 22:38 James_F: Zuul: Update archived repo failure message to point to new policy page
* 19:33 James_F: Zuul: Mega re-ordering
* 18:22 James_F: Add node10-docker to ProveIt
* 00:16 Reedy: rm /srv/mediawiki-staging.git/gc.log && git gc --aggressive
== 2019-11-06 ==
* 22:46 Krinkle: Fix closing pattern of "setup quibble" Console Section in Jenkins to not break on composertest jobs.
* 20:31 brennen: Updating dev-images docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/c/releng/dev-images/+/548941
* 17:43 mdholloway: deployment-prep dropped and rebuilt MachineVision extension tables
* 15:40 James_F: Upgraded mediawiki-quibble-composer-mysql-php72-docker as a test flight for T236907 T237049
* 11:31 hauskater: Deleted https://github.com/wikimedia/labs-tools-discordwiki T237491
* 09:04 hashar: Restarted CI Jenkins
* 09:02 hashar: Jenkins: upgrade Monitoring plugin (javamelody) to 1.80.0 + misc updates
== 2019-11-05 ==
* 18:40 twentyafterfour: branching 1.35.0-wmf.5
* 18:39 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/548827
* 16:16 Lucas_WMDE: lucaswerkmeister-wmde@deployment-deploy01:~$ mwscript createAndPromote.php deploymentwiki 'Lucas Werkmeister (WMDE)' --force --custom-groups=steward # make myself steward to remove an erroneous global lock
* 15:14 James_F: Zuul: Adding InterwikiExtracts extension
* 12:16 Lucas_WMDE: lucaswerkmeister-wmde@deployment-deploy01:~$ mwscript createAndPromote.php dewiki 'Lucas Werkmeister (WMDE)' --force --sysop # make myself admin for T235152
== 2019-11-04 ==
* 16:52 James_F: Zuul: Make debian-glue voting for prometheus-ipsec-exporter
* 16:49 James_F: Zuul: Add coverage reporting to Cite
== 2019-11-02 ==
* 19:07 James_F: Zuul: Adding phan to ULS and GettingStarted
* 17:55 Urbanecm: deployment-prep: mwscript initSiteStats.php --wiki=cswiki --update (T236823)
* 17:52 Urbanecm: deployment-prep: mwscript rebuildrecentchanges.php --wiki=cswiki (T236823)
* 17:52 Urbanecm: deployment-prep: mwscript importDump.php --wiki=cswiki /tmp/Wikipedie-20191031004003.xml (T236823)
* 16:44 Urbanecm: deployment-prep: mwscript createAndPromote.php --wiki=cswiki --force --bureaucrat --sysop 'Martin Urbanec'
* 16:26 Urbanecm: mwscript extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --wiki=cswiki (T237077)
* 16:16 Amir1: Created cswiki (T237077)
* 00:15 paladox: ssh -p 29418 paladox@gerrit.wikimedia.org replication start --url gerrit2001.wikimedia.org --all
== 2019-11-01 ==
* 21:06 paladox: ssh -p 29418 paladox@gerrit.wikimedia.org replication start --url gerrit2001.wikimedia.org --all
* 16:40 Amir1: ladsgroup@deployment-deploy01:~$ mwscript sql.php --wiki=wikidatawiki /srv/mediawiki-staging/php-master/extensions/Wikibase/repo/sql/increase_wbxl_language.sql (T237102)
* 16:39 James_F: Zuul: Reloading with pipelines with _s not .s
== 2019-10-31 ==
* 20:29 tgr: importing a bunch of pages from production cswiki via importDump.php for T236823 (for reals now)
* 18:38 James_F: Docker: Pushing updates for T236907
* 18:06 James_F: Docker: Pushing php71:0.1.6 php72:0.2.3 php73:0.2.4
* 16:25 James_F: Zuul: [puppet] Complete the un-submodule cleanup T230206
* 15:15 James_F: Zuul: Archive operations/puppet/nginx repo in CI
* 15:15 James_F: JJB: puppet-nginx-rake-docker deleted
* 14:46 hashar: Building CI containers for php7.2 php-xdebug upgrade #T234418
* 13:15 hauskater: Diffusion Set default branch for Diffusion {rAPIOS} GitHub mirror from 'master' to 'develop' ('master' is not updated c. 2015; thus serving very outdated data).
* 01:14 tgr: importing a bunch of pages from production cswiki via importDump.php for T236823
== 2019-10-30 ==
* 22:22 Krenair: added jdl to deployment-prep https://www.mediawiki.org/w/index.php?title=Wikimedia_Product/Analytics_Infrastructure&diff=3487111&oldid=3487110&diffmode=source
* 21:52 hashar: Rolled back all CI WMCS Docker to 18.06.2
* 21:51 James_F: Zuul: Add Echo as phan dependency for CentralAuth
* 21:48 hashar: Triggering build of mwcore-phpunit-coverage-master , should use Docker 18.06.2 and be faster # T232706
* 21:17 hashar: Rolling back docker on CI Stretch instances to 18.06.2~ce~3-0~debian (matching jessie)
* 20:44 hashar: integration-agent-docker-1005 downgrade docker to docker-ce 18.06.2~ce~3-0~debian , the one we had on Jessie # T236675
* 19:11 hauskater: Disabled explicit mirroring of rCICF to GitHub via Diffusion URI; let Gerrit replication take over
* 18:51 James_F: Zuul: Add CI for new XSL extension
* 16:58 James_F: Zuul: Add Echo as a phan and real dependency for MachineVision
* 15:56 hauskater: gerrit : created https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/extensions/XLS # T236914
* 11:17 hashar: Bring back online a couple instances I have put offline. That was preventing some jobs to run :/
* 09:45 elukey: mediawiki mcrouter's on deployment-prep are now using deployment-memc-08 (Buster node with memcached) - to report any issue T213089
* 09:44 elukey: manual rebase fix on deployment-puppetmaster03 to unblock git-sync - T236876
== 2019-10-29 ==
* 19:06 paladox: updaed ownership of https://gerrit.wikimedia.org/r/#/admin/projects/openstack/horizon/deploy,access to wmcs-trusted
* 18:55 paladox: granted ldap/ops ownership of https://gerrit.wikimedia.org/r/#/admin/projects/openstack/horizon/deploy,access
* 17:26 hashar: Reloading zuul for https://gerrit.wikimedia.org/r/546987
* 16:31 hashar: integration-agent-docker-1005 change sudo and ldap client (to sssd) # T236675
* 14:34 hashar: integration-slave-jessie-* : rm /var/cache/pbuilder/base-buster-amd64.cow/var/lib/apt/lists/mirrors.wikimedia.org_debian_dists_buster_InRelease && /usr/sbin/cowbuilder --update --basepath "/srv/pbuilder/base-buster-amd64.cow" # T236796
* 14:24 hashar: integration-slave-jessie-1004 : updating buster cow image to verify the cron updater actually works: /usr/sbin/cowbuilder --update --basepath "/srv/pbuilder/base-buster-amd64.cow"
* 12:14 Urbanecm: deployment-prep: delete from user_properties where up_user=11942 and up_property like "%growth%" and up_property like "%suggested%";
* 09:45 hashar: Purging Docker images on CI instances
== 2019-10-28 ==
* 19:04 James_F: Zuul: Add CI for operations/software/httpbb T236699
* 16:11 James_F: Zuul: Add BlueSpiceUserInfo
* 14:57 hashar: Updating mediawiki-quibble-api-testing-vendor-mysql-php72-docker mediawiki-quibble-api-testing-vendor-sqlite-php72-docker # T236333
* 14:49 hashar: contint1001 for https://gerrit.wikimedia.org/r/#/c/integration/config/+/546505/ sudo docker tag docker-registry.discovery.wmnet/releng/quibble-stretch-php72:0.0.40 docker-registry.discovery.wmnet/releng/quibble-stretch-php72:0.0.39-s2
* 14:47 hashar: contint1001: docker image prune
* 14:45 James_F: Zuul: Drop PHP56 jobs for SmashPig T224906
* 14:12 hashar: Reloading Zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/546470/
== 2019-10-27 ==
* 16:47 James_F: Zuul: Add seccheck and phan to GitGadgets extension
* 00:18 Krinkle: krinkle@contint Replay various mediawiki/core git tags to zuul 'publish' pipeline for T232055
* 00:13 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/546339
== 2019-10-26 ==
* 21:25 Krinkle: krinkle@contint `zuul enqueue-ref --trigger gerrit --pipeline publish --project mediawiki/core --ref 'refs/tags/1.30.1' --newrev 'a0b31d95610c8f415a2cc29349cdbf14c23c7d4b' --oldrev '0000000000000000000000000000000000000000'` for T232055
* 21:20 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/546343
* 20:54 Krinkle: krinkle@contint Replay oojs/core git tags to zuul 'publish' pipeline for T232055
* 20:51 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/546342
* 20:47 Krinkle: krinkle@contint1001$ `zuul enqueue-ref --trigger gerrit --pipeline publish --project unicodejs --ref 'refs/tags/v12.0.0' --newrev '63ab6302019d3b6b1b762b5c0094370e3030e94c' --oldrev '0000000000000000000000000000000000000000'` for T232055
* 20:35 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/546340
== 2019-10-25 ==
* 20:59 paladox: created cloud/instance-puppet-dev gerrit repo per andrewbogott
* 18:30 brennen: Updating docker-pkg files on contint1001 for T236333
* 09:49 hashar: Restarting CI Jenkins for PAM plugin upgrade
* 09:20 hashar: Updated integration/jenkins-job-builder.git to 2.9.1 # T236036
* 09:19 hashar: Reconfiguring all Jenkins jobs defined by JJB due to upgrade to jjb 2.9.1 # T236036
== 2019-10-24 ==
* 14:17 James_F: Zuul: Drop BlueSpiceBlog from CI, archived. T231853
* 14:05 James_F: Zuul: Drop BlueSpiceUserPreferences from CI, archived.
* 10:04 hashar: Updated integration/jenkins-job-builder.git to 2.0.3 # T188398
* 10:00 hashar: Reconfiguring most Jenkins jobs due to PostBuildScript plugin change https://gerrit.wikimedia.org/r/#/c/integration/config/+/544907/
== 2019-10-23 ==
* 18:09 paladox: create wikimedia/fundraising/analytics per T236306
* 17:00 hashar: Reloading Zuul for https://gerrit.wikimedia.org/r/545580 Standalone Selenium jobs for Wikibase # T232759
* 09:12 hashar: Pooling back integration-agent-docker-1006 , went offline for unknown reason?
* 09:09 hashar: CI Jenkins: downgrading AnsiColor plugin from 0.6.2 to 0.5.3 # T236222
* 08:33 hashar: contint1001: backing up Zuul logs to /var/log/zuul/backup-T236114 so we get gerrit activities traces for T236114
== 2019-10-22 ==
* 13:01 James_F: Limited pywikibot to one concurrent job in Wikimedia TravisCI to avoid resource exhaustion
== 2019-10-21 ==
* 19:00 Krinkle: Pruning old image from contint1001: 0.0.38 
* 19:00 Krinkle: Pruning old image from contint1001: 0.0.38-s1 
* 18:42 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/544992
* 18:38 hashar: Jenkins: upgraded logstash plugin to 2.3.1 , overriding a custom build snapshot of 1.2.1. # T78705
* 12:06 hashar: Draining merger:merge request. On contint2001 : sudo chown -R root:root /srv/zuul/git/integration/config
* 12:04 hashar: Draining merger:merge request. On contint1001 : sudo chown -R root:root /srv/zuul/git/integration/config
* 10:23 hashar: Restored integration/config on zuul-mergers : sudo chown -R zuul:zuul /srv/zuul/git/integration/config
* 10:15 hashar: contint1001 / contint2001 : marking integration/config zuul merger repo readonly: sudo chown -R root:root /srv/zuul/git/integration/config
* 09:48 hashar: jenkins-job-builder : remove cherry-pick which is no more needed (to support multiple macro in conditional-step builder https://review.openstack.org/#/c/471030/1 ) == rollback from to tag 2.0.2
* 09:12 hashar: update Quibble jobs to container 0.0.39-s1
* 09:04 hashar: Restarting CI Jenkins for plugin upgrades
== 2019-10-20 ==
* 21:28 Krinkle: Click "Disable Publishing" for extensions/OAuth and extensions/RevisionSlider in Phabricator, T143162 ,T219604
* 09:38 James_F: Zuul: Run phan job for Translate T224756
* 09:37 James_F: Zuul: Add BlueSpiceReminder as dependency for BlueSpiceExpiry
== 2019-10-19 ==
* 20:48 Krinkle: Permissions are broken again on doc1001 for /srv/docroot (various files wrongly chmoded as 660 or wrongly owned by nobody, or hashar). Unable to git-pull.
== 2019-10-18 ==
* 21:26 paladox: create cloud/instance-puppet repo per andrewbogott request
* 20:34 hashar: integration: sudo cumin --force 'name:docker' 'docker image prune -f'
* 20:34 hashar: integration: sudo cumin --force 'name:docker' 'docker container prune -f'
* 18:42 James_F: Docker: Rebuilding the world onto composer-php70:0.1.1
* 18:32 James_F: Zuul clean sync as the last patch was seemingly already deployed?
* 18:19 James_F: Docker: Building lintr:0.2.1 and php70-compile:0.1.4
* 17:09 thcipriani: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/544219
* 13:51 hashar: Building Quibble 0.0.39 CI containers
* 13:44 hashar: Tag Quibble 0.0.39 @ # T235023 T235564
* 12:43 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/544186
== 2019-10-17 ==
* 20:48 hauskater: Arm eventlogging key via keyholder for beta cluster following the instructions on Wikitech for T235674
* 17:10 James_F: Zuul: OOUI] Drop generic-node10-rundoc-docker and …-npmaudit-docker experiments
* 16:33 James_F: Zuul: Moving OOUI from node 6 to node 10 job T235570
* 16:11 James_F: Docker: Pushing node10-test-browser-php72-composer:0.1.1
== 2019-10-16 ==
* 22:42 thcipriani: deployment-deploy02:sudo vipw to remove mwdeploy user
* 22:10 James_F: https://integration.wikimedia.org/ci/job/wmf-quibble-core-vendor-mysql-php72-docker/8502/console got stuck in quibble's composer install step for half an hour; manually aborted. :-(
* 18:27 James_F: Zuul: [mediawiki/tools/phan/SecurityCheckPlugin] Move to PHP72+
* 18:26 James_F: Zuul: Activate tests for new repo BlueSpiceDistributionConnector
* 17:44 James_F: Marking deployment-deplog01 offline temporarily for T235674
* 08:57 elukey: created deployment-memc08 in deployment-prep as memcached test host for Buster - T213089
== 2019-10-15 ==
* 21:55 James_F: Docker: Provide node10-test-browser-php72-composer T235570
* 19:40 James_F: Created the wmf.N milestones for 1.35.0 out until wmf.15 for ReleaseTaggerBot: https://phabricator.wikimedia.org/project/subprojects/4306/
* 19:19 James_F: Zuul: Enable Phan for FlaggedRevs T224769
* 18:55 hashar: Jenkins: deleted no more used jobs operations-mw-config-hhvm-composer-lint-docker and operations-mw-config-hhvm-composer-test-docker
* 18:01 James_F: Zuul: Enable Phan for OpenStackManager T224765
* 17:33 James_F: Zuul: Enable Phan for ReadingLists T224762
* 16:49 James_F: Zuul: Phan dependencies for ReadingLists and OpenStackManager.
* 15:36 James_F: Docker: Rebuild images for PHP 7.2.22 T232165
* 13:04 hashar: Building docker-registry.wikimedia.org/releng/tox-homer:0.1.0 for XioNoX / operations/software/homer
* 12:08 paladox: create group “bluespice-admin” in gerrit per T234224
* 09:31 hashar: CI Jenkins: uninstalling BlueOcean meta plugin to let us uninstall plugins we do not care about # T233695
== 2019-10-14 ==
* 11:40 hashar: contint1001: deleting old Quibble containers
* 09:34 hashar: Upgraded CI jobs to Quibble 0.0.38
* 09:32 hashar: Deleting old release-quibble* jobs from Jenkins. They are no more used.
== 2019-10-12 ==
* 16:43 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/542690
== 2019-10-11 ==
* 17:58 Krinkle: Click "Disable Publishing" on extension-Elastica Phab mirror, T235233, T143162
* 16:45 James_F: Zuul: Add dependency "BlueSpiceExtendedSearch" to BlueSpiceStatistics
* 16:03 paladox: ssh -p 29418 gerrit.wikimedia.org gerrit create-project cloud.git --permissions-only - T235279
* 15:59 paladox: ssh -p 29418 gerrit.wikimedia.org gerrit create-project cloud.git --permissions-only --description="Container for all other cloud/* projects for permissions, etc." - T235279
* 13:32 hashar: Build docker-registry.discovery.wmnet/releng/tox-pyspark:0.5.1 for ebernhardson
* 01:37 Reedy: Killed the job that was waiting for executor for beta-scap-eqiad, it's now running after 4 hours not
== 2019-10-10 ==
* 21:08 Krenair: Ran apt upgrades on deployment-mediawiki-0[79] to bring beta cluster back online, MW started requiring a version higher than we were running
* 20:52 James_F: Zuul: Make mediawiki-quibble-api-testing* voting T230340
* 16:22 James_F: Zuul: Also enable phan and phan-seccheck on DiscussionTools T234481
* 16:15 James_F: Zuul: Add CI configuration for new DiscussionTools repo T234481
* 10:35 hashar: gerrit: added onimisionipe and gehel to wikidata-query-gui groups # T235159
== 2019-10-09 ==
* 20:01 hashar: gerrit set-account owl --active # T234328
* 19:55 urandom: Upgrading deployment-sessionstore01.deployment-prep.eqiad.wmflabs to Cassandra 3.11.4 -- T200803
* 19:49 urandom: Upgrading deployment-restbase02.deployment-prep.eqiad.wmflabs to Cassandra 3.11.4 -- T200803
* 19:46 urandom: Upgrading deployment-restbase01.deployment-prep.eqiad.wmflabs to Cassandra 3.11.4 -- T200803
* 17:52 hashar: Build Quibble 0.0.38 CI Docker images
* 17:52 hashar: Tagged Quibble 0.0.38 @ earlier today to fix $wgServer # T235023
* 15:55 James_F: Docker: [mediawiki-phan] Avoid unbound variable -> 0.5.3
* 07:37 hashar: Build Quibble 0.0.37 docker containers
* 07:28 hashar: Tag Quibble 0.0.37 @ 
== 2019-10-08 ==
* 22:13 James_F: Zuul: Actually restrict mwcore-codehealth-master-non-voting and mwext-codehealth-master-non-voting to master
* 20:19 hashar: Tag Quibble 0.0.36 @ # T181942 T190829 T199939 T230340 T233140 T233143
* 16:27 James_F: Docker: [mediawiki-phan] Fix phan version validation
== 2019-10-07 ==
* 20:59 hashar: deployment-prep: on a few mediawiki deployment targets: sudo rm -fR /srv/mediawiki/.~tmp~/
* 20:32 brennen: https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/540701 is merged and parsoid dev images are being published
* 20:22 thcipriani: remove shadow mwdeploy user from a bunch of other deployment-prep hosts to (hopefully) fix beta-scap-eqiad
* 19:29 thcipriani: deployment-deploy01:sudo rm -rf /srv/mediawiki/.~tmp~
* 19:29 James_F: Docker: [mediawiki-phan] Fix syntax error in shell scripts
* 18:50 thcipriani: deployment-deploy01 vipw && vipw -s to remove local mwdeploy user (use user 603 from ldap instead)
* 17:39 James_F: Zuul: Move mediawiki/tools/release to php72+
* 17:35 James_F: Zuul: Add phan dependencies for FlaggedRevs T224769
* 17:25 James_F: Zuul: [mediawiki/tools/phan,PerfCheckPlugin] Move to PHP72+
* 16:51 James_F: Zuul: Add phan dependencies for Translate T224756
* 15:34 James_F: Docker: [mediawiki-phan] Allow requiring phan like normal in composer.json T220589
* 15:13 James_F: Docker: [mediawiki-phan] Drop back-compatibility hacks for old phan
* 13:37 hashar: integration/quibble : grants kostajh +2 (in addition to awight and hashar)
* 08:33 hashar: gerrit: `gerrit index start changes --force` # T233989
* 08:32 hashar: gerrit: `gerrit index start changes` # T233989
== 2019-10-04 ==
* 21:13 hauskater: gerrit: Ran replication start mediawiki/extensions/WikiSEO --wait # New repository
* 19:34 marxarelli: re-running `mediawiki/tools/release/make-release/branch.py --bundle '*' ...` due to https://gerrit.wikimedia.org/r/c/mediawiki/tools/release/+/540953 cc: T232024
* 19:17 Krinkle: Fix Jenkins Console section "setup quibble prepare" pattern to account for recent changes on "mwext-php72-phan-docker" which was eating the pattern incorrectly
* 18:55 marxarelli: branching REL1_34 for remaining extensions/skins and core, `mediawiki/tools/release/make-release/branch.py REL1_34 --bundle '*' --core --branchpoint master --core-version 1.34.0` (note: core branchpoint is overridden in settings.yaml at base of wmf/1.34.0-wmf.25) cc: T232024
* 18:34 marxarelli: branching REL1_34 for all wmf-deployed extensions, `mediawiki/tools/release/make-release/branch.py REL1_34 --bundle wmf --branchpoint wmf/1.34.0-wmf.25 --core-version 1.34.0` cc: T232024
* 17:45 brennen: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/540699 for parsoid publication pipeline
* 07:44 hashar: Adding labels to Jenkins agent "Docker blubber m4executor" on integration-agent-jessie-docker-1001 Jessie based VM to potentially compare speed of jobs with the Stretch based ones
== 2019-10-03 ==
* 21:03 James_F: Docker: Bump things over to use php70 which adds intl
* 20:57 James_F: Docker: Rename php to php70 and add intl
* 19:12 James_F: [wikimedia/fundraising/crm] AAdd experimental php70 job T230446
* 18:25 James_F: Zuul: [mediawiki/tools/codesniffer] Move to PHP7.2+
* 11:09 hashar: integration-castor03: pruning HHVM jobs caches: rm -fR /srv/jenkins-workspace/caches/*/*/*hhvm* # T234384
* 11:07 hashar: jenkins: removing global environment variable: HHVM_REPO_CENTRAL_PATH=$WORKSPACE/central.hhbc https://integration.wikimedia.org/ci/configure T234384
== 2019-10-02 ==
* 21:06 James_F: Zuul: Misc HHVM test clean-ups.
* 20:37 James_F: Zuul: Migrate fundraising tests from HHVM to PHP70 T234384
* 19:16 James_F: Zuul: Drop all HHVM testing except for Fundraising T234384
* 19:01 James_F: Zuul: Drop -nohhvm jobs, no longer needed
* 18:55 James_F: Zuul: Drop HHVM testing from all quibble jobs T234384
* 17:57 hashar: Fixed puppet ssl issue on integration-agent-docker-1004 and integration-agent-docker-1008 ( rm -fR /var/lib/puppet/ssl )
* 03:07 Krinkle: Fix "setup quibble mw-install" Jenkins Console section to not eat all output on new quibble-selenium jobs (T232759)
== 2019-10-01 ==
* 15:17 hashar: Deleted integration-agent-docker-1015 (was too slow)
* 15:17 hashar: Pooled unused Jessie agent https://integration.wikimedia.org/ci/computer/integration-agent-jessie-docker-1001/
* 13:55 hashar: Gerrit: disabled bot account "owl" , see whether that helps on the JVM gc issue T234328
* 13:43 hashar: Gerrit: disabling bot account "owl" , see whether that helps on the JVM gc issue
* 11:13 hashar: shutting down integration-castor03 # T232646
== 2019-09-30 ==
* 20:47 James_F: Zuul: Add phan job for WikibaseQualityConstraints T224773 and some phan dependencies for WikimediaMaintenance T224770.
* 19:03 hashar: Update jobs wmf-quibble-core-vendor-mysql-hhvm-docker wmf-quibble-core-vendor-mysql-php72-docker wmf-quibble-vendor-mysql-hhvm-docker wmf-quibble-vendor-mysql-php72-docker to stop running selenium tests https://gerrit.wikimedia.org/r/#/c/integration/config/+/539630/
* 19:01 hashar: Reloading Zuul for https://gerrit.wikimedia.org/r/539630
* 18:53 hashar: Create jobs wmf-quibble-selenium-hhvm-docker wmf-quibble-selenium-php72-docker https://gerrit.wikimedia.org/r/#/c/integration/config/+/539630/
* 15:23 James_F: Zuul: Added TheSandDoctor to the CI whitelist.
* 13:10 hashar: integration-castor03 : sudo mkdir -p /usr/local/lib/python2.7/dist-packages
* 12:49 hashar: Fixed cumin on integration project # T234203
* 11:08 hashar: Restarting Docker on CI agents to clear out some docker/iptables oddity # T234197
* 10:59 hashar: Restarted Docker on integration-agent-docker-1001 T234197
== 2019-09-28 ==
* 18:35 hauskatze: gerrit: Ran gerrit gc labs/tools/stewardbots --show-progress --aggressive
== 2019-09-27 ==
* 21:01 hashar: T233989 I have failed to run (must be run offline): /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -jar /var/lib/gerrit2/review_site/bin/gerrit.war reindex --list
* 20:01 hashar: Marking integration-agent-docker-1015 offline due to cloudvirt1004 being wayyyyy too slow T223971
* 18:23 brennen: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/539447
== 2019-09-26 ==
* 23:42 brennen: Updating dev-images docker-pkg files on contint1001 for stretch-php72-fpm-apache2 publication
* 22:47 James_F: Zuul: Re-enable trigger-mediawiki-pipeline-dev now that we're ready T218360
* 12:43 hashar: Pooling integration-agent-docker-1015 and integration-agent-docker-1016
* 11:48 hashar: Pooling integration-agent-docker-1014
* 11:45 hashar: Pooling integration-agent-docker-1013
== 2019-09-25 ==
* 23:01 andrewbogott: moving deployment-mwmaint01 and deployment-ircd to cloudvirt1021
* 20:14 hashar: Pooled integration-agent-docker-1007
* 19:36 hashar: Deleting integration-agent-docker-1007 it is too slow ( T223971 )
* 15:15 andrewbogott: moving deployment-snapshot01 to cloudvirt1021
* 15:02 andrewbogott: moving deployment-dumps-puppetmaster02 to cloudvirt1021
* 11:01 Lucas_WMDE: lucaswerkmeister-wmde@deployment-deploy01:~$ mwscript createAndPromote.php enwiki --force --sysop 'Lucas Werkmeister (WMDE)' # making myself admin for Wikidata Bridge testing
* 07:14 hashar: Restarted memcached on deployment-memc04, deployment-memc05 , deployment-memc06, deployment-memc07, deployment-imagescaler01 # T232796
== 2019-09-24 ==
* 15:27 hashar: deployment-logstash2: removing class profile::elastalert , does not exist and breaks puppet # T233134
== 2019-09-23 ==
* 20:42 hashar: Added integration-agent-docker-1012 # T226233
* 20:38 hashar: Added integration-agent-docker-1011 # T226233
* 20:37 hashar: Added integration-agent-docker-1010 # T226233
* 20:23 hashar: Added integration-agent-docker-1009 # T226233
* 19:15 hashar: Buidling Docker image releng/gerrit:0.1.0-s2 (bazel upgrade)
* 19:11 hashar: Disconnected and reconnected integration-agent-docker-1006 to have jenkins-deploy to be a meember of docker group
* 19:09 hashar: Added integration-agent-docker-1008 # T226233
* 19:06 hashar: Added integration-agent-docker-1007 # T226233
* 19:03 hashar: Buidling Docker image releng/bazel:0.3.0
* 18:43 hashar: Added integration-agent-docker-1006 # T226233
== 2019-09-21 ==
* 19:36 James_F: Zuul: [WikibaseMediaInfo] Add phan dependency T224772
== 2019-09-20 ==
* 21:47 James_F: jjb: Deleting mediawiki-tarball-builder; follow-up can happen in 
* 21:45 James_F: jjb: All executors moved from DebianJessieDocker to Docker
* 21:35 James_F: jjb: Dropping differential-docker-test
* 21:34 James_F: jjb: Dropping wmf-sonar-*
* 21:33 James_F: jjb: Dropping wikimedia-fundraising-crm-composer-php56-docker
* 21:32 James_F: jjb: Dropping puppet-zookeeper-rake-docker
* 21:31 James_F: jjb: Dropping mwgate-npm-node-6-docker
* 21:31 James_F: jjb: Dropping php-compile-hhvm-jessie-docker
* 21:30 James_F: jjb: Dropping Schedule a Build
* 19:41 James_F: Zuul: Drop ortiz-rlang (merged but not deployed)
* 19:41 James_F: Zuul: [JsonConfig] Add phan dependency and phan
* 15:16 hashar: Pooled integration-agent-docker-1005 based on Stretch # T226233
* 15:14 hashar: Pooled integration-agent-docker-1004 based on Stretch # T226233
* 14:47 hashar: Pooled integration-agent-docker-1002 integration-agent-docker-1003 based on Stretch # T226233
* 13:23 hashar: Deleting Jenkins agent integration-r-lang-01 (unused) # T153856
* 13:14 hashar: Pooled integration-agent-docker-1001 based on Stretch # T226233
* 01:12 James_F: Zuul: [PropertySuggester] Add phan
* 01:00 James_F: Zuul: [PropertySuggester] Add phan dependency T224774
== 2019-09-19 ==
* 16:52 hashar: Building releng/helm-linter:0.2.0 T233291
* 09:31 hashar: Upgrading Jenkins plugins on CI and release hosts
== 2019-09-18 ==
* 19:22 hashar: Keeping https://integration.wikimedia.org/ci/job/mediawiki-quibble-vendor-mysql-php72-docker/5393/ forever for T233143
* 18:54 James_F: Zuul: Drop extension-seccheck-non-voting and skin-seccheck-non-voting, unused
* 18:47 James_F: T231928 terminating endlessly-running attempts to build kartotherian and tilerator
* 17:19 James_F: Zuul: T231928 Migrate kartotherian to custom pipeline jobs
* 07:43 hashar: Raised integration-castor03 egress trafic shaping from 50mbps to 100mbps # T213223
* 07:36 hashar: Raised integration-castor03 egress trafic shaping from 50mbps to 100mbps # T232644
== 2019-09-17 ==
* 20:52 brennen: Updating dev-images docker-pkg files on contint1001 for new restbase image
* 18:32 James_F: jjb: Updated the rest of the quibble jobs to 0.0.35.
* 18:14 James_F: jjb: Upgrading wmf-quibble-vendor-mysql-php72-docker
* 17:56 James_F: jjb: Upgrading mediawiki-quibble-vendor-mysql-php72-docker
* 17:51 James_F: jjb: Updating parsoidsvc-parsertests-docker
* 17:49 James_F: jjb: Updating mwext-phpunit-coverage*
* 17:40 James_F: jjb: Updated wikibase-client-docker, wikibase-repo-docker
* 17:36 James_F: jjb: Updating mediawiki-fresnel-patch-docker
* 17:30 James_F: jjv: Updated mwext-php72-phan-docker, mwext-php72-phan-seccheck-docker, mwskin-php72-phan-docker, mwskin-php72-phan-seccheck-docker
* 17:29 James_F: jjb: Updated mwext-codehealth-patch and mwext-codehealth-master-non-voting
* 17:16 James_F: jjv: Updating mwcore-phpunit-coverage-master,mwcore-phpunit-coverage-patch
* 16:58 James_F: jjb: Updating mwcore-codehealth-patch experimentally
* 16:51 James_F: jjb: Experimentally updated mwext-codehealth-master-non-voting
* 16:46 James_F: jjb: Experimentally updated mwext-php72-phan-seccheck-docker-non-voting
* 16:04 hashar: Quibble 0.0.35 containers for CI have been build.
* 15:40 hashar: Building docker images for Quibble 0.0.35
* 15:32 hashar: Tagging quibble 0.0.35 # T225730 T231862 T219694
* 14:31 hashar: Raise egress filter trafic shaping on integration-castor03 from 240Mbit to 400Mbit ( T232644 / T188375 )
* 12:11 hashar: Remove Jenkins CI webproxy configuration , that breaks jobs using the git plugin!
* 11:26 hashar: Configured CI jenkins to use a webproxy for the update center due to mysterious "connection reset" when downloading plugins
* 09:24 hashar: Manually triggered https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/9920/ (switch to php7.2)
* 09:16 hashar: Rebuilding releng/doxygen container to use php7.2
* 09:15 hashar: On doxygen jobs, change container to use php7.2 instead of php7.0
* 08:19 hashar: Building docker container releng/quibble-stretch-bundle:0.0.34-2 (switch to php7.2)
* 00:29 James_F: Zuul: Dropping php70 and php71 from quibble jobs for master and REL1_34 T228342
== 2019-09-16 ==
* 23:54 James_F: Zuul: Initial CI for WebToolsManager
* 18:52 hashar: Gerrit: tweaked members of the Analytics group # T233026
* 18:51 hashar: Gerrit: tweaked members of the Analytics group # T233026
* 16:00 James_F: Zuul: Restore maps/kartotherian/package to unarchived CI-less midden
== 2019-09-14 ==
* 20:45 Krenair: shifted deployment-cache-upload05 to using unified cert from acme-chief
== 2019-09-13 ==
* 23:40 Krinkle: deployment-deploy01: cleaning disk: rm /var/cache/hhvm/cli.hhbc.sq3
* 16:19 James_F: Zuul: Add initial jobs for mediawiki/tools/api-testing T230340
* 12:39 hashar: integration-slave-docker-1043 : killed stall container # T232706
== 2019-09-12 ==
* 23:37 James_F: Zuul: Define pipelines for REL1_34 T232025
* 22:19 hashar: integration-slave-jessie-1002: rm /srv/pbuilder/build/* /srv/pbuilder/aptcache/*/*
* 22:16 hashar: Pooling back integration-slave-jessie-1002 in Jenkins ( /srv was 95% full )
* 22:15 hashar: integration-slave-jessie-1002: apt-get clean
* 21:45 James_F: Temporarily taking deployment-deploy01 offline as CI is stuck
* 20:54 James_F: Zuul: Drop operations-mw-config-hhvm-composer-test-docker, the PHP72 one suffices
* 20:31 James_F: Zuul: Add seccheck to PageNotice
* 16:36 halfak: deploying ores 
* 15:53 hauskatze: Phabricator: Switched articlequality, drafttopic, draftquality and editquality Diffusion (ORES project) to mirror to Gerrit via SSH/K19 refs. T232661
* 08:14 hashar: Marking integration-slave-docker-1043 offline , triggering mwcore-phpunit-coverage-master again # T232706
== 2019-09-11 ==
* 22:31 Krinkle: Disable "publishing" for mediawiki/skins/Vector in Phabricator Diffusion mirror - T143162
* 21:06 James_F: Zuul: Reload to add operations-mw-config-php72-composer-test-docker and operations-mw-config-hhvm-composer-lint-docker
* 14:27 hashar: puppet cert clean integration-agent-docker-1001.integration.eqiad.wmflabs # T232619 might unbreak puppet
== 2019-09-10 ==
* 11:57 hashar: Gerrit: deleting mediawiki/services/service-template-node and mediawiki/services/service-runner (unused) # T231953
* 08:03 hashar: Updated Phan jobs for T232413 https://gerrit.wikimedia.org/r/#/c/integration/config/+/535507 phan: stop installing MediaWiki
== 2019-09-09 ==
* 15:05 hashar: Reloading Zuul for https://gerrit.wikimedia.org/r/535198 Shibboleth disable selenium tests
* 12:26 hashar: Reloading Zuul to add multiple extension dependencies
* 12:15 hashar: Reloading Zuul for https://gerrit.wikimedia.org/r/535161 " FundraiserLandingPage requires EventLogging"
== 2019-09-06 ==
* 19:59 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/534872
* 19:51 hashar: Reloading zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/534862/ https://gerrit.wikimedia.org/r/#/c/integration/config/+/534863/ https://gerrit.wikimedia.org/r/#/c/integration/config/+/534864/
* 19:37 hashar: Reloading zuul for Add UniversalLanguageSelector to ExternalGuidance's dependencies
== 2019-09-05 ==
* 19:36 James_F: Zuul: Drop debian-glue-non-voting from cassandra-table-properties
== 2019-09-04 ==
* 18:48 James_F: Zuul: Move quibble and mwgate basic test jobs from php70 to php72
* 16:49 James_F: Zuul: Replacing basic quibble HHVM test with PHP72 https://gerrit.wikimedia.org/r/533973
* 16:25 hashar: Reloading Zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/534481/
* 12:20 hashar: Dropped Phan jobs for extensions from release branches T226945
== 2019-09-03 ==
* 21:59 James_F: Zuul: Move ORES and CirrusSearch to codehealth T224990
* 20:47 James_F: Zuul: Suspend trigger-mediawiki-pipeline-dev for now T218360
* 15:57 James_F: Zuul: Enable the publish pipeline for mediawiki/services/kartotherian T223275
* 15:36 James_F: Zuul: Add phan dependencies for MachineVision
* 14:34 James_F: Zuul: And wmf/ branches for VisualEditor core.
* 14:21 James_F: Zuul: Drop EducationProgram from dependencies
* 09:17 hashar: Reloaded Zuul for https://gerrit.wikimedia.org/r/533916 "Apply REL based pipelines"
* 09:16 hashar: Deploy refactor of Zuul pipelines which might mean that some repos/branches would miss jobs or have extra unwanted jobs. In such case please fill in a task against #continuous-integration-config
* 08:32 hashar: reloading zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/533905/ "Remove wmf pipelines from non mediawiki repos"
* 08:32 hashar: err
* 08:32 hashar: reloading zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/533909/ "zuul: add pipelines for MediaWiki releases"
== 2019-09-02 ==
* 17:08 hashar: Reloading Zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/533885/ zuul: introduce test-wmf pipeline
* 17:05 hashar: Reloading Zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/533880/ "zuul: rename gate-and-submit-swat to use -wmf"
* 08:30 hashar: Reloading Zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/533536/ "zuul: drop skip-if for fundraising/REL branch"
* 08:22 hashar: Reloading Zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/533532/ "zuul: pipelines for fundraising branches"
== 2019-08-31 ==
* 16:20 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/533628/
* 12:28 hauskatze: gerrit: created https://gerrit.wikimedia.org/r/#/admin/projects/operations/debs/prometheus-swagger-exporter T231696
* 12:22 hauskatze: gerrit: created https://gerrit.wikimedia.org/r/#/admin/projects/operations/debs/prometheus-swagger-exporter T 231696
* 12:18 hauskatze: github: created https://github.com/wikimedia/operations-debs-prometheus-swagger-exporter T231696
== 2019-08-30 ==
* 18:28 James_F: Zuul: Move Popups and MobileFrontend to codehealth
* 15:40 Krinkle: Updated "Prod Error" task form to have Title at the top and Trace below Description.
* 10:53 hashar: deployment-mediawiki-07: nuke /mnt/mediawiki , last touched in April 2018 and using space on /
== 2019-08-29 ==
* 23:23 James_F: Zuul: Move all archived repos to their own section at the end (no-op)
* 18:38 James_F: Zuul: Adding Wikibase back into dependencies for MachineVision
* 15:47 James_F: Zuul: Drop Wikibase dependency from MachineVision
* 02:32 Krinkle: Delete erroneous branch 'REL1_34-5' () from mediawiki/extensions/SoftRedirector. ref https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/SoftRedirector/+/532274/
== 2019-08-28 ==
* 23:00 James_F: Zuul: add tests for the labs/tools/maintain-kubeusers repo T228499
* 23:00 James_F: Zuul: [TimedMediaHandler] Run phan job T224766 (merged but not deployed?)
* 21:08 thcipriani: restarting zuul server since it's out-of-sync with reality
* 16:11 hashar: Upgraded Jenkins for security update (2.176.3)
* 16:08 hashar: integration: docker container prune -f ; docker image prune -f
* 00:07 James_F: JJB: Delete mwext-npm-doc-publish, no longer used
* 00:07 James_F: Zuul: Migrate MobileFrontend to extension-javascript-documentation T230831
== 2019-08-27 ==
* 20:25 James_F: Zuul: TwnMainPage: switch to extension-quibble-composer-nohhvm T231289
== 2019-08-26 ==
* 22:31 James_F: Zuul: Move Popups to extension-javascript-documentation
* 22:12 James_F: Zuul: Move MinervaNeue to extension-javascript-documentation
* 18:23 James_F: Zuul: [MinervaNeue, MobileFrontend, Popups] Add extension-javascript-documentation template
* 15:35 hashar: deployment-mwmaint01: /var/cache/hhvm/cli.hhbc.sq3 (1.3GB) - T161598
* 13:51 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/532379
== 2019-08-25 ==
* 21:27 Reedy: centralauth.globalblocks `ALTER TABLE /*_*/globalblocks MODIFY gb_reason varbinary(767) NOT NULL;
* 16:43 Urbanecm: Revert 'Revert locally on beta cluster to test possible root cause of T231162'
* 16:33 Urbanecm: Revert locally on beta cluster to test possible root cause of T231162
== 2019-08-23 ==
* 11:12 Urbanecm: deployment-prep: run scap sync to ensure everything is deployed (T231058)
* 07:12 rxy: Applied SQL queries per phab:T231058#5433197
== 2019-08-21 ==
* 15:49 James_F: Zuul: Add phan for LiquidThreads
* 08:41 hashar: Build docker-registry.discovery.wmnet/releng/quibble-coverage:0.0.34-4 # T230423
== 2019-08-20 ==
* 15:55 James_F: Zuul: Update Shreyas Minocha's whitelist e-mail
* 15:45 urandom: creating Parsoid/PHP storage schema in deployment-prep -- T230792
* 09:39 James_F: Zuul: [kartotherian] Disable pipeline on gate as it doesn't pass T223275
== 2019-08-19 ==
* 18:43 thcipriani: git -C /srv/mediawiki-staging/php-master/extensions/ConfirmEdit checkout maintenance/GenerateFancyCaptchas.php
== 2019-08-15 ==
* 12:13 James_F: Zuul: Make GrowthExperiments depend on Echo for non-Phan too.
* 10:50 James_F: Zuul: Make GrowthExperiments depend on Echo
* 09:05 James_F: Zuul: Kartotherian pipeline enablement T223275
== 2019-08-14 ==
* 18:40 hauskatze: github: Created https://github.com/wikimedia/operations-debs-prometheus-ipsec-exporter && forced replication to mirror changes # T230443
* 18:29 hauskatze: Created new gerrit project: `operations/debs/prometheus-ipsec-exporter.git` # T230443
* 14:42 James_F: Move all BlueSpice extensions to not run HHVM T230482
== 2019-08-13 ==
* 22:56 thcipriani: Reloading zuul to deploy https://gerrit.wikimedia.org/r/530012
== 2019-08-08 ==
* 17:14 thcipriani: updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/529108
* 03:18 thcipriani: cancelled stuck beta-scap-eqiad job to unblock postmerge
== 2019-08-07 ==
* 16:00 thcipriani: restarting jenkins for updates
== 2019-08-06 ==
* 21:24 James_F: Zuul: [TheWikipediaLibrary] Enable plan
* 21:05 James_F: Zuul: [TheWikipediaLibrary] Enable phan-seccheck
* 21:01 James_F: Zuul: [TheWikipediaLibrary] Add phan dependencies
* 19:26 paladox: ssh -p 29418 gerrit.wikimedia.org replication start --url github mediawiki/core --wait --now
* 17:09 accraze: deploying ores 
* 11:42 paladox: ssh -p 29418 gerrit.wikimedia.org replication start operations/puppet --wait
== 2019-08-05 ==
* 22:24 accraze: deploying ores 
* 19:22 James_F: Zuul: [mediawiki/services/chromium-render] Enable pipeline testing and publishing T217114
== 2019-08-04 ==
* 12:11 hauskatze: gerrit: ssh -p 29418 gerrit.wikimedia.org replication start mediawiki/extensions/OATHAuth --wait # T229756
== 2019-08-02 ==
* 17:30 James_F: Zuul: [operations/mediawiki-config] Stop running php70 jobs"
== 2019-08-01 ==
* 20:19 Krinkle: Resolve Jenkins admin warning about legacy token by revoking it from the only user id "<unknown>" that seems to have one
* 19:00 James_F: Zuul: [BlueSpiceExtendedSearch] Disable HHVM tests, this doesn't work there
== 2019-07-31 ==
* 19:20 Amir1: ladsgroup@deployment-deploy01:/srv/mediawiki-staging/php-master$ mwscript extensions/Wikibase/repo/maintenance/rebuildItemTerms.php --wiki=wikidatawiki (T225056)
* 18:27 James_F: Zuul: Move three repos to generic tox-docker job
* 17:50 James_F: Zuul: Move cassandra-table-properties to use cassandra-table-properties-tox-docker
* 16:27 James_F: Zuul: Archive SemanticTitle T229422
* 15:55 James_F: Docker: Pushing operations-puppet:0.5.4
* 15:46 James_F: Docker: Pushing quibble-coverage:0.0.34-3
* 14:21 andrewbogott: moving deployment-sca02 to cloudvirt1030
* 12:59 andrewbogott: moving deployment-elastic05, deployment-kafka-main-2, deployment-mx02, deployment-webperf11 to new cloudvirts
== 2019-07-30 ==
* 21:17 James_F: Docker: Push operations-puppet:0.5.3
* 16:17 James_F: Docker: Pushing quibble-coverage:0.0.34-2
* 15:39 cdanis: manually set dbctl etcd key to dummy value: cdanis@deployment-etcd-01:~$ etcdctl -u root -C http://127.0.0.1:2378 set /conftool/v1/mediawiki-config/eqiad/dbconfig
* 14:44 James_F: Docker: Pushing quibble-coverage:0.0.34-1
== 2019-07-29 ==
* 18:23 James_F: Zuul: [translatewiki] T228376 Make PHP72 job voting, drop HHVM one
* 18:13 James_F: Zuul: [translatewiki] T228376 Provide experimental PHP72 job to replace HHVM one
== 2019-07-27 ==
* 20:38 hauskatze: github: deleted https://github.com/wikimedia/mediawiki-extensions-SemanticHighcharts refs. T199421
== 2019-07-26 ==
* 21:10 legoktm: deploying https://gerrit.wikimedia.org/r/525871
* 18:38 James_F: Zuul: operations/software/cassandra-table-properties initial jobs
* 16:02 hauskatze: Deleted https://github.com/wikimedia/mediawiki-extensions-Copyvio refs. T228087
* 16:00 James_F: Zuul: [Copyvio] Mark as archived for CI T228087
* 15:49 Krinkle: Fix Jenkins Console output section "Selenium" to account for Quibble 0.34 changes (multiple Selenium sections, one implicitly closing the other)
* 15:45 James_F: doc1001: sudo -u doc-uploader rm -rf /srv/docroot/org/wikimedia/doc/cover-extensions/Copyvio/ # T228087
* 04:43 James_F: Zuul: [mediawiki/libs/LangConv] Make this PHP7.2+ only
* 04:36 James_F: Zuul: Add testing for mediawiki/libs/LangConv
== 2019-07-25 ==
* 23:12 hashar: Purging all docker images on Jenkins instances: sudo cumin --trace --force 'name:docker' 'docker images awk "{ print \$3}"xargs docker rmi'
* 22:52 hashar: Upgrading Jenkins jobs to Quibble 0.0.34 for https://gerrit.wikimedia.org/r/525673
* 22:30 hashar: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/525673
* 21:28 hashar: Upgrading Jenkins jobs to Quibble 0.0.34 (second attempt) for T229020 using https://gerrit.wikimedia.org/r/#/c/integration/config/+/525570/
* 21:00 James_F: Zuul: Add phan to EventBus
* 18:17 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/525608 - T227004)
* 17:38 greg-g: Added Jforrester to the Release Engineering group in Gerrit
* 17:17 hashar: Upgrading Jenkins jobs to Quibble 0.0.34 for T229020
* 16:56 hashar: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/525591
* 16:54 hashar: Tagged Quibble 0.0.34 # T229020
* 16:41 James_F: Zuul: [analytics-wikistats2] Drop node 6, switch to node10-browser-docker T211784
* 16:20 hashar: Rolling back Quibble based jobs to 0.0.32 :-(
* 15:38 hashar: Upgrading Jenkins jobs to Quibble 0.0.33
* 15:09 hashar: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/525559
* 14:52 hashar: Tagged Quibble 0.0.33 T193824 T87781 T199116
== 2019-07-24 ==
* 20:00 James_F: Zuul: Mark VIKI, VikiSemanticTitle, and VikiTitleIcon as broken.
* 10:55 hauskatze: Dry-running extensions/AbuseFilter/maintenance/fixOldLogEntries.php refs. T228655
* 10:54 hauskatze: deployment-prep Dry-running extensions/AbuseFilter/maintenance/fixOldLogEntries.php refs. T228655
== 2019-07-23 ==
* 17:01 James_F: Zuul: [trigger-mediawiki-pipeline-dev] Only trigger on master
* 16:00 hashar: gerrit: adding Jakob_WMDE to wmf-deployment group ( https://gerrit.wikimedia.org/r/#/admin/groups/21,members ) # T227193
* 08:00 hashar: Restore puppet class contint::packages::javascript on permanent slave ( reverting https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/524221/ ) # T228639
== 2019-07-22 ==
* 22:43 James_F: Zuul: [wikimedia/iegreview] Adding mediawiki-i18n-check-docker template
* 10:49 Amir1: ladsgroup@deployment-deploy01:/srv/mediawiki-staging/php-master/extensions$ sudo rm -rf Wikidata
== 2019-07-19 ==
* 19:11 brennen: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/524565
== 2019-07-18 ==
* 22:18 James_F: Zuul: Archive maps/kartotherian and maps/tilerator
* 21:17 mutante: rebooting gerrit server
* 19:18 James_F: Docker: Creating docker-registry.discovery.wmnet/releng/node10-kartotherian:0.1.1
* 19:03 James_F: Zuul: [analytics-mediawiki-storage] Drop node 6, switch to node10-browser-docker
* 18:25 mutante: contint2001 - puppet run reinstalled python-dateutil and ruby-safe-yeaml, stopped zuul service but no more failures now
* 18:25 mutante: contint2001 /etc/init.d/zuul-merger stop; apt install zuul (this time it worked to configure zuul after also stopping zuul-merge) ; apt-get autoremove (removed unused python packages); puppet agent -tv
* 18:22 mutante: contint2001 /usr/sbin/dpkg-reconfigure: zuul is broken or not fully installed
* 18:22 mutante: contint2001 systemctl stop zuul ; apt install zuul ... still fails
* 18:10 James_F: Zuul: [wikimedia/services/kartotherian] Switch to custom job T228170
* 18:06 James_F: Docker: Creating docker-registry.discovery.wmnet/releng/node10-kartotherian:0.1.0
* 13:51 hashar: integration: in hiera project config, remove classes: [ contint::packages::apt ] T225735
* 13:48 hashar: integration: cherry picked https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/523150/ for apt unattended_upgrades T225735
* 13:37 hashar: integration: rm /etc/apt/sources.list.d/sury-php.list and remove php packages https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/523148/ T225735
== 2019-07-17 ==
* 16:49 James_F: Zuul: [PageForms] Add phan dependencies on AdminLinks, Cargo, PageSchemas T228155
* 16:48 James_F: Zuul: Register initial tests for eventgate-wikimedia T226668
* 16:08 hashar: Jenkins: updating wikidata-query-gui-build to get rid of slave-scripts/bin/npm-setup.sh which requires an obsolete/non existing tmpfs # T227818
* 09:48 hashar: Reloaded Zuul [Convert2Wiki] Archive extension
* 00:42 James_F: Docker: Deploy ci-buster and lintr:0.2.0
* 00:04 James_F: Docker: Complete rebuild finished and published; only took 2.5 hours. But now T228196 is fixed elsewise anyway, oh well.
== 2019-07-16 ==
* 21:58 hauskatze: GitHub: deleted `wikimedia/mediawiki-extensions-Convert2Wiki` refs. T228198
* 21:29 James_F: Docker: Publishing a whole new set of RelEng images for T228196
* 20:25 James_F: Docker: Running a general rebuild for all missing RelEng images T228196
* 16:49 James_F: Zuul: [ContentTranslation] Adding CentralAuth to phan dependencies
* 16:48 James_F: Zuul: Adding first tasks for mediawiki/services/kartotherian T228170
* 15:19 James_F: Ran manual scap pull on deployment-deploy01
* 15:10 James_F: beta-scap-eqiad failing because mw-config repo is out of date?
* 14:57 James_F: Taking deployment-deploy01 offline
* 14:50 James_F: Jenkins deadlock? Beta jenkins jobs haven't successfully run for ~20 hours.
== 2019-07-15 ==
* 22:40 James_F: Zuul: [CentralAuth] Add phan T189371
* 18:05 James_F: Zuul: [MinervaNeue] Make phan voting T133664
* 17:00 James_F: Zuul: [GrowthExperiments] Add MobileFrontend to phan dependencies
* 16:49 James_F: Zuul: [MinervaNeue] Add phan as non-voting T133664
* 16:04 Urbanecm: Run urbanecm@deployment-deploy01:~$ mwscript migrateUserGroup.php --wiki=commonswiki Image-reviewer image-reviewer (T216406)
* 15:59 James_F: Zuul: Adding Echo and MobileFrontend dependencies to skins/MinervaNeue T133664
* 15:38 James_F: Zuul: [SecurePoll] Make seccheck voting T205972
* 15:37 James_F: Zuul: [CentralAuth] Add seccheck T189227
* 13:30 hashar: Deleting compiler1003.puppet-diffs.eqiad.wmflabs .. insanely slow for some reason (was on cloudvirt1002) # T221969 )
* 12:55 hashar: Creating compiler1003.puppet-diffs.eqiad.wmflabs [172.16.2.46] m1.large / 4vCPUs # T221969
== 2019-07-14 ==
* 00:55 Krinkle: krinkle@doc1001: Remove https://doc.wikimedia.org/cover-extensions/$DOC_BASENAME/ - a failed build for GrowthExperiments
== 2019-07-12 ==
* 23:07 James_F: Zuul: Instead of hacking around for WebAuthn, define bespoke quibble php72+ template
* 22:10 James_F: Zuul: Make seccheck voting on WebAuthn T226737
* 19:28 James_F: Zuul: Add MobileFrontend and MinervaNeue to Wikibase's dependencies
* 16:56 James_F: Zuul: Add Flow as phan dependency to GrowthExperiments
* 00:32 James_F: Zuul: Migrate from php70-phan-seccheck to php72-phan-seccheck T226420
== 2019-07-11 ==
* 23:33 James_F: Pushing docker mediawiki-phan-seccheck:0.4.0
* 21:34 James_F: Pushing new php-based docker images with php-bcmath extension for T227043
* 20:30 mutante: add project member cdanis
* 18:57 Krinkle: Fix "setup quibble prepare" Console Section pattern in Jenkins to account for changes in 'mediawiki-quibble-composertest-php72-docker' job
* 18:22 James_F: Zuul: Archive ZeroBanner and ZeroPortal; drop from dependencies
* 17:50 James_F: jforrester@doc1001:/srv/docroot/org/wikimedia/doc/cover-extensions$ rm -rf WikibaseQuality/ # T205065
* 16:15 James_F: Zuul: Archiving WikibaseQuality
* 09:48 hashar: jenkins: add more log details to hudson.model.Queue (FINER > FINEST) https://integration.wikimedia.org/ci/log/Jenkins%20Queue/configure # T72597
* 09:06 hashar: beta cluster jobs are dead locked. Taking a thread dump in case it helps figure out what is going on. T72597
== 2019-07-10 ==
* 21:33 James_F: Zuul: Fix gerrit-docker pointer
* 18:27 James_F: Zuul: Adding phan to StructuredDiscussions T224759
* 17:55 James_F: Zuul: Disabling mwgate-rake for mediawiki/extensions/CentralNotice
* 13:30 hashar: on zuul-merger: rm /srv/zuul/git/operations/software/gerrit due to T157818 # T189549
== 2019-07-09 ==
* 18:47 James_F: Zuul: Move mediawiki-quibble-composertest from php70 to php72
* 18:18 James_F: Zuul: [mediawiki/tools/phan/SecurityCheckPlugin] Move back to php72 as primary
* 18:08 James_F: Push docker composer-package-php72:0.2.2-s2
* 17:21 James_F: Push docker mediawiki-phan-seccheck:0.3.6-2
* 16:18 James_F: Push docker image +mediawiki-phan-seccheck:0.3.6-1
* 15:53 James_F: Pushing docker images php-ast:1.0.1-2 and mediawiki-phan-seccheck:0.3.6
* 15:29 thcipriani: contint{1,2}001: rm -rf /srv/zuul/git/mediawiki/services/restbase due to T157818
* 15:03 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/520437
* 14:08 James_F: Pushing mediawiki-phan-seccheck:0.3.6 docker image.
* 09:48 hashar: contint1001: removing /var/lib/docker , no more needed since we now use /mnt/docker # T207707
* 00:04 James_F: Pushing php-ast:1.0.1-1 and related composer-package* docker image updates
== 2019-07-08 ==
* 23:54 James_F: Zuul: Move wikimania-scholarships from php56 to php72
* 23:35 James_F: Dropping mwext-MobileFrontend-npm-run-lint-modules-docker from Zuul T224997
* 21:39 thcipriani: reenable jenkins jobs service-pipeline-test and service-pipeline-test-and-publish following docker mount move
* 21:27 thcipriani: docker-pkg failed building docker-registry.discovery.wmnet/releng/composer-php56:0.1.5
* 21:18 thcipriani: fab deploy_docker to pull down ci docker-pkg images (I hope)
* 21:15 thcipriani: restart docker on contint1001 to pickup config change for https://gerrit.wikimedia.org/r/520738/
* 21:14 thcipriani: disabled jenkins jobs service-pipeline-test and service-pipeline-test-and-publish pre-contint1001 docker restart
* 13:38 hashar: Build docker-registry.wikimedia.org/releng/bazel:0.1.0 # T189549
== 2019-07-06 ==
* 11:20 hauskatze: maurelio@deployment-deploy01:~$ mwscript extensions/AbuseFilter/maintenance/normalizeThrottleParameters.php --wiki=deploymentwiki T209565
* 11:09 hauskatze: beta cluster: maurelio@deployment-deploy01 foreachwiki extensions/AbuseFilter/maintenance/normalizeThrottleParameters.php --dry-run T209565
== 2019-07-05 ==
* 21:17 James_F: Adding node10-docker for react.ooui
* 19:08 James_F: Pushing java8-sonar-scanner:0.4.11
== 2019-07-03 ==
* 21:01 accraze: deploying ores 
* 13:12 Urbanecm: deployment-prep Manually ran sudo -u jenkins-deploy scap sync-file wmf-config/CommonSettings-labs.php 'Not synced yet?' to sync file that's mysteriously not synced, but present on deployment host
== 2019-07-02 ==
* 20:34 James_F: Zuul: Adding quibble to the PopupAlerts, UnblockMe and Scribe extensions
== 2019-07-01 ==
* 23:24 James_F: Pushing quibble-coverage:0.0.32-1 image
* 22:30 thcipriani: cleaned contint1001 docker images using https://phabricator.wikimedia.org/P8604
* 20:40 James_F: Rebuilt codehealth jjbs using node10-test with https://gerrit.wikimedia.org/r/519652
* 18:49 James_F: [SecurityCheckPlugin] Revert blocking jobs to php70; move php71+ to experimental
* 18:43 James_F: Reloading Zuul for mediawiki/tools/phan/SecurityCheckPlugin php72 tests
== 2019-06-30 ==
* 17:03 Reedy: reloading zuul to deploy https://gerrit.wikimedia.org/r/519793
== 2019-06-28 ==
* 08:58 godog: cherry pick https://gerrit.wikimedia.org/r/c/operations/puppet/+/519603
* 00:07 James_F: Restoring php-gmp in quibble docker images
== 2019-06-27 ==
* 21:27 James_F: Pushing new docker php base images with gmp, and all their descendants
* 17:39 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/519455
* 17:04 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/519454
* 15:29 legoktm: building composer-package-*, php-ast images for https://gerrit.wikimedia.org/r/519424 (try #2)
* 15:22 legoktm: building composer-package-*, php-ast images for https://gerrit.wikimedia.org/r/519424
== 2019-06-26 ==
* 17:57 James_F: Adding the experimental helm-lint job to deployment-charts T216049
* 17:55 James_F: Pushing initial helm-linter docker image for T216049
* 01:29 James_F: Moving labs/tools/stewardbots's composer test to php72+ T226585
== 2019-06-25 ==
* 22:00 hauskatze: Ran `replication start operations/debs/coredns --wait`
* 21:07 hauskatze: gerrit: Created new repo: operations/debs/coredns - requested by fsero on mediawiki
* 19:11 Krinkle: Update Jenkins collapsible section patterns to account for "quibble.cmd" => "quibble.commands" changing in the output
* 15:58 James_F: Adding CentralAuth to FileImporter required extensions in Zuul
* 15:51 awight: restart php7.2-fpm for wikidiff2 upgrade (T223391)
* 15:50 awight: deployment-prep restart php7.2-fpm for wikidiff2 upgrade (T223391)
* 14:39 hauskatze: Ran `replication start operations/debs/file-read-backwards --wait` on gerrit.wikimedia.org refs. T226449
* 14:34 hauskatze: github: created https://github.com/wikimedia/operations-debs-file-read-backwards mirror refs. T226449
* 14:14 hauskatze: gerrit: created `operations/debs/file-read-backwards.git` refs. T226449
* 11:42 hashar: Deleting Jenkins jobs puppet-cdh-rake-docker and puppet-cdh4-rake-docker # T226474
== 2019-06-24 ==
* 23:31 thcipriani: cleaned contint1001 docker images using https://phabricator.wikimedia.org/P8604
* 22:08 James_F: Zuul: Restore graphoid deploy repo tests, and comment on change-propagation ones.
* 22:02 James_F: Pushed graphoid-deploy-npm-node-6-docker into jjb
* 22:01 James_F: Re-publishing npm-test-graphoid, whoops.
* 15:17 James_F: Reloading Zuul to drop wikiba.se-composer-php56-docker T224905
* 13:04 hashar: Updating git mirrored repos: integration-cumin:~$ sudo cumin --trace --force 'name:docker' 'find /srv/git -name "*.git" -exec git -C {} fetch --prune \;'
* 12:00 hashar: Updating Quibble based jobs to use version 0.0.32
* 10:09 hashar: Tag Quibble 0.0.32 @ # T211701 T218357 T220199 T223752
* 09:50 hashar: Building container releng/zuul-cloner:0.2.0 and switching jobs # T224908
* 08:25 hashar: Removing mysql-server from the CI slaves # https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/517095/ T225735
== 2019-06-22 ==
* 11:36 hauskatze: github: deleted `wikimedia/mediawiki-extensions-FirstSteps` GitHub mirror refs. T226149
* 00:31 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/518344
== 2019-06-21 ==
* 20:52 Krinkle: Add "post-build scripts" Jenkins Console section to collapse most unrelated output and cascading failures from artefact readers etc. which are confusing when looking for "the" error
* 20:48 Krinkle: Fix Jenkins Console Section pattern for "setup quibble mw-install" to not eat all of output from phan (differentiate from phpunit, composer-test and phan). regexcomplexity++
* 16:58 James_F: Updated all four codehealth jjb jobs to use java8-sonar-scanner 0.4.10
* 16:55 James_F: Docker: Publishing java8-sonar-scanner 0.4.10.
* 06:37 legoktm: deployed https://gerrit.wikimedia.org/r/518205 https://gerrit.wikimedia.org/r/518206
* 06:34 legoktm: deployed https://gerrit.wikimedia.org/r/518204
* 06:04 legoktm: deleted remexhtml-phpunit-coverage-docker-publish jenkins job
* 05:51 legoktm: deployed https://gerrit.wikimedia.org/r/518195 https://gerrit.wikimedia.org/r/518196
== 2019-06-20 ==
* 22:03 James_F: Pushing new java8-sonar-scanner:0.4.9 image.
* 21:29 James_F: Pushing new java8-sonar-scanner:0.4.8 image
* 20:06 James_F: Add new mwcore-codehealth-* config to Zuul
* 20:04 James_F: Pushed new jjb mwcore-codehealth-* jobs and updated mwext-codehealth-* ones
* 19:54 James_F: Pushing new java8-sonar-scanner:0.4.7 image
* 17:45 James_F: Updated JJB definitions for mwext-php7\d-phan(-seccheck)?-docker for new docker-setup-mwext-for-phan macro.
* 17:35 James_F: Updated JJB definitions for wikibase-{clientrepo}-docker to use php72 not php70.
* 17:12 James_F: Updated JJB definitions for all six coverage jobs
* 15:09 hashar: Nuking castor caches for Quibble jobs integration-castor03 : rm -fR /srv/jenkins-workspace/caches/castor-mw-ext-and-skins/master/*quibble*/{composer,npm}
* 14:44 hashar: Reloading Zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/518033/
* 14:16 hashar: building some container for codehealth pipeline https://gerrit.wikimedia.org/r/518034
== 2019-06-19 ==
* 08:58 hashar: Removed Debian packages for python / tox etc from all slaves T225735 https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/517092/
* 08:46 hashar: Removed Debian package colordiff from all slaves T225735 https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/517091/
* 03:43 legoktm: deploying https://gerrit.wikimedia.org/r/517785 https://gerrit.wikimedia.org/r/517786
== 2019-06-18 ==
* 23:19 James_F: Making extension-seccheck non-voting on all 195 repos, hopefully temporarily. T226064
* 22:14 Reedy: Apply https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/OATHAuth/+/496451/53/sql/mysql/patch-add_generic_fields.sql to centralauth on beta T218210
* 21:59 James_F: Manually pruned some old quibble-coverage Docker images from contint1001.
* 21:51 James_F: Updating docker-pkg files on contint1001 to fix quibble-stretch-php72
* 21:43 James_F: Zuul: Adding php72 variants of phan-seccheck jobs as experimental
* 21:35 James_F: Pushing docker image quibble-stretch-php72:0.3.3
* 20:40 James_F: Reload Zuul to point MediaWiki's phan jobs at php72 instead of php70 T225456
* 20:24 hashar: Switching jobs to SRE php7.2 package # T226028 https://gerrit.wikimedia.org/r/#/c/integration/config/+/517733/
* 20:24 hashar: Switching jobs to SRE php7.2 package # T226028
* 20:22 James_F: Reload Zuul to point Parsoid's experimental phan job at php72 instead of php70 T225456
* 20:12 James_F: Reload Zuul to point extension-phan and skin-phan templates at php72 instead of php70 T225456
* 19:51 hashar: building containers for php7.2 from SRE # T226028 https://gerrit.wikimedia.org/r/#/c/integration/config/+/517684/
* 19:38 James_F: Bumped jjb mwext-codehealth-*, mediawiki-*coverage*docker*, mwext*coverage*docker* jobs to use quibble-coverage:0.0.31-4
* 19:28 James_F: Pushing quibble-coverage:0.0.31-4 docker image.
* 17:02 James_F: Updated jjb jobs for mwext-codehealth-patch and mwext-codehealth-master-non-voting.
* 16:54 James_F: Pushing new quibble-coverage docker image
* 11:32 hauskatze: github: deleted https://github.com/wikimedia/mediawiki-extensions-PureWikiDeletion T225991
== 2019-06-15 ==
* 00:17 Krinkle: Disable "Publish/Notify" for mediawiki/libs/ObjectFactory.git in Diffusion – T222409, T143162
== 2019-06-14 ==
* 22:15 James_F: Dropped jjb jobs trending-edits?(-deploy)-npm-node-6-docker
* 22:14 James_F: Reloading Zuul to archive the trending-edits repos
* 21:50 James_F: Dropped docker-registry.discovery.wmnet/releng/hhvm-jessie:latest, unused
* 19:57 James_F: Ran `docker rmi docker-registry.discovery.wmnet/releng/quibble-fresnel:0.0.31-4` on contint1001; unused.
* 19:41 James_F: Zuul: Deploying Codehealth: Add Translate extension to pipeline
* 19:39 James_F: Zuul: Deploying Remove coverage ci step for extensions with codehealth
* 19:35 James_F: Zuul: Deploying Add sonar CodeHealth checks to Wikibase repos
* 18:44 James_F: Publishing image docker-registry.discovery.wmnet/releng/quibble-fresnel:0.0.31-5
* 16:58 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/517087
* 16:08 James_F: Updating docker-pkg to provide quibble-coverage on php7.2
* 16:04 James_F: Dropped docker-registry.discovery.wmnet/releng/quibble-stretch-php70:0.0.31-1 from contint1001, unused
* 15:50 James_F: Bumped jjb mediawiki-phpunit-coverage-patch-docker
== 2019-06-13 ==
* 22:30 James_F: Deploying quibble-stretch-php70 update.
* 22:17 James_F: Deploying quibble-coverage fix with chmod 0755 files.
* 21:49 James_F: Reverted JJB jobs mediawiki-core-code-coverage-docker and mediawiki-phpunit-coverage-patch-docker back to quibble-stretch-php70
* 21:43 James_F: Updated JJB job mediawiki-phpunit-coverage-patch-docker to point to quibble-coverage
* 21:42 James_F: Update JJB job mediawiki-core-code-coverage-docker to point to quibble-coverage
* 21:10 brennen: restarted jenkins agent on integration-trigger-01 to pick up new root configuration
* 20:54 hashar: ¡Learning to log one step at a time!
* 20:54 James_F: Updating docker-pkg files to create quibble-coverage image
* 20:40 hashar: deleting docker-registry.wikimedia.org/releng/hhvm-jessie-compile from contint1001 # T224908
* 18:17 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/514016
* 17:05 brennen: changing jenkins remote root for integration-trigger-01 to /mnt/home/jenkins-deploy per T224069
* 16:07 James_F: Dropped docker-registry.discovery.wmnet/releng/npm-test-mathoid:latest, unused
* 16:06 James_F: Dropped mathoid?(-deploy)-npm-node-6-docker from jjb
* 16:01 James_F: Dropped docker-registry.discovery.wmnet/releng/npm6:latest from contint1001, unused
* 15:33 Krinkle: Disable "Publish/Notify" for quibble.git in diffusion, T143162, T196347
* 00:13 James_F: Dropped jjb jobs for (change-propagationcitoidcxservergraphoid)?(-deploy)-npm-node-6-docker
== 2019-06-12 ==
* 23:34 brennen: deleting and rebuilding integration-trigger-01 (again) to use jessie instead of stretch
* 23:20 James_F: Zuul: Disable service-pipeline-test from 3d2png, chromium-render, and trending-edits for now
* 22:44 James_F: Enabling service-pipeline-test on four services: 3d2png, chromium-render, recommendation-api, trending-edits
* 22:40 James_F: Dropped jjb jobs wikimedia-cz-tracker-npm-node-6-docker fundraising-dash-npm-node-6-docker wikimedia-portals-npm-browser-node-6-docker wikimedia-portals-deploy-npm-browser-node-6-docker
* 22:38 brennen: deleting and re-creating integration-trigger-01
* 22:22 James_F: Dropped mapdata-npm-node-6-docker and marvin-npm-node-6-docker from jjb
* 22:21 James_F: Reloading Zuul to make mpadata use generic-node10-browser-docker
* 21:54 James_F: Reloading Zuul to deploy, umm, 12 patches. 🤷🏽‍♂️
* 21:07 James_F: Updating Zuul with https://gerrit.wikimedia.org/r/516674 https://gerrit.wikimedia.org/r/516675 https://gerrit.wikimedia.org/r/516676
* 18:35 James_F: Reloaded Zuul to use node10 things for OOUI experimentally
* 18:32 James_F: Deployed new jjb job ooui-npm-special-node10-plus-composer-docker
* 16:28 Krinkle: Fix Jenkins Console section "setup quibble mw-install" pattern to not slurp the whole output on 'mediawiki-quibble-composertest' jobs
* 16:27 James_F: Reloading Zuul to add service-pipeline-test?(-and-publish) to mediawiki/services/mobileapps
* 00:49 James_F: Updated Zuul phan dependencies for ContentTranslation, adding GlobalPreferences
== 2019-06-11 ==
* 23:51 James_F: Updated Zuul phan dependencies for WikibaseLexeme, adding Wikibase T225524
* 23:37 James_F: Dropped unused jjb jobs mwext-jsduck-publish mwext-VisualEditor-jsduck-docker, and mwext-VisualEditor-docker-publish
* 23:34 James_F: Updating Zuul to drop extension-jsduck, no longer used (yay)
* 23:14 James_F: Reloading Zuul to have Wikibase use extension-javascript-documentation
* 22:39 James_F: PHP55 docker images deleted; needed to specify :0.… and also :latest because we're not just dropping one tag, we're dropping the double pair. All now done. T224907
* 22:34 James_F: PHP55 docker images not deleted because they're still referenced (by what?).
* 22:25 James_F: Running `docker rmi` for quibble-jessie-php55 () composer-test-php55 () composer-package-php55 () php55-compile () composer-php55 () and php55 ()
* 22:22 James_F: Updating docker-pkg files on contint1001 to drop all php55 images T224907
* 22:18 James_F: Dropped from jjb: mwgate-composer-php55-docker, composer-php55-docker, composer-package-php55-docker, php-compile-php55-docker
* 22:06 James_F: Drop unused jjb job integration-composer-check-php55-docker
* 22:06 James_F: Reloading Zuul to drop integration-composer-check-php55-docker from integration/composer
* 22:01 James_F: Reloading Zuul to add integration-composer-check-php72-docker to integration/composer
* 21:59 James_F: Push new jjb job integration-composer-check-php72-docker
* 21:56 James_F: Reloading Zuul to drop all *php55* jobs except integration-composer
* 21:49 James_F: Reloading Zuul to add composer-php56-docker to integration/docroot
* 21:45 James_F: Reloading Zuul to drop quibble*php55*
* 21:42 James_F: Reloading Zuul to add composer-php72-docker to labs/tools/heritage
* 21:07 brennen: creating integration-trigger-01 for T224069
* 19:34 hashar: building docker-registry.wikimedia.org/releng/java8-sonar-scanner:0.4.5 for https://gerrit.wikimedia.org/r/#/c/integration/config/+/516511/
* 19:04 halfak: deployment-prep deploying ores 
* 19:03 mdholloway: deployment-prep created deployment-wikifeeds01, added role::beta::docker_services and service config
* 18:29 halfak: deployment-prep deploying ores 
* 13:17 hashar: Building container docker-registry.wikimedia.org/releng/civicrm:0.1.0 # T210287
== 2019-06-10 ==
* 19:50 James_F: Restarting zuul to drop quibble experimental php70-phan-docker jobs; php72 suffices
* 19:39 thcipriani: restarting jenkins
* 19:37 James_F: Restarting jenkins with `sudo service restart jenkins` on contint1001 due to out-of-space issue earlier.
* 19:10 Krinkle: Fix "setup quibble mw-install" collapsible section pattern in Jenkins administration, so that it doesn't match the "PHP 7.1+" warning from Phan. The pattern is meant to detect the first line of mw/maint/install.php output
* 19:01 thcipriani: cleaning contint1001 docker images using https://phabricator.wikimedia.org/P8604
* 17:48 James_F: Reloading zuul to set zuul parameters for PHP 7.2 variants of phan jobs too
* 17:41 James_F: Reloading zuul to Add experimental mwext-php72-phan-docker jobs to quibble template
* 17:38 James_F: Pushed new mwext-php72-phan-docker with jjb
* 16:59 James_F: Deployed https://gerrit.wikimedia.org/r/516300
* 16:31 legoktm: deployed https://gerrit.wikimedia.org/r/516298
* 04:38 legoktm: deploying https://gerrit.wikimedia.org/r/516065 https://gerrit.wikimedia.org/r/516066
== 2019-06-08 ==
* 00:49 James_F: Updating docker-pkg files on contint1001 to build mediawiki-phan:0.2.0 based on php72 T223847
== 2019-06-07 ==
* 23:29 marxarelli: deploying zuul changes for https://gerrit.wikimedia.org/r/c/integration/config/+/510602
* 23:28 marxarelli: updating jenkins jobs for https://gerrit.wikimedia.org/r/c/integration/config/+/510602
* 13:30 godog: cherry pick https://gerrit.wikimedia.org/r/c/operations/puppet/+/515058
* 00:10 James_F: jjb: Migrated docker-setup-mwext-for-phan to node10 quibble
== 2019-06-06 ==
* 18:06 James_F: Reloading Zuul to drop EventBus dependency on CentralNotice for now T225195
* 15:12 hashar: reloading zuul for T222802
* 15:12 hashar: reloading Zuul for T224301
* 08:13 hashar: Reloading Zuul for which drops mwselenium-quibble-docker from Wikibase
== 2019-06-05 ==
* 19:34 andrewbogott: moving deployment-imagescaler03 to cloudvirt1029
* 17:35 James_F: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/514547
* 16:01 James_F: Reloading Zuul to deploy 
* 09:07 hashar: Pooled in integration-slave-docker-1058 and integration-slave-docker-1059
== 2019-06-04 ==
* 20:39 James_F: Updating jjb PHP code coverage jobs to quibble-stretch-php70:0.0.31-5 T220917
* 20:31 James_F: Updating docker-pkg files on contint1001 for quibble-stretch-php70 T220917
* 17:44 James_F: Updating jjb PHP code coverage jobs to node10 quibble T224983
* 17:31 James_F: Reloading Zuul to make mwext-MobileFrontend-npm-run-lint-modules-docker non-voting T224997
* 16:10 hashar: Deleting integration-slave-docker-1021 and integration-slave-docker-1049 / too small disk (20G partition) and not enough ram (2G) # T221872
* 13:24 hashar: Update all selenium-daily* jobs to use NodeJS 10 instead of NodeJS 6. T217545
* 13:01 hashar: Building docker-registry.discovery.wmnet/releng/node10-test-browser:0.6.0 # T217545
* 11:32 hashar: Upgrading Jenkins Pipeline plugins
* 11:31 hashar: Upgrading Jenkins Warnings Next Generation Plugin # T224745
* 11:30 hashar: Upgrading Jenkins BlueOcean and all its dependencies
== 2019-06-03 ==
* 19:32 James_F: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/514089
* 19:29 James_F: Generated mwgate-node10-docker and deployed via jjb.
* 18:34 hashar: hswitch most Quibble jobs to node 10 T222406 - ttps://gerrit.wikimedia.org/r/#/c/integration/config/+/514034/ T222406
* 17:40 James_F: Reloading Zuul to switch most Quibble jobs to node 10 T222406
* 15:58 hashar: Deleting integration-slave-docker-1055 and integration-slave-docker-1056 . CPU is way too slow T223971
* 15:46 hashar: reduce number of executors on all integration slave docker, they are somehow starving on CPU and/or IO when lot of mediawiki builds are running in parallel
* 14:38 James_F: hashar and I are temporarily disabling running selenium tests in CI. See T211784 T222406 for more details.
== 2019-06-01 ==
* 22:54 James_F: Reloading Zuul to re-add Kartographer dependency for JsonConfig T224785
* 21:14 James_F: Reloading Zuul to add phan for CentralNotice and add DannyS712 to whitelist
== 2019-05-31 ==
* 23:34 James_F: Reloading Zuul to remove Kartographer dependency for JsonConfig T224785
* 22:12 James_F: Reloading Zuul to add phan for LdapAuthentication
* 21:18 James_F: Reloading Zuul to add dependencies for TimedMediaHandler, ReadingLists, JsonConfig, and FlaggedRevs.
* 21:13 James_F: Reloading Zuul to add phan for Collection, ContentTranslation, and Jade
* 19:43 James_F: Reloading Zuul to add phan for Math(s)
* 19:31 James_F: Reloading Zuul to add phan for MobileFrontend
* 19:20 James_F: Reloading Zuul to add phan for Sentry
* 18:35 James_F: Reloading Zuul to deploy CentralNotice dependency to EventBus
* 18:17 James_F: Reloading Zuul to deploy extra dependencies for MobileFrontend
* 18:10 James_F: Reloading Zuul to deploy phan for cldr and GlobalPreferences
* 18:02 James_F: Reloading Zuul to deploy 
* 17:56 James_F: Reloading Zuul to deploy and 
== 2019-05-30 ==
* 23:19 James_F: Pruned releng/quibble-fresnel:0.0.31-3 from contint1001 to save space
* 23:02 James_F: Updating docker-pkg files on contint1001 for +quibble-fresnel (0.0.31-4)
* 21:33 Krinkle: Jenkins admin says ""Your Jenkins data directory /var/lib/jenkins (AKA JENKINS_HOME) is almost full. You should act on it before it gets completely full.""
* 21:27 greg-g: back in business (ugh)
* 21:21 greg-g: doing the zuul deadlock dance again
* 20:51 legoktm: deploying https://gerrit.wikimedia.org/r/513168
* 02:48 legoktm: deploying https://gerrit.wikimedia.org/r/512474 https://gerrit.wikimedia.org/r/512507 https://gerrit.wikimedia.org/r/513153 https://gerrit.wikimedia.org/r/513157 https://gerrit.wikimedia.org/r/512729 https://gerrit.wikimedia.org/r/513206 https://gerrit.wikimedia.org/r/513214 https://gerrit.wikimedia.org/r/513217 https://gerrit.wikimedia.org/r/513218 https://gerrit.wikimedia.org/r/513211
* 02:35 legoktm: deployed https://gerrit.wikimedia.org/r/513129 https://gerrit.wikimedia.org/r/513071
== 2019-05-29 ==
* 21:46 greg-g: doing the executor deadlock dance: https://www.mediawiki.org/wiki/Continuous_integration/Zuul#Jenkins_execution_lock
== 2019-05-28 ==
* 12:43 paladox: delete extensions/3D repo in gerrit - T224463
* 01:35 legoktm: deployed https://gerrit.wikimedia.org/r/512794
== 2019-05-26 ==
* 18:50 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/512544
== 2019-05-22 ==
* 07:21 hashar: Updating Jenkins job to have castor use rsync --delay-updates # T203506 https://gerrit.wikimedia.org/r/#/c/integration/config/+/511672/
== 2019-05-21 ==
* 22:05 marxarelli: generating new blubber-pipeline-* and trigger-blubber-pipeline-* jenkins jobs defined by https://gerrit.wikimedia.org/r/c/integration/config/+/510602
* 11:23 hashar: Depooling integration-slave-docker-1055 and integration-slave-docker-1056 : CPU is too slow # T223971
* 08:24 hashar: Updated phan jobs to no more install mediawiki development dependencies (potentially conflicting with the extensions code) https://gerrit.wikimedia.org/r/#/c/integration/config/+/511447/ # T223397
* 07:29 legoktm: deployed https://gerrit.wikimedia.org/r/511627
* 07:10 legoktm: manually rebuilding mediawiki-core-code-coverage-docker with 5 hour timeout
== 2019-05-20 ==
* 13:11 hashar: updating phan jobs to use docker-registry.wikimedia.org/releng/mediawiki-phan:0.1.15 # T219114
* 01:43 legoktm: deploying https://gerrit.wikimedia.org/r/511361
== 2019-05-19 ==
* 15:43 hashar: Purging webperformance.integration.eqiad.wmflabs /tmp directory (inodes full)
* 15:43 hashar: Purging all images on Docker CI slaves
* 08:03 legoktm: deployed https://gerrit.wikimedia.org/r/511122
== 2019-05-18 ==
* 18:14 Amir1: cherry-picking 511078 on puppetmaster
* 12:07 Reedy: reload zuul to deploy https://gerrit.wikimedia.org/r/510967
== 2019-05-17 ==
* 22:55 paladox: created mediawiki/extensions/Scribe gerrit repo - T223662
* 00:35 brennen: Updating dev-images docker-pkg files on contint1001
== 2019-05-16 ==
* 09:36 hashar: Successfully tagged docker-registry.discovery.wmnet/releng/composer-test-php72:0.1.0 # T223428
* 07:35 hashar: integration-slave-jessie-1002: purging all php Debian packages and rerunning puppet. There is some package conflict somewhere :-(
== 2019-05-15 ==
* 11:55 hashar: Bring back https://integration.wikimedia.org/ci/computer/integration-castor03/ to restore the central cache behavior. In turn unblocking a wide range of builds
* 11:32 hashar: bringing back castor, integration-castor03 is back
* 09:35 hashar: Regenerating all CI jobs to disable castor saving entirely
* 02:35 Krenair: Logged into deployment-sca0[12] as root and given them the correct nameservers to try to unbreak things. T221654
* 01:00 thcipriani: /usr/local/sbin/keyholder arm
== 2019-05-14 ==
* 09:43 hashar: Upgraded all tox jobs to tox 3.10.0
* 09:19 hashar: Building docker containers releng/tox-*:0.4.0
== 2019-05-13 ==
* 20:21 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/509930
* 19:47 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/502606
* afk: updating docker-pkg images on contint1001 for https://gerrit.wikimedia.org/r/508019
* 14:48 hashar: if you build Docker containers, there is a long delay between it being build/published and it actually being available https://phabricator.wikimedia.org/T222210#5176863 known issue
* 13:49 hashar: Building docker image releng/tox:0.4.0
* 10:37 hashar: Rolling CI config change https://gerrit.wikimedia.org/r/508512 which caused some patches to not be processed last week # https://wikitech.wikimedia.org/wiki/Incident_documentation/20190506-zuul / T105474
== 2019-05-10 ==
* 18:28 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/509483
* 18:08 Reedy: Reloading Zuul to deploy dependancies, new tests, phan
* 17:53 Reedy: Reloading Zuul to deploy 5 more phan patches
* 17:37 Reedy: Reloading Zuul to deploy 5 phan enabling patches
* 17:23 Reedy: Reloading Zuul to deploy various phan additions and one dependancy
* 17:08 Reedy: Reloading Zuul to deploy patches adding phab
* 16:46 Reedy: Reloading Zuul to deploy various dependancy patches
== 2019-05-09 ==
* 08:17 elukey: remove mediawiki memcached nutcracker config from deployment-prep (should be unused) - T214275
* 00:19 thcipriani: updating docker images on contint1001 for https://gerrit.wikimedia.org/r/508929
* 00:19 thcipriani: clean docker images on contint1001
== 2019-05-07 ==
* 22:23 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/507871
* 10:57 hashar: Upgraded Zuul to 2.5.1-wmf8 # T105474 T140297
== 2019-05-06 ==
* 21:57 thcipriani: update docker-pkg on contint1001 for https://gerrit.wikimedia.org/r/508086
* 19:07 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/508012
* 18:37 marxarelli: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/c/integration/config/+/508370
* 17:44 marxarelli: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/c/integration/config/+/508036
* 13:31 hashar: Jenkins: installed Least Load plugin T218458
== 2019-05-03 ==
* 20:58 thcipriani: updating docker-pkg on contint1001 for https://gerrit.wikimedia.org/r/508006
* 20:58 thcipriani: clean docker images from contint1001
* 17:02 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/507992
* 09:22 hashar: removed zuul debian package from integration-castor03
* 08:30 hashar: Building Docker image releng/gradle:0.1.0 https://gerrit.wikimedia.org/r/#/c/integration/config/+/507872/ T222199
== 2019-05-01 ==
* 17:53 halfak: deploying ores:52e9759
== 2019-04-30 ==
* 18:00 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/494778
* 17:01 bearND: (beta): Update mobileapps to 
* 02:01 hashar: Polled in integration-slave-docker-1055 and ntegration-slave-docker-1056
* 01:35 hashar: Deleting integration-slave-docker-1037 (bigram) it is too slow for some reason # T222023
== 2019-04-29 ==
* 14:04 godog: add dsharpe user
== 2019-04-26 ==
* 13:33 Krenair: shut off deployment-conf03 after discussion with otto.mata and elu.key - it seems ancient, broken, unused. T218729
== 2019-04-25 ==
* 16:35 Krenair: shutting down deployment-ms-fe02 and deployment-poolcounter04 T218729
== 2019-04-23 ==
* 15:33 paladox: merging the 2.15.13 release into stable-2.15 following https://wikitech.wikimedia.org/wiki/Gerrit#Update_our_repository
* 14:41 Krenair: Shut down deployment-ms-be03 and deployment-ms-be04 T218729
* 10:22 Amir1: ores:060fc37 going beta
== 2019-04-19 ==
* 19:56 mutante: phab1003 - editing /srv/deployment/phabricator/deployment-cache/.config manually to replace tin.eqiad.wmnet with deploy1001.eqiad.wmnet to fix git cloning issue on first puppet run on new host where somehow tin.eqiad still shows up. fixes puppet run on T221389
== 2019-04-18 ==
* 19:25 Reedy: reloading zuul to deploy https://gerrit.wikimedia.org/r/504824
== 2019-04-17 ==
* 17:17 andrewbogott: cherry-picking https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/504580/ to move off of soon-to-be-shutdown dns recursors
== 2019-04-16 ==
* 17:36 Lucas_WMDE: lucaswerkmeister-wmde@deployment-deploy01:~$ mwscript extensions/WikibaseQualityConstraints/maintenance/ImportConstraintEntities.php --wiki=wikidatawiki --config-format=wgConf tee T221107.php
== 2019-04-15 ==
* 10:16 Amir1: ores:8f01d40 going beta
* 08:51 hashar: castor: nuked /srv/jenkins-workspace/caches/castor-mw-ext-and-skins/master/mwselenium-quibble-docker # T220948
== 2019-04-13 ==
* 21:05 Krinkle: Deleting a bunch of job config+history from Jenkins for jobs that no longer exist in JJB/Zuul. T91410
* 21:00 Krinkle: Deleting a bunch of job config+history from Jenkins for jobs that no longer exist in JJB/Zuul.
* 21:00 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/503669
* 18:52 Krinkle: "Your JENKINS_HOME (/var/lib/jenkins) is almost full. "
* 18:16 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/503664
* 17:44 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/502807 (postgres php72)
* 00:06 Krenair: transferred /home from deployment-cache-upload04 to deployment-cache-upload05 and shut down old one
* 00:06 Krenair: transferred /home from deployment-cumin to deployment-cumin02 and shut down old one
== 2019-04-12 ==
* 15:09 Krenair: upload traffic now through cache-upload05
== 2019-04-11 ==
* 18:29 Reedy: reloading zuul to deploy https://gerrit.wikimedia.org/r/480463
* 18:23 Reedy: reloading zuul to deploy https://gerrit.wikimedia.org/r/499334
* 17:50 Reedy: deployed 'quibble-donationinterface-REL1_31-php70-docker' to jenkins
* 16:31 Krenair: deleting deployment-db04, unused and shut down since 28th March. T219087
* 16:18 mateusbs17: deployment-chromium0[1-2] Update Proton on to (T213362, T216191, T212322)
== 2019-04-10 ==
* 23:32 James_F: Manually created REL1_33 branches for the core, vendor, and tarball extensions and skins. Eurgh. T220653
* 22:36 James_F: Deleted faulty REL1_33 branches for the Timeless, Vector and Monobook skins; they were duplicates of the REL1_32 branches.
* 20:27 paladox: create operations/software/gerrit/plugins/MassBranchCreation repository
* 14:16 hashar: contint1001: sudo -u zuul git -C /srv/zuul/git/mediawiki/core remote prune origin # T220606
* 14:13 hashar: contint2001: sudo -u zuul git -C /srv/zuul/git/mediawiki/core remote prune origin # T220606
* 13:00 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/502790
* 12:46 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/502786
* 12:40 hashar: contint2001: stopped puppet and zuul-merger for debugging
== 2019-04-09 ==
* 23:08 Krinkle: Reloading Zuul to deploy https://phabricator.wikimedia.org/T220561
* 17:59 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/502560
* 16:38 Krenair: deleting deployment-db03, unused since T216635 and shut down since 24th March. T219087
* 16:21 bearND: (beta): Update mobileapps to (T220045 T219411 T219667) - 3rd time is the charm
== 2019-04-08 ==
* 22:46 hashar: cleaned docker images on integration-slave-docker-1030 and integration-slave-docker-1043 :)
* 20:06 bearND: (beta): Update mobileapps to (T220045 T219411 T219667)
* 14:47 hashar: hard rebooting integration-slave-docker-1053 OOM / deadlocked
* 13:39 hashar: Deleting integration-slave-docker-1045 again. It uses Stretch instead of Jessie
* 13:21 hashar: integration: fix cumin profiles that got renamed by 
* 10:15 hashar: Building Docker image releng/quibble-stretch-php73:0.0.31-2 # T220237
== 2019-04-07 ==
* 00:30 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/501793
== 2019-04-06 ==
* 02:17 legoktm: rebuilding npm-php image again https://gerrit.wikimedia.org/r/501848
* 01:52 legoktm: rebuilding npm-php image https://gerrit.wikimedia.org/r/501847
* 01:10 legoktm: deploying https://gerrit.wikimedia.org/r/501786 https://gerrit.wikimedia.org/r/501714 https://gerrit.wikimedia.org/r/501707 https://gerrit.wikimedia.org/r/501782 https://gerrit.wikimedia.org/r/501709 https://gerrit.wikimedia.org/r/500111 https://gerrit.wikimedia.org/r/500106 https://gerrit.wikimedia.org/r/500127 https://gerrit.wikimedia.org/r/500119
* 00:53 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/501813
== 2019-04-05 ==
* 23:53 Krinkle: Beta cluster puppetmaster is stalled behind origin/production as of 24 hours ago (57 patches behind) due to a local merge conflict
* 22:24 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/501789
* 22:02 legoktm: rebuilding mediawiki-phan docker image https://gerrit.wikimedia.org/r/501794
* 20:11 thcipriani: updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/501465
* 08:56 hashar: Reloaded Zuul for operations/software/gerrit/plugins/barricade https://gerrit.wikimedia.org/r/#/c/integration/config/+/501507/
* 08:21 legoktm: deploying https://gerrit.wikimedia.org/r/501416
== 2019-04-04 ==
* 23:37 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/501442
* 22:27 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/500360
* 21:48 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/501428
* 20:47 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/501407
* 17:17 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/500719 (
== 2019-04-03 ==
* 06:13 hashar: gerrit: renamed group "scholarships" to "wikimedia-wikimania-scholarships". Made it owned by "Gerrit Managers" # T218864
== 2019-04-02 ==
* 22:24 hauskatze: maurelio@deployment-deploy01:~$ mwscript extensions/PageAssessments/maintenance/purgeUnusedProjects.php --wiki=enwikivoyage T219935
* 09:46 hashar: Upgrading CI Quibble jobs to 0.0.31
== 2019-04-01 ==
* 21:42 hauskatze: Imported tool-ldap from Diffusion to Gerrit with full history T219703
* 21:33 hauskatze: Created https://gerrit.wikimedia.org/r/#/admin/projects/labs/tools/ldap T219703
* 20:53 hashar: ssh contint1001.wikimedia.org sudo rm /tmp/docker-pkg-build.log
* 20:44 hashar: Building Quibble 0.0.31 containers again # T219647 T219786
* 20:10 hashar: gerrit: flush-caches --cache git_tags # some tag got stalled when querying over https - T219786
* 18:50 hauskatze: Created mediawiki/extensions/ContributionCredits.git per request on mediawiki.org
* 18:00 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/#/c/integration/config/+/500504/
* 17:25 Krinkle: fresnel-node10-browser-docker failing with ENOMEM. Depooled integration-slave-docker-1049 as precaution.
* 17:04 hashar: Building CI docker images for Quibble 0.0.31 (yes it is a long day...)
* 13:50 hashar: Reverted CI Jenkins jobs to Quibble 0.0.28 # T219647
* 13:11 hashar: Upgraded CI Jenkins jobs to Quibble 0.0.30 # T219647
* 10:50 hashar: Manually triggering postmerge step of citoid due to T219017 for mvolz. On contint1001: zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/services/citoid --change 497315,1
* 08:08 hashar: Rebuilding Quibble Jessie containers that failed to build last week due to wikimedia-jessie container. # T219647
* 08:07 hashar: Rebuilding container docker-registry.wikimedia.org/wikimedia-jessie # T219683
== 2019-03-29 ==
* 19:12 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/497975/ + https://gerrit.wikimedia.org/r/498887/
* 17:30 hashar: some quibble 0.0.30 images fail to build for an unknown reason . Left to figure out after the week-end has passed..
* 17:22 hashar: Building Ci docker images for quibble 0.0.30 T219647 T219645
* 12:03 Krenair: added alaasarhan to deployment-prep T219621
== 2019-03-28 ==
* 18:49 Krenair: shut off deployment-db04 instance per T219087
* 18:20 thcipriani: reload zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/496387/
* 17:36 paladox: forking plugins/quota from upstream
* 17:24 Krenair: deployment-prep T219087 beginning master switch
* 10:45 hashar: Tagged Quibble 0.0.30 
* 08:08 legoktm: deployed https://gerrit.wikimedia.org/r/c/integration/config/+/499539 (no-op) and https://gerrit.wikimedia.org/r/499717 (SecurePoll phan)
== 2019-03-27 ==
* 22:27 James_F: Altered Wikimedia GitHub settings to require 2FA; see T198810
* 14:16 hashar: contint: refreshed all git caches manually from cumin: cumin --force 'name:docker' 'find /srv/git -name '*.git' -type d -print -exec git -C {} fetch --prune \;'
* 14:16 hashar: contint: added repositories to the git caches https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/499482/
== 2019-03-26 ==
* 23:04 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/499344 (part 1, 2 and 3)
* 20:36 paladox: create gerrit repo operations/software/gerrit/plugins/WikimediaBlocks T219300
* 20:00 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/499312
* 17:43 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/499262
* 17:42 Krinkle: Reloading Zuul to deploy merged-but-not-deployed patches https://gerrit.wikimedia.org/r/#/c/integration/config/+/497800/ and https://gerrit.wikimedia.org/r/#/c/498684/
== 2019-03-25 ==
* 14:46 mateusbs17: sunset deployment-maps03
== 2019-03-24 ==
* 16:06 Krenair: shut off old deployment-db03 instance per T219087
* 05:39 Krenair: cleaned up old puppet certs/nodes -certcentral-testclient03 -certcentral-testdns -certcentral03 -zotero01 -eventgate-analytics -t153468-test -rd3-cptest-master01 -maps05
* 04:12 Krenair: removed php7.0-fpm package (conflicting with php7.2-fpm) and removed /etc/nginx/sites-enabled/default (conflicting with apache, puppet will remove the available copy too) from -deploy02, -jobrunner03, -mwmaint01, and -mediawiki-07 hosts to try to get puppet there happy again
== 2019-03-23 ==
* 23:44 legoktm: deploying https://gerrit.wikimedia.org/r/498681 https://gerrit.wikimedia.org/r/498682 https://gerrit.wikimedia.org/r/498683
* 23:33 legoktm: deploying https://gerrit.wikimedia.org/r/498247 https://gerrit.wikimedia.org/r/498249 https://gerrit.wikimedia.org/r/498680
* 20:52 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/498664 / T215562
== 2019-03-22 ==
* 19:59 hashar: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/498466/ # T219017
* 19:34 thcipriani: updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/#/c/integration/config/+/494548/
* 19:24 thcipriani: clean old docker-pkg image from contint1001
* 03:08 legoktm: rebuilding mediawiki-phan for https://gerrit.wikimedia.org/r/498297
* 00:36 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/498276 / T218963
* 00:36 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/498276 / T215562)
== 2019-03-21 ==
* 23:34 legoktm: rebuilding mediawiki-phan docker image for https://gerrit.wikimedia.org/r/498266
* 22:34 legoktm: onlined integration-slave-jessie-1002
* 22:33 legoktm: legoktm@integration-slave-jessie-1002:/srv/jenkins-workspace/workspace$ sudo rm -rf *
* 21:03 legoktm: deploying https://gerrit.wikimedia.org/r/498178 https://gerrit.wikimedia.org/r/498185 https://gerrit.wikimedia.org/r/498187 https://gerrit.wikimedia.org/r/498189 https://gerrit.wikimedia.org/r/498182
* 15:22	<hashar>: pruning images/containers on integration-slave-docker-1021
* 10:13	<hashar>: deployment-deploy01: sudo rm -fR /tmp/mw-cache-master | files were from Mar 15 10:21
* 00:33	<legoktm>: deploying https://gerrit.wikimedia.org/r/497600
== 2019-03-20 ==
* 13:20	<hashar>: Scheduled update of Diffusion repository wikibase-termbox via https://phabricator.wikimedia.org/source/wikibase-termbox/manage/ on request of Pablo_WMDE
== 2019-03-19 ==
* 22:47	<Krinkle>: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/497322 / T218553
* 21:00	<paladox>: hiding operations/software/gerrit/plugins/WikimediaWebSessions from normal users
* 20:59	<paladox>: create operations/software/gerrit/plugins/WikimediaWebSessions project T218739
* 20:23	<hashar>: integration: sudo cumin --force '*' 'rm /etc/apt/preferences.d/jessie_mitaka_pinning_*' # T218559
* 19:56	<hashar>: integration: sudo cumin --force '*' 'rm /etc/apt/sources.list.d/openstack-mitaka-jessie.list' # T218559
* 16:56	<Reedy>: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/494802
* 15:48	<Krinkle>: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/497437
* 14:01	<hashar>: Removed Zuul "check" pipeline | https://gerrit.wikimedia.org/r/#/c/integration/config/+/493188/ | T192217
* 02:21	<Krenair>: sudo aptly publish --architectures="all,amd64" --skip-signing repo buster-deployment-prep
* 02:09	<Krenair>: sudo aptly repo create -component="main" -distribution="buster-deployment-prep" buster-deployment-prep
== 2019-03-18 ==
* 16:39 Krenair: yet*
* 16:38 Krenair: created deployment-acme-chief01 and a client instance for further acme-chief testing + dev. used stretch, would be buster like prod but not sure that's easily available outside testlabs yes
* 10:09 hashar: contint1001: rm -fR /srv/doc1001.eqiad.wmnet
* 10:06 hashar: github: deleting https://github.com/wikimedia/wikidata-gremlin # archived T155829
* 09:58 hashar: arming keyholder on integration-cumin
* 09:55 hashar: deleting shutdowned instance integration-publisher02 , we do not use it anymore since doc publishing got overhauled ( T137890 ) # T218146
* 09:12 hashar: deployment-deploy01: cleaning disk: rm /var/cache/hhvm/cli.hhbc.sq3
* 07:37 legoktm: deployed https://gerrit.wikimedia.org/r/496610
== 2019-03-17 ==
* 16:29 Krenair: deactivated and cleaned certs for deployment-redis3-changeprop02 and deployment-prometheus01 (which no longer appear to exist but were causing cumin to be upset)
* 16:27 Krenair: armed deployment-cumin keyholder
* 00:08 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/496887
== 2019-03-16 ==
* 21:56 Krinkle: krinkle@contint1001:~$ zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/extensions/OOJsUIAjaxLogin --change 490979,1
* 21:49 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/497077
* 21:41 legoktm: deploying https://gerrit.wikimedia.org/r/496507 https://gerrit.wikimedia.org/r/496524 https://gerrit.wikimedia.org/r/496520 https://gerrit.wikimedia.org/r/496516 https://gerrit.wikimedia.org/r/496510
* 21:21 Krinkle: krinkle@contint1001$ zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/extensions/EventLogging --change 264494,4
* 21:20 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/497076
* 21:20 Krinkle: Removing doc1001:/srv/docroot/org/wikimedia/doc/mediawiki-extensions-EventLogging (created by accident)
== 2019-03-15 ==
* 21:58 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/496978
* 21:52 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/496882
* 20:10 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/496688
* 18:21 thcipriani: clean old docker images from contint1001
* 17:40 thcipriani: rearm beta keyholder
* 16:54 thcipriani: reenable beta-scap-eqiad
* 16:08 thcipriani: disable beta-scap-eqiad to test new php, back shortly
== 2019-03-14 ==
* 23:48 Krinkle: Abort job quibble-vendor-mysql-hhvm-docker/39874/ for mwext-CentralAuth (stuck after 59 minutes)
* 21:25 hashar: Manually triggered tests for 12 ContentTranslation changes that had label:verified=-1 # T216689
* 21:21 hashar: Updated quibble-vendor-mysql-hhvm-docker with latest libc6 hopefully fixing HHVM segfault within libpthread # T216689
* 19:14 ebernhardson: restart logstash on deployment-logstash2 to re-read and re-create apifeatureusage template
* 16:34 hashar: rollback quibble-vendor-mysql-hhvm-docker job to no more capture core files, we have enough and a good lead ( reverting https://gerrit.wikimedia.org/r/#/c/integration/config/+/496392/ ) # T216689
* 12:31 hashar: Updated quibble-vendor-mysql-hhvm-docker to hopefully allow core dumps and capture them https://gerrit.wikimedia.org/r/#/c/integration/config/+/496392/4 # T216689
* 12:12 hashar: Updated quibble-vendor-mysql-hhvm-docker to hopefully allow core dumps and capture them https://gerrit.wikimedia.org/r/#/c/integration/config/+/496392/3 # T216689
* 10:49 hashar: triggering tests for all ContentTranslation pending changes # T216689
* 09:54 hashar: ci: live hacked job https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-hhvm-docker/ in attempt to capture 'core' files from hhvm https://gerrit.wikimedia.org/r/#/c/integration/config/+/496392/ T216689
== 2019-03-13 ==
* 21:26 thcipriani: pool new bigram CI instance integration-slave-docker-1054
* 20:58 thcipriani: deleting bigram CI instance integration-slave-docker-1046 due to corrupt disk cf: T218245
* 20:44 thcipriani: marking integration-slave-docker-1046 offline
* 20:38 hashar: Added integration-slave-docker-1045 to Jenkins. The instance existed in WMCS but was not in Jenkins
* 20:13 bearND: (beta): Update mobileapps to ( )
* 19:13 hashar: integration-slave-docker-1046 is back online # T218245
* 19:04 hashar: hard rebooting integration-slave-docker-1046 , not reachable over ssh # T218245
* 18:48 hashar: Building containers releng/quibble-jessie-hhvm and releng/quibble-stretch-hhvm with HHVM core_dump_report enabled # T216689
* 18:09 ebernhardson: restart elasticsearch on deployment-elastic* to deploy apifeature usage fix (T183156)
* 10:39 hashar: Triggered tests for ContentTranslation changes that had label:verified=-1 # T217654
* 10:03 hashar: Bump Quibble tmpfs disk space used to hold the database from 256MBytes to 320Mbytes. l10n_cache causes an overflow # T217654
== 2019-03-12 ==
* 18:38 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/479558 / T203506
* 18:23 thcipriani: bring integration-slave-jessie-1001 back online, /srv disk space now at 20% (not sure if someone cleared disk and forgot to repool)
* 16:45 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/495931
* 13:50 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/493139/
== 2019-03-11 ==
* 15:59 Krenair: previous !log was for deployment-prep
* 15:56 Krenair: added MSantos to projectadmin, per chat in -infrastructure he'll use it to work on maps. user is a new foundation software engineer
* 01:20 legoktm: deploying https://gerrit.wikimedia.org/r/495593
== 2019-03-10 ==
* 02:50 legoktm: deploying https://gerrit.wikimedia.org/r/495443 https://gerrit.wikimedia.org/r/493741
* 02:45 legoktm: deploying https://gerrit.wikimedia.org/r/495332 https://gerrit.wikimedia.org/r/495470
== 2019-03-08 ==
* 22:18 thcipriani: Reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/490950/
* 20:20 legoktm: deploying https://gerrit.wikimedia.org/r/495302
* 20:16 legoktm: deploying https://gerrit.wikimedia.org/r/495300 https://gerrit.wikimedia.org/r/495298 https://gerrit.wikimedia.org/r/493502
* 11:36 hasharLunch: integration: deleting old java8 docker images: sudo cumin --force 'name:docker' 'docker imagesgrep java8awk "{ print $3 }"xargs docker rmi'
* 10:04 hashar: Updating maven based jobs to latest java8 containers https://gerrit.wikimedia.org/r/#/c/integration/config/+/495022/ T208938
== 2019-03-06 ==
* 18:25 hasharAway: contint1001: restart Jenkins for plugins upgrade
* 18:24 hasharAway: reloading zuul for 
* 18:02 hashar: Upgrading plugins on https://releases-jenkins.wikimedia.org
* 17:37 hashar: Reloading Zuul for mediawiki/libs/Zest https://gerrit.wikimedia.org/r/#/c/integration/config/+/494609/2/zuul/layout.yaml
* 16:27 Krinkle: doc1001 had bad permissions set on /org/wikimedia/doc (chmod 755 instead of 775). Making it impossible to git pull in the way that post-merge Jenkins job on integration/docroot recommends. Fixed with `sudo -u doc-uploader chmod 775 /srv/wikimedia/org/wikimedia/doc`.
* 15:58 andrewbogott: deleting deployment-prometheus01 on Filippo's advice
* 13:31 gehel: upgrading logstash to 5.6.14 on deployment-logstash2
* 13:15 gehel: upgrading elasticsearch to 5.6.14 on deployment-logstash2
* 11:33 Lucas_WMDE: lucaswerkmeister-wmde@deployment-mediawiki-09:~$ sudo systemctl restart php7.2-fpm # T217323
== 2019-03-05 ==
* 20:10 thcipriani: reenable beta-scap-eqiad
* 19:17 thcipriani: disable beta-scap-eqiad due to T217587
* 10:13 hashar: integration: fixed erroneous ssh key restriction for cumin T217642
== 2019-03-04 ==
* 17:36 hashar: Build Docker containers for chromium=v71 pin ( https://gerrit.wikimedia.org/r/#/c/integration/config/+/494243/ )
* 15:14 herron: cherry picking https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/492390/ to deployment-puppetmaster03
* 13:49 hauskatze: GitHub: deleted wikimedia/mediawiki-extensions-UploadLocal T213011
* 09:59 hashar: cleaned docker images on integration-slave-docker-1037
* 07:38 legoktm: deploying https://gerrit.wikimedia.org/r/494167
== 2019-03-03 ==
* 02:26 Krinkle: tried rebooting or shutting down integration-slave-docker-1021, no response on horizon. Did pause/resume instead, which did work, after which shutdown/start worked. Jenkins agent has been relaunched and seems online again.
* 02:20 Krinkle: integration-slave-docker-1021 (ci1.medium) has jobs failing on it due to ENOMEM. Horizon shows in log: integration-slave-docker-1021 login: [4961938.696837] Out of memory: Kill process 21770 (chromium) score 841 or sacrifice child; [4961938.699176] Killed process 21770 (chromium) total-vm:3171496kB, anon-rss:1379288kB, file-rss:0kB, shmem-rss:1636kB
== 2019-03-02 ==
* 22:12 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/493959
* 20:44 hauskatze: Renamed https://github.com/wikimedia/wikimedia-github-community-health-defaults to https://github.com/wikimedia/.github
* 20:42 hauskatze: ssh -p 29418 gerrit.wikimedia.org replication start wikimedia/github-community-health-defaults --wait
* 20:40 hauskatze: github created https://github.com/wikimedia/wikimedia-github-community-health-defaults
* 20:31 Reedy: reloading zuul to deploy https://gerrit.wikimedia.org/r/493881
* 20:30 Krinkle: Failure on integration-slave-docker-1021 (ENOMEM) https://integration.wikimedia.org/ci/job/fresnel-node10-browser-docker/61/console
* 19:51 legoktm: deploying https://gerrit.wikimedia.org/r/493872
* 19:37 legoktm: deploying https://gerrit.wikimedia.org/r/493862
* 18:26 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/493808
* 18:21 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/493837
== 2019-03-01 ==
* 19:17 thcipriani: integration-slave-docker-1021:/# docker rmi $(docker images grep " months " grep -v " [1-2] months " awk '{print $3}')
* 17:02 thcipriani: integration-slave-jessie-1004 back online
* 16:58 thcipriani: integration-slave-jessie-1002 back online (disk space looked fine); rebooting integration-slave-jessie-1004 -- can't ssh to machine
* 16:11 Lucas_WMDE: delete refs/master and refs/gerrit/master on WikibaseQualityConstraints repository T217408
* 15:49 hashar: wikidata/query/blazegraph change Gerrit config to require a change-id # T216855
* 14:28 hashar: Upgrading integration/jenkins-job-builder to version 2.0.2 + one custom hack 11aa5de4...a06d173e # T143731
* 14:18 hashar: integration/jenkins-job-builder : importing upstream code to new branch "upstream". Push all upstream tags to our repository
== 2019-02-28 ==
* 18:14 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/493261
* 03:42 duh: deploying https://gerrit.wikimedia.org/r/493357
* 03:42 duh: deploying
* 03:30 duh: deploying https://gerrit.wikimedia.org/r/c/integration/config/+/493355
== 2019-02-27 ==
* 22:02 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/490678/
* 19:20 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/492758
* 19:08 thcipriani: updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/c/integration/config/+/492758 (take II)
* 18:27 thcipriani: updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/c/integration/config/+/492758
== 2019-02-26 ==
* 22:52 ebernhardson: delete logstash logs in /var/log/logstash generated prior to 2019
* 22:51 ebernhardson: restart logstash on deployment-logstash2 while hacking around to see why apifeatureusage doesn't work
* 18:01 dcausse: deployement-prep: installing elastic 6.5.4 to deployment-elastic* machines
* 16:46 addshore: added Cparle to deployment-prep
* 16:11 hashar: Generating 1.33.0-wmf.19 deploy notes https://integration.wikimedia.org/ci/job/train-deploy-notes/9/console T206673
* 09:32 godog: remove now-merged node-exporter timer disable, cherry pick https://gerrit.wikimedia.org/r/c/operations/puppet/+/492632
== 2019-02-25 ==
* 23:32 twentyafterfour: root@deployment-db05# mariabackup --innobackupex --apply-log --use-memory=10G /srv/sqldata # T216067
* 22:14 thcipriani: docker rmi images without "latest" tag on contint1001 to free space -- should have kept all current docker-pkg images as well as images with children -- T217094
* 13:39 hashar: Rebuilding some CI Docker images using PHP sury.org to switch the sury.org component from jessie to stretch ( https://gerrit.wikimedia.org/r/#/c/integration/config/+/492666/ )
== 2019-02-24 ==
* 21:09 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/492561, T216964
* 04:54 legoktm: rebuilding docker image for https://gerrit.wikimedia.org/r/485241
* 04:41 legoktm: legoktm@contint1001:/srv/zuul/git/mediawiki/tools$ sudo -u zuul rm -rf phan
== 2019-02-23 ==
* 22:25 legoktm: deploying https://gerrit.wikimedia.org/r/492497
* 02:32 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/492427
* 01:59 Krinkle: Reloading Zuul to deploy ttps://gerrit.wikimedia.org/r/492425
== 2019-02-22 ==
* 20:59 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/492377
* 16:48 paladox: created "block-users" group for https://bugs.chromium.org/p/gerrit/issues/detail?id=10507 (reported by a wmf user)
* 13:38 Reedy: restarting zuul because nothing is being run
== 2019-02-21 ==
* 21:02 paladox: branch deploy/wmf/stable-2.16 from in operations/software/gerrit
* 19:49 Amir1: ores:5d937b1 is going beta
* 18:22 Amir1: ores:2d84709 going beta
* 17:45 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/483225/
* 16:35 hashar: Adjusting IP of https://integration.wikimedia.org/ci/computer/compiler1002.puppet-diffs.eqiad.wmflabs/ and adding it back # T216513
* 14:10 Amir1: deploying ores:5d50713 to beta
== 2019-02-20 ==
* 19:45 hashar: deployment-db03: restored some old /var/lib/dpkg/status file : sudo zcat /var/backups/dpkg.status.2.gz sudo tee /var/lib/dpkg/status # T216635
* 19:45 hashar: deployment-db03: restored some old /var/lib/dpkg/status file : sudo zcat /var/backups/dpkg.status.2.gz sudo tee /var/lib/dpkg/status
* 17:26 hashar: For beta cluster the MySQL master database has some innodb issue T216635 , the MySQL slave has an issue as well T216067
* 17:09 hashar: reloading zuul for 
* 17:09 hashar: contint1001: fix broken root ownership on zuul git deploy repo: sudo find /etc/zuul/wikimedia/.git -not -user zuul -exec chown zuul:zuul {} +
* 17:05 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/490606/
* 16:57 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/490559
* 16:36 hauskatze: Ran replication start mediawiki/extensions/PageViewInfo --wait on gerrit.wikimedia to populate GitHub mirror (success messages afterwards) T180864
* 00:25 greg-g: disabled beta-update-databases-eqiad in the jenkins UI - T216067
== 2019-02-19 ==
* 21:23 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/491577, also deploys https://gerrit.wikimedia.org/r/490492 / T198495
* 20:28 hashar: jenkins: disable Android Emulator plugin T198495
* 20:27 hashar: gerrit: archived repository integration/jenkinsci/android-emulator-plugin T198495
* 20:23 hashar: Deleting Jenkins job apps-android-wikipedia-periodic-test T198495
* 15:55 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/491513
* 15:46 hashar: Creating integration-slave-jessie-android m1.large it got deleted last week T216517
* 08:55 hashar: Cleaning contint1001 / partition
* 04:55 bd808: Removed stale cherry-pick of from deployment-puppetmaster03 that was blocking rebase on origin/production
* 04:50 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/491405
* 04:14 Krinkle: Disabled spam account on Phab - https://phabricator.wikimedia.org/people/manage/18915/
* 03:55 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/491403
* 00:54 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/491381 / / T133646
== 2019-02-18 ==
* 22:32 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/491379 / T211784
* 21:51 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/491378 / T133646)
* 21:37 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/459268 / T133646
* 15:38 elukey: kill/spawn deployment-aqs0[2,3] in deployment-prep with Debian Stretch
* 14:12 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/490590
* 12:00 Krenair: T216067 Stopping mysql on -db04 to begin copy to -db05. Note crashed tables centralauth.globaluser and centralauth.localuser
* 11:57 elukey: kill/spawn deployment-aqs01 with Debian Stretch in deployment-prep
* 11:45 arturo: manually start deployment-db03 per Krenair request
* 11:29 hasharAway: beta: tried to start instance deployment-db03 172.16.5.23 --> ERROR T216067
== 2019-02-17 ==
* 07:21 legoktm: deploying https://gerrit.wikimedia.org/r/491029
* 07:10 legoktm: Building image docker-registry.discovery.wmnet/releng/tox-acme-chief:0.3.4
* 06:28 legoktm: building new tox-acme-chief docker image https://gerrit.wikimedia.org/r/489725
* 01:12 Krinkle: beta-scap-eqiad (cron) failing with "sudo: a password is required"
== 2019-02-16 ==
* 19:44 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/490937 / T216275)
* 17:23 thcipriani: installed php7.0-curl on deployment-deploy01 (why was that suddenly necessary?)
== 2019-02-15 ==
* 17:28 thcipriani: integration-slave-jessie-1002:/srv/jenkins-workspace/workspace$ `sudo rm -rf *` due to full disk
== 2019-02-14 ==
* 20:50 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/490528 / T216102
* 20:50 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/490528 / T133646
* 13:24 thcipriani: rearm keyholder on deployment-deploy01: sudo keyholder arm, passwords on https://wikitech.wikimedia.org/wiki/Keyholder
== 2019-02-13 ==
* 21:32 marxarelli: dduvall@integration-slave-jessie-1001:/mnt/home/jenkins-deploy$ `rm -rf .gradle/ .m2/` due to full disk
* 21:21 marxarelli: bringing integration-slave-docker-1046 and integration-slave-jessie-1001 back online
* 21:20 marxarelli: dduvall@integration-slave-jessie-1001:/srv/jenkins-workspace/workspace$ `sudo rm -rf *` due to full disk
* 21:15 marxarelli: removing old docker images on integration-slave-docker-1046
* 21:10 marxarelli: starting migrated integration-slave-docker-1046 instance
* 21:01 marxarelli: pooling new jenkins node for integration-slave-docker-1052
* 20:46 marxarelli: pooling jenkins node for integration-slave-docker-1051
* 20:45 marxarelli: launching replacement instance integration-slave-docker-1052
* 20:35 marxarelli: launching replacement instance integration-slave-docker-1051
* 20:32 marxarelli: pooling jenkins node for integration-slave-docker-1050
* 
* }
* 19:48 marxarelli: pooling replacement jenkins node integration-slave-docker-1049
* 19:34 marxarelli: deleting integration-slave-jessie-android jenkins node and instance
* 19:33 marxarelli: deleting integration-slave-jessie-1003 jenkins node and instance
* 19:32 marxarelli: deleting integration-slave-docker-1033 jenkins node and instance
* 19:25 marxarelli: deleting integration-slave-docker-1017 jenkins node and instance
* 18:45 Krinkle: integration-slave-jessie-1003 seems to be consitently unable to start jobs, marking as offline manually
* 18:32 thcipriani: bringing up new integration-castor03, re-enabling castor-save* jobs
* 18:15 marxarelli: adding new jenkins node integration-slave-docker-1048
* 18:02 marxarelli: launching new integration-slave-docker-1048 instance
* 17:59 marxarelli: deleting integration-slave-docker-1038 node and deleting instance
* 17:50 marxarelli: bringing integration-slave-docker-1033 back online after clearing out old docker images
* 17:33 thcipriani: rebuilding integration-castor03
* 17:21 thcipriani: stopping rsync server on castor03
* 17:21 twentyafterfour: stopped rsync on castor03
* 17:16 twentyafterfour: disconnected castor03 from jenkins
* 16:48 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/487880/
* 14:34 thcipriani: modified castor-save-workspace-cache to exit 0 and run on blubber nodes while integration-castor03 is down
* 14:26 dcausse: deployement-prep: upgrading to elastic 5.6.14
== 2019-02-12 ==
* 23:15 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/490130/1
== 2019-02-11 ==
* 21:05 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/489689
* 20:39 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/489688/
* 19:46 marxarelli: installing/enabling HTTP Request jenkins plugin on integration.wikimedia.org/ci to support https://gerrit.wikimedia.org/r/c/integration/pipelinelib/+/480689 changes
* 16:04 addshore: bring integration-slave-docker-1040 back online
* 16:04 addshore: addshore@integration-slave-docker-1040:~$ sudo docker image prune -a --force --filter "until=2191h" // (3 months?) Total reclaimed space: 17.52GB
* 10:47 godog: shut deployment-prometheus01, unused now
* 03:06 Reedy: graceful restart of zuul as no jobs were running
== 2019-02-10 ==
* 03:22 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/489434 (Create quibble-stretch-hhvm, replacing jessie)
* 02:06 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/489430
== 2019-02-08 ==
* 20:20 Krinkle: Delete various jobs on Jenkins that no longer exist in JJB config, ref T91410
* 15:59 addshore: this reload also included "Switch npm-audit job to node10"? T211784, which did touch the zuul file
* 15:58 addshore: reloaded zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/489241/
* 03:10 Krinkle: Delete various jobs on Jenkins that no longer exist in JJB config
* 00:28 Krinkle: krinkle@doc1001: sudo -u doc-uploader chmod 775 /srv/docroot/org/wikimedia/doc/
* 00:12 marxarelli: removed old docker images on contint1001 to free up space
== 2019-02-07 ==
* 23:17 thcipriani: integration-slave-jessie-1003:sudo rm -rf /srv/jenkins-workspace/workspace/*
* 23:15 thcipriani: integration-slave-docker-1033:sudo docker image prune and bring back online
* 22:28 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/467550 (
* 19:09 paladox: created integration/zuul/build gerrit repo for T215458
* 19:05 paladox: created integration/zuul/wheels gerrit repo for T215458
* 15:48 addshore: brought integration-slave-docker-1043 back online
* 15:48 addshore: addshore@integration-slave-docker-1043:~$ sudo docker image prune -a --force --filter "until=2191h" // (3 months?) Total reclaimed space: 14.86GB
* 08:49 hashar: cleaning docker images on integration-slave-docker-1021
== 2019-02-06 ==
* 22:34 shdubsh: Deploy node-exporter 0.17 T213708
* 14:12 godog: shut off deployment-prometheus01 - T215272
* 14:00 godog: switch beta-prometheus to deployment-prometheus02 - T215272
== 2019-02-05 ==
* 20:07 ebernhardson: jobrunner port 9006 is firewalled, revert to 9005 and created T215339 to fix job queue in beta cluste
* 19:36 ebernhardson: Update profile::cpjobqueue::{jobrunner,videoscaler}_host in horizon hiera from port 9005 to 9006 to match new restrictions in gerrit.wikimedia.org/r/481866
* 16:29 addshore: T215288 added mirrys to deployment-prep as a user
* 15:32 addshore: T215278 addshore@integration-slave-docker-1037:~$ sudo docker image prune -a --force --filter "until=2191h" // (3 months?) Total reclaimed space: 16.59GB
== 2019-02-04 ==
* 23:13 thcipriani: integration-slave-docker-1040:sudo docker image prune and bring back online
* 23:12 thcipriani: integration-slave-docker-1038:sudo docker image prune and bring back online
* 21:48 ebernhardson: restart logstash on deployment-logstash2
* 15:25 hashar: removed Jenkins user "nodepoolmanager" as well as related authorizations T209361
== 2019-02-03 ==
* 06:15 legoktm: deployed https://gerrit.wikimedia.org/r/487627
* 05:44 legoktm: deployed https://gerrit.wikimedia.org/r/485967
* 04:36 legoktm: deploying https://gerrit.wikimedia.org/r/487534
== 2019-02-02 ==
* 22:17 legoktm: legoktm@integration-slave-jessie-1004:/srv/jenkins-workspace/workspace$ sudo rm -rf *
== 2019-01-31 ==
* 15:03 thcipriani: rearm keyholder on deployment-deploy01
* 12:05 arturo: VM instances deployment-deploy01,deployment-deploy02,deployment-fluorine02,deployment-kafka-jumbo-2,deployment-kafka-main-1,deployment-maps04,deployment-mcs01,deployment-mediawiki-09,deployment-memc04,deployment-ms-be03,deployment-ms-fe02,deployment-parsoid09,deployment-sca04,deployment-webperf12, were stopped briefly due to issue in hypervisor (T215012)
== 2019-01-30 ==
* 08:35 legoktm: deploying https://gerrit.wikimedia.org/r/486439 https://gerrit.wikimedia.org/r/481570 https://gerrit.wikimedia.org/r/481571
== 2019-01-29 ==
* 07:41 legoktm: legoktm@integration-slave-jessie-1001:/srv/jenkins-workspace/workspace$ sudo rm -rf * b/c full disk
== 2019-01-28 ==
* 16:33 hashar: contint1001: cleaning up disk space on /
* 13:07 addshore: bringing integration-slave-docker-1041 back online
* 13:07 addshore: addshore@integration-slave-docker-1041:~$ sudo docker image prune -a --force --filter "until=2191h" // (3 months?) Total reclaimed space: 16.12GB
* 09:37 Amir1: ores:ad160b0 is going beta
== 2019-01-27 ==
* 19:57 addshore: bringing integration-slave-docker-1034 back online
* 19:50 addshore: addshore@integration-slave-docker-1034:~$ sudo docker image prune -a --force --filter "until=2191h" // (3 months?) Total reclaimed space: 17.12GB
== 2019-01-26 ==
* 22:48 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/486791
* 21:21 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/486737
== 2019-01-25 ==
* 19:25 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/486503/
* 19:07 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/486501/
* 14:56 hashar: contint1001: systemctl stop zuul-merger && find /srv/zuul/git -name .git -type d -print -execdir git gc --prune=now \;
* 13:35 hashar: flake8 broken under python2.7 due to configparser==3.5.2 https://github.com/jaraco/configparser/issues/27 https://github.com/jaraco/configparser/issues/27
* 00:58 thcipriani: integration-slave-jessie-1002:sudo rm -rf /srv/jenkins-workspace/workspace/* clean gradle cache, bring back online
== 2019-01-23 ==
* 22:15 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/486170/
== 2019-01-22 ==
* 23:46 thcipriani: Updating (more) tox docker image on contint1001 for https://gerrit.wikimedia.org/r/#/c/integration/config/+/485971/
* 23:24 thcipriani: integration-slave-docker-1017 clean docker images and repool
* 21:06 thcipriani: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/#/c/integration/config/+/485895/
* 20:18 legoktm: rebuilding php, composer, phan images for apt security update https://gerrit.wikimedia.org/r/485883
* 19:59 thcipriani: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/#/c/integration/config/+/485882/
* 19:28 legoktm: rebuilding ci support images for apt security upgrade https://gerrit.wikimedia.org/r/485874
* 19:09 legoktm: rebuilding ci-stretch/ci-jessie images https://gerrit.wikimedia.org/r/485873
* 19:08 legoktm: manually docker pulled docker-registry.wikimedia.org/wikimedia-jessie and docker-registry.wikimedia.org/wikimedia-stretch on contint1001
* 18:56 bearND: (beta) Update mobileapps to (fix pronunciation detection in mobile-sections T214338)
* 18:46 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/485870/ and https://gerrit.wikimedia.org/r/#/c/integration/config/+/485331/
* 14:31 addshore: reloaded zuul for REVERT: Add ArticlePlaceholder to gated extensions [integration/config] - https://gerrit.wikimedia.org/r/485754
* 13:26 addshore: reloaded zuul for Add ArticlePlaceholder to gated extensions [integration/config] - https://gerrit.wikimedia.org/r/485754
* 13:03 addshore: reload zuul for Add ArticlePlaceholder to Wikibase Tests [integration/config] - https://gerrit.wikimedia.org/r/485753
== 2019-01-21 ==
* 19:49 hashar: integration: update sudo rule for debian-glue to keep env variable EXTRAPACKAGES. Would let us get eatmydata included T214328
* 15:40 hashar: contint1001: removing all generated doc/cover from /srv/org/wikimedia/doc T137890
== 2019-01-18 ==
* 23:22 hashar: contint1001: sudo docker image prune # Total reclaimed space: 3.592GB
* 23:00 Krinkle: Some docker builds on integration-slave-docker-1021 failing with ENOMEM
* 23:00 mutante: contint1001 - gzipping more files in /var/log/zuul/
* 22:57 mutante: contint1001 - moved zuul logs from 2018 and gzipped zuul logs from /var/log/zuul to /srv/logs/zuul to free disk space on /
* 22:39 mutante: contint1001 - apt-get clean - disk space low
* 22:31 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/482527 / T212602
== 2019-01-17 ==
* 19:34 thcipriani: integration-slave-jessie-1002:sudo rm -rf /srv/jenkins-workspace/workspace/* and bring back online
* 08:39 legoktm: deploying composer docker image - https://gerrit.wikimedia.org/r/484853
== 2019-01-16 ==
* 21:11 bearND: (beta): Update mobileapps to page summary changes
== 2019-01-15 ==
* 09:00 hashar: Deleting Docker images on integration-slave-docker-1021
== 2019-01-14 ==
* 22:02 bearND: (beta): Update mobileapps to 
* 21:47 mutante: deployment-mcs01 - sudo su deploy-service; cd /srv/deployment/mobileapps/deploy-cache/revs/1182b3b8f288df0221257b929ca43fb86862c2f8/scap ; touch log (for debugging permission problem reported by bearND)
* 14:31 hashar: Nuked Castor cache for all *tox* jobs. Some might have cached binary wheels compiled against a lib that is no more existing (eg libmysqlclient.so.18 for mysql-python). Follow up the jessie -> stretch upgrade # T191764
* 14:28 hashar: Deleted Castor cache for wikimedia-cz/tracker mysql-python got cached as a wheel but compiled against libmysqlclient.so.18. That fails with the new tox...:0.3.0 containers which uses mariadb / libmysqlclient.so compat symlink
== 2019-01-11 ==
* 20:48 thcipriani: repooling integration-slave-jessie-1003 after cleaning mvn and gradle cache
* 19:29 legoktm: deploying https://gerrit.wikimedia.org/r/483832
* 19:20 legoktm: deployed https://gerrit.wikimedia.org/r/483824
* 04:37 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/483674
* 03:48 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/483671
* 03:15 twentyafterfour: mediawiki-1.32.0 uploaded to releases.wikimedia.org (archives should be downloadable in a few minutes after puppet runs and the proxy's cache expires)
* 03:09 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/483667 Switch unicodejs jobs to Node 10 - T211784
* 02:37 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/483666
* 02:01 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/483656
* 01:22 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/483644
* 00:12 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/482755 - T211784
== 2019-01-09 ==
* 19:59 legoktm: deploying https://gerrit.wikimedia.org/r/482766
* 18:05 thcipriani: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/#/c/integration/config/+/483189 https://gerrit.wikimedia.org/r/#/c/integration/config/+/481947/
* 13:27 hasharAway: shutting down instance integration-publishing02 [172.16.4.5] No more used T137890
* 13:26 hasharAway: deleting https://integration.wikimedia.org/ci/job/publish-on-contint1001/ no more used T137890
== 2019-01-08 ==
* 21:52 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/476600/
* 21:31 Hauskatze: github: @niedzielski updated @jdlrobson permission on Wikimedia from `read` to `admin`
* 21:30 Hauskatze: github:
* 20:46 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/482855/
* 19:53 mutante: deployment-prep adjusting puppet config on deployment-mwmaint01. remove "mediawiki_maintenance" role from "other classes" section and apply "mediawiki::maintenance" instead after role rename in gerrit:479131 for consistency with other mediawiki:: roles
* 19:53 mutante: adjusting puppet config on deployment-mwmaint01. remove "mediawiki_maintenance" role from "other classes" section and apply "mediawiki::maintenance" instead after role rename in gerrit:479131 for consistency with other mediawiki:: roles
* 14:25 hashar: Upgrading plugins on https://releases-jenkins.wikimedia.org/
* 09:19 hashar: gerrit: resaved configuration for All-Projects by changing "Max Reviewers" from 3 to 4. Might enable adding reviewers automatically based on git blame. See task for config diff # T 101131
* 05:37 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/482752
* 02:45 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/482751
== 2019-01-07 ==
* 20:29 Krinkle: Reloading Zuul to apply https://gerrit.wikimedia.org/r/482695
* 20:12 Krinkle: Delete job 'fresnel-npm6-browser-node-6-docker' and 'fresnel-node-10-docker'
* 20:08 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/482572
* 19:32 awight: T212530: ORES revscoring 2.3.0
* 19:22 legoktm: building node10 docker images w/ https://gerrit.wikimedia.org/r/482573 this time
* 00:30 legoktm: deploying https://gerrit.wikimedia.org/r/482392 / create node10 mage
== 2019-01-06 ==
* 00:39 Krinkle: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/482392
== 2019-01-03 ==
* 20:15 Hauskatze: gerrit: created https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/extensions/SparkPost
* 20:03 Hauskatze: github: created https://github.com/wikimedia/mediawiki-extensions-SparkPost
* 18:46 bearND: (beta): Update mobileapps to Set timeout for backend /page/html requests, part 2
* 18:32 bearND: (beta): Update mobileapps to Set timeout for backend /page/html requests
* 16:44 thcipriani: remove old docker images and bring integration-slave-docker-1021 back online
== 2019-01-02 ==
* 10:19 hashar: updating all debian-glue jobs and creating new ones with hardcoded distributions (trusty, jessie, stretch, unstable) T210780
== 2019-01-01 ==
* 15:33 hashar: contint1001: deleting some extensions documentation for wmf branches: rm -fR /srv/org/wikimedia/doc/{Kartographer,MinervaNeue,MobileFrontend,Wikibase}/wmf # T118599