Wikimedia Cloud Services team/EnhancementProposals/DNS domain usage

Years after the rebranding project we are still using domains with labs in them: *.wmflabs.org, *.eqiad.wmflabs. The WMCS team would like to design and begin implementing a sane replacement for these legacy domains both to remove this vestige of the former project name and to help make separations of various zones of operation more clear.
== Problem statement ==
===Things to name===
*Physical hardware: cloud*.{codfw,eqiad}.wmnet, cloud*.wikimedia.org
*Cloud VPS instances: *.{codfw,eqiad}.wmflabs
*Public HTTPS proxies: *.wmflabs.org
*Toolforge webservices: tools.wmflabs.org
*Virtual services: *.svc.{codfw,eqiad}.wmflabs
We have two different DNS systems to work with as well, the production gdnsd system and the Designate/PDNS system inside Cloud VPS.
===Candidate domains===
Domain names we already own related to Cloud Services:
*wikimediacloudservices.org
*wikimediacloud.org
*wmcloud.org
*wikimedia.cloud
*toolforge.org
*pseudo TLDs we can resolve locally (for example *.{codfw,eqiad}.wmcloud or *.{codfw,eqiad}.wmcs)
== Resolution ==
The #SRE offsite proposal has been chosen as the "winning" proposal.
=== DNS naming ===
; *.wikimediacloud.org
: Naming for openstack API endpoints and other shared service endpoints like elasticsearch. Subdomains may be added as needed to differentiate services and service regions/zones/clusters.
: DNS will be managed in the main Wikimedia DNS servers. Subdomains may be delegated to OpenStack Designate when appropriate.
:Examples:
:* openstack.eqiad1.wikimediacloud.org
:* cloudinstances2b-gw.openstack.codfw1dev.wikimediacloud.org
:* nat.openstack.eqiad1.wikimediacloud.org
; *.wmcloud.org
: Naming for public exposure of Cloud VPS instances either directly (public IPv4/IPv6) or via proxy (HTTPS proxies).
: Replaces deprecated *.wmflabs.org
: DNS will be managed via OpenStack Designate in eqiad1. Subdomains may be delegated to other deployments (for example codfw1dev) as needed.
: Examples:
:* bastion.wmcloud.org
:* space.wmcloud.org
:* outreachdashboard.wmcloud.org
:* bastion.tools.wmcloud.org
; *.{deployment}.wikimedia.cloud
: Naming for internal Cloud VPS instances and services. Intended for private IP addresses.
: Replaces deprecated *.{data center}.wmflabs
: DNS will be managed via OpenStack Designate in the associated deployment.
: Examples:
:* bastion-01.bastion.eqiad1.wikimedia.cloud
:* k8s-control.svc.tools.wikimedia.cloud
:* enwiki.analytics.db.svc.wikimedia.cloud
; *.toolforge.org
: Naming for public exposure of Toolforge tools and other 'native' Toolforge services
: Replaces deprecated tools.wmflabs.org
: DNS will be managed via OpenStack Designate.
: Examples:
:* admin.toolforge.org
:* openstack-browser.toolforge.org
=== HA / Load balancing ===
Production LVS servers MUST NOT be used for *.wikimediacloud.org services. Alternate load balancing systems will be designed and implemented for HA of Cloud Services core services.
See Wikimedia Cloud Services team/EnhancementProposals/OpenStackHA for further design discussion.
==Proposals==
===Arturo===
Proposal raised by in phab:T223902#5210949:
*Toolforge tools and other stuff native to Toolforge: we plan to use toolforge.org for them, as stated in the original docs
*cloudVPS VMs and other virtual stuff managed by openstack: we plan to use wmcloud.org for them, as stated in the original docs. This domain will be entirely managed by Designate.
*databases, like wikireplicas, toolsdb, etc. They are (or will be soon) all cloudVPS VMs. So they are covered by wmcloud.org already.
*for Openstack endpoints (the original issue in this tickets) I think we have 2 options:
**we could use $subdomain.wmcloud.org if this subdomain is not hosted by desginate (to avoid chicken-egg problems)
**we could use wikimedia.cloud, host this domain in prod DNS and dedicate it to services only. Examples: keystone.eqiad1.wikimedia.cloud, nova.codfw1dev.wikimedia.org
*supporting services, like NFS or Ceph. We are in the middle of doing a PoC for this. It's not clear what the final networking topology/layout for this would be. We could probably either:
**reuse standard prod domains (be it .wmnet or .wikimedia.org)
**try to introduce them to wikimedia.cloud as well: cloudosd1001.wikimedia.cloud? Perhaps this will be the first case of real HW being assigned anything other than .wmnet|.wikimedia.org.
===Brandon===
Proposal raised by in phab:T223902#5211181:
*wmcloud.org - replaces wmflabs.org, hosted directly on Designate, covers everything that's inside instances
*wikimediacloud.org - Used for the official public API service (nevermind that they're not yet fully public, initially) - The ~5 API endpoints in this ticket, and the DNS is hosted by prod DNS and not designate.
*wikimedia.cloud - Used for supporting services (NFS, Ceph, etc) or any other semi-internal usage that's sitting in some bridging position between prod and wmcs. Probably hosted by prod DNS, but could delegate subdomains to Designate if appropriate.
=== SRE offsite proposal ===
Proposal raised by Brandon, Jason, Jonh, Andrew, Bryan and Arturo @ Dublin SRE offsite Jun 2019:
*openstack.eqiad1.wikimediacloud.org --- for openstack endpoints, and other endpoints like elasticsearch, using different subdomain.
*wmcloud.org --- replaces wmflabs.org
*wikimedia.cloud --- replaces .wmflabs
*toolforge.org --- replaces tools.wmflabs.org
HA proxy in cloudcontrol servers instead of using prod LVS servers.