Portal:Cloud VPS/Admin/IPv6/initial deploy

This page tracks project information for the initial IPv6 deployment in Cloud VPS.
== Implementation details ==
Some implementation details.
=== user facing options ===
The IPv6 initial deploy is paired with the VXLAN migration. For some time, there will be an option in horizon to create virtual machines attached to 3 different kind of networks:
* VLAN/legacy
* VXLAN/IPv4-only (a 1:1 replacement of the old legacy VLAN, but based on VXLAN)
* VXLAN/IPv6-dualstack (VMs will get 1 IPv4 and 1 IPv6)
As the migration progresses, the options available in Horizon will get trimmed down, until everyone is on VXLAN/IPv6-dualstack.
See also Wikimedia_Cloud_Services_team/EnhancementProposals/Decision_record_T377467_cloud_vps_vxlan_ipv6_migration
=== network topology ===
For VMs in VXLAN/IPv6-dualstack, neutron will allocate an IPv4 and an IPv6.
* The IPv4 is private as usual. It will get the usual egress NAT, can get associated a floating IP, etc.
* The IPv6 is public and global. There are no NAT or floating IP options. The VM can contact the internet (and WMF wikis) with it, freely.
The edge network routing is dual-stacked. Network path for IPv4 and IPv6 is similar, barring NATs.
We have a primary neutron virtual router, then cloudgw router, then cloudsw, then the Wikimedia core routers. This is as usual, just with IPv6 support, and with no NATs in case of IPv6.
To clarify, IPv6 traffic on a VM:
* egress: open by default, to other VMs in the same project, other VMs in other projects, the WMF networks and the internet.
* ingress: restricted in default security group to VMs in the same project. Manual change in security groups can be made to allow ingress from VMs in other projects, the WMF networks and the internet.
=== firewalling ===
There are some special semantics here.
* IPv6 egress traffic is allowed by default for VMs with IPv6.
* IPv6 ingress traffic is controlled purely by local security groups on each VM. Additional firewalling can be done inside the VM via whatever method.
* There are options for additional perimeter firewalling on cloudgw nodes, similar to IPv4, but is not active unless there is need to.
=== DNS ===
If using VXLAN/IPv6-dualstack, each VM will get a FQDN with two DNS recods, for example some-vm.some-project.eqiad1.wikimedia.cloud will get:
* A 172.16.x.y its IPv4, as usual
* AAAA 2a02:ec80:a000:1::x:y:z its IPv6
Additionally, using https://gitlab.wikimedia.org/repos/cloud/cloud-vps/designate-sink-wmcs-nova-fixed-ptr we will create two PTR records for each IPv4/IPv6.
=== addressing plan ===
Remember, source of truth is Netbox.
IPv6
In codfw:
* 2a02:ec80:a100::/40 -- wmcs codfw https://netbox.wikimedia.org/ipam/prefixes/1077/
** 2a02:ec80:a100::/55 -- wmcs codfw openstack https://netbox.wikimedia.org/ipam/prefixes/1092/
*** 2a02:ec80:a100:1::/64 -- cloud-flat-1-codfw1dev-v6 https://netbox.wikimedia.org/ipam/prefixes/1084/
In eqiad:
* 2a02:ec80:a000::/40 -- wmcs eqiad https://netbox.wikimedia.org/ipam/prefixes/1090/
** 2a02:ec80:a000::/55 -- wmcs eqiad openstack https://netbox.wikimedia.org/ipam/prefixes/1093/
*** 2a02:ec80:a000:1::/64 -- cloud-flat-1-dualstack-ipv6 https://netbox.wikimedia.org/ipam/prefixes/1102/
IPv4
In codfw:
* 172.16.130.0/24 -- cloud-flat-codfw1dev-ipv4only https://netbox.wikimedia.org/ipam/prefixes/1101/
* 172.16.129.0/24 -- cloud-flat-codfw1dev https://netbox.wikimedia.org/ipam/prefixes/928/
In eqiad:
* 172.16.8.0/21 -- cloud-flat-1-ipv4only https://netbox.wikimedia.org/ipam/prefixes/1076/
* 172.16.16.0/21 -- cloud-flat-1-dualstack-ipv4 https://netbox.wikimedia.org/ipam/prefixes/1103/
=== other details ===
Some additional information.
* openstack APIs: can only work on IPv4 at the moment. We will need to enable IPv6 for it, see also phab:T379282.
* nova-proxy: can only work on IPv4 at the moment. We will need to enable IPv6 for it, see also phab:T379175.
* toolforge: we will need to figure out a number of things before introducing IPv6 inside Kubernetes, see also phab:T380060.
* tenant networks: we have plenty of options available to us at the moment. But it was decided that this was out of the scope for this migration.
== Log ==
Log of relevant events related to this project, including decisions, or others.
Date 	Event 	Who 	Additional comments
2024-11-25	Attempt to enable VXLAN/IPv6 on eqiad1 failed. There was a rollback. 	Arturo 	
2024-11-22	User-facing migration documentation bootstrapped 	Arturo 	
2024-11-21	Migration start date decided to be on 2025-01-06 	WMCS 	WMF policy to don't announce new projects during December.
2024-11-07	Decided to keep 1500 MTU for the physical network 	Cathal & Arturo	tricky to change -- not causing any trouble today
2024-11-06	Decision record - How to do the Cloud VPS VXLAN/IPv6 migration 	WMCS 	
2024-10-17	Decision Request - How to do the Cloud VPS VXLAN/IPv6 migration started	WMCS 	
2024-10-16	IPv6 virtual machine firewalling semantics clarified 	WMCS 	based purely on neutron security groups
2024-10-11	IPv6 edge routing enabled in codfw1dev 	WMCS & NetOps 	north-south traffic
2024-10-02	Initial IPv6 support enabled in codfw1dev 	WMCS 	east-west traffic
2024-09-16	IPv6 addressing plan agreement reached 	WMCS & NetOps 	
2024-09-13	IPv6 rollout should be paired with the VXLAN migration 	Cathal & Arturo	
2024-05-13	Initial stages of the VXLAN migration started 	Taavi 	
2024-04-24	IPv6 project declared non-stalled 	WMCS & NetOps 	Both teams declare interest in re-starting work on this "soon"
2021-08-27	IPv6 project declared stalled 	 	
2018-02-21	Initial attempt to decide on IPv6 CIDR allocations for WMCS started 	NetOps 	
== See also ==
* T364725 Migrate Cloud VPS instances to VXLAN based networks
* News/2025 Cloud VPS VXLAN IPv6 migration