Chapter 16. Deploying routed provider networks 16.1. Advantages of routed provider networks In Red Hat OpenStack Platform (RHOSP), administrators can create routed provider networks. Routed provider networks are typically used in edge deployments, and rely on multiple layer 2 network segments instead of traditional networks that have only one segment. Routed provider networks simplify the cloud for end users because they see only one network. For administrators, routed provider networks deliver scalabilty and fault tolerance. For example, if a major error occurs, only one segment is impacted instead of the entire network failing. Before routed provider networks, administrators typically had to choose from one of the following architectures: A single, large layer 2 network Multiple, smaller layer 2 networks Single, large layer 2 networks become complex when scaling and reduce fault tolerance (increase failure domains). Multiple, smaller layer 2 networks scale better and shrink failure domains, but can introduce complexity for end users. Starting with RHOSP 16.2 and later, you can deploy routed provider networks using the Modular Layer 2 plug-in with the Open Virtual Network mechanism driver (ML2/OVN). (Routed provider network support for the ML2/Open vSwitch (OVS) and SR-IOV mechanism drivers was introduced in RHOSP 16.1.1.) Additional resources Section 16.2, "Fundamentals of routed provider networks" 16.2. Fundamentals of routed provider networks A routed provider network is different from other types of networks because of the one-to-one association between a network subnet and a segment. In the past, the Red Hat OpenStack (RHOSP) Networking service has not supported routed provider networks, because the Networking service required that all subnets must either belong to the same segment or to no segment. With routed provider networks, the IP addresses available to virtual machine (VM) instances depend on the segment of the network available on the particular compute node. The Networking service port can be associated with only one network segment. Similar to conventional networking, layer 2 (switching) handles transit of traffic between ports on the same network segment and layer 3 (routing) handles transit of traffic between segments. The Networking service does not provide layer 3 services between segments. Instead, it relies on physical network infrastructure to route subnets. Thus, both the Networking service and physical network infrastructure must contain configuration for routed provider networks, similar to conventional provider networks. You can configure the Compute scheduler to filter Compute nodes that have affinity with routed network segments, so that the scheduler places instances only on Compute nodes that are in the required routed provider network segment. If you require a DHCP-metadata service, you must define an availability zone for each edge site or network segment, to ensure that the local DHCP agent is deployed. Additional resources Section 16.1, "Advantages of routed provider networks" 16.3. Limitations of routed provider networks The known constraints of routed provider networks in Red Hat OpenStack Platform include: North-south routing with central SNAT or a floating IP is not supported. When using SR-IOV or PCI pass-through, physical network (physnet) names must be the same in central and remote sites or segments. You cannot reuse segment IDs. 16.4. Preparing for a routed provider network To create a routed provider network in Red Hat OpenStack Platform (RHOSP), you must first gather the network information that is required to create it. You must configure the overcloud to create a custom role that deploys a RHOSP Networking service (neutron) metadata agent for the Compute nodes that contain the network segments. For environments that use the ML2/OVS mechanism driver, in addition to the metadata agent, you must also include the NeutronDhcpAgent service on the Compute nodes. On the Controllers that are running the Compute scheduler services, you must enable scheduling support for routed provider networks. Prerequisites You must be a RHOSP user with the admin role. Procedure Gather the VLAN IDs from the tripleo-heat-templates/network_data.yaml file for the network you want to create the routed provider network on, and assign unique physical network names for each segment that you will create on the routed provider network. This enables reuse of the same segmentation details between subnets. Create a reference table to visualize the relationships between the VLAN IDs, segments, and physical network names: Table 16.1. Example - routed provider network segment definitions Routed provider network VLAN ID Segment Physical network multisegment1 128 segment1 provider1 multisegment1 129 segment2 provider2 Plan the routing between segments. Each subnet on a segment must contain the gateway address of the router interface on that particular subnet. You need the subnet address in both IPv4 and IPv6 formats. Table 16.2. Example - routing plan for routed provider network segments Routed provider network Segment Subnet address Gateway address multisegment1 segment1 (IPv4) 203.0.113.0/24 203.0.113.1 multisegment1 segment1 (IPv6) fd00:203:0:113::/64 fd00:203:0:113::1 multisegment1 segment2 (IPv4) 198.51.100.0/24 198.51.100.1 multisegment1 segment2 (IPv6) fd00:198:51:100::/64 fd00:198:51:100::1 Routed provider networks require that Compute nodes reside on different segments. Check the templates/overcloud-baremetal-deployed.yaml file to ensure that every Compute host in a routed provider network has direct connectivity to one of its segments. For more information, see Provisioning bare metal nodes for the overcloud in the Installing and managing Red Hat OpenStack Platform with director guide. Ensure that the NeutronMetadataAgent service is included in templates/roles_data-custom.yaml for the Compute nodes containing the segments: For more information, see Composable services and custom roles in the Customizing your Red Hat OpenStack Platform deployment guide. When using the ML2/OVS mechanism driver, in addition to the NeutronMetadataAgent service, also ensure that the NeutronDhcpAgent service is included in templates/roles_data-custom.yaml for the Compute nodes containing the segments: Tip Unlike conventional provider networks, a DHCP agent cannot support more than one segment within a network. Deploy DHCP agents on the Compute nodes containing the segments rather than on the network nodes to reduce the node count. Create an routed provider network environment file, for example, rpn_env.yaml . Configure DHCP to enable metadata support on isolated networks: Ensure that the segments service plug-in is loaded into the Networking service: If the segments plug-in is missing, add it to the NeutronServicePlugins parameter: Example Important When you add new values to the NeutronServicePlugins parameter, RHOSP director overwrites any previously declared values with the ones that you are adding. Therefore, when you are adding segments , you must also include any previously declared Networking service plug-ins. To verify the network with the Placement service before scheduling an instance on a host, enable scheduling support for routed provider networks on the Controllers that are running the Compute scheduler services. Example Add your routed provider network environment file to the stack with your other environment files and deploy the overcloud: steps Creating a routed provider network Additional resources Provisioning bare metal nodes for the overcloud in the Installing and managing Red Hat OpenStack Platform with director guide Composable services and custom roles in the Customizing your Red Hat OpenStack Platform deployment guide 16.5. Creating a routed provider network Routed provider networks simplify the Red Hat OpenStack Platform (RHOSP) cloud for end users because they see only one network. For administrators, routed provider networks deliver scalabilty and fault tolerance. When you perform this procedure, you create an routed provider network with two network segments. Each segment contains one IPv4 subnet and one IPv6 subnet. Prerequisites Complete the steps in Section 16.4, "Preparing for a routed provider network" . You must be a RHOSP user with the admin role. Procedure Create a VLAN provider network that includes a default segment. In this example, the VLAN provider network is named multisegment1 and uses a physical network called provider1 and a VLAN whose ID is 128 : Example Sample output Rename the default network segment to segment1 . Obtain the segment ID: Sample output Using the segment ID, rename the network segment to segment1 : Create a second segment on the provider network. In this example, the network segment uses a physical network called provider2 and a VLAN whose ID is 129 : Example Sample output Verify that the network contains the segment1 and segment2 segments: Sample output Create one IPv4 subnet and one IPv6 subnet on the segment1 segment. In this example, the IPv4 subnet uses 203.0.113.0/24 : Example Sample output In this example, the IPv6 subnet uses fd00:203:0:113::/64 : Example Sample output Note By default, IPv6 subnets on provider networks rely on physical network infrastructure for stateless address autoconfiguration (SLAAC) and router advertisement. Create one IPv4 subnet and one IPv6 subnet on the segment2 segment. In this example, the IPv4 subnet uses 198.51.100.0/24 : Example Sample output In this example, the IPv6 subnet uses fd00:198:51:100::/64 : Example Sample output Verification Verify that each IPv4 subnet associates with at least one DHCP agent: Sample output Verify that inventories were created for each segment IPv4 subnet in the Compute service placement API. Run this command for all segment IDs: Sample output In this sample output, only one of the segments is shown: Verify that host aggregates were created for each segment in the Compute service: Sample output In this example, only one of the segments is shown: Launch one or more instances. Each instance obtains IP addresses according to the segment it uses on the particular compute node. Note If a fixed IP is specified by the user in the port create request, that particular IP is allocated immediately to the port. However, creating a port and passing it to an instance yields a different behavior than conventional networks. If the fixed IP is not specified on the port create request, the Networking service defers assignment of IP addresses to the port until the particular compute node becomes apparent. For example, when you run this command: Sample output Additional resources Section 16.4, "Preparing for a routed provider network" network create in the Command line interface reference network segment create in the Command line interface reference subnet create in the Command line interface reference port create in the Command line interface reference 16.6. Migrating a non-routed network to a routed provider network You can migrate a non-routed network to a routed provider network by associating the subnet of the network with the ID of the network segment. Prerequisites The non-routed network you are migrating must contain only one segment and only one subnet. Important In non-routed provider networks that contain multiple subnets or network segments it is not possible to safely migrate to an routed provider network. In non-routed networks, addresses from the subnet allocation pools are assigned to ports without consideration of the network segment to which the port is bound. Procedure For the network that is being migrated, obtain the ID of the current network segment. Example Sample output For the network that is being migrated, obtain the ID of the current subnet. Example Sample output Verify that the current segment_id of the subnet has a value of None . Example Sample output Change the value of the subnet segment_id to the network segment ID. Here is an example: Verification Verify that the subnet is now associated with the desired network segment. Example Sample output Additional resources subnet show in the Command line interface reference subnet set in the Command line interface reference

Commands:
- name: Compute ServicesDefault: - OS::TripleO::Services::NeutronMetadataAgent
- name: Compute ServicesDefault: - OS::TripleO::Services::NeutronDhcpAgent - OS::TripleO::Services::NeutronMetadataAgent
parameter_defaults: NeutronEnableIsolatedMetadata: true
openstack extension list --network --max-width 80 | grep -E "Segment"
parameter_defaults: NeutronEnableIsolatedMetadata: true NeutronServicePlugins: 'router,qos,segments,trunk,placement'
parameter_defaults: NeutronEnableIsolatedMetadata: true NeutronServicePlugins: 'router,qos,segments,trunk,placement' NovaSchedulerQueryPlacementForRoutedNetworkAggregates: true
openstack overcloud deploy --templates -e <your_environment_files> -e /home/stack/templates/rpn_env.yaml
openstack network create --share --provider-physical-network provider1 --provider-network-type vlan --provider-segment 128 multisegment1
+---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | UP | | id | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 | | ipv4_address_scope | None | | ipv6_address_scope | None | | l2_adjacency | True | | mtu | 1500 | | name | multisegment1 | | port_security_enabled | True | | provider:network_type | vlan | | provider:physical_network | provider1 | | provider:segmentation_id | 128 | | revision_number | 1 | | router:external | Internal | | shared | True | | status | ACTIVE | | subnets | | | tags | [] | +---------------------------+--------------------------------------+
openstack network segment list --network multisegment1
+--------------------------------------+----------+--------------------------------------+--------------+---------+ | ID | Name | Network | Network Type | Segment | +--------------------------------------+----------+--------------------------------------+--------------+---------+ | 43e16869-ad31-48e4-87ce-acf756709e18 | None | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 | vlan | 128 | +--------------------------------------+----------+--------------------------------------+--------------+---------+
openstack network segment set --name segment1 43e16869-ad31-48e4-87ce-acf756709e18
openstack network segment create --physical-network provider2 --network-type vlan --segment 129 --network multisegment1 segment2
+------------------+--------------------------------------+ | Field | Value | +------------------+--------------------------------------+ | description | None | | headers | | | id | 053b7925-9a89-4489-9992-e164c8cc8763 | | name | segment2 | | network_id | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 | | network_type | vlan | | physical_network | provider2 | | revision_number | 1 | | segmentation_id | 129 | | tags | [] | +------------------+--------------------------------------+
openstack network segment list --network multisegment1
+--------------------------------------+----------+--------------------------------------+--------------+---------+ | ID | Name | Network | Network Type | Segment | +--------------------------------------+----------+--------------------------------------+--------------+---------+ | 053b7925-9a89-4489-9992-e164c8cc8763 | segment2 | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 | vlan | 129 | | 43e16869-ad31-48e4-87ce-acf756709e18 | segment1 | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 | vlan | 128 | +--------------------------------------+----------+--------------------------------------+--------------+---------+
openstack subnet create --network multisegment1 --network-segment segment1 --ip-version 4 --subnet-range 203.0.113.0/24 multisegment1-segment1-v4
+-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | allocation_pools | 203.0.113.2-203.0.113.254 | | cidr | 203.0.113.0/24 | | enable_dhcp | True | | gateway_ip | 203.0.113.1 | | id | c428797a-6f8e-4cb1-b394-c404318a2762 | | ip_version | 4 | | name | multisegment1-segment1-v4 | | network_id | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 | | revision_number | 1 | | segment_id | 43e16869-ad31-48e4-87ce-acf756709e18 | | tags | [] | +-------------------+--------------------------------------+
openstack subnet create --network multisegment1 --network-segment segment1 --ip-version 6 --subnet-range fd00:203:0:113::/64 --ipv6-address-mode slaac multisegment1-segment1-v6
+-------------------+------------------------------------------------------+ | Field | Value | +-------------------+------------------------------------------------------+ | allocation_pools | fd00:203:0:113::2-fd00:203:0:113:ffff:ffff:ffff:ffff | | cidr | fd00:203:0:113::/64 | | enable_dhcp | True | | gateway_ip | fd00:203:0:113::1 | | id | e41cb069-9902-4c01-9e1c-268c8252256a | | ip_version | 6 | | ipv6_address_mode | slaac | | ipv6_ra_mode | None | | name | multisegment1-segment1-v6 | | network_id | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 | | revision_number | 1 | | segment_id | 43e16869-ad31-48e4-87ce-acf756709e18 | | tags | [] | +-------------------+------------------------------------------------------+
openstack subnet create --network multisegment1 --network-segment segment2 --ip-version 4 --subnet-range 198.51.100.0/24 multisegment1-segment2-v4
+-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | allocation_pools | 198.51.100.2-198.51.100.254 | | cidr | 198.51.100.0/24 | | enable_dhcp | True | | gateway_ip | 198.51.100.1 | | id | 242755c2-f5fd-4e7d-bd7a-342ca95e50b2 | | ip_version | 4 | | name | multisegment1-segment2-v4 | | network_id | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 | | revision_number | 1 | | segment_id | 053b7925-9a89-4489-9992-e164c8cc8763 | | tags | [] | +-------------------+--------------------------------------+
openstack subnet create --network multisegment1 --network-segment segment2 --ip-version 6 --subnet-range fd00:198:51:100::/64 --ipv6-address-mode slaac multisegment1-segment2-v6
+-------------------+--------------------------------------------------------+ | Field | Value | +-------------------+--------------------------------------------------------+ | allocation_pools | fd00:198:51:100::2-fd00:198:51:100:ffff:ffff:ffff:ffff | | cidr | fd00:198:51:100::/64 | | enable_dhcp | True | | gateway_ip | fd00:198:51:100::1 | | id | b884c40e-9cfe-4d1b-a085-0a15488e9441 | | ip_version | 6 | | ipv6_address_mode | slaac | | ipv6_ra_mode | None | | name | multisegment1-segment2-v6 | | network_id | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 | | revision_number | 1 | | segment_id | 053b7925-9a89-4489-9992-e164c8cc8763 | | tags | [] | +-------------------+--------------------------------------------------------+
openstack network agent list --agent-type dhcp --network multisegment1
+--------------------------------------+------------+-------------+-------------------+-------+-------+--------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+------------+-------------+-------------------+-------+-------+--------------------+ | c904ed10-922c-4c1a-84fd-d928abaf8f55 | DHCP agent | compute0001 | nova | :-) | UP | neutron-dhcp-agent | | e0b22cc0-d2a6-4f1c-b17c-27558e20b454 | DHCP agent | compute0101 | nova | :-) | UP | neutron-dhcp-agent | +--------------------------------------+------------+-------------+-------------------+-------+-------+--------------------+
SEGMENT_ID=053b7925-9a89-4489-9992-e164c8cc8763 openstack resource provider inventory list USDSEGMENT_ID
+----------------+------------------+----------+----------+-----------+----------+-------+ | resource_class | allocation_ratio | max_unit | reserved | step_size | min_unit | total | +----------------+------------------+----------+----------+-----------+----------+-------+ | IPV4_ADDRESS | 1.0 | 1 | 2 | 1 | 1 | 30 | +----------------+------------------+----------+----------+-----------+----------+-------+
openstack aggregate list
+----+---------------------------------------------------------+-------------------+ | Id | Name | Availability Zone | +----+---------------------------------------------------------+-------------------+ | 10 | Neutron segment id 053b7925-9a89-4489-9992-e164c8cc8763 | None | +----+---------------------------------------------------------+-------------------+
openstack port create --network multisegment1 port1
+-----------------------+--------------------------------------+ | Field | Value | +-----------------------+--------------------------------------+ | admin_state_up | UP | | binding_vnic_type | normal | | id | 6181fb47-7a74-4add-9b6b-f9837c1c90c4 | | ip_allocation | deferred | | mac_address | fa:16:3e:34:de:9b | | name | port1 | | network_id | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 | | port_security_enabled | True | | revision_number | 1 | | security_groups | e4fcef0d-e2c5-40c3-a385-9c33ac9289c5 | | status | DOWN | | tags | [] | +-----------------------+--------------------------------------+
openstack network segment list --network my_network
+--------------------------------------+------+--------------------------------------+--------------+---------+ | ID | Name | Network | Network Type | Segment | +--------------------------------------+------+--------------------------------------+--------------+---------+ | 81e5453d-4c9f-43a5-8ddf-feaf3937e8c7 | None | 45e84575-2918-471c-95c0-018b961a2984 | flat | None | +--------------------------------------+------+--------------------------------------+--------------+---------+
openstack network segment list --network my_network
+--------------------------------------+-----------+--------------------------------------+---------------+ | ID | Name | Network | Subnet | +--------------------------------------+-----------+--------------------------------------+---------------+ | 71d931d2-0328-46ae-93bc-126caf794307 | my_subnet | 45e84575-2918-471c-95c0-018b961a2984 | 172.24.4.0/24 | +--------------------------------------+-----------+--------------------------------------+---------------+
openstack subnet show my_subnet --c segment_id
+------------+-------+ | Field | Value | +------------+-------+ | segment_id | None | +------------+-------+
openstack subnet set --network-segment 81e5453d-4c9f-43a5-8ddf-feaf3937e8c7 my_subnet
openstack subnet show my_subnet --c segment_id
+------------+--------------------------------------+ | Field | Value | +------------+--------------------------------------+ | segment_id | 81e5453d-4c9f-43a5-8ddf-feaf3937e8c7 | +------------+--------------------------------------+