Chapter 26. Configuring policy-based routing to define alternative routes By default, the kernel in RHEL decides where to forward network packets based on the destination address using a routing table. Policy-based routing enables you to configure complex routing scenarios. For example, you can route packets based on various criteria, such as the source address, packet metadata, or protocol. 26.1. Routing traffic from a specific subnet to a different default gateway by using nmcli You can use policy-based routing to configure a different default gateway for traffic from certain subnets. For example, you can configure RHEL as a router that, by default, routes all traffic to internet provider A using the default route. However, traffic received from the internal workstations subnet is routed to provider B. The procedure assumes the following network topology: Prerequisites The system uses NetworkManager to configure the network, which is the default. The RHEL router you want to set up in the procedure has four network interfaces: The enp7s0 interface is connected to the network of provider A. The gateway IP in the provider's network is 198.51.100.2 , and the network uses a /30 network mask. The enp1s0 interface is connected to the network of provider B. The gateway IP in the provider's network is 192.0.2.2 , and the network uses a /30 network mask. The enp8s0 interface is connected to the 10.0.0.0/24 subnet with internal workstations. The enp9s0 interface is connected to the 203.0.113.0/24 subnet with the company's servers. Hosts in the internal workstations subnet use 10.0.0.1 as the default gateway. In the procedure, you assign this IP address to the enp8s0 network interface of the router. Hosts in the server subnet use 203.0.113.1 as the default gateway. In the procedure, you assign this IP address to the enp9s0 network interface of the router. The firewalld service is enabled and active. Procedure Configure the network interface to provider A: The nmcli connection add command creates a NetworkManager connection profile. The command uses the following options: type ethernet : Defines that the connection type is Ethernet. con-name <connection_name> : Sets the name of the profile. Use a meaningful name to avoid confusion. ifname <network_device> : Sets the network interface. ipv4.method manual : Enables to configure a static IP address. ipv4.addresses <IP_address> / <subnet_mask> : Sets the IPv4 addresses and subnet mask. ipv4.gateway <IP_address> : Sets the default gateway address. ipv4.dns <IP_of_DNS_server> : Sets the IPv4 address of the DNS server. connection.zone <firewalld_zone> : Assigns the network interface to the defined firewalld zone. Note that firewalld automatically enables masquerading for interfaces assigned to the external zone. Configure the network interface to provider B: This command uses the ipv4.routes parameter instead of ipv4.gateway to set the default gateway. This is required to assign the default gateway for this connection to a different routing table ( 5000 ) than the default. NetworkManager automatically creates this new routing table when the connection is activated. Configure the network interface to the internal workstations subnet: This command uses the ipv4.routes parameter to add a static route to the routing table with ID 5000 . This static route for the 10.0.0.0/24 subnet uses the IP of the local network interface to provider B ( 192.0.2.1 ) as hop. Additionally, the command uses the ipv4.routing-rules parameter to add a routing rule with priority 5 that routes traffic from the 10.0.0.0/24 subnet to table 5000 . Low values have a high priority. Note that the syntax in the ipv4.routing-rules parameter is the same as in an ip rule add command, except that ipv4.routing-rules always requires specifying a priority. Configure the network interface to the server subnet: Verification On a RHEL host in the internal workstation subnet: Install the traceroute package: Use the traceroute utility to display the route to a host on the internet: The output of the command displays that the router sends packets over 192.0.2.1 , which is the network of provider B. On a RHEL host in the server subnet: Install the traceroute package: Use the traceroute utility to display the route to a host on the internet: The output of the command displays that the router sends packets over 198.51.100.2 , which is the network of provider A. Troubleshooting steps On the RHEL router: Display the rule list: By default, RHEL contains rules for the tables local , main , and default . Display the routes in table 5000 : Display the interfaces and firewall zones: Verify that the external zone has masquerading enabled: Additional resources nmcli(1) and nm-settings(5) man pages on your system 26.2. Routing traffic from a specific subnet to a different default gateway by using the network RHEL system role You can use policy-based routing to configure a different default gateway for traffic from certain subnets. For example, you can configure RHEL as a router that, by default, routes all traffic to internet provider A using the default route. However, traffic received from the internal workstations subnet is routed to provider B. By using Ansible and the network RHEL system role, you can automate this process and remotely configure connection profiles on the hosts defined in a playbook. You can use the network RHEL system role to configure the connection profiles, including routing tables and rules. This procedure assumes the following network topology: Prerequisites You have prepared the control node and the managed nodes You are logged in to the control node as a user who can run playbooks on the managed nodes. The account you use to connect to the managed nodes has sudo permissions on them. The managed nodes use NetworkManager and the firewalld service. The managed nodes you want to configure has four network interfaces: The enp7s0 interface is connected to the network of provider A. The gateway IP in the provider's network is 198.51.100.2 , and the network uses a /30 network mask. The enp1s0 interface is connected to the network of provider B. The gateway IP in the provider's network is 192.0.2.2 , and the network uses a /30 network mask. The enp8s0 interface is connected to the 10.0.0.0/24 subnet with internal workstations. The enp9s0 interface is connected to the 203.0.113.0/24 subnet with the company's servers. Hosts in the internal workstations subnet use 10.0.0.1 as the default gateway. In the procedure, you assign this IP address to the enp8s0 network interface of the router. Hosts in the server subnet use 203.0.113.1 as the default gateway. In the procedure, you assign this IP address to the enp9s0 network interface of the router. Procedure Create a playbook file, for example ~/playbook.yml , with the following content: --- - name: Configuring policy-based routing hosts: managed-node-01.example.com tasks: - name: Routing traffic from a specific subnet to a different default gateway ansible.builtin.include_role: name: rhel-system-roles.network vars: network_connections: - name: Provider-A interface_name: enp7s0 type: ethernet autoconnect: True ip: address: - 198.51.100.1/30 gateway4: 198.51.100.2 dns: - 198.51.100.200 state: up zone: external - name: Provider-B interface_name: enp1s0 type: ethernet autoconnect: True ip: address: - 192.0.2.1/30 route: - network: 0.0.0.0 prefix: 0 gateway: 192.0.2.2 table: 5000 state: up zone: external - name: Internal-Workstations interface_name: enp8s0 type: ethernet autoconnect: True ip: address: - 10.0.0.1/24 route: - network: 10.0.0.0 prefix: 24 table: 5000 routing_rule: - priority: 5 from: 10.0.0.0/24 table: 5000 state: up zone: trusted - name: Servers interface_name: enp9s0 type: ethernet autoconnect: True ip: address: - 203.0.113.1/24 state: up zone: trusted The settings specified in the example playbook include the following: table: <value> Assigns the route from the same list entry as the table variable to the specified routing table. routing_rule: <list> Defines the priority of the specified routing rule and from a connection profile to which routing table the rule is assigned. zone: <zone_name> Assigns the network interface from a connection profile to the specified firewalld zone. For details about all variables used in the playbook, see the /usr/share/ansible/roles/rhel-system-roles.network/README.md file on the control node. Validate the playbook syntax: Note that this command only validates the syntax and does not protect against a wrong but valid configuration. Run the playbook: Verification On a RHEL host in the internal workstation subnet: Install the traceroute package: Use the traceroute utility to display the route to a host on the internet: The output of the command displays that the router sends packets over 192.0.2.1 , which is the network of provider B. On a RHEL host in the server subnet: Install the traceroute package: Use the traceroute utility to display the route to a host on the internet: The output of the command displays that the router sends packets over 198.51.100.2 , which is the network of provider A. On the RHEL router that you configured using the RHEL system role: Display the rule list: By default, RHEL contains rules for the tables local , main , and default . Display the routes in table 5000 : Display the interfaces and firewall zones: Verify that the external zone has masquerading enabled: Additional resources /usr/share/ansible/roles/rhel-system-roles.network/README.md file /usr/share/doc/rhel-system-roles/network/ directory 26.3. Overview of configuration files involved in policy-based routing when using the legacy network scripts If you use the legacy network scripts instead of NetworkManager to configure your network, you can also configure policy-based routing. Note Configuring the network using the legacy network scripts provided by the network-scripts package is deprecated in RHEL 8. Use NetworkManager to configure policy-based routing. For an example, see Routing traffic from a specific subnet to a different default gateway by using nmcli . The following configuration files are involved in policy-based routing when you use the legacy network scripts: /etc/sysconfig/network-scripts/route- interface : This file defines the IPv4 routes. Use the table option to specify the routing table. For example: /etc/sysconfig/network-scripts/route6- interface : This file defines the IPv6 routes. /etc/sysconfig/network-scripts/rule- interface : This file defines the rules for IPv4 source networks for which the kernel routes traffic to specific routing tables. For example: /etc/sysconfig/network-scripts/rule6- interface : This file defines the rules for IPv6 source networks for which the kernel routes traffic to specific routing tables. /etc/iproute2/rt_tables : This file defines the mappings if you want to use names instead of numbers to refer to specific routing tables. For example: Additional resources ip-route(8) and ip-rule(8) man pages on your system 26.4. Routing traffic from a specific subnet to a different default gateway by using the legacy network scripts You can use policy-based routing to configure a different default gateway for traffic from certain subnets. For example, you can configure RHEL as a router that, by default, routes all traffic to internet provider A using the default route. However, traffic received from the internal workstations subnet is routed to provider B. Important Configuring the network using the legacy network scripts provided by the network-scripts package is deprecated in RHEL 8. Follow the procedure only if you use the legacy network scripts instead of NetworkManager on your host. If you use NetworkManager to manage your network settings, see Routing traffic from a specific subnet to a different default gateway by using nmcli . The procedure assumes the following network topology: Note The legacy network scripts process configuration files in alphabetical order. Therefore, you must name the configuration files in a way that ensures that an interface, that is used in rules and routes of other interfaces, are up when a depending interface requires it. To accomplish the correct order, this procedure uses numbers in the ifcfg-* , route-* , and rules-* files. Prerequisites The NetworkManager package is not installed, or the NetworkManager service is disabled. The network-scripts package is installed. The RHEL router you want to set up in the procedure has four network interfaces: The enp7s0 interface is connected to the network of provider A. The gateway IP in the provider's network is 198.51.100.2 , and the network uses a /30 network mask. The enp1s0 interface is connected to the network of provider B. The gateway IP in the provider's network is 192.0.2.2 , and the network uses a /30 network mask. The enp8s0 interface is connected to the 10.0.0.0/24 subnet with internal workstations. The enp9s0 interface is connected to the 203.0.113.0/24 subnet with the company's servers. Hosts in the internal workstations subnet use 10.0.0.1 as the default gateway. In the procedure, you assign this IP address to the enp8s0 network interface of the router. Hosts in the server subnet use 203.0.113.1 as the default gateway. In the procedure, you assign this IP address to the enp9s0 network interface of the router. The firewalld service is enabled and active. Procedure Add the configuration for the network interface to provider A by creating the /etc/sysconfig/network-scripts/ifcfg-1_Provider-A file with the following content: The configuration file uses the following parameters: TYPE = Ethernet : Defines that the connection type is Ethernet. IPADDR = IP_address : Sets the IPv4 address. PREFIX = subnet_mask : Sets the subnet mask. GATEWAY = IP_address : Sets the default gateway address. DNS1 = IP_of_DNS_server : Sets the IPv4 address of the DNS server. DEFROUTE = yes|no : Defines whether the connection is a default route or not. NAME = connection_name : Sets the name of the connection profile. Use a meaningful name to avoid confusion. DEVICE = network_device : Sets the network interface. ONBOOT = yes : Defines that RHEL starts this connection when the system boots. ZONE = firewalld_zone : Assigns the network interface to the defined firewalld zone. Note that firewalld automatically enables masquerading for interfaces assigned to the external zone. Add the configuration for the network interface to provider B: Create the /etc/sysconfig/network-scripts/ifcfg-2_Provider-B file with the following content: Note that the configuration file for this interface does not contain a default gateway setting. Assign the gateway for the 2_Provider-B connection to a separate routing table. Therefore, create the /etc/sysconfig/network-scripts/route-2_Provider-B file with the following content: This entry assigns the gateway and traffic from all subnets routed through this gateway to table 5000. Create the configuration for the network interface to the internal workstations subnet: Create the /etc/sysconfig/network-scripts/ifcfg-3_Internal-Workstations file with the following content: Add the routing rule configuration for the internal workstation subnet. Therefore, create the /etc/sysconfig/network-scripts/rule-3_Internal-Workstations file with the following content: This configuration defines a routing rule with priority 5 that routes all traffic from the 10.0.0.0/24 subnet to table 5000 . Low values have a high priority. Create the /etc/sysconfig/network-scripts/route-3_Internal-Workstations file with the following content to add a static route to the routing table with ID 5000 : This static route defines that RHEL sends traffic from the 10.0.0.0/24 subnet to the IP of the local network interface to provider B ( 192.0.2.1 ). This interface is to routing table 5000 and used as the hop. Add the configuration for the network interface to the server subnet by creating the /etc/sysconfig/network-scripts/ifcfg-4_Servers file with the following content: Restart the network: Verification On a RHEL host in the internal workstation subnet: Install the traceroute package: Use the traceroute utility to display the route to a host on the internet: The output of the command displays that the router sends packets over 192.0.2.1 , which is the network of provider B. On a RHEL host in the server subnet: Install the traceroute package: Use the traceroute utility to display the route to a host on the internet: The output of the command displays that the router sends packets over 198.51.100.2 , which is the network of provider A. Troubleshooting steps On the RHEL router: Display the rule list: By default, RHEL contains rules for the tables local , main , and default . Display the routes in table 5000 : Display the interfaces and firewall zones: Verify that the external zone has masquerading enabled: Additional resources Overview of configuration files involved in policy-based routing when using the legacy network scripts ip-route(8) and ip-rule(8) man pages on your system /usr/share/doc/network-scripts/sysconfig.txt file

Commands:
nmcli connection add type ethernet con-name Provider-A ifname enp7s0 ipv4.method manual ipv4.addresses 198.51.100.1/30 ipv4.gateway 198.51.100.2 ipv4.dns 198.51.100.200 connection.zone external
nmcli connection add type ethernet con-name Provider-B ifname enp1s0 ipv4.method manual ipv4.addresses 192.0.2.1/30 ipv4.routes "0.0.0.0/0 192.0.2.2 table=5000" connection.zone external
nmcli connection add type ethernet con-name Internal-Workstations ifname enp8s0 ipv4.method manual ipv4.addresses 10.0.0.1/24 ipv4.routes "10.0.0.0/24 table=5000" ipv4.routing-rules "priority 5 from 10.0.0.0/24 table 5000" connection.zone trusted
nmcli connection add type ethernet con-name Servers ifname enp9s0 ipv4.method manual ipv4.addresses 203.0.113.1/24 connection.zone trusted
yum install traceroute
traceroute redhat.com traceroute to redhat.com (209.132.183.105), 30 hops max, 60 byte packets 1 10.0.0.1 (10.0.0.1) 0.337 ms 0.260 ms 0.223 ms 2 192.0.2.1 (192.0.2.1) 0.884 ms 1.066 ms 1.248 ms
yum install traceroute
traceroute redhat.com traceroute to redhat.com (209.132.183.105), 30 hops max, 60 byte packets 1 203.0.113.1 (203.0.113.1) 2.179 ms 2.073 ms 1.944 ms 2 198.51.100.2 (198.51.100.2) 1.868 ms 1.798 ms 1.549 ms
ip rule list 0: from all lookup local 5 : from 10.0.0.0/24 lookup 5000 32766: from all lookup main 32767: from all lookup default
ip route list table 5000 0.0.0.0/0 via 192.0.2.2 dev enp1s0 proto static metric 100 10.0.0.0/24 dev enp8s0 proto static scope link src 192.0.2.1 metric 102
firewall-cmd --get-active-zones external interfaces: enp1s0 enp7s0 trusted interfaces: enp8s0 enp9s0
firewall-cmd --info-zone=external external (active) target: default icmp-block-inversion: no interfaces: enp1s0 enp7s0 sources: services: ssh ports: protocols: masquerade: yes
--- - name: Configuring policy-based routing hosts: managed-node-01.example.com tasks: - name: Routing traffic from a specific subnet to a different default gateway ansible.builtin.include_role: name: rhel-system-roles.network vars: network_connections: - name: Provider-A interface_name: enp7s0 type: ethernet autoconnect: True ip: address: - 198.51.100.1/30 gateway4: 198.51.100.2 dns: - 198.51.100.200 state: up zone: external - name: Provider-B interface_name: enp1s0 type: ethernet autoconnect: True ip: address: - 192.0.2.1/30 route: - network: 0.0.0.0 prefix: 0 gateway: 192.0.2.2 table: 5000 state: up zone: external - name: Internal-Workstations interface_name: enp8s0 type: ethernet autoconnect: True ip: address: - 10.0.0.1/24 route: - network: 10.0.0.0 prefix: 24 table: 5000 routing_rule: - priority: 5 from: 10.0.0.0/24 table: 5000 state: up zone: trusted - name: Servers interface_name: enp9s0 type: ethernet autoconnect: True ip: address: - 203.0.113.1/24 state: up zone: trusted
ansible-playbook --syntax-check ~/playbook.yml
ansible-playbook ~/playbook.yml
yum install traceroute
traceroute redhat.com traceroute to redhat.com (209.132.183.105), 30 hops max, 60 byte packets 1 10.0.0.1 (10.0.0.1) 0.337 ms 0.260 ms 0.223 ms 2 192.0.2.1 (192.0.2.1) 0.884 ms 1.066 ms 1.248 ms
yum install traceroute
traceroute redhat.com traceroute to redhat.com (209.132.183.105), 30 hops max, 60 byte packets 1 203.0.113.1 (203.0.113.1) 2.179 ms 2.073 ms 1.944 ms 2 198.51.100.2 (198.51.100.2) 1.868 ms 1.798 ms 1.549 ms
ip rule list 0: from all lookup local 5 : from 10.0.0.0/24 lookup 5000 32766: from all lookup main 32767: from all lookup default
ip route list table 5000 0.0.0.0/0 via 192.0.2.2 dev enp1s0 proto static metric 100 10.0.0.0/24 dev enp8s0 proto static scope link src 192.0.2.1 metric 102
firewall-cmd --get-active-zones external interfaces: enp1s0 enp7s0 trusted interfaces: enp8s0 enp9s0
firewall-cmd --info-zone=external external (active) target: default icmp-block-inversion: no interfaces: enp1s0 enp7s0 sources: services: ssh ports: protocols: masquerade: yes
192.0.2.0/24 via 198.51.100.1 table 1 203.0.113.0/24 via 198.51.100.2 table 2
from 192.0.2.0/24 lookup 1 from 203.0.113.0/24 lookup 2
1 Provider_A 2 Provider_B
TYPE=Ethernet IPADDR=198.51.100.1 PREFIX=30 GATEWAY=198.51.100.2 DNS1=198.51.100.200 DEFROUTE=yes NAME=1_Provider-A DEVICE=enp7s0 ONBOOT=yes ZONE=external
TYPE=Ethernet IPADDR=192.0.2.1 PREFIX=30 DEFROUTE=no NAME=2_Provider-B DEVICE=enp1s0 ONBOOT=yes ZONE=external
0.0.0.0/0 via 192.0.2.2 table 5000
TYPE=Ethernet IPADDR=10.0.0.1 PREFIX=24 DEFROUTE=no NAME=3_Internal-Workstations DEVICE=enp8s0 ONBOOT=yes ZONE=internal
pri 5 from 10.0.0.0/24 table 5000
10.0.0.0/24 via 192.0.2.1 table 5000
TYPE=Ethernet IPADDR=203.0.113.1 PREFIX=24 DEFROUTE=no NAME=4_Servers DEVICE=enp9s0 ONBOOT=yes ZONE=internal
systemctl restart network
yum install traceroute
traceroute redhat.com traceroute to redhat.com (209.132.183.105), 30 hops max, 60 byte packets 1 10.0.0.1 (10.0.0.1) 0.337 ms 0.260 ms 0.223 ms 2 192.0.2.1 (192.0.2.1) 0.884 ms 1.066 ms 1.248 ms
yum install traceroute
traceroute redhat.com traceroute to redhat.com (209.132.183.105), 30 hops max, 60 byte packets 1 203.0.113.1 (203.0.113.1) 2.179 ms 2.073 ms 1.944 ms 2 198.51.100.2 (198.51.100.2) 1.868 ms 1.798 ms 1.549 ms
ip rule list 0: from all lookup local 5 : from 10.0.0.0/24 lookup 5000 32766: from all lookup main 32767: from all lookup default
ip route list table 5000 default via 192.0.2.2 dev enp1s0 10.0.0.0/24 via 192.0.2.1 dev enp1s0
firewall-cmd --get-active-zones external interfaces: enp1s0 enp7s0 internal interfaces: enp8s0 enp9s0
firewall-cmd --info-zone=external external (active) target: default icmp-block-inversion: no interfaces: enp1s0 enp7s0 sources: services: ssh ports: protocols: masquerade: yes