Chapter 10. Configuring Quality of Service (QoS) policies You can offer varying service levels for instances by using quality of service (QoS) policies to apply rate limits to egress and ingress traffic. You can apply QoS policies to individual ports. You can also apply QoS policies to a project network, where ports with no specific policy attached inherit the policy. Note Internal network owned ports, such as DHCP and internal router ports, are excluded from network policy application. You can apply, modify, or remove QoS policies dynamically. However, for guaranteed minimum bandwidth QoS policies, you can only apply modifications when there are no instances that use any of the ports the policy is assigned to. 10.1. QoS rules The following rule types define the limits of a particular quality of service (QoS) policy: bandwidth_limit : Provides bandwidth limitations on networks, ports or floating IPs. If implemented, any traffic that exceeds the specified rate is dropped. minimum_bandwidth : Provides minimum bandwidth constraints on certain types of traffic. If implemented, best efforts are made to provide no less than the specified bandwidth to each port on which the rule is applied. dscp_marking : Marks network traffic with a Differentiated Services Code Point (DSCP) value. Related procedures The following procedures describe how to create a QoS policy with each rule type: Creating and applying a bandwidth limit QoS policy and rule Creating and applying a guaranteed minimum bandwidth QoS policy and rule Creating and applying a DSCP marking QoS policy and rule for egress traffic 10.2. Creating and applying a QoS policy and rule To create a quality of service (QoS) policy and rule and apply the policy to a port, complete the following steps: Procedure Identify the ID of the project you want to create the QoS policy for: Create the new QoS policy: Create new rules for the QoS policy: Table 10.1. Rule properties Property Description max_kbps The maximum rate (in Kbps) that the instance can send. max_burst_kbps The maximum amount of data (in kbits) that the port can send in an instant if the token buffer is full. The token buffer replenishes at a "max_kbps" rate. The burst value for TCP traffic can be set as 80% of desired bandwidth limit value. For example, if the bandwidth limit is set to 1000kbps then a burst value of 800kbps is sufficient. Note If the burst value is set too low, bandwidth usage is throttled even with a proper bandwidth limit setting, resulting in a lower than expected bandwidth. If the configured burst value is too high, too few packets could be limited, resulting in a higher than expected bandwidth limit. min-kbps The minimum bandwidth (in Kbps) guaranteed to an instance. ingress/egress The direction of traffic the rule is applied to. From the perspective of a cloud server, ingress indicates download, and egress indicates upload. dscp-mark Specifies the decimal value of a DSCP mark. Configure a port or network to apply the policy to. You can update an existing port or network, or create a new port or network to apply the policy to: Apply the policy to an existing port: Create a new port: Apply the policy to an existing network: Create a new network: 10.2.1. Creating and applying a bandwidth limit QoS policy and rule You can create a QoS policy that limits the bandwidth on networks, ports or floating IPs, and drops any traffic that exceeds the specified rate. To create and apply a bandwidth limit QoS policy and rule, complete the following steps: Procedure If the qos extension is not already enabled for OpenStack Networking in /etc/neutron/plugins/ml2/<agent_name>_agent.ini , then perform the following steps: Create a custom environment file, and add the following configuration: To apply this configuration, deploy the overcloud, adding your custom environment file to the stack along with your other environment files: For more information, see "Modifying the Overcloud Environment" in the Director Installation and Usage guide. Identify the ID of the project you want to create the QoS policy for: Create a QoS policy named "bw-limiter" in the admin project: Configure the rules for the "bw-limiter" policy: Configure a port to apply the "bw-limiter" policy to: 10.2.2. Creating and applying a guaranteed minimum bandwidth QoS policy and rule You can request bandwidth guarantees for ports on networks with segmentation_type=flat or segmentation_type=vlan , which are backed by a physical network (physnet). Note Do not mix ports with and without bandwidth guarantees on the same physical interface, as the ports without a guarantee may starve. Create host aggregates to separate ports with bandwidth guarantees from those without. You can only modify a guaranteed minimum bandwidth QoS policy when there are no instances using any of the ports the policy is assigned to. Supported drivers and agents¶ SR-IOV (sriovnicswitch) vnic_types: direct, macvtap ML2/OVS (openvswitch) vnic_types: normal, direct Note ML2/OVN does not support minimum bandwidth. Prerequisites The Placement service must support microversion 1.29. The Compute (nova) service must support microversion 2.72. The Networking (neutron) service must support the following API extensions: agent-resources-synced port-resource-request qos-bw-minimum-ingress To query placement information using the OpenStack CLI, install the Placement service package, python3-osc-placement , on the undercloud. Procedure If the Placement service plugin is not already configured for Openstack networking, then perform the following steps: If the NeutronServicePlugins is already provided in a custom environment file, update the parameter to include "placement", otherwise create a custom environment file and add the following configuration: To apply this configuration, deploy the overcloud, adding your custom environment file to the stack along with your other environment files: For more information, see "Modifying the Overcloud Environment" in the Director Installation and Usage guide. (Optional) To black list vnic_types when multiple ML2 mechanism drivers support them by default and multiple agents are being tracked in Placement, add vnic_type_blacklist to /etc/neutron/plugins/ml2/ml2_conf.ini and restart the agent: Configure the resource provider ingress and egress bandwidths for the relevant agents on each Compute node that needs to provide a minimum bandwidth. You can configure only ingress or egress, or both, using the following formats: Configure only egress bandwidth, in kbps: Configure only ingress bandwidth, in kbps: Configure both egress and ingress bandwidth, in kbps: For example: To configure the resource provider ingress and egress bandwidths for the OVS agent, add resource_provider_bandwidths to /etc/neutron/plugins/ml2/openvswitch_agent.ini : To configure the resource provider ingress and egress bandwidths for the SRIOV agent, add resource_provider_bandwidths to /etc/neutron/plugins/ml2/sriov_agent.ini : To implement the resource provider bandwidths, restart the configured agents. Identify the ID of the project you want to create the QoS policy for: Create a QoS policy named "guaranteed_min_bw" in the admin project: Configure the rules for the "guaranteed_min_bw" policy: Configure a port to apply the "guaranteed_min_bw" policy to: Verification List all of the available resource providers: Example output: Check the bandwidth a specific resource provides: The following example output shows the bandwidth provided by interface enp6s0f1 on dell-r730-014: To check claims against the resource provider when instances are running, run the following command: Example output: 10.2.3. DSCP marking for egress traffic You can use differentiated services code point (DSCP) to implement quality-of-service (QoS) policies on your network by embedding relevant values in the IP headers. OpenStack Networking (neutron) QoS policies can use DSCP marking to manage egress traffic on neutron ports and networks. At present, DSCP is available only for VLAN and flat provider networks using Open vSwitch (OVS). Complete the steps in this example procedure to create a policy, define DSCP rules, and apply the rules to the policy. These rules use the --dscp-mark parameter, which specifies the decimal value of a DSCP mark. Create a new QoS policy: Create a DSCP rule and apply it to the qos-web-servers policy, using DSCP mark 18 : View the DSCP rules for QoS policy, qos-web-servers : View the details of the DSCP rule assigned to policy, qos-web-servers : Change the DSCP value assigned to a rule: Delete a DSCP rule: 10.2.4. How to verify the application of your QoS policy and rules Use the following commands to verify the creation and application of your quality of service (QoS) policy and rules. Action Command List the available QoS policies USD openstack network qos policy list Show details of a specific QoS policy USD openstack network qos policy show <policy_name> List the available QoS rules USD openstack network qos rule type list List the rules of a specific QoS policy USD openstack network qos rule list <policy_name> Show details of a specific rule USD openstack network qos rule type show <rule_id> List the available ports USD openstack port list Show details of a specific port USD openstack port show <port_ID/port_name> 10.3. RBAC for QoS Policies You can add a role-based access control (RBAC) for quality-of-service (QoS) policies. As a result, you can now make QoS policies available to certain projects. For example, you can now create a QoS policy that allows for lower-priority network traffic, and have it only apply to certain projects. Run the following command to assign the bw-limiter policy to the project, demo :

Commands:
(overcloud) USD openstack project list
---------------------------------- ---------+ | ID | Name | ---------------------------------- ---------+ | 8c409e909fb34d69bc896ab358317d60 | admin | | 92b6c16c7c7244378a062be0bfd55fa0 | service | ---------------------------------- ---------+
(overcloud) USD openstack network qos policy create --share --project <project_ID> <policy_name>
(overcloud) USD openstack network qos rule create --type <rule-type> [rule properties] <policy_name>
(overcloud) USD openstack port set --qos-policy <policy_name> <port_name|port_ID>
(overcloud) USD openstack port create --qos-policy <policy_name> --network <network_name|network_ID> <port_name|port_ID>
(overcloud) USD openstack network set --qos-policy <policy_name> <network_name|network_ID>
(overcloud) USD openstack network create --qos-policy <policy_name> <network_name>
parameter_defaults: NeutronSriovAgentExtensions: 'qos'
(undercloud) USD openstack overcloud deploy --templates -e [your environment files] -e /home/stack/templates/<custom-environment-file>.yaml
(overcloud) USD openstack project list +----------------------------------+----------+ | ID | Name | +----------------------------------+----------+ | 4b0b98f8c6c040f38ba4f7146e8680f5 | auditors | | 519e6344f82e4c079c8e2eabb690023b | services | | 80bf5732752a41128e612fe615c886c6 | demo | | 98a2f53c20ce4d50a40dac4a38016c69 | admin | +----------------------------------+----------+
(overcloud) USD openstack network qos policy create --share --project 98a2f53c20ce4d50a40dac4a38016c69 bw-limiter
(overcloud) USD openstack network qos rule create --type bandwidth-limit --max_kbps 3000 --max_burst_kbps 300 bw-limiter
(overcloud) USD openstack port set --qos-policy bw-limiter <port_name|port_ID>
parameter_defaults: NeutronServicePlugins: 'router,qos,segments,trunk,placement'
(undercloud) USD openstack overcloud deploy --templates -e [your environment files] -e /home/stack/templates/network-environment.yaml
[ovs_driver] vnic_type_blacklist = direct vnic_type_blacklist = direct
resource_provider_bandwidths = <bridge0>:<egress_kbps>:,<bridge1>:<egress_kbps>:,...,<bridgeN>:<egress_kbps>:
resource_provider_bandwidths = <bridge0>::<ingress_kbps>,<bridge1>::<ingress_kbps>,...,<bridgeN>::<ingress_kbps>
resource_provider_bandwidths = <bridge0>:<egress_kbps>:<ingress_kbps>,<bridge1>:<egress_kbps>:<ingress_kbps>,...,<bridgeN>:<egress_kbps>:<ingress_kbps>
[ovs] bridge_mappings = physnet0:br-physnet0 resource_provider_bandwidths = br-physnet0:10000000:10000000
[sriov_nic] physical_device_mappings = physnet0:ens5,physnet0:ens6 resource_provider_bandwidths = ens5:40000000:40000000,ens6:40000000:40000000
(overcloud) USD openstack project list +----------------------------------+----------+ | ID | Name | +----------------------------------+----------+ | 4b0b98f8c6c040f38ba4f7146e8680f5 | auditors | | 519e6344f82e4c079c8e2eabb690023b | services | | 80bf5732752a41128e612fe615c886c6 | demo | | 98a2f53c20ce4d50a40dac4a38016c69 | admin | +----------------------------------+----------+
(overcloud) USD openstack network qos policy create --share --project 98a2f53c20ce4d50a40dac4a38016c69 guaranteed_min_bw
(overcloud) USD openstack network qos rule create --type minimum-bandwidth --min-kbps 40000000 --ingress guaranteed_min_bw (overcloud) USD openstack network qos rule create --type minimum-bandwidth --min-kbps 40000000 --egress guaranteed_min_bw
(overcloud) USD openstack port set --qos-policy guaranteed_min_bw <port_name|port_ID>
(undercloud) USD openstack --os-placement-api-version 1.17 resource provider list
-------------------------------------- ----------------------------------------------------- ------------ -------------------------------------- -------------------------------------- | uuid | name | generation | root_provider_uuid | parent_provider_uuid | -------------------------------------- ----------------------------------------------------- ------------ -------------------------------------- -------------------------------------- | 31d3d88b-bc3a-41cd-9dc0-fda54028a882 | dell-r730-014.localdomain | 28 | 31d3d88b-bc3a-41cd-9dc0-fda54028a882 | None | | 6b15ddce-13cf-4c85-a58f-baec5b57ab52 | dell-r730-063.localdomain | 18 | 6b15ddce-13cf-4c85-a58f-baec5b57ab52 | None | | e2f5082a-c965-55db-acb3-8daf9857c721 | dell-r730-063.localdomain:NIC Switch agent | 0 | 6b15ddce-13cf-4c85-a58f-baec5b57ab52 | 6b15ddce-13cf-4c85-a58f-baec5b57ab52 | | d2fb0ef4-2f45-53a8-88be-113b3e64ba1b | dell-r730-014.localdomain:NIC Switch agent | 0 | 31d3d88b-bc3a-41cd-9dc0-fda54028a882 | 31d3d88b-bc3a-41cd-9dc0-fda54028a882 | | f1ca35e2-47ad-53a0-9058-390ade93b73e | dell-r730-063.localdomain:NIC Switch agent:enp6s0f1 | 13 | 6b15ddce-13cf-4c85-a58f-baec5b57ab52 | e2f5082a-c965-55db-acb3-8daf9857c721 | | e518d381-d590-5767-8f34-c20def34b252 | dell-r730-014.localdomain:NIC Switch agent:enp6s0f1 | 19 | 31d3d88b-bc3a-41cd-9dc0-fda54028a882 | d2fb0ef4-2f45-53a8-88be-113b3e64ba1b | -------------------------------------- ----------------------------------------------------- ------------ -------------------------------------- --------------------------------------
(undercloud) USD openstack --os-placement-api-version 1.17 resource provider inventory list <rp_uuid>
[stack@dell-r730-014 nova]USD openstack --os-placement-api-version 1.17 resource provider inventory list e518d381-d590-5767-8f34-c20def34b252 ---------------------------- ------------------ ---------- ------------ ---------- ----------- ---------- | resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total | ---------------------------- ------------------ ---------- ------------ ---------- ----------- ---------- | NET_BW_EGR_KILOBIT_PER_SEC | 1.0 | 1 | 2147483647 | 0 | 1 | 10000000 | | NET_BW_IGR_KILOBIT_PER_SEC | 1.0 | 1 | 2147483647 | 0 | 1 | 10000000 | ---------------------------- ------------------ ---------- ------------ ---------- ----------- ----------
(undercloud) USD openstack --os-placement-api-version 1.17 resource provider show --allocations <rp_uuid>
[stack@dell-r730-014 nova]USD openstack --os-placement-api-version 1.17 resource provider show --allocations e518d381-d590-5767-8f34-c20def34b252 -f value -c allocations { 3cbb9e07-90a8-4154-8acd-b6ec2f894a83 : { resources : { NET_BW_EGR_KILOBIT_PER_SEC : 1000000, NET_BW_IGR_KILOBIT_PER_SEC : 1000000}}, 8848b88b-4464-443f-bf33-5d4e49fd6204 : { resources : { NET_BW_EGR_KILOBIT_PER_SEC : 1000000, NET_BW_IGR_KILOBIT_PER_SEC : 1000000}}, 9a29e946-698b-4731-bc28-89368073be1a : { resources : { NET_BW_EGR_KILOBIT_PER_SEC : 1000000, NET_BW_IGR_KILOBIT_PER_SEC : 1000000}}, a6c83b86-9139-4e98-9341-dc76065136cc : { resources : { NET_BW_EGR_KILOBIT_PER_SEC : 3000000, NET_BW_IGR_KILOBIT_PER_SEC : 3000000}}, da60e33f-156e-47be-a632-870172ec5483 : { resources : { NET_BW_EGR_KILOBIT_PER_SEC : 1000000, NET_BW_IGR_KILOBIT_PER_SEC : 1000000}}, eb582a0e-8274-4f21-9890-9a0d55114663 : { resources : { NET_BW_EGR_KILOBIT_PER_SEC : 3000000, NET_BW_IGR_KILOBIT_PER_SEC : 3000000}}}
openstack network qos policy create --project 98a2f53c20ce4d50a40dac4a38016c69 qos_policy_name
openstack network qos rule create --type dscp-marking --dscp-mark 18 qos_policy_name Created a new dscp_marking_rule: +-----------+--------------------------------------+ | Field | Value | +-----------+--------------------------------------+ | dscp_mark | 18 | | id | d7f976ec-7fab-4e60-af70-f59bf88198e6 | +-----------+--------------------------------------+
openstack network qos rule list qos-web-servers +-----------+--------------------------------------+ | dscp_mark | id | +-----------+--------------------------------------+ | 18 | d7f976ec-7fab-4e60-af70-f59bf88198e6 | +-----------+--------------------------------------+
openstack network qos rule show qos-web-servers d7f976ec-7fab-4e60-af70-f59bf88198e6 +-----------+--------------------------------------+ | Field | Value | +-----------+--------------------------------------+ | dscp_mark | 18 | | id | d7f976ec-7fab-4e60-af70-f59bf88198e6 | +-----------+--------------------------------------+
openstack network qos rule set --dscp-mark 22 qos-web-servers d7f976ec-7fab-4e60-af70-f59bf88198e6
openstack network qos rule delete qos-web-servers d7f976ec-7fab-4e60-af70-f59bf88198e6
openstack network rbac create --type qos_policy --target-project 80bf5732752a41128e612fe615c886c6 --action access_as_shared bw-limiter