17.14. Applying Network Filtering This section provides an introduction to libvirt's network filters, their goals, concepts and XML format. 17.14.1. Introduction The goal of the network filtering, is to enable administrators of a virtualized system to configure and enforce network traffic filtering rules on virtual machines and manage the parameters of network traffic that virtual machines are allowed to send or receive. The network traffic filtering rules are applied on the host physical machine when a virtual machine is started. Since the filtering rules cannot be circumvented from within the virtual machine, it makes them mandatory from the point of view of a virtual machine user. From the point of view of the guest virtual machine, the network filtering system allows each virtual machine's network traffic filtering rules to be configured individually on a per interface basis. These rules are applied on the host physical machine when the virtual machine is started and can be modified while the virtual machine is running. The latter can be achieved by modifying the XML description of a network filter. Multiple virtual machines can make use of the same generic network filter. When such a filter is modified, the network traffic filtering rules of all running virtual machines that reference this filter are updated. The machines that are not running will update on start. As previously mentioned, applying network traffic filtering rules can be done on individual network interfaces that are configured for certain types of network configurations. Supported network types include: network ethernet -- must be used in bridging mode bridge Example 17.1. An example of network filtering The interface XML is used to reference a top-level filter. In the following example, the interface description references the filter clean-traffic. Network filters are written in XML and may either contain: references to other filters, rules for traffic filtering, or hold a combination of both. The above referenced filter clean-traffic is a filter that only contains references to other filters and no actual filtering rules. Since references to other filters can be used, a tree of filters can be built. The clean-traffic filter can be viewed using the command: # virsh nwfilter-dumpxml clean-traffic . As previously mentioned, a single network filter can be referenced by multiple virtual machines. Since interfaces will typically have individual parameters associated with their respective traffic filtering rules, the rules described in a filter's XML can be generalized using variables. In this case, the variable name is used in the filter XML and the name and value are provided at the place where the filter is referenced. Example 17.2. Description extended In the following example, the interface description has been extended with the parameter IP and a dotted IP address as a value. In this particular example, the clean-traffic network traffic filter will be represented with the IP address parameter 10.0.0.1 and as per the rule dictates that all traffic from this interface will always be using 10.0.0.1 as the source IP address, which is one of the purpose of this particular filter. 17.14.2. Filtering Chains Filtering rules are organized in filter chains. These chains can be thought of as having a tree structure with packet filtering rules as entries in individual chains (branches). Packets start their filter evaluation in the root chain and can then continue their evaluation in other chains, return from those chains back into the root chain or be dropped or accepted by a filtering rule in one of the traversed chains. Libvirt's network filtering system automatically creates individual root chains for every virtual machine's network interface on which the user chooses to activate traffic filtering. The user may write filtering rules that are either directly instantiated in the root chain or may create protocol-specific filtering chains for efficient evaluation of protocol-specific rules. The following chains exist: root mac stp (spanning tree protocol) vlan arp and rarp ipv4 ipv6 Multiple chains evaluating the mac, stp, vlan, arp, rarp, ipv4, or ipv6 protocol can be created using the protocol name only as a prefix in the chain's name. Example 17.3. ARP traffic filtering This example allows chains with names arp-xyz or arp-test to be specified and have their ARP protocol packets evaluated in those chains. The following filter XML shows an example of filtering ARP traffic in the arp chain. The consequence of putting ARP-specific rules in the arp chain, rather than for example in the root chain, is that packets protocols other than ARP do not need to be evaluated by ARP protocol-specific rules. This improves the efficiency of the traffic filtering. However, one must then pay attention to only putting filtering rules for the given protocol into the chain since other rules will not be evaluated. For example, an IPv4 rule will not be evaluated in the ARP chain since IPv4 protocol packets will not traverse the ARP chain. 17.14.3. Filtering Chain Priorities As previously mentioned, when creating a filtering rule, all chains are connected to the root chain. The order in which those chains are accessed is influenced by the priority of the chain. The following table shows the chains that can be assigned a priority and their default priorities. Table 17.1. Filtering chain default priorities values Chain (prefix) Default priority stp -810 mac -800 vlan -750 ipv4 -700 ipv6 -600 arp -500 rarp -400 Note A chain with a lower priority value is accessed before one with a higher value. The chains listed in Table 17.1, "Filtering chain default priorities values" can be also be assigned custom priorities by writing a value in the range [-1000 to 1000] into the priority (XML) attribute in the filter node. Section 17.14.2, "Filtering Chains" filter shows the default priority of -500 for arp chains, for example. 17.14.4. Usage of Variables in Filters There are two variables that have been reserved for usage by the network traffic filtering subsystem: MAC and IP. MAC is designated for the MAC address of the network interface. A filtering rule that references this variable will automatically be replaced with the MAC address of the interface. This works without the user having to explicitly provide the MAC parameter. Even though it is possible to specify the MAC parameter similar to the IP parameter above, it is discouraged since libvirt knows what MAC address an interface will be using. The parameter IP represents the IP address that the operating system inside the virtual machine is expected to use on the given interface. The IP parameter is special in so far as the libvirt daemon will try to determine the IP address (and thus the IP parameter's value) that is being used on an interface if the parameter is not explicitly provided but referenced. For current limitations on IP address detection, consult the section on limitations Section 17.14.12, "Limitations" on how to use this feature and what to expect when using it. The XML file shown in Section 17.14.2, "Filtering Chains" contains the filter no-arp-spoofing , which is an example of using a network filter XML to reference the MAC and IP variables. Note that referenced variables are always prefixed with the character USD . The format of the value of a variable must be of the type expected by the filter attribute identified in the XML. In the above example, the IP parameter must hold a legal IP address in standard format. Failure to provide the correct structure will result in the filter variable not being replaced with a value and will prevent a virtual machine from starting or will prevent an interface from attaching when hot plugging is being used. Some of the types that are expected for each XML attribute are shown in the example Example 17.4, "Sample variable types" . Example 17.4. Sample variable types As variables can contain lists of elements, (the variable IP can contain multiple IP addresses that are valid on a particular interface, for example), the notation for providing multiple elements for the IP variable is: This XML file creates filters to enable multiple IP addresses per interface. Each of the IP addresses will result in a separate filtering rule. Therefore, using the XML above and the following rule, three individual filtering rules (one for each IP address) will be created: As it is possible to access individual elements of a variable holding a list of elements, a filtering rule like the following accesses the 2nd element of the variable DSTPORTS . Example 17.5. Using a variety of variables As it is possible to create filtering rules that represent all of the permissible rules from different lists using the notation USDVARIABLE[@<iterator id="x">] . The following rule allows a virtual machine to receive traffic on a set of ports, which are specified in DSTPORTS , from the set of source IP address specified in SRCIPADDRESSES . The rule generates all combinations of elements of the variable DSTPORTS with those of SRCIPADDRESSES by using two independent iterators to access their elements. Assign concrete values to SRCIPADDRESSES and DSTPORTS as shown: Assigning values to the variables using USDSRCIPADDRESSES[@1] and USDDSTPORTS[@2] would then result in all variants of addresses and ports being created as shown: 10.0.0.1, 80 10.0.0.1, 8080 11.1.2.3, 80 11.1.2.3, 8080 Accessing the same variables using a single iterator, for example by using the notation USDSRCIPADDRESSES[@1] and USDDSTPORTS[@1] , would result in parallel access to both lists and result in the following combination: 10.0.0.1, 80 11.1.2.3, 8080 Note USDVARIABLE is short-hand for USDVARIABLE[@0] . The former notation always assumes the role of iterator with iterator id="0" added as shown in the opening paragraph at the top of this section. 17.14.5. Automatic IP Address Detection and DHCP Snooping This section provides information about automatic IP address detection and DHCP snooping. 17.14.5.1. Introduction The detection of IP addresses used on a virtual machine's interface is automatically activated if the variable IP is referenced but no value has been assigned to it. The variable CTRL_IP_LEARNING can be used to specify the IP address learning method to use. Valid values include: any , dhcp , or none . The value any instructs libvirt to use any packet to determine the address in use by a virtual machine, which is the default setting if the variable CTRL_IP_LEARNING is not set. This method will only detect a single IP address per interface. Once a guest virtual machine's IP address has been detected, its IP network traffic will be locked to that address, if for example, IP address spoofing is prevented by one of its filters. In that case, the user of the VM will not be able to change the IP address on the interface inside the guest virtual machine, which would be considered IP address spoofing. When a guest virtual machine is migrated to another host physical machine or resumed after a suspend operation, the first packet sent by the guest virtual machine will again determine the IP address that the guest virtual machine can use on a particular interface. The value of dhcp instructs libvirt to only honor DHCP server-assigned addresses with valid leases. This method supports the detection and usage of multiple IP address per interface. When a guest virtual machine resumes after a suspend operation, any valid IP address leases are applied to its filters. Otherwise the guest virtual machine is expected to use DHCP to obtain a new IP addresses. When a guest virtual machine migrates to another physical host physical machine, the guest virtual machine is required to re-run the DHCP protocol. If CTRL_IP_LEARNING is set to none , libvirt does not do IP address learning and referencing IP without assigning it an explicit value is an error. 17.14.5.2. DHCP Snooping CTRL_IP_LEARNING= dhcp (DHCP snooping) provides additional anti-spoofing security, especially when combined with a filter allowing only trusted DHCP servers to assign IP addresses. To enable this, set the variable DHCPSERVER to the IP address of a valid DHCP server and provide filters that use this variable to filter incoming DHCP responses. When DHCP snooping is enabled and the DHCP lease expires, the guest virtual machine will no longer be able to use the IP address until it acquires a new, valid lease from a DHCP server. If the guest virtual machine is migrated, it must get a new valid DHCP lease to use an IP address (for example by bringing the VM interface down and up again). Note Automatic DHCP detection listens to the DHCP traffic the guest virtual machine exchanges with the DHCP server of the infrastructure. To avoid denial-of-service attacks on libvirt, the evaluation of those packets is rate-limited, meaning that a guest virtual machine sending an excessive number of DHCP packets per second on an interface will not have all of those packets evaluated and thus filters may not get adapted. Normal DHCP client behavior is assumed to send a low number of DHCP packets per second. Further, it is important to setup appropriate filters on all guest virtual machines in the infrastructure to avoid them being able to send DHCP packets. Therefore, guest virtual machines must either be prevented from sending UDP and TCP traffic from port 67 to port 68 or the DHCPSERVER variable should be used on all guest virtual machines to restrict DHCP server messages to only be allowed to originate from trusted DHCP servers. At the same time anti-spoofing prevention must be enabled on all guest virtual machines in the subnet. Example 17.6. Activating IPs for DHCP snooping The following XML provides an example for the activation of IP address learning using the DHCP snooping method: 17.14.6. Reserved Variables Table 17.2, "Reserved variables" shows the variables that are considered reserved and are used by libvirt: Table 17.2. Reserved variables Variable Name Definition MAC The MAC address of the interface IP The list of IP addresses in use by an interface IPV6 Not currently implemented: the list of IPV6 addresses in use by an interface DHCPSERVER The list of IP addresses of trusted DHCP servers DHCPSERVERV6 Not currently implemented: The list of IPv6 addresses of trusted DHCP servers CTRL_IP_LEARNING The choice of the IP address detection mode 17.14.7. Element and Attribute Overview The root element required for all network filters is named <filter> with two possible attributes. The name attribute provides a unique name of the given filter. The chain attribute is optional but allows certain filters to be better organized for more efficient processing by the firewall subsystem of the underlying host physical machine. Currently, the system only supports the following chains: root , ipv4 , ipv6 , arp and rarp . 17.14.8. References to Other Filters Any filter may hold references to other filters. Individual filters may be referenced multiple times in a filter tree but references between filters must not introduce loops. Example 17.7. An Example of a clean traffic filter The following shows the XML of the clean-traffic network filter referencing several other filters. To reference another filter, the XML node <filterref> needs to be provided inside a filter node. This node must have the attribute filter whose value contains the name of the filter to be referenced. New network filters can be defined at any time and may contain references to network filters that are not known to libvirt, yet. However, once a virtual machine is started or a network interface referencing a filter is to be hot-plugged, all network filters in the filter tree must be available. Otherwise the virtual machine will not start or the network interface cannot be attached. 17.14.9. Filter Rules The following XML shows a simple example of a network traffic filter implementing a rule to drop traffic if the IP address (provided through the value of the variable IP) in an outgoing IP packet is not the expected one, thus preventing IP address spoofing by the VM. Example 17.8. Example of network traffic filtering The traffic filtering rule starts with the rule node. This node may contain up to three of the following attributes: action is mandatory can have the following values: drop (matching the rule silently discards the packet with no further analysis) reject (matching the rule generates an ICMP reject message with no further analysis) accept (matching the rule accepts the packet with no further analysis) return (matching the rule passes this filter, but returns control to the calling filter for further analysis) continue (matching the rule goes on to the rule for further analysis) direction is mandatory can have the following values: in for incoming traffic out for outgoing traffic inout for incoming and outgoing traffic priority is optional. The priority of the rule controls the order in which the rule will be instantiated relative to other rules. Rules with lower values will be instantiated before rules with higher values. Valid values are in the range of -1000 to 1000. If this attribute is not provided, priority 500 will be assigned by default. Note that filtering rules in the root chain are sorted with filters connected to the root chain following their priorities. This allows to interleave filtering rules with access to filter chains. See Section 17.14.3, "Filtering Chain Priorities" for more information. statematch is optional. Possible values are '0' or 'false' to turn the underlying connection state matching off. The default setting is 'true' or 1 For more information, see Section 17.14.11, "Advanced Filter Configuration Topics" . The above example Example 17.7, "An Example of a clean traffic filter" indicates that the traffic of type ip will be associated with the chain ipv4 and the rule will have priority= 500 . If for example another filter is referenced whose traffic of type ip is also associated with the chain ipv4 then that filter's rules will be ordered relative to the priority= 500 of the shown rule. A rule may contain a single rule for filtering of traffic. The above example shows that traffic of type ip is to be filtered. 17.14.10. Supported Protocols The following sections list and give some details about the protocols that are supported by the network filtering subsystem. This type of traffic rule is provided in the rule node as a nested node. Depending on the traffic type a rule is filtering, the attributes are different. The above example showed the single attribute srcipaddr that is valid inside the ip traffic filtering node. The following sections show what attributes are valid and what type of data they are expecting. The following datatypes are available: UINT8 : 8 bit integer; range 0-255 UINT16: 16 bit integer; range 0-65535 MAC_ADDR: MAC address in dotted decimal format, for example 00:11:22:33:44:55 MAC_MASK: MAC address mask in MAC address format, for instance, FF:FF:FF:FC:00:00 IP_ADDR: IP address in dotted decimal format, for example 10.1.2.3 IP_MASK: IP address mask in either dotted decimal format (255.255.248.0) or CIDR mask (0-32) IPV6_ADDR: IPv6 address in numbers format, for example FFFF::1 IPV6_MASK: IPv6 mask in numbers format (FFFF:FFFF:FC00::) or CIDR mask (0-128) STRING: A string BOOLEAN: 'true', 'yes', '1' or 'false', 'no', '0' IPSETFLAGS: The source and destination flags of the ipset described by up to 6 'src' or 'dst' elements selecting features from either the source or destination part of the packet header; example: src,src,dst. The number of 'selectors' to provide here depends on the type of ipset that is referenced Every attribute except for those of type IP_MASK or IPV6_MASK can be negated using the match attribute with value no . Multiple negated attributes may be grouped together. The following XML fragment shows such an example using abstract attributes. Rules behave evaluate the rule as well as look at it logically within the boundaries of the given protocol attributes. Thus, if a single attribute's value does not match the one given in the rule, the whole rule will be skipped during the evaluation process. Therefore, in the above example incoming traffic will only be dropped if: the protocol property attribute1 does not match both value1 and the protocol property attribute2 does not match value2 and the protocol property attribute3 matches value3 . 17.14.10.1. MAC (Ethernet) Protocol ID: mac Rules of this type should go into the root chain. Table 17.3. MAC protocol types Attribute Name Datatype Definition srcmacaddr MAC_ADDR MAC address of sender srcmacmask MAC_MASK Mask applied to MAC address of sender dstmacaddr MAC_ADDR MAC address of destination dstmacmask MAC_MASK Mask applied to MAC address of destination protocolid UINT16 (0x600-0xffff), STRING Layer 3 protocol ID. Valid strings include [arp, rarp, ipv4, ipv6] comment STRING text string up to 256 characters The filter can be written as such: 17.14.10.2. VLAN (802.1Q) Protocol ID: vlan Rules of this type should go either into the root or vlan chain. Table 17.4. VLAN protocol types Attribute Name Datatype Definition srcmacaddr MAC_ADDR MAC address of sender srcmacmask MAC_MASK Mask applied to MAC address of sender dstmacaddr MAC_ADDR MAC address of destination dstmacmask MAC_MASK Mask applied to MAC address of destination vlan-id UINT16 (0x0-0xfff, 0 - 4095) VLAN ID encap-protocol UINT16 (0x03c-0xfff), String Encapsulated layer 3 protocol ID, valid strings are arp, ipv4, ipv6 comment STRING text string up to 256 characters 17.14.10.3. STP (Spanning Tree Protocol) Protocol ID: stp Rules of this type should go either into the root or stp chain. Table 17.5. STP protocol types Attribute Name Datatype Definition srcmacaddr MAC_ADDR MAC address of sender srcmacmask MAC_MASK Mask applied to MAC address of sender type UINT8 Bridge Protocol Data Unit (BPDU) type flags UINT8 BPDU flagdstmacmask root-priority UINT16 Root priority range start root-priority-hi UINT16 (0x0-0xfff, 0 - 4095) Root priority range end root-address MAC _ADDRESS root MAC Address root-address-mask MAC _MASK root MAC Address mask roor-cost UINT32 Root path cost (range start) root-cost-hi UINT32 Root path cost range end sender-priority-hi UINT16 Sender priority range end sender-address MAC_ADDRESS BPDU sender MAC address sender-address-mask MAC_MASK BPDU sender MAC address mask port UINT16 Port identifier (range start) port_hi UINT16 Port identifier range end msg-age UINT16 Message age timer (range start) msg-age-hi UINT16 Message age timer range end max-age-hi UINT16 Maximum age time range end hello-time UINT16 Hello time timer (range start) hello-time-hi UINT16 Hello time timer range end forward-delay UINT16 Forward delay (range start) forward-delay-hi UINT16 Forward delay range end comment STRING text string up to 256 characters 17.14.10.4. ARP/RARP Protocol ID: arp or rarp Rules of this type should either go into the root or arp/rarp chain. Table 17.6. ARP and RARP protocol types Attribute Name Datatype Definition srcmacaddr MAC_ADDR MAC address of sender srcmacmask MAC_MASK Mask applied to MAC address of sender dstmacaddr MAC_ADDR MAC address of destination dstmacmask MAC_MASK Mask applied to MAC address of destination hwtype UINT16 Hardware type protocoltype UINT16 Protocol type opcode UINT16, STRING Opcode valid strings are: Request, Reply, Request_Reverse, Reply_Reverse, DRARP_Request, DRARP_Reply, DRARP_Error, InARP_Request, ARP_NAK arpsrcmacaddr MAC_ADDR Source MAC address in ARP/RARP packet arpdstmacaddr MAC _ADDR Destination MAC address in ARP/RARP packet arpsrcipaddr IP_ADDR Source IP address in ARP/RARP packet arpdstipaddr IP_ADDR Destination IP address in ARP/RARP packet gratuitous BOOLEAN Boolean indicating whether to check for a gratuitous ARP packet comment STRING text string up to 256 characters 17.14.10.5. IPv4 Protocol ID: ip Rules of this type should either go into the root or ipv4 chain. Table 17.7. IPv4 protocol types Attribute Name Datatype Definition srcmacaddr MAC_ADDR MAC address of sender srcmacmask MAC_MASK Mask applied to MAC address of sender dstmacaddr MAC_ADDR MAC address of destination dstmacmask MAC_MASK Mask applied to MAC address of destination srcipaddr IP_ADDR Source IP address srcipmask IP_MASK Mask applied to source IP address dstipaddr IP_ADDR Destination IP address dstipmask IP_MASK Mask applied to destination IP address protocol UINT8, STRING Layer 4 protocol identifier. Valid strings for protocol are: tcp, udp, udplite, esp, ah, icmp, igmp, sctp srcportstart UINT16 Start of range of valid source ports; requires protocol srcportend UINT16 End of range of valid source ports; requires protocol dstportstart UNIT16 Start of range of valid destination ports; requires protocol dstportend UNIT16 End of range of valid destination ports; requires protocol comment STRING text string up to 256 characters 17.14.10.6. IPv6 Protocol ID: ipv6 Rules of this type should either go into the root or ipv6 chain. Table 17.8. IPv6 protocol types Attribute Name Datatype Definition srcmacaddr MAC_ADDR MAC address of sender srcmacmask MAC_MASK Mask applied to MAC address of sender dstmacaddr MAC_ADDR MAC address of destination dstmacmask MAC_MASK Mask applied to MAC address of destination srcipaddr IP_ADDR Source IP address srcipmask IP_MASK Mask applied to source IP address dstipaddr IP_ADDR Destination IP address dstipmask IP_MASK Mask applied to destination IP address protocol UINT8, STRING Layer 4 protocol identifier. Valid strings for protocol are: tcp, udp, udplite, esp, ah, icmpv6, sctp scrportstart UNIT16 Start of range of valid source ports; requires protocol srcportend UINT16 End of range of valid source ports; requires protocol dstportstart UNIT16 Start of range of valid destination ports; requires protocol dstportend UNIT16 End of range of valid destination ports; requires protocol comment STRING text string up to 256 characters 17.14.10.7. TCP/UDP/SCTP Protocol ID: tcp, udp, sctp The chain parameter is ignored for this type of traffic and should either be omitted or set to root. Table 17.9. TCP/UDP/SCTP protocol types Attribute Name Datatype Definition srcmacaddr MAC_ADDR MAC address of sender srcipaddr IP_ADDR Source IP address srcipmask IP_MASK Mask applied to source IP address dstipaddr IP_ADDR Destination IP address dstipmask IP_MASK Mask applied to destination IP address scripto IP_ADDR Start of range of source IP address srcipfrom IP_ADDR End of range of source IP address dstipfrom IP_ADDR Start of range of destination IP address dstipto IP_ADDR End of range of destination IP address scrportstart UNIT16 Start of range of valid source ports; requires protocol srcportend UINT16 End of range of valid source ports; requires protocol dstportstart UNIT16 Start of range of valid destination ports; requires protocol dstportend UNIT16 End of range of valid destination ports; requires protocol comment STRING text string up to 256 characters state STRING comma separated list of NEW,ESTABLISHED,RELATED,INVALID or NONE flags STRING TCP-only: format of mask/flags with mask and flags each being a comma separated list of SYN,ACK,URG,PSH,FIN,RST or NONE or ALL ipset STRING The name of an IPSet managed outside of libvirt ipsetflags IPSETFLAGS flags for the IPSet; requires ipset attribute 17.14.10.8. ICMP Protocol ID: icmp Note: The chain parameter is ignored for this type of traffic and should either be omitted or set to root. Table 17.10. ICMP protocol types Attribute Name Datatype Definition srcmacaddr MAC_ADDR MAC address of sender srcmacmask MAC_MASK Mask applied to the MAC address of the sender dstmacaddr MAD_ADDR MAC address of the destination dstmacmask MAC_MASK Mask applied to the MAC address of the destination srcipaddr IP_ADDR Source IP address srcipmask IP_MASK Mask applied to source IP address dstipaddr IP_ADDR Destination IP address dstipmask IP_MASK Mask applied to destination IP address srcipfrom IP_ADDR start of range of source IP address scripto IP_ADDR end of range of source IP address dstipfrom IP_ADDR Start of range of destination IP address dstipto IP_ADDR End of range of destination IP address type UNIT16 ICMP type code UNIT16 ICMP code comment STRING text string up to 256 characters state STRING comma separated list of NEW,ESTABLISHED,RELATED,INVALID or NONE ipset STRING The name of an IPSet managed outside of libvirt ipsetflags IPSETFLAGS flags for the IPSet; requires ipset attribute 17.14.10.9. IGMP, ESP, AH, UDPLITE, 'ALL' Protocol ID: igmp, esp, ah, udplite, all The chain parameter is ignored for this type of traffic and should either be omitted or set to root. Table 17.11. IGMP, ESP, AH, UDPLITE, 'ALL' Attribute Name Datatype Definition srcmacaddr MAC_ADDR MAC address of sender srcmacmask MAC_MASK Mask applied to the MAC address of the sender dstmacaddr MAD_ADDR MAC address of the destination dstmacmask MAC_MASK Mask applied to the MAC address of the destination srcipaddr IP_ADDR Source IP address srcipmask IP_MASK Mask applied to source IP address dstipaddr IP_ADDR Destination IP address dstipmask IP_MASK Mask applied to destination IP address srcipfrom IP_ADDR start of range of source IP address scripto IP_ADDR end of range of source IP address dstipfrom IP_ADDR Start of range of destination IP address dstipto IP_ADDR End of range of destination IP address comment STRING text string up to 256 characters state STRING comma separated list of NEW,ESTABLISHED,RELATED,INVALID or NONE ipset STRING The name of an IPSet managed outside of libvirt ipsetflags IPSETFLAGS flags for the IPSet; requires ipset attribute 17.14.10.10. TCP/UDP/SCTP over IPV6 Protocol ID: tcp-ipv6, udp-ipv6, sctp-ipv6 The chain parameter is ignored for this type of traffic and should either be omitted or set to root. Table 17.12. TCP, UDP, SCTP over IPv6 protocol types Attribute Name Datatype Definition srcmacaddr MAC_ADDR MAC address of sender srcipaddr IP_ADDR Source IP address srcipmask IP_MASK Mask applied to source IP address dstipaddr IP_ADDR Destination IP address dstipmask IP_MASK Mask applied to destination IP address srcipfrom IP_ADDR start of range of source IP address scripto IP_ADDR end of range of source IP address dstipfrom IP_ADDR Start of range of destination IP address dstipto IP_ADDR End of range of destination IP address srcportstart UINT16 Start of range of valid source ports srcportend UINT16 End of range of valid source ports dstportstart UINT16 Start of range of valid destination ports dstportend UINT16 End of range of valid destination ports comment STRING text string up to 256 characters state STRING comma separated list of NEW,ESTABLISHED,RELATED,INVALID or NONE ipset STRING The name of an IPSet managed outside of libvirt ipsetflags IPSETFLAGS flags for the IPSet; requires ipset attribute 17.14.10.11. ICMPv6 Protocol ID: icmpv6 The chain parameter is ignored for this type of traffic and should either be omitted or set to root. Table 17.13. ICMPv6 protocol types Attribute Name Datatype Definition srcmacaddr MAC_ADDR MAC address of sender srcipaddr IP_ADDR Source IP address srcipmask IP_MASK Mask applied to source IP address dstipaddr IP_ADDR Destination IP address dstipmask IP_MASK Mask applied to destination IP address srcipfrom IP_ADDR start of range of source IP address scripto IP_ADDR end of range of source IP address dstipfrom IP_ADDR Start of range of destination IP address dstipto IP_ADDR End of range of destination IP address type UINT16 ICMPv6 type code UINT16 ICMPv6 code comment STRING text string up to 256 characters state STRING comma separated list of NEW,ESTABLISHED,RELATED,INVALID or NONE ipset STRING The name of an IPSet managed outside of libvirt ipsetflags IPSETFLAGS flags for the IPSet; requires ipset attribute 17.14.10.12. IGMP, ESP, AH, UDPLITE, 'ALL' over IPv6 Protocol ID: igmp-ipv6, esp-ipv6, ah-ipv6, udplite-ipv6, all-ipv6 The chain parameter is ignored for this type of traffic and should either be omitted or set to root. Table 17.14. IGMP, ESP, AH, UDPLITE, 'ALL' over IPv6 protocol types Attribute Name Datatype Definition srcmacaddr MAC_ADDR MAC address of sender srcipaddr IP_ADDR Source IP address srcipmask IP_MASK Mask applied to source IP address dstipaddr IP_ADDR Destination IP address dstipmask IP_MASK Mask applied to destination IP address srcipfrom IP_ADDR start of range of source IP address scripto IP_ADDR end of range of source IP address dstipfrom IP_ADDR Start of range of destination IP address dstipto IP_ADDR End of range of destination IP address comment STRING text string up to 256 characters state STRING comma separated list of NEW,ESTABLISHED,RELATED,INVALID or NONE ipset STRING The name of an IPSet managed outside of libvirt ipsetflags IPSETFLAGS flags for the IPSet; requires ipset attribute 17.14.11. Advanced Filter Configuration Topics The following sections discuss advanced filter configuration topics. 17.14.11.1. Connection tracking The network filtering subsystem (on Linux) makes use of the connection tracking support of IP tables. This helps in enforcing the direction of the network traffic (state match) as well as counting and limiting the number of simultaneous connections towards a guest virtual machine. As an example, if a guest virtual machine has TCP port 8080 open as a server, clients may connect to the guest virtual machine on port 8080. Connection tracking and enforcement of the direction and then prevents the guest virtual machine from initiating a connection from (TCP client) port 8080 to the host physical machine back to a remote host physical machine. More importantly, tracking helps to prevent remote attackers from establishing a connection back to a guest virtual machine. For example, if the user inside the guest virtual machine established a connection to port 80 on an attacker site, the attacker will not be able to initiate a connection from TCP port 80 back towards the guest virtual machine. By default the connection state match that enables connection tracking and then enforcement of the direction of traffic is turned on. Example 17.9. XML example for turning off connections to the TCP port The following shows an example XML fragment where this feature has been turned off for incoming connections to TCP port 12345. This now allows incoming traffic to TCP port 12345, but would also enable the initiation from (client) TCP port 12345 within the VM, which may or may not be desirable. 17.14.11.2. Limiting number of connections To limit the number of connections a guest virtual machine may establish, a rule must be provided that sets a limit of connections for a given type of traffic. If for example a VM is supposed to be allowed to only ping one other IP address at a time and is supposed to have only one active incoming ssh connection at a time. Example 17.10. XML sample file that sets limits to connections The following XML fragment can be used to limit connections Note Limitation rules must be listed in the XML prior to the rules for accepting traffic. According to the XML file in Example 17.10, "XML sample file that sets limits to connections" , an additional rule for allowing DNS traffic sent to port 22 go out the guest virtual machine, has been added to avoid ssh sessions not getting established for reasons related to DNS lookup failures by the ssh daemon. Leaving this rule out may result in the ssh client hanging unexpectedly as it tries to connect. Additional caution should be used in regards to handling timeouts related to tracking of traffic. An ICMP ping that the user may have terminated inside the guest virtual machine may have a long timeout in the host physical machine's connection tracking system and will therefore not allow another ICMP ping to go through. The best solution is to tune the timeout in the host physical machine's sysfs with the following command:# echo 3 > /proc/sys/net/netfilter/nf_conntrack_icmp_timeout . This command sets the ICMP connection tracking timeout to 3 seconds. The effect of this is that once one ping is terminated, another one can start after 3 seconds. If for any reason the guest virtual machine has not properly closed its TCP connection, the connection to be held open for a longer period of time, especially if the TCP timeout value was set for a large amount of time on the host physical machine. In addition, any idle connection may result in a timeout in the connection tracking system which can be re-activated once packets are exchanged. However, if the limit is set too low, newly initiated connections may force an idle connection into TCP backoff. Therefore, the limit of connections should be set rather high so that fluctuations in new TCP connections do not cause odd traffic behavior in relation to idle connections. 17.14.11.3. Command-line tools virsh has been extended with life-cycle support for network filters. All commands related to the network filtering subsystem start with the prefix nwfilter . The following commands are available: nwfilter-list : lists UUIDs and names of all network filters nwfilter-define : defines a new network filter or updates an existing one (must supply a name) nwfilter-undefine : deletes a specified network filter (must supply a name). Do not delete a network filter currently in use. nwfilter-dumpxml : displays a specified network filter (must supply a name) nwfilter-edit : edits a specified network filter (must supply a name) 17.14.11.4. Pre-existing network filters The following is a list of example network filters that are automatically installed with libvirt: Table 17.15. ICMPv6 protocol types Protocol Name Description allow-arp Accepts all incoming and outgoing Address Resolution Protocol (ARP) traffic to a guest virtual machine. no-arp-spoofing , no-arp-mac-spoofing , and no-arp-ip-spoofing These filters prevent a guest virtual machine from spoofing ARP traffic. In addition, they only allows ARP request and reply messages, and enforce that those packets contain: no-arp-spoofing - the MAC and IP addresses of the guest no-arp-mac-spoofing - the MAC address of the guest no-arp-ip-spoofing - the IP address of the guest low-dhcp Allows a guest virtual machine to request an IP address via DHCP (from any DHCP server). low-dhcp-server Allows a guest virtual machine to request an IP address from a specified DHCP server. The dotted decimal IP address of the DHCP server must be provided in a reference to this filter. The name of the variable must be DHCPSERVER . low-ipv4 Accepts all incoming and outgoing IPv4 traffic to a virtual machine. low-incoming-ipv4 Accepts only incoming IPv4 traffic to a virtual machine. This filter is a part of the clean-traffic filter. no-ip-spoofing Prevents a guest virtual machine from sending IP packets with a source IP address different from the one inside the packet. This filter is a part of the clean-traffic filter. no-ip-multicast Prevents a guest virtual machine from sending IP multicast packets. no-mac-broadcast Prevents outgoing IPv4 traffic to a specified MAC address. This filter is a part of the clean-traffic filter. no-other-l2-traffic Prevents all layer 2 networking traffic except traffic specified by other filters used by the network. This filter is a part of the clean-traffic filter. no-other-rarp-traffic , qemu-announce-self , qemu-announce-self-rarp These filters allow QEMU's self-announce Reverse Address Resolution Protocol (RARP) packets, but prevent all other RARP traffic. All of them are also included in the clean-traffic filter. clean-traffic Prevents MAC, IP and ARP spoofing. This filter references several other filters as building blocks. These filters are only building blocks and require a combination with other filters to provide useful network traffic filtering. The most used one in the above list is the clean-traffic filter. This filter itself can for example be combined with the no-ip-multicast filter to prevent virtual machines from sending IP multicast traffic on top of the prevention of packet spoofing. 17.14.11.5. Writing your own filters Since libvirt only provides a couple of example networking filters, you may consider writing your own. When planning on doing so there are a couple of things you may need to know regarding the network filtering subsystem and how it works internally. Certainly you also have to know and understand the protocols very well that you want to be filtering on so that no further traffic than what you want can pass and that in fact the traffic you want to allow does pass. The network filtering subsystem is currently only available on Linux host physical machines and only works for QEMU and KVM type of virtual machines. On Linux, it builds upon the support for ebtables, iptables and ip6tables and makes use of their features. Considering the list found in Section 17.14.10, "Supported Protocols" the following protocols can be implemented using ebtables: mac stp (spanning tree protocol) vlan (802.1Q) arp, rarp ipv4 ipv6 Any protocol that runs over IPv4 is supported using iptables, those over IPv6 are implemented using ip6tables. Using a Linux host physical machine, all traffic filtering rules created by libvirt's network filtering subsystem first passes through the filtering support implemented by ebtables and only afterwards through iptables or ip6tables filters. If a filter tree has rules with the protocols including: mac, stp, vlan arp, rarp, ipv4, or ipv6; the ebtable rules and values listed will automatically be used first. Multiple chains for the same protocol can be created. The name of the chain must have a prefix of one of the previously enumerated protocols. To create an additional chain for handling of ARP traffic, a chain with name arp-test, can for example be specified. As an example, it is possible to filter on UDP traffic by source and destination ports using the ip protocol filter and specifying attributes for the protocol, source and destination IP addresses and ports of UDP packets that are to be accepted. This allows early filtering of UDP traffic with ebtables. However, once an IP or IPv6 packet, such as a UDP packet, has passed the ebtables layer and there is at least one rule in a filter tree that instantiates iptables or ip6tables rules, a rule to let the UDP packet pass will also be necessary to be provided for those filtering layers. This can be achieved with a rule containing an appropriate udp or udp-ipv6 traffic filtering node. Example 17.11. Creating a custom filter Suppose a filter is needed to fulfill the following list of requirements: prevents a VM's interface from MAC, IP and ARP spoofing opens only TCP ports 22 and 80 of a VM's interface allows the VM to send ping traffic from an interface but not let the VM be pinged on the interface allows the VM to do DNS lookups (UDP towards port 53) The requirement to prevent spoofing is fulfilled by the existing clean-traffic network filter, thus the way to do this is to reference it from a custom filter. To enable traffic for TCP ports 22 and 80, two rules are added to enable this type of traffic. To allow the guest virtual machine to send ping traffic a rule is added for ICMP traffic. For simplicity reasons, general ICMP traffic will be allowed to be initiated from the guest virtual machine, and will not be specified to ICMP echo request and response messages. All other traffic will be prevented to reach or be initiated by the guest virtual machine. To do this a rule will be added that drops all other traffic. Assuming the guest virtual machine is called test and the interface to associate our filter with is called eth0 , a filter is created named test-eth0 . The result of these considerations is the following network filter XML: 17.14.11.6. Sample custom filter Although one of the rules in the above XML contains the IP address of the guest virtual machine as either a source or a destination address, the filtering of the traffic works correctly. The reason is that whereas the rule's evaluation occurs internally on a per-interface basis, the rules are additionally evaluated based on which (tap) interface has sent or will receive the packet, rather than what their source or destination IP address may be. Example 17.12. Sample XML for network interface descriptions An XML fragment for a possible network interface description inside the domain XML of the test guest virtual machine could then look like this: To more strictly control the ICMP traffic and enforce that only ICMP echo requests can be sent from the guest virtual machine and only ICMP echo responses be received by the guest virtual machine, the above ICMP rule can be replaced with the following two rules: Example 17.13. Second example custom filter This example demonstrates how to build a similar filter as in the example above, but extends the list of requirements with an ftp server located inside the guest virtual machine. The requirements for this filter are: prevents a guest virtual machine's interface from MAC, IP, and ARP spoofing opens only TCP ports 22 and 80 in a guest virtual machine's interface allows the guest virtual machine to send ping traffic from an interface but does not allow the guest virtual machine to be pinged on the interface allows the guest virtual machine to do DNS lookups (UDP towards port 53) enables the ftp server (in active mode) so it can run inside the guest virtual machine The additional requirement of allowing an FTP server to be run inside the guest virtual machine maps into the requirement of allowing port 21 to be reachable for FTP control traffic as well as enabling the guest virtual machine to establish an outgoing TCP connection originating from the guest virtual machine's TCP port 20 back to the FTP client (FTP active mode). There are several ways of how this filter can be written and two possible solutions are included in this example. The first solution makes use of the state attribute of the TCP protocol that provides a hook into the connection tracking framework of the Linux host physical machine. For the guest virtual machine-initiated FTP data connection (FTP active mode) the RELATED state is used to enable detection that the guest virtual machine-initiated FTP data connection is a consequence of ( or 'has a relationship with' ) an existing FTP control connection, thereby allowing it to pass packets through the firewall. The RELATED state, however, is only valid for the very first packet of the outgoing TCP connection for the FTP data path. Afterwards, the state is ESTABLISHED, which then applies equally to the incoming and outgoing direction. All this is related to the FTP data traffic originating from TCP port 20 of the guest virtual machine. This then leads to the following solution: Before trying out a filter using the RELATED state, you have to make sure that the appropriate connection tracking module has been loaded into the host physical machine's kernel. Depending on the version of the kernel, you must run either one of the following two commands before the FTP connection with the guest virtual machine is established: modprobe nf_conntrack_ftp - where available OR modprobe ip_conntrack_ftp if above is not available If protocols other than FTP are used in conjunction with the RELATED state, their corresponding module must be loaded. Modules are available for the protocols: ftp, tftp, irc, sip, sctp, and amanda. The second solution makes use of the state flags of connections more than the solution did. This solution takes advantage of the fact that the NEW state of a connection is valid when the very first packet of a traffic flow is detected. Subsequently, if the very first packet of a flow is accepted, the flow becomes a connection and thus enters into the ESTABLISHED state. Therefore, a general rule can be written for allowing packets of ESTABLISHED connections to reach the guest virtual machine or be sent by the guest virtual machine. This is done writing specific rules for the very first packets identified by the NEW state and dictates the ports that the data is acceptable. All packets meant for ports that are not explicitly accepted are dropped, thus not reaching an ESTABLISHED state. Any subsequent packets sent from that port are dropped as well. 17.14.12. Limitations The following is a list of the currently known limitations of the network filtering subsystem. VM migration is only supported if the whole filter tree that is referenced by a guest virtual machine's top level filter is also available on the target host physical machine. The network filter clean-traffic for example should be available on all libvirt installations and thus enable migration of guest virtual machines that reference this filter. To assure version compatibility is not a problem make sure you are using the most current version of libvirt by updating the package regularly. Migration must occur between libvirt insallations of version 0.8.1 or later in order not to lose the network traffic filters associated with an interface. VLAN (802.1Q) packets, if sent by a guest virtual machine, cannot be filtered with rules for protocol IDs arp, rarp, ipv4 and ipv6. They can only be filtered with protocol IDs, MAC and VLAN. Therefore, the example filter clean-traffic Example 17.1, "An example of network filtering" will not work as expected.

Commands:
<devices> <interface type='bridge'> <mac address='00:16:3e:5d:c7:9e'/> <filterref filter='clean-traffic'/> </interface> </devices>
<devices> <interface type='bridge'> <mac address='00:16:3e:5d:c7:9e'/> <filterref filter='clean-traffic'> <parameter name='IP' value='10.0.0.1'/> </filterref> </interface> </devices>
<filter name='no-arp-spoofing' chain='arp' priority='-500'> <uuid>f88f1932-debf-4aa1-9fbe-f10d3aa4bc95</uuid> <rule action='drop' direction='out' priority='300'> <mac match='no' srcmacaddr='USDMAC'/> </rule> <rule action='drop' direction='out' priority='350'> <arp match='no' arpsrcmacaddr='USDMAC'/> </rule> <rule action='drop' direction='out' priority='400'> <arp match='no' arpsrcipaddr='USDIP'/> </rule> <rule action='drop' direction='in' priority='450'> <arp opcode='Reply'/> <arp match='no' arpdstmacaddr='USDMAC'/> </rule> <rule action='drop' direction='in' priority='500'> <arp match='no' arpdstipaddr='USDIP'/> </rule> <rule action='accept' direction='inout' priority='600'> <arp opcode='Request'/> </rule> <rule action='accept' direction='inout' priority='650'> <arp opcode='Reply'/> </rule> <rule action='drop' direction='inout' priority='1000'/> </filter>
<devices> <interface type='bridge'> <mac address='00:16:3e:5d:c7:9e'/> <filterref filter='clean-traffic'> <parameter name='IP' value='10.0.0.1'/> <parameter name='IP' value='10.0.0.2'/> <parameter name='IP' value='10.0.0.3'/> </filterref> </interface> </devices>
<rule action='accept' direction='in' priority='500'> <tcp srpipaddr='USDIP'/> </rule>
<rule action='accept' direction='in' priority='500'> <udp dstportstart='USDDSTPORTS[1]'/> </rule>
<rule action='accept' direction='in' priority='500'> <ip srcipaddr='USDSRCIPADDRESSES[@1]' dstportstart='USDDSTPORTS[@2]'/> </rule>
SRCIPADDRESSES = [ 10.0.0.1, 11.1.2.3 ] DSTPORTS = [ 80, 8080 ]
<interface type='bridge'> <source bridge='virbr0'/> <filterref filter='clean-traffic'> <parameter name='CTRL_IP_LEARNING' value='dhcp'/> </filterref> </interface>
<filter name='clean-traffic'> <uuid>6ef53069-ba34-94a0-d33d-17751b9b8cb1</uuid> <filterref filter='no-mac-spoofing'/> <filterref filter='no-ip-spoofing'/> <filterref filter='allow-incoming-ipv4'/> <filterref filter='no-arp-spoofing'/> <filterref filter='no-other-l2-traffic'/> <filterref filter='qemu-announce-self'/> </filter>
<filter name='no-ip-spoofing' chain='ipv4'> <uuid>fce8ae33-e69e-83bf-262e-30786c1f8072</uuid> <rule action='drop' direction='out' priority='500'> <ip match='no' srcipaddr='USDIP'/> </rule> </filter>
[...] <rule action='drop' direction='in'> <protocol match='no' attribute1='value1' attribute2='value2'/> <protocol attribute3='value3'/> </rule> [...]
[...] <mac match='no' srcmacaddr='USDMAC'/> [...]
[...] <rule direction='in' action='accept' statematch='false'> <cp dstportstart='12345'/> </rule> [...]
[...] <rule action='drop' direction='in' priority='400'> <tcp connlimit-above='1'/> </rule> <rule action='accept' direction='in' priority='500'> <tcp dstportstart='22'/> </rule> <rule action='drop' direction='out' priority='400'> <icmp connlimit-above='1'/> </rule> <rule action='accept' direction='out' priority='500'> <icmp/> </rule> <rule action='accept' direction='out' priority='500'> <udp dstportstart='53'/> </rule> <rule action='drop' direction='inout' priority='1000'> <all/> </rule> [...]
<filter name='test-eth0'> <!- - This rule references the clean traffic filter to prevent MAC, IP and ARP spoofing. By not providing an IP address parameter, libvirt will detect the IP address the guest virtual machine is using. - -> <filterref filter='clean-traffic'/> <!- - This rule enables TCP ports 22 (ssh) and 80 (http) to be reachable - -> <rule action='accept' direction='in'> <tcp dstportstart='22'/> </rule> <rule action='accept' direction='in'> <tcp dstportstart='80'/> </rule> <!- - This rule enables general ICMP traffic to be initiated by the guest virtual machine including ping traffic - -> <rule action='accept' direction='out'> <icmp/> </rule>> <!- - This rule enables outgoing DNS lookups using UDP - -> <rule action='accept' direction='out'> <udp dstportstart='53'/> </rule> <!- - This rule drops all other traffic - -> <rule action='drop' direction='inout'> <all/> </rule> </filter>
[...] <interface type='bridge'> <source bridge='mybridge'/> <filterref filter='test-eth0'/> </interface> [...]
<!- - enable outgoing ICMP echo requests- -> <rule action='accept' direction='out'> <icmp type='8'/> </rule>
<!- - enable incoming ICMP echo replies- -> <rule action='accept' direction='in'> <icmp type='0'/> </rule>
<filter name='test-eth0'> <!- - This filter (eth0) references the clean traffic filter to prevent MAC, IP, and ARP spoofing. By not providing an IP address parameter, libvirt will detect the IP address the guest virtual machine is using. - -> <filterref filter='clean-traffic'/> <!- - This rule enables TCP port 21 (FTP-control) to be reachable - -> <rule action='accept' direction='in'> <tcp dstportstart='21'/> </rule> <!- - This rule enables TCP port 20 for guest virtual machine-initiated FTP data connection related to an existing FTP control connection - -> <rule action='accept' direction='out'> <tcp srcportstart='20' state='RELATED,ESTABLISHED'/> </rule> <!- - This rule accepts all packets from a client on the FTP data connection - -> <rule action='accept' direction='in'> <tcp dstportstart='20' state='ESTABLISHED'/> </rule> <!- - This rule enables TCP port 22 (SSH) to be reachable - -> <rule action='accept' direction='in'> <tcp dstportstart='22'/> </rule> <!- -This rule enables TCP port 80 (HTTP) to be reachable - -> <rule action='accept' direction='in'> <tcp dstportstart='80'/> </rule> <!- - This rule enables general ICMP traffic to be initiated by the guest virtual machine, including ping traffic - -> <rule action='accept' direction='out'> <icmp/> </rule> <!- - This rule enables outgoing DNS lookups using UDP - -> <rule action='accept' direction='out'> <udp dstportstart='53'/> </rule> <!- - This rule drops all other traffic - -> <rule action='drop' direction='inout'> <all/> </rule> </filter>
<filter name='test-eth0'> <!- - This filter references the clean traffic filter to prevent MAC, IP and ARP spoofing. By not providing and IP address parameter, libvirt will detect the IP address the VM is using. - -> <filterref filter='clean-traffic'/> <!- - This rule allows the packets of all previously accepted connections to reach the guest virtual machine - -> <rule action='accept' direction='in'> <all state='ESTABLISHED'/> </rule> <!- - This rule allows the packets of all previously accepted and related connections be sent from the guest virtual machine - -> <rule action='accept' direction='out'> <all state='ESTABLISHED,RELATED'/> </rule> <!- - This rule enables traffic towards port 21 (FTP) and port 22 (SSH)- -> <rule action='accept' direction='in'> <tcp dstportstart='21' dstportend='22' state='NEW'/> </rule> <!- - This rule enables traffic towards port 80 (HTTP) - -> <rule action='accept' direction='in'> <tcp dstportstart='80' state='NEW'/> </rule> <!- - This rule enables general ICMP traffic to be initiated by the guest virtual machine, including ping traffic - -> <rule action='accept' direction='out'> <icmp state='NEW'/> </rule> <!- - This rule enables outgoing DNS lookups using UDP - -> <rule action='accept' direction='out'> <udp dstportstart='53' state='NEW'/> </rule> <!- - This rule drops all other traffic - -> <rule action='drop' direction='inout'> <all/> </rule> </filter>