Project: MetalLB (Orchestration & Management / Service Proxy)
File: vrf-announcement.md

The purpose of this enhancement is allow MetalLB to peer with external BGP peers using interfaces belonging to , and to announce traffic from within VRFs. The VRF support will be available only via the . Having interfaces wrapped in VRFs allow to set a different default gateway than the regular one. This helps to avoid asymmetrical routing in scenarios where the nodes have multiple interfaces, providing that either the CNI is able to receive the traffic inside the VRF or the host networking is set up in a way that the VRF is able to communicate with the default one for the traffic related to the service's IP. This is the default behaviour for a client coming from a router connected via an interface different from the default one. ```none eth0 eth1 DCGW Client Node ``` And this is how it would look like with VRF: ```none eth0 VRF eth1 DCGW Client Node ``` Other advantages of having VRFs on the node are: being able to have more complex routing tables (i.e. multiple DCGWs for HA) without impacting the node networking ease the troubleshooting of the traffic directed to the service making it possible to have multiple clients (coming from different interfaces) sharing the same IP VRF for BGP is supported by FRR . Establish a BGP session with a BGP peer using an interface with VRF as master Announce a LoadBalancer service to a BGP peer peered via an interface belonging to a VRF Eventually, provide a guideline and an example on how to setup the VRF and the host routing, knowing that the routing logic might be CNI dependant Creating the VRF Having MetalLB configure the host networking routing to drive the traffic towards / from the VRF Having MetalLB to check the existence of the VRF on the host. What MetalLB will require to do is to expose the information that the BGP session can't be established, both via the current Prometheus metrics and via CRDs, in case we are going to extend the CRDs to report the state of the session. As a cluster administrator, I want to declare a BGP session specifying the host VRF to establish the session from. As a cluster administrator, I want to have the LoadBalancer IP announced to BGP peers connected from within a VRF FRR is able to run a bgpd process inside a VRF, so what needs to be done is to enable the specific configuration. From the : ```none router bgp ASN vrf VRFNAME VRFNAME is matched against VRFs configured in the

When vrf VRFNAME is not specified, the BGP protocol process belongs to the default VRF. ``` The proposal here is to define a new `vrf` field in the BGPPeer CR. By setting it, the speaker will instruct FRR to create a new router section related to the router and will configure the neighbour inside of it. ```yaml apiVersion: metallb.io/v1beta2 kind: BGPPeer metadata: name: peer namespace: metallb-system spec: myASN: 64512 peerASN: 64512 peerAddress: 10.2.2.254 vrf: red ``` Any announcement meant to be performed towards the vrf-ed BGPPeer will be added in the related router's section. ```none router bgp 64512 vrf red no bgp ebgp-requires-policy no bgp network import-check no bgp default ipv4-unicast bgp router-id 10.1.1.255 neighbor 10.2.2.254 remote-as 64512 neighbor 10.2.2.254 ebgp-multihop neighbor 10.2.2.254 port 179 neighbor 10.2.2.254 timers 1 1 neighbor 10.2.2.254 password password neighbor 10.2.2.254 update-source 10.1.1.254 address-family ipv4 unicast neighbor 10.2.2.254 activate neighbor 10.2.2.254 route-map 10.2.2.254-red-in in network 172.16.1.10/24 neighbor 10.2.2.254 route-map 10.2.2.254-red-out out exit-address-family ``` Note: We must be careful in handling the FRR configuration rendering, because all the route maps are handled in terms of the peer ip. With VRFs, we can have different BGP peers with the same IP. Note 2: Having multiple BGP peers reached via different VRFs will allow them to share the same IP. For this reason, we will have to change the validating webhook logic in order to allow it. The right combination of host routing through ip rules and iptables rules is strongly dependant on the CNI. Because of this reason, we don't think it is safe to have a one size-fit all solution. What we will provide with this change is a sample script (and possibly a controller) that works for kindnet (in order to enable CI) and calico, with the idea of extending / validating it with the other major CNIs. e2e tests: the dev-env will be extended with an option to have additional interfaces inside a VRF, and to configure the host network in such a way that the service is accessible from inside a VRF. unit tests: Unit tests will be added (mostly to the frr package) to ensure we are producing valid configurations. The alternative is to embed all the logic inside MetalLB, but this means that we might end up with per-cni logic and feels a bit less modular. There are strictly no phases. We want to: validate that FRR is able to implement what we think we require extend dev-env to allow traffic directed to a LB to flow from / to a VRF implement the feature have running CI that validates it document the feature and how to setup host networking