Performance Isolation inMulti-tenant Cloud Datacenters
ベノワ ヌニャンケ

第93回 TechTrend Talk Series
vol.7

2023 / 11 / 21

Outline● Historical Perspective● Multi-tenant Cloud Data Centers● PerfIsol Initiative2

Outline● Historical Perspective● Multi-tenant Cloud Data Centers● PerfIsol Initiative3

From Mainframes to Cloud: Application-Network Synergy
Mainframes & Time-Sharing

1960-

1980-

Client-Server & LAN●Web Apps & Internet

1990-

2010-

Microservices & CloudApplication evolution puts different and increased challenges on the network 4

Evolution of Application Architecture: Application-Network Journey

Cloud-native Apps
5

Modern Applications●●Cloud-native → agility, velocity○Microservices, using RPCs to communicate between services○Containers & orchestration → Kubernetes ⇐ in the cloud○

Automation: DevOps, CI/CDMore services/tenants to the Cloud⇒ Multi-Tenant Cloud Data Centers (MTCDC) and its Networking challenges
Image source: https://blog.knoldus.com/cloud-native-future-of-software-architecture/6

Outline● Historical Perspective● Multi-tenant Cloud Data Centers● PerfIsol Initiative7

Challenges on Multi-tenant Cloud DC Networking●Management: flexible and efficient to operate●Bandwidth: High server-to-server capacity○●

Network Virtualization at scale○●

Topology, Routing, Load Balancing

Accommodate different services/tenants simultaneously

Performance Guarantees○

predictable and reliable network ← efficient resource sharing
8Multi-tenant Cloud Data Center Network Fabric●Non-blocking fabric○High bandwidth●Portland, VL2: Virtualization, scale [SIGCOMM 09]●

Andromeda: Network virtualization stack [NSDI 18]

IP Fabric

The Rise of IP Fabrics and Corresponding Novel Routing Paradigms [https://www.comsoc.org/publications/ctn/rise-ip-fabrics-and-corresponding-novel-routing-paradigms-part-1] & RFC 7938

9

Multi-tenancy in Cloud Data Center Networks●●NVO3: Data-Center Network Virtualization over Layer 3○

Programmatically create, provision, and manage networks completely within software○

While leveraging the underlying physical network as the packet-forwarding backplane

Overlay Network:○

A virtual network■○

The separation of tenants is hidden from the underlying physical infrastructure

e.g., VXLAN, GENEVE ⇒ tunnel encapsulations

RFC 8014 “An Architecture for Data-Center Network Virtualization over Layer 3 (NVO3) ” 201610

Overlays and Underlay: Packet Journey●

UDP over IP encapsulation → L2 segment for a tenant
IP FabricImage source: Network Virtualization For Dummies, VMware 3rd Special Edition11

Logical Isolation vs. Performance Isolation●●Overlays offer logical isolation only:○

Traffic Isolation○

Address Isolation

Ideally: tenant virtual network (DC) could support the illusion that eachapplication is running on its own isolated network○

Not satisfied by overlays only

12

Performance Isolation Matters●

Mechanisms ensuring tenants' resource usage doesn't impact other tenants.

●The “illusion of feeling alone”: each service/tenant operates as if
connected to a separate physical switch○

An app facing attack/bug does not adversely impact the performance of other apps/tenants○

Taking into account the possibility of selfish or malicious behavior from tenants●Contention at shared resources in the MTCDC underlay fabric●Tenants should get end-to-end guarantees → Isolate tenants
across the network
13

Noormohammadpour M., and Cauligi S. R. "Datacenter traffic control: Understanding techniques and tradeoffs." IEEE Communications Surveys & Tutorials 20.2 (2017)

How to share the cloud resources ?

14

Resources Sharing (1/2)●Resources:○

Compute, Memory, Network ⇒ links bandwidth & buffers of switch/vswitch (queuedpackets)●

Traditional resource allocation on packet-switched network○

Queuing Disciplines: e.g. FIFO → Packet-level Control○

Congestion Control: e.g. TCP → Flow-level Control●

Packet-Level & Flow-Level Controls ⇒ Best-effort service to MTCDC ☹

●We need a Tenant-level Control mechanism !

15
Computer Networks: A Systems Approach [https://book.systemsapproach.org/congestion/problem.html]

Resources Sharing (2/2)●Tenant-level Control Layer to cope with the multi-tenancy specificities○

With SLA/SLO: pricing, bandwidth guarantees○

Scale: number of tenants, VMs, workloads, physical topology○

Additional network layer:■

Overlays → the virtual networks for tenants* SLA = Service-Level Agreement* SLO = service-Level Objective
16

Intuition and Illustration

Infinite Bwd
demands

2.5Gbps
1Gbps
5Gbps
2Gbps

VM
VM

2Gbps

VM

8Gbps

Sharing the network is challenging: local-, remote-, traffic-pattern dependent

2Gbps●10Gbps LinkUnderutilization

5Gbps
1Gbps
2.5Gbps●Min Bwd
Guarantees

VM

How to implement this tenant-level control layer (Rate limiting) efficiently ??
8Gbps
5Gbps
5Gbps

8Gbps

VM

8Gbps

VM
17
Jeyakumar, Vimalkumar, et al. "EyeQ: Practical network performance isolation at the edge." USENIX NSDI 2013

Tenant-level Control Layer Design●Service Model (Abstractions)○●●

How tenants express SLAs / SLOs ?

Well Defined Goals○

Establish measurable objectives for performance isolation○

Navigating the objectives tradeoff space ⇒ resource sharing policies

Flexible and Efficient Implementation

18Service Model
Tenant specifies performance requirements as bandwidth capacity of the vNICVM1sBn

VM2

psb8G

psb2G

VM3…

VMn

VM18Gb2Gbpsbp

Tenant 2Switch
ps

Tenant 1
Switch2G

●VM2

psBm

b8G

VM3…

VMm

Duffield, Nick G., et al. "A flexible model for resource management in virtual private networks." ACM SIGCOMM 1999

19

Performance Isolation Goals● Predictable performance● SLOs guarantees in regard to SLAs
● Optimal resource utilization● Fair QoS impact during overload

20

Tenant-Level Control Layer: Existing Works●●●EyeQ○

A Sender EyeQ Module (SEM) and Receiver EyeQ Module (REM) at every end-host → WFQ○Convergence time (~ 5-10ms), qdisc implemented as Linux kernel (v. 3.0.0) module

Carousel → EDT: Early Departure Time○

Single queue with timestamping as the basis for releasing packets ← Scalable packet sched.

○

How to make such timestamping consistent for all packets ??

vFabric → Programmable dataplane○●

active edge (smart NICs) + informative Core (Tofino) → path selection and traffic AdmissionNetHint○White box approach providing relevant underlay network info (hints) to tenants applications○

eBP-based monitoring and hints computation at end-hosts
21

Tenant-level Control (Rate Limiting) Design Considerations●Guaranty Granularities:○●

per-tenant, per-VM, per-flow ?

SLA and Pricing:○flat-price vs. fixed + dynamic (for bandwidth allocation beyond the min.)●Nice vs disruptive tenants consideration●

Deployment/Implementation: Host/Switch capabilities

22

Outline● Historical Perspective● Multi-tenant Cloud Data Centers● PerfIsol Initiative23

PerfIsol Research Goal●

Complete system design comprising:○

Well defined service abstractions to tenants○

Sharing models related to SLA/SLO○

Low level operations in the multi-tenant data plane

As a unifiedframework● Without SLA/SLO (payment) Considerations● Performance Isolation (fairness) ⇒ Abstract concept hard to justify
24PerfISol System Architecture25

Rate-Limit Agent: Motivation●

Traffic Control (Rate limiting) at end-host○

Typically implemented in software in the kernel networking stack○Some Limitations:■Non-precise rate limiting (for Performance isolation)■

Configurable Qdisc (rate/ceil != minBwd/maxBwd)●Not directly programmable● Need for simplified and programmable Qdisc● Leverage eBPF Kernel programmability capability
26

ePerfIsol: eBPF-based Rate-Limit Agent (programmable qdisc)● Tenant-level mechanism○ Counting○ Thresholding → packet drops

27Resource Allocation: Motivation●

How to find the optimal rates consumed by the rate-limiters ?
○○●Linear programming
Multi-commodity flow problem■ MTCDC topology + VMs-pair traffic matrix

Computing time Scalability issues○with huge number of nodes (VMs in MTCDCs)● Need of accelerated rate-limits allocation mechanism● Leverage Neural Network on graphs (GNNs) learning ability
28

gPerfIsol: GNN-based Rate-Limits Allocation●Admission Control○Accept or refuse a VM-pair demand29Summary (Takeaways)●

Application evolution puts different and increased challenges on the network○●Performance isolation is needed to address tenants interference in MTCDCs○●monoliths / mainframes → client-server / LAN → Web / Internet → microservices / MT-Cloud

How tenant virtual networks share the underlying physical infrastructure ?

PerfIsol Approach for performance isolation○

Unified framework (management & data planes) for optimal resource sharing for multi-tenancy○

Rate Limiting: allocation with gPerfIsol (GNN) & ePerfIsol (eBPF) for the execution

ご清聴ありがとうございました30