Project: Kepler (Observability and Analysis / Observability)
File: OSS-NA22.pdf

Sustainability 
the Container Native Way 
Huamin Chen 
Red Hat 
Twitter @root_fs 
Github rootfs 
#ossummit Chen Wang 
IBM Research 
Twitter @wangchen615 
Github wangchen615 #ossummit 
Agenda 
•Background 
•Introduction to Project Kepler 
•Next Steps 
•Community 
•Demo #ossummit 
Energy Measurement 
•Energy measurement theory  and 
methodology 
–How to measure energy consumption, 
indirectly? 
•Energy consumption attribution problem  
and methodology 
–How much energy used by the Pods? #ossummit 
Power Measurement Theory 
There are several factors contributing to the CPU power consumption; they include dynamic  
power consumption, short-circuit power consumption, and power loss due to transistor leakage  
currents :
The dynamic power consumption originates from the activity of logic gates inside a CPU. When  
the logic gates toggle, energy is flowing as the capacitors inside them are charged and  
discharged. The dynamic power consumed by a CPU is approximately proportional to the CPU  
frequency, and to the square of the CPU voltage: 
where C is the switched load capacitance, f is frequency, V is voltage 
Source: https://en.wikipedia.org/wiki/Processor_power_dissipation 
leakage current is a 
function of thermal 
temperature 
Frequency based 
tuning to optimize 
dynamic power 
consumption. #ossummit 
Energy Measurement Methodology : Ideal vs Reality 
Measure Target Ideal Way Reality Solution 
Frequency Monitor each circuit 
frequency Linux kernel CPU 
frequency governor 
dynamically changes 
frequency for energy 
saving or performance 
objectives Use average 
frequency  (aperf) to 
approximate 
Capacitance Monitor the number 
of circuits  powered 
onThere is no such way 
on CPUs Use the number of 
CPU instructions  to 
approximate 
Execution Time Monitor the duration 
of circuits  staying 
powered on There is no such way 
on CPUs Use the CPU cycles  
to approximate 
Indirect measurement to approach the truth in an imperfect world #ossummit 
Energy Consumption Attribution Problem 
Power Supply 
Process 
 Process 
 Process 
 Process 
 Process 
 Process Container Container Container Pod Pod
How to attribute 
power consumption 
on shared resources  
to processes, 
containers, or Pods? 10% CPU usage 50% CPU usage #ossummit 
Energy Consumption Attribution Methodology 
Power Supply 
Process 
 Process 
 Process 
 Process 
 Process 
 Process Container Container Container Pod Pod
Power consumption 
is attributed to the 
resource usage by 
processes, 
containers. and Pods 10% CPU usage 
means 10% CPU 
power consumption 50% CPU usage means 50% 
CPU power consumption #ossummit 
Put Together 
•Use software counters to measure power 
consumption by hardware resources 
–ML models are used for approximation. 
•Use hardware resource utilization to 
attribute power consumption by 
processes, containers, and Pods. 
#ossummit 
9 - Reduced computational 
resource  used by the 
probe 
- Using eBPF R egr e ssion R educ tion 
- Support ML models to 
estimate energy 
consumption 
- Science based 
approach R eporting 
- Per Pod level energy 
consumption reporting, 
including CPU/GPU, RAM 
- Support bare metal  as well 
as VM 
- Support Prometheus 
https://github.com/sustainable-computing-io/kepler 
Kepler: Kubernetes-based Efficient Power Level Export er#ossummit 
Data Collection Data Aggregation Data Modeling Data Presentation 
Linux Kernel 
Tracepoint Performance Counter RAPL 
hwmon stats cgroupfs 
stats GPU 
stats(nvml) 
PID, Cgroup ID 
CPU cycles 
CPU time 
CPU instructions 
Cache misses 
….EnergyPo
d= (CPU stats, Memory stats, GPU stats, cgroupfs stats, 
hwmon stats)
Energy stats as Metrics Counters Kepler Architecture 
Model Server Power 
Estimate 
Model 
RAPL 
Per Container 
Data Aggregate 
Data Aggregate 
Estimate Metrics Per Container 
Estimate  
Metrics 
dataset = 
tfio.experimental.IODataset.stream.from_prometheus(...) 
                         …… 
model = tf.keras.models.Sequential(...) 
                         …… 
model.fit(dataset….) 
 Kepler 
Model Server Metrics Scraping 
Download 
Model 
Model Server trains ML models that use software counters to approximate power consumption. #ossummit 
Setup 
-Bare Metal Machine 
-Intel Xeon 4210 
-NVIDIA T4 Graphics Card / 20 cores / 2.20 GHz / 32 GB RAM / 20 TB bandwidth 
-MicroShift 
-NVIDIA GPU Operator 
-Kepler  
-kube-prometheus Demo #ossummit 
Demo 
Steps: 
1.Deploy Kepler Exporter as a DaemonSet 
2.Check Kepler exported data in Kepler logs and endpoint 
3.Configure Prometheus and Metric Scape 
4.Load Kepler Grafana Dashboard 
5.Test CPU intensive workload 
6.Test GPU intensive workload #ossummit 
Supported Data Stats 
•CPU Stats 
–Energy, model, time and frequency 
–Hardware Performance Counters 
•Memory 
–Cache misses, resident memory size 
•GPU 
–Energy, resident memory size 
•Block Device 
–Read/write stats #ossummit 
Next Steps 
•Energy consumption and carbon emissions 
telemetry and dashboard. 
•Kepler and Model Server provide tuning and 
scheduling heuristics. 
–Dynamic power reduction 
•Energy Aware Pod Scheduling 
•DVFS based Vertical Pod Scaling 
•Energy Efficient Node Tuning 
–Leaky power reduction 
•Thermal Temperature Aware Scheduling and Scaling 
#ossummit 
Community 
https://github.com/sustainable-computing-io https://sustainableco-ese6814.slack.com/archives/C02VCJ8K8LT 
Backup Slides #ossummit 
Existing Landscape and Use Cases 
•Open Source Projects 
–PowerAPI 
•https://github.com/powerapi-ng 
–Scaphandre 
•https://github.com/hubblo-org/scaphandre  
•Kepler Design Principle: 
–Container Native, Cloud Native 
–Lightweight, Expansible 
–Accurate and Fair #ossummit 
Performance Per Watt 
DVFS based performance per watt tuning yields promising energy savings Workload: sysbench 
Platform: Intel Xeon Sandy Bridge