# Algorithmic Techniques for GPU Scheduling: A Comprehensive Survey
## 4 Looking Forward: LLM Focus
### Energy Efficiency and Cost-Aware Scheduling  
The escalating energy footprint of modern GPU workloads is impossible to ignore: training a single foundation model--GPT-3, for instance--has been estimated to consume several gigawatt-hours of electricity [216]. Environmental concerns and operational cost pressures therefore elevate _energy_ and _cost_ to first-class scheduling objectives alongside traditional metrics such as makespan, latency, and fairness. While classical HPC systems relied on coarse controls such as power capping or shifting batch jobs to align with renewable-rich intervals, deep learning clusters expose far richer knobs. Many DL tasks tolerate throughput-latency trade-offs, enabling schedulers to under-clock GPUs for non-urgent training or to steer low-accuracy inference queries to lightweight CPU models, realizing savings without violating SLOs. Recent work confirms the promise of _budget-aware_ optimization. PowerFlow, for example, learns job-specific performance-energy trade-offs over configuration spaces (GPU count, clock frequency) and minimizes JCT subject to a cluster-wide energy cap, achieving1.5\(\sim\)3.4\(\times\) JCT reductions without raising total consumption [104]. Such results motivate multi-objective formulations that (i) minimize makespan under explicit energy constraints, (ii) minimize energy given per-job deadlines, or (iii) jointly optimize both. The inverse question--What is the fastest feasible schedule under a strict power envelope?--is pressing for data centers limited by delivery, cooling, or volatile electricity prices. Inference workloads shift the emphasis from aggregate consumption to _energy per query_. Dynamic batching, request consolidation, and selective idling can concentrate load during low-traffic periods and scale out elastically as demand surges. Early CNN studies report 5% savings from adaptive batching and up to 28% when combined with DVFS [217; 218]; whether similar gains extend to transformer-based LLMs remains an open question. Beyond raw usage, the _carbon intensity_ of power sources now shapes scheduling: deferring non-urgent training to low-emission intervals can markedly cut footprint, provided frameworks maintain convergence under intermittent execution. Energy and cost constraints further motivate _geo-distributed_ and decentralized scheduling. Volunteer or spot-market GPUs, often heterogeneous and ephemeral, offer attractive monetary cost per FLOP but introduce churn, stragglers, and limited global visibility [219]. Meanwhile, emerging silicon supports microsecond-scale DVFS adjustments [220], yet no production schedulers fully exploit per-GPU power-performance curves or real-time telemetry, leaving a rich integration opportunity. Collectively, these trends herald a shift toward holistic schedulers that transcend single-objective optimization. Throughput, latency, energy, fairness, and reliability interact in complex ways, and the rise of LLMs has stretched demands along every axis. Novel systems such as Frenzy (serverless heterogeneous training) [215], Lluminix (adaptive inference serving) [208], Helix (distributed fine-tuning) [211], and TRAIL (preemptive scheduling at scale) [210] each attack one facet of the broader challenge, from minimizing memory fragmentation to enabling live migration. Important open problems persist. How can schedulers guarantee performance amid uncertainty in job profiles, resource availability, and energy pricing? What are the theoretical utilization limits under heavy-tailed workloads and heterogeneous hardware? How should systems remain robust to tail events such as correlated stragglers or sudden spot revocations? As GPU fleets approach exascale and serve an increasingly diverse model mix, answers to these questions will be pivotal. Taken together, these questions re-activate the \"Energy-aware scheduling\" row that Table 8 left marked open.