# Algorithmic Techniques for GPU Scheduling: A Comprehensive Survey
## 4 Looking Forward: LLM Focus
### Hybrid Scheduling Strategies and Learning-Augmented Algorithms  
Revisiting the \"Robust hybrid scheduling\" row left open in Section 3, this subsection crystallizes the obstacles, payoffs, and performance targets for robust hybrid algorithms--those that preserve worst-case guarantees while still leveraging learned hints when they are accurate. Classical scheduling theory is increasingly intersecting with ML, particularly in GPU clusters, where high-dimensional job behavior defies closed-form modeling. _Learning-augmented_ (or prediction-enhanced) algorithms equip online schedulers with forecasts--such as job sizes or speed-up curves--and analyze two regimes: one where predictions are accurate and another where they are adversarially wrong [212]. The key metric, the _price of misprediction_, captures how far performance can degrade when forecasts are noisy [113]. Translating this framework to accelerator-rich data centers is both natural and challenging,as deep learning workloads interleave compute, communication, and memory stalls in patterns poorly captured by classical queueing models.  
In practice, most systems adopt lightweight hybrid designs that pair heuristic search with small learned components. Sia, for example, performs fast placement enumeration and uses a throughput predictor to score candidates [55]. RL appears in early-stage prototypes--such as Google's cluster-DAG scheduler and Decima--but end-to-end RL often struggles with safety, generalization, and inference-time constraints. A more robust pattern confines learning to a well-bounded subproblem: Harmony, for instance, trains an RL agent only to select tensors for offload, while a rule-based outer loop enforces fairness and SLA compliance [213; 214]. Multi-policy hybrids generalize this approach: short jobs may enter a size-based queue, while long-running training tasks are pooled under fair-share policies, echoing multi-level feedback queues. Profiling-first systems go further. Frenzy runs a short trial to estimate an LLM's execution footprint, feeds the resulting vector to a predictor, and then invokes classical optimization for placement [215]. With lightweight, systematic profiling in the first seconds of execution, this architecture could become a general design pattern.  
Still, prediction-driven scheduling raises robustness concerns. Forecasting errors can cascade, leading to GPU fragmentation or inflated tail latency. Robust optimization offers a hedge--minimizing regret or optimizing high-confidence quantiles--but GPU-centric cost models (e.g., throughput versus memory fragmentation) remain underexplored. So too do learning methods that can operate within sub-second controller loops without incurring significant latency overhead.  
Three research questions currently define the frontier. First, what should be learned and what should remain hard-coded? Features like speed-up curves, interference patterns, and failure probabilities offer rich signal, but may resist accurate prediction. Second, how can systems respect the tight control-loop latencies of modern clusters--often tens of milliseconds--while still benefiting from ML-driven decisions? Third, how can worst-case performance be guaranteed when predictions fail? Establishing competitive ratios or tail-latency bounds under adversarial or noisy inputs is key to ensuring that learning-augmented methods never underperform traditional heuristics.  
Progress will depend on an iterative dialogue between systems practice--demonstrating real-world benefit--and algorithmic theory, which provides performance guarantees when predictions go awry. As GPU fleets grow and workload diversity increases, hybrid schedulers that blend analytic insights with judicious learning are poised to become the rule rather than the exception.