# Algorithmic Techniques for GPU Scheduling: A Comprehensive Survey
## 1 Introduction
### Survey Methodology  
A rigorous, multi-stage screening pipeline ensured that the survey is systematic rather than anecdotal. Figure 2 summarizes the workflow; the text below details each decision point and the approximate record counts. Stage 0: scope definition and rubric design. Before any database queries, we drafted the four--dimension quality--weighting rubric in Table 1. Pilot screening on a conve  
Figure 2: Multi-stage screening pipeline. Counts rounded to nearest 10.  
nience sample of 20 papers confirmed that the rubric discriminated well between position statements, theoretical work, and large-scale empirical studies. Stage 1: background and systems context (1960-2025). Databases searched: Google Scholar, IEEE Xplore, ACM DL, manufacturer websites for official specifications. Query families:  
\"GPU AND history\", \"CPU-GPU architectural comparison\", \"cluster topology evolution\", \"GPU architecture\". Roughly 200 candidate records were identified by title screening; abstract screening reduced this to \(\approx\)100. After full-text reading we retained 70 sources that (i) provided quantitative historical context or (ii) framed scheduling problems from a systems perspective. These sources underpin the Introduction. Stage 2: algorithmic techniques (2010-2025). Databases searched: ACM DL, USENIX, IEEE Xplore, Google Scholar. Canonical query:  
\"GPU\" AND \"cluster\" AND (scheduler OR scheduling) AND (coflow OR multi-objective OR reinforcement OR prediction OR heuristic OR energy-aware OR techniques OR metaheuristic). Title screening surfaced more than 400 unique records. Abstract screening--applying the inclusion criteria listed below--reduced the pool to \(\approx\)250 (ten papers overlapped with Stage 1 and were carried forward). Full-text review against the rubric yielded roughly 100 high-quality papers, weighted in proportion to their rubric scores when allocating discussion space in Sections 2 and 3. A study was included if it (i) addressed the scheduling of multiple GPUs, (ii) tackled heterogeneous hardware, multi-objective optimization, prediction-enhanced policies, or formal guarantees, and (iii) provided empirical evidence or rigorous theory. Position papers without such support and single-GPU runtime optimizations were excluded. Stage 3: LLM-specific scheduling (2020-2025). Given the rapid rise of large language models (LLMs), we executed a dedicated search phase focused on inference/training at scale. Query example:  
Given the rapid rise of large language models, we executed a dedicated search phase focused on inference/training at scale. **Query example: \"LLM inference\" AND \"GPU scheduling\" AND coflow. Approximately 120 titles were screened; abstract review left \(\approx\) 40. Full-text assessment against the Stage 2 criteria produced a final set of about 20 papers, complemented by Stage 2 studies that offered immediately applicable techniques. \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Dimension Evidence Required** & **Score Range** & \\ \hline Strong theoretical guarantees & 0–2 & Formal bounds or proofs (e.g., competitive ratio). \\ \hline Hardware realism & 0–2 & Evaluation on clusters with at least 8 GPUs or production-scale traces. \\ \hline Comparative analysis & 0–2 & Benchmarking against standard baselines. \\ \hline Modern DL relevance & 0–2 & Targets large-scale deep learning or LLM workloads. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Quality-weighting rubric used to guide the depth of coverage (maximum possible score: 8). Stage 4: citation snowballing and gap filling. Backward and forward snowballing on the highest-weighted papers unearthed about 30 additional works, often preprints that had not yet appeared in indexed databases. Where these filled clear topical gaps (e.g., secure multi-tenant isolation), they were assimilated and scored. Stage 5: weighting and depth of coverage.