# Algorithmic Techniques for GPU Scheduling: A Comprehensive Survey
## 1 Introduction
### GPU vs. CPU: Similarities and Differences in Architecture and Fundamentals  
A meaningful comparison between CPUs and GPUs spans architectural design, performance characteristics, memory hierarchy, and resource management. Traditional CPUs prioritize sequential instruction throughput and general-purpose flexibility, while modern GPUs consist of thousands of smaller, specialized cores engineered for massive data parallelism [15; 16; 17; 18]. For example, while a high-end server CPU may reach roughly 192 cores, the NVIDIA A100 GPU boasts 6912 cores, dramatically illustrating the divergent scaling strategies of these platforms [19; 20]. CPUs often encounter diminishing returns from increase in core count--constrained by Amdahl's law and the overhead of sequential resource management--while GPUs excel in scheduling and running thousands of parallel threads with minimal latency [18; 21; 22; 23; 24]. Consequently, for highly parallelizable workloads such as scientific simulations and deep learning, GPUs can achieve speedups ranging from \(55\times\) to over \(100\times\) compared to CPUs, a performance gap that further widens with larger model sizes and batch dimensions [25; 26; 27]. Memory architecture further accentuates the contrast between CPUs and GPUs. GPU workloads--particularly in graphics rendering and deep learning--rely on extremely high memory bandwidth to support their massive parallel operations [28]. To meet these demands, GPUs employ specialized on-board memory (e.g., VRAM) and allocate a much larger proportion of registers relative to main memory than CPUs do [29; 30]. Consequently, modern data-center GPUs can deliver up to 54 \(\times\) the memory bandwidth of comparable CPUs, highlighting a profound disparity in data-movement capability [27]. Resource management also diverges sharply between the two architectures. In standalone systems, the GPU driver orchestrates memory allocation, context switching, and command handling, serving as the critical interface between the operating system and the hardware [31; 32]. Meanwhile, runtime frameworks such as CUDA and OpenCL furnish low-level scheduling and execution controls [33; 34]. At scale, high-performance computing clusters layer in sophisticated orchestration platforms--SLURM, Borg, Yarn, and Kubernetes--to allocate GPU resources, track utilization, and optimize throughput across nodes [35; 36; 37; 38; 39]. These systems enforce workload isolation, priority policies, and cluster-wide performance tuning [40], yet the GPU driver remains indispensable for fine-grained, device-level control of the hardware [41]. The performance of GPUs is typically evaluated across several key metrics, as outlined below. _Floating-point operations per second_ (FLOPs) measure raw arithmetic throughput and are broken out by precision--16-bit (half), 32-bit (single), 64-bit (double), and, more recently, 8-bit operations [42; 43; 44; 45]. _Tensor performance_, reported in TOPS (tera operations per second), reflects the capabilities of specialized tensor cores for AI/ML (machine learning) workloads (e.g., INTS TOPS for inference or FP16 TFLOPS for training) [46]. _Memory bandwidth_, in GB/s or TB/s, gauges how quickly data can be moved on and off the GPU--an essential factor for large-scale deep learning [45]. The _number of CUDA cores_ (Nvidia) or stream processors (AMD) indicates parallelism potential [19; 47], while _VRAM capacity_ determines the size of models and datasets that can be held in memory [45]. Additionally, _clock speed_ (MHz/GHz) impacts per-core instruction throughput [48], _PCIe (peripheral component interconnect express) bandwidth_ governs CPU-GPU data transfer rates [49], and _power consumption_ (Watts) affects overall energy efficiency and cooling requirements [45; 48]. Beyond raw numbers, a GPU's compute capability defines its supported feature set--such as specific CUDA compute capability levels on Nvidia hardware [45].