ATLAS: Alibaba Dataset and Benchmark for Learning-Augmented Scheduling
Authors: Zhiyun Jiang
Tianming Zhao
Chunqiu xia
Albert Zomaya
Venue: ICLR 2026 Poster (2026)
Keywords: Scheduling with predictions
Dataset and benchmark
Machine learning
Learning augmented scheduling
Non-clairvoyant scheduling
Primary area: datasets and benchmarks

Abstract: Learning-augmented scheduling uses ML predictions to improve decision-making under uncertainty. Many algorithms in this class have been proposed with better theoretical guarantees than the classic methods. Translating these theoretical results into practice, however, requires an understanding of real workloads. Such an understanding is hard to develop because existing production traces either lack the ground-truth processing times or are not publicly available, while synthetic benchmarks fail to represent real-world complexity. We fill this gap by introducing *Alibaba Trace for Learning-Augmented Scheduling (ATLAS)*, a research-ready dataset derived from Alibaba's Platform of Artificial Intelligence (PAI) cluster trace—a production system that processes hundreds of thousands of ML jobs per day. The ATLAS dataset has been cleaned and features engineered to represent the inputs and constraints of non-clairvoyant scheduling, including user tags, resource requests (CPU/GPU/memory), and job structures with ground-truth processing times. We develop a prediction benchmark reporting prediction error metrics, along with feature importance analysis, and introduce a novel multiple-stage ML model. We also provide a scheduling benchmark for minimizing the total completion time, max-stretch, and makespan. ATLAS is a reproducible foundation for researchers to study learning-augmented scheduling on real workloads, available at https://github.com/zhiyunjiang0810/non-clairvoyant-with-predictions.

Published as a conference paper at ICLR 2026
ATLAS: ALIBABA DATASET AND BENCHMARK FOR SEARNING-AUGMENTER SCHEDULING
Zhiyun Jiang Tianming Zhao
Chunqiu Xia
Albert Y. Zomaya
University of Sydney
(zj1a5638, tianming ng. zhao, chunqiu. xia, albert zomaya) @sydney edu.
ABSTRACT
-ugmmnete scheduling uses ML predictions to improve decision maaiingg-maknnggg under uncertainty. Many algorithms in this class have been proposed with bet- ter theoretical guarantees than the classic methods. Translating these theoreti- cal results into practice, however, requires understanding of real workloads. Such an understanding is hard to develop because existing production traces ei- ther lack the ground-truth processing times are not publicly available, while synthetic benchmarks fail to represent real-world complexity. We fill this gap by introducing Alibaba Trace for Saarning-Augmenter Scheduling (ATLAS), research-ready dataset derived from Alibaba's 's Platform of Artificial Intelligence (PAI) cluster trace-a production system that processes hundreds of thousands of ML jobs per day. The ATLAS dataset has been cleaned and features engi- neered to represent the inputs and constraints of non-clairvoyant scheduling, in- cluding user tags, resource requests (CPU/GPU/memory)) and job structures with ground-truth processing times. We develop a prediction benchmark reporting prediction error metries, along with feature importance analysis, and introduce a novel multi-stage ML model. We also provide a scheduling benchmark for min- imizing the total completion time, max-stretch, and makespan. ATLAS is re- scheduling on
producible foundation for researchers to study sournin--augmenne real workloads, available at https: //github. com/ zhi yun jiang0810/ hon-cle rvoyant - w th cttredicciinns.
INTRODUCTION
Modern computing systems have to schedule millions of jobs across without knowing job sizes (i.e., processing time) at submission, challenge known non-clairvoyant scheduling. As job sizes are unknown at arrival, the scheduler cannot implement an optimal clairvoyant strategy such SRPT for total completion time; consequently, non-clairvoyant algorithms achieve suboptimal scheduling performance (Motwani et al.) 1994). Learning- augmented algorithms address this per- formance degradation by incorporating ML job size predictions into online algorithms, improving performance while maintaining worst-case guarantees (Kumar et al. 2018). This framework applies to many domains, and now has grown into active community (Lindermayr & Megow 2022).
To illustrate the effect of predictions, consider the single-machine scheduling to minimize total com- pletion time Σ Cj. where Cj is the completion time of job Jj. in Figure | Suppose have four jobs released at ry with unknown sizes p7. Without job predictions, First-In-First-Out (FIFO) runs jobs in arrival order, yielding a total completion time of 51. Round Robin(RR), another good default for non- non-clair alncciirvoyy scheduling (Motwani et al. 1994), achieves C of 46.3. With predictions p;, Shortest Predicted Job First (SPJF) runs jobs by job size predictions. Even imperfect predictions add value when they roughly reflect relative job order. With true sizes, Shortest Remaining Processing Time (SRPT) is optimal for this problem (Schrage 1968), yielding total completion time of 33. Be- yond Σ Cj. other interesting objectives include maximum stretch, which measures faimess via the maximum ratio of job response time to size max, C,-ry and makespan Cmax = max, Cj repre- pr
senting the completion time of the last job, classic objective in auralll-mmachnne scheduling (Zheng (et al. 2023). For each specific objective, recent theoretical work has developed learning-ant -uggmented algorithms with provable guarantees (Zhao al. 2024 [ Lattanzi et al. 2020 Kumar al. 2018).

---

Published as a conference paper at ICLR 2026
compltton
Streks
Ralose thas
SPAF
hediced 19
SUP
Figure 1: A toy example showing the comparison of online/offtine scheduling algorithms single machine with four johs arriving at 0,0,1, with job sizes 4, 10, 1,3 and job size predictions 11, Despite pre- diction errors, SPJF, a widely known icarning-augmentt
algorithm, achieves a total completion time of 37, remaining close to SRPT's optimum of 33 and outperforming both Round Robin's ' 46.3 and FIFO's
However, real production clusters often violate core assumptions underlying these theoretical mod- els. In practice, jobs execute multi-step workflows (e.g.. preprocessing before training) where early-stage failures can terminate the sequence. Furthermore, hardware is heterogeneous, and ar- rivals are stochastic, undermining standard analyses (Weng et al.) 2022). These disconnects between theory and practice raise a central question: How well do learning cetrningg-uggwan augmented schedulers perform in real-world environments? ? We address the problem with Alibaba Trace for Learning-Augmente Augmented Scheduling (ATLAS), the first dataset for -agminnted scheduling derived from PAI produc- tion clusters, covering over 730,000 jobs with complete execution histories and resource profiles.
Issues with current datasets and benchmarks. First, existing production traces offer limited data for training and evaluating predictors for job processing times. Google's 's Borg traces (Tirmazi ct al. 2020) normalize processing times and obfuscate job identities, removing rich context like user patterns, job types, resource requests, and historical behavior. Azure public datasets (Cortez et al.) 2017) and Microsoft's Virtual Machine (VM) allocation traces (Lu et al., 2017) focus primarily on VM provisioning, exposing utilization rates while omitting, job structures or exact completion times. The Alibaba trace (Weng et al. (022)) provides job structures but was designed for workload char- acterization rather than scheduling evaluations. Second, most theoretical studies rely exclusively synthetic workloads (Zhao et al. 2022; Benomar & Perchet 2024), limiting job sizes to standard exponential, Pareto, uniform distributions that miss the complex patterns found in real systems. Third, the field lacks a standardized evaluation benchmark: a clear, reproducible specification of (a) the scheduling framework (online/offtine, jpremmppee,, number of machines), (h) how pre- dictors are trained and validated, and (c) how results are reported and normalized. Consequently, different studies adopt incompatible problem formulations, metrics, and experimental setups, such as work by Fan et al. (2022); Im et al. (2023); |Bampis et al. (2023). making cross cross-paper algorithm comparisons difficult. Furthermore, many overlook temporal constraints (training on past, testing on future), failing to restrict features to historical information, or skip calibration-test separation, risk- ing information leakage that violates non-clairvoyant assumptions (Kapoor & Narayanan 2023).
1.1
OUR WORK
The ATLAS Dataset. ATLAS transforms raw production traces from Alibaba's Platform of Ar- tificial Intelligence cluster into a dataset designed for scheduling research. The dataset contains completed ML. jobs collected from a cluster with over 6,500 GPUs across 1,800 machines. The overview and statistics are shown in Table ATLAS dataset has three defining characteristies:
(1) Non-clairvoyant dataset: Our dataset is non-clairvoyant, where models access only information available at submission time, such as resource plans (resource requests) for CPU, memory, GPU, and instance counts for each task, and identity fields needed to build signatures such as user, group, and workload. We exclude post-execution metries, actual resource usage, utilization rates, and machine placements, ensuring the ATLAS dataset is research-ready. which replicates a real scheduling envi- ronment. (2) Complete ground-truth job sizes: Unlike other production traces that omit or normalize runtime information for privacy reasons, ATLAS provides verified processing times for all tasks and jobs. Jobs, with a mean of 1.5 hours, range from seconds to days, with 73.8% completing within one hour and 5.7% exceeding six hours, showing full spectrum of production workloads. (3) Rich workload diversity: ATLAS dataset encompasses jobs with resource requirements spanning from single-CPI tasks to distributed jobs using multiple GPU and GPU clusters, reflecting realis- tie heterogeneity in CPU/GPU/memory requests. The dataset includes 74.4% single-instane jobs, multi-instance jobs, and distributed jobs with up 1,050 instances, capturing job scale complexity.
2

---

Published as a conference paper at ICLR 2026
Table 1: ATLAS dataset contains jobs from two-month Alibaba trace with selected tummit-time known and engineered without data-leakage features. prediction feature; ground truth label; excluded.
Source
Field
Use
Description
Dataset Statistics
Submission time
Release time ry: cnables temporal patterms Anonymized submitter (1,314 users)
Dataset splits
Job
User
Training
512.649 (70%)
Processing time
I Label p3 ming 5:
max; fz
Validation
109,853 (15%) 109,853 (15%)
Test
Task count
Number of roles [1,20 20] (modian: 1)
Job size
Planned CPU
E Rett, [0, (108K]K] (median: 6)
Mean
5,382 (1.5 h) 663 (11 min) 17,095 [3 s. 626,384 s]
Task
Planned GPU
1,2 [0, 40K] (median:
Median
Planned Memory
Retks (0.4, 47K] GiB (median: 29)
Std. Dev.
Instance count
Total parallelism Σ, the [L. 1050]
Range
Group tag
Semantic cluster (65% recurring tasks)
Job duration
Group-tag
Workload tag
Application type when known
< 10 min
355,328 (48.5%)
GPU spec
Submit-time constraint (V100/100//4)
10 min-6
335,121 (45.8%)
Recurrence count
Historical submission frequency
>6
41,906 (5.7%)
CPU per GPU
Resource ratio Yepu//ine
Job-scale Single-instanee()
Memory per GPU
Resource ratio /mem/ Fape
544,881 (74,4%)
Engineered
Distributed flag
Binary indicator (r > 1)
Small (2 (2-10)
93,951 (12.8%)
User history
Mcan/std of user's past submissions Mean/std/count of group's past runs
Medium (11-100)
86,380 (118%%
Group history
Large (> 100)
7,143 (1.0%)
Scale category
Composite job-scale classification
Assigned GPU
Actual placement (V100/P000/4MMisc)
GPU categories
Instance IDs
Worker/comtainer identifiers
No GPU
17,452 (2.4%)
Excluded
Sensor metrics
Runtime CPU/GPU/memory utilization
Single GPU
554,179 (75.7%)
Network usage
Bandwidth and I/O measurements
Multi-GUU
110,474 (15.1%)
Machine specs
Hardwure configuration details
GPU cluster
50,250 (6.9%)
The LASched Benchmark. Built on top of ATLAS, LASched (Learning-Augett Augmented Scheduling Benchmark) provides a standardized evaluation for the Prediction Task on job sizes and Schedul- ing Task of jobs with job size predictions. The prediction benchmark implements multiple baseline models and evaluation metrics, showing that jobs exhibit recurring patterns that can be leveraged for accuracy. LASched enforces leakage-safe construction and restricts features to past-only in- formation available at decision time, preventing future leakage. The scheduling benchmark evalu- ates classic and rraming-augmenttrr schedulers three metrics-tor completion time, maximum stretch, and makespan. With a scheduling benchmark including various baselines, using job size predictions to schedule, the objective values normalized against the optimum. Together, ATLAS and LASched form a complete platform that enables researchers to develop, evaluate, and compare leamnig-augmmented scheduling methods on real production workloads with reproducible results.
2 THE ATLAS DATASET
2.1 DATA SOURCE AND FORMALIZATION
Alibaba PAI-2020 Trace ATLAS is built on the publicly available Alibaba PAI-2020 GPU- cluster trace, which captures two months of MLaaS activity a large heterogeneous GPU cluster over 6,500 GPUs across ~ ~,00 machines (Weng et al. 2022). The trace captures the job life-cycle, including submission, queuing, scheduling, and execution. The trace is relational and organized hierarchically into jobs, tasks, and instances, consistent with established system frameworks such as Google Borg (Verma al., 2015)) and Facebook's Hadoop workloads (Zaharia al.] 2008). Users submit ML jobs through frameworks (e.g.. TensorFlow, PyTorch, Graph-Learn); each job assigned to a scheduler (Fuxi), which translates into multiple tasks with different roles. e.g., worker, parameter-server (PS), and then instantiates them into Docker containers that are distributed across multiple machines based on resource availability and locality requirements (Weng et al. 2022). Once started, jobs run to completion without preemption. The trace contains only start and end timestamps for each instance, not suspension or resumption events. PAI's monitoring collects per-instance system metrics: CPU/GPU utilization and host/GPU memory, at every 15 seconds via daemon agents that query the Linux kernel and NVIDIA's NVML; the release data also includes machine-level statistics such network receive throughput. Figure 2 summarizes the job schema, showing hierarchical structure, and the relationship between jobs, planned resources, observed uti- lization, and machine specifications. We formalize the PAI trace columns that time semanties,
3

---

Published as a conference paper at ICLR 2026
Job schema
Plan va Dsage
Machine Spec
C-) Aliaaa Cluu
Instance
GPPpp
Tink A
fir que GOR
pi in 100% in 2205 (m (UPUiA))
Michie ID CPU typa Vimo Cip GPI
(
per (a
Instance_ 110
Canea'' Caty
Off - 1000 9305 C
Cip ETU Wieerm Cap Ma2
Taak
pkr_qu--
of
Ne Na Seweee
Roy EORLL
Instance
020 5T% (0.37 GPGeqcrrakes)
GFU tinccin) 85% CPl 18.5%
XXL
P2
TAX
S0 plaz
plr -00
QU Toggee
5005 (5ceees)
Netrocthe 112 NIB
June lead 183
Figure 2: Structure of the Alibaba PAT-2020 GPU cluster trace. Jobs decompose hierarchically into tasks and instances with per-instance resource requirements for GPU and CPU. Job processing time p} cTX) de equals the maximum task duration. The center panel shows planned versus actual resource usage, revealing users requesting more resources than they actually use. Instances can request fractional resources (e.g., 0.5 GPU), supporting resource sharing across multiple jobs. Machine specifications and utilization metrics appear on the right. Task illustrates gang scheduling g. where more than instances start the same time.
label construction, and reproducibility checks are unambiguous, whereas prior analyses of this trace reported workload behavior without unified mathematical specification.
Job life-cycle: arrivals, queuing, launching. We formalise job life-cycle mathematically as fol- lows. A job Jj arrives at time ry and consists of tasks E T(j). Each task t declares per- instance demand vector r, (rr,1, Tr,2; Fr,3) E R3 representing per-instance GPU, CPU, and memory requests. To be specific, a distributed training worker might request r, = (1,8, 32) for GPU, 8 CPUs, and 32GB memory. Also, each task includes an instance count ne, and constraints: an admissible GPU-type set r,, a gang flag Jt E {0,1}, and an optional local- ity flag l E 10, The cluster comprises machines and each machine m with capacity vec- tor Cm = (Gm, I: 2,,22,m,.3) Cm,3) RR. corresponding to GPU, CPU and memory capacity. While the job waits, the scheduler seeks, for each task t, the carliest time T 2 ry at which its n, in- stances admit a feasible placement. Let I(t) be the instances of t; let t(i) denote the task of in- stance é; let (i,m(T) E 1) indicate that instance i is assigned to machine m at time T (i.e., Timm (T) 1 if instance i runs m at time 7, O otherwise). Feasibility requires, for every ma- chine m. the resource capacity constraint: E, (. Te(e) S Cm Where the inequality holds for each resource dimension: GPU, CPU and memory. If gt
then all instances of must start together, i.e., Σm EVET()) (T) = nyi if l = 1 then all instances of t must be co-located on some machine m,, i.c. LiEI(e) Lt,m, (T) n;; for GPU-type admissibility, let g(m) denote the GPU type of machine m. We enforee Ft.m (7) = 0, wheneverg(m) & I,, Vi E I(t), Vm, Vr, i.c. instances are ineligible for machines of the wrong GPU type. Define the task-ready time 8, := inf(r 2 rj a feasible placement for / exists at T). The job's start time the earliest task launch 8j mil 8t, and the queuing delay is 1 8, - -r-- In the PAI trace, the job table's start time stores the submission timer,, while task and instance tables record the real- ized launches and finishes. In production, PAI uses reserving-and-Pai aad-ciknn scheduling: it reserves high-end 100M32 (NVLink) nodes for high-GPU or strict gang/locality tasks, and packs lower-GPU tasks onto T4/older 'Misc' machines via fractional-GPe sharing (Weng et al. 2022).
Job processing time. For every instance i we record its start and end time (ss, es) and the duration as d, E Si. For each task with instances I(t), define s, min $t, et max4ez(() Ei and de = et- St. Let = min; 8t and E = max; ét. The job-level processing time is:
Dj E
For any task i, de- C2 St F S pj. hence max, de S p) Prior work by Weng et al. (2022) trains a regressor on instance records and predicts a per-instance duration d; scheduling and error evaluation are performed at the instance granularity. When a value summarized for task, is E
the mean across that task's s instances, d, = [[(0] EvErae) d. In contrast, we define the job-level label without using the mean task duration: max; é min, 8t . which matches the classic fork-join completion rule: a task completes when all of its instances complete, and a job completes when all of its tasks complete (Blumofe & Leiserson
1999
Ko & Serfozo 2004). Under gang scheduling
4

---

Published as a conference paper at ICLR 2026
Table 2: Three representative job examples from the ATLAS tubmit-time dataset, and the processing time serves as the prediction label representing the uninterrupted execution duration.
Type
tablee worbload
Procoiing Time
Joh pare
uer 16
subeol Hime
Instances
<pe (14
mem (GB)
17rep Dag
recurmaee
(Labll)
Suaall Inforenee
30334555...
09:2:15
600
25
20.3
602875677
47
136
Desctleuted Truinieg
$4ade520...
dkatlece Lasttec.....
1045:30
26
15,100
635
52.0
abwB22ek...
ctr
12
5,003
Larae
#8.454b3...
1412245
$5,900
4,000
2000.
46,822
iiteration
Madiize qudificatie
Schucallag
compliiiss
Tel: a
Jen 1pc
Seall rikenee
1364
Unittberd tuiwng
Lege wade
Medioa STSDR
Midinal
44552>
Madina (200) mort CPCS
Teal moorm
35800 EPU
Madion IVTOR 1VB8 C11
25
100
(Mad EMi CMUL PEC
Figure 3: SRPT scheduling of three real PAT jobs. The table shows job characteristics. The timeline illustrates how johs are allocated: Job (small inference) completes quickly on Machine 1; Job (distributed training) runs tasks in parallel on Machines 2 and 3; Job (large scale) has Task 1 starting immediately on Machine 4 while Task 2 waits for Machine 3, demonstrating the impact of resource heterogeneity on scheduling decisions.
@ 1). EiEzO (T) = nt, instances of a task start together (s: = 8t), so de = max; d, and per-instance predictions can be used as task size directly. Weng et al. (022) report that 85% of task instances the PAI trace require gang scheduling; in the remaining cases, instances may start at different times, such as one at Os and one at 50s, but both lasting 100s. The same policies apply to job level. We propose p that is reconstructed from timestamps in the task table, not just using the mean task size, which is more pessimistic method. Admittedly, neither definition decouples the true job demand from Fuxi's historical allocation decisions; allocation preferences and CPU-bound contention remain embedded in the recorded durations. However, the trade-off lies in safety. de de filters skew systematically underestimates occupancy for gang-scheduled tasks where tail determines release. Conversely, p3 captures actual system constraints. We explicitly enforce robustness.
Resource metrics and utilization. Let resource coordinates & E (1, denote GPU, CPU, and memory, respectively. We distinguish submit-time requests from post-execution utilization metrics. Each task t declares a per-instance request = T1,2,, Fr.3) and instance count n4, yielding total request R2 ng For job j with tasks T(j), the submit-time request known at arrival is R₃ = EIET() R. These submit-time quantities {r,, nr, n,,R, R2} constitute the pre- diction features available at scheduling time. Post-execution, instance i runs on interval [st, es) with duration d, = e, - s; on machine m(i) having capacity vector Cm(i). Let ūg, E [0,1] denote the time-averaged utilization fraction of resource A for instance i; the resource-time sumed is Ai, = Ui,k Cmtt).. d, Aggregating to tasks and jobs yields A = Eiez(o) Ai, and Aj.k = ΣIETO) ALk. Over reporting horizon H > O, the average utilization of machine m on resource is Unost, (m) = Hmm, EiET(m)) Ai, With cluster capacity Cs = Σm Cm,k. the cluster-level utilization is ucluster, k HH. Ej Aj,. These realized usage metrics {ūr, A. k: u. computed after job completion for workload characterization and data validation, not for predic- tion (Verma et al. 2015 Jeon et al. 2019 Weng et al. 2022).
2.2 DATASET DESCRIPTION
Revised data columns. We extract submit-time features from three of the original Alibaba's PAI joint tables to build dataset for [] From the job table, we keep job's 'ss start time and user ID as features, while task and instance timestamps are used solely to compute the ground-truth processing time p = max; 6 min, 8t- The task table provides resource requirements (CPU, GPU, memory) and parallelism metrics (task and instance counts), aggregated to job level; we exclude assigned GPU types as these reflect post- submission scheduling decisions. The group-tag table contributes semantic identifiers and GPU specifications that encode submission-time time constraints and recurrence patterns. The remaining ta- bles, sensor and instance, are excluded as they contain either redundant information, post-execution outcomes incompatible with non-ci olacciirvoyant scheduling. Table 55
non-clairvoyant scheduling, specified by Ta-
metrics, or senecullingdeppendee illustrates data columns with real information for representative jobs derived from the ATLAS.
5

---

Published as a conference paper at ICLR 2026
1.6
(N)
(N)
(4)
0.4
362
12"0
12:2 10
CPU Cures
GPU Unitt
Mtemsry (G30)
Proceseing Time (houra))
Figure 4: Resource demand and processing time distributions in the PAI workload. Panels (a) (c) plot cumu- lative distribution function (CDF) per-job requested resources on a log-scaled x-axis with common y-axis, Cumulative Probability. (a) total requested CPU cores, (b) total requested GPUs, and (c) total requested mem- ory (GiB). Requests are computed as per-instance plans multiplied by instance count and aggregated per job. (d) shows the CDF of per-joh Pz. defined as the elapsed time from the first task launch to the joh's completion.
Salk 1ly u0h
Nbwr USEWE
Figure 5: Temporal patterns of submissions and planned resource demand.(a) Heatmap of cumulative job sub- missions by weekday (rows) and hour (columns). (b) Weekly mean job submissions per hour over complete Souday-Sunday weeks (jobs and tasks)((e) Weekly means of total planned requests from obtained from per- task, per-instance plans ner, (CPU, GPU, memory); shaded bands show 95% confidence intervals across weeks.
Case study. Figure illustrates heterogeneous-GGU scheduling on representative PAI jobs. For this case study we assume SRPT with known processing times; the goal is to illustrate how trace semantics and resource constraints impact on execution. The small inference job (136 s) is priori- tized and finishes quickly, minimizing its impact on throughput. The distributed training job, though longer, exploits parallelism with tasks running concurrently on different nodes. The large-scale job, arriving at t=100, shows how task-level scheduling adds flexibility:: Task 2 starts immediately available machine 4, while Task 1 queues for a V100 due to GPU-type locality. This yields asym- metric task completions (44,732 vs. 54,125 s), with the job's finish dictated by the slower task.
2.3
WORK LOAD CHARACTERIZATION
Data cleaning. We keep only terminated jobs/tasks and drop rows with missing timestamps or non-positive instance counts. In the trace, timestamps are converted to UTC+8, and for any time-series statistics we restrict to complete Monday-Sunday weeks, discarding partial weeks to avoid edge cases; this dataset choice improves comparability and reduces noise. We reproduced the instance-anchore plot in Weng et al. (2022) for verification, but our scope is on job- and task-level.
Heavy-skewed distribution. The PAI workload exhibits extreme heterogeneity at the job level, with distributions showing severe right-skew. Skewness is 11.02 for processing time and coefficient of variation is larger than 3 for resources. As shown in Figure resource requests size 3-5 orders of magnitude: CPUs (0-8,100 cores). GPUs (0-400 units), and memory (0.4 GB TB). Processing times vary from seconds to days, with the 99th percentile (19.1 hours) being 106x the median: 11 minutes. The workload stratifies into distinct scales: 74% single-instance jobs versus distributed jobs with up to 1,050 instances, and 66% tiny jobs (median 7-minute runtime) versus 1% massive jobs (median 77-minute runtime). This compound heterogeneity- where small fraction of jobs dominate resource
centumpttioonncessaaate log transformation, which reduces skewness from 11.02 to 0. 17 and scale-aware modeling for effective learning-augment scheduling.
Temporal pattern. The trace exhibits strong diurnal/weekly regularity: weekends have fewer sub- missions, and late-nigh hours show higher arrivals, consistent with prior production studies (Tir- mazi et al. 22200 Reiss et al. 201T). Whereas Weng et al. (2022) visualize a single week, we report the weekly average across complete weeks, shown in Figure which smooths episodic spikes and

---

Published as a conference paper at ICLR 2026
yields a more stable signature. Concretely, letting h E {0, ........ ,0....,,677 167) index Hfur-of-week and index weeks, we average hourly request series (e.g., task aaccored (h) and job-anchored Ju Jw(h) formed from planned totals R, n,r;)) as T(h) = [W] wwew Tu (h) and analogously for Jh)). In our data, T(h) and Jhh are nearly identical, reflecting that most tasks launch within the job's start hour, while the averaged curves remain less sharp, but more robust than single-week analysis.
BENCHMARK
3.1 PREPARATION
Data pre-processing. We use the ATLAS dataset, which constructs a submit-time, job--eell table by joining the job, task, and group-tag relations from ALIBABA PAI trace and retaining only ter- minated records. Timestamps are parsed as seconds, and empty rows are removed. For each task I with instances, we set St min; St,é, et max; etyi; and define the job processing time as Pj max; & min, 8t- Jobs with pj < 0 are discarded. The submission time ry anchors chronol- ogy and all causal features. Submit-time resource declarations are aggregated per job by summing the times of per-task plans multiplied by their multiplicity, and details are in Table [ We join the group tag, user identifier, workload tag. and requested GPU specification via the instance identifier; assigned hardware and any sust-submission outcomes are excluded. To avoid leakage, we split by ry: the earliest 70% for training, next 15% for validation, and final 15% for testing. Before training, we run simple checks and use log- tegtrransforeed pj the prediction target to stabilize heavy tails.
Feature eeeeeeeeeiiiinn engineering. From raw PAI ATLAS data. including 13 columns, we engineer 40 addi- tional features into 53 data frame columns, filtering to 33 model features after removing identifiers and intermediates. All encoders and statistics use training data only. (1) Resources: log-transformed totals (CPU, GPU, memory, instances, tasks) and per-instance ratios (CPU/instance, GPU/instance, memory/instance, tasks/instancee, addressing the heavy-tailed PAI distributions (Weng et al.) 2022) (Figure 4. (2) Temporal: sine-cosine hour-of-day encoding to preserve cyclic continuity (Jiang & Zhang 2009). plus day-of-week and weekend flags. (3) Recurrence signatures: concatenate
group, workload, and decile-bucketed resources; match to historical executions and attach the same train-only statistics such as mean, median, quartiles, standard deviation, and counts. expanding statistics for users and groups on y = log(1 + cumulatiive means, counts, and exponentially weighted moving averages (span=10) all with one-step lags via shift (1); low-support groups Empirical Bayes shrink- age (A = 5) toward the training-set mean. (5) Categorical: user, group, workload, and GPU speci- fication are label-encoded from the training set vocabulary with unseen values mapped to -1.
(4) Historical: strictly causal, aubmit-time-ordere
Ablation study and overfitting analysis. The ablation study, using LightGBM, reveals that work- load recurrence and group-level execution patterns are the dominant predictive signals (+20.2% over a resource-only baseline), while individual user behaviors provide secondary refinement. The re- sults validate our benchmark design and demonstrate that all using features contribute meaningfully to prediction accuracy, with group-level patterns generalizable across users and resource features transferable across datasets. Our overfitting analysis shows minimal 1.1% Cov @ 25% gap between 5-fold cross-validation training and test results, indicating negligible overfitting to the training data. While performance naturally drops for unseen users, 5.8% gap, due to missing user-specific his- tory, the model maintains robust accuracy by relying on generalizable group and resource features.
3.2 PREDICTION TASK
Prediction models. We model y = log(1 + p²), where p1 is from earliest task start to latest task end, from submit-time features x, using gradient boosting with validation- based calibration. We in- clude four baselines (Single-Stage, CRR++ without task-signature features, and six advanced methods specified in Appendix D (1) Conformal quantile regression (CQR) training quantile regressors at a E {0.1, 0.5, 0.9} with Ridge-blended blended fi- nal predictions (Romano et al. 2 ;; (2) Isotonic calibration ensuring monotonic mapping adapted for regression calibration Zadrozny & Elkan 2002; Kuleshov et al,, 2018);; (3) Meta-stacking com- bines diverse base models (I.2, regularized, quantile, Huber) via gradient boosting on validation
Recency-Fixed) that use rolling history
7

---

Published as a conference paper at ICLR 2026
predictions (Wolpert [992); (4) Gated experts (two-stage): a classifier routes examples capacity- matched regressors and aggregates them by soft probabilities (Jordan & Jacobs 1994); (5) Weighted recency uses exponential time-decay w, (XT - t)) across multiple temporal windows with MAB-weighted blending for drift adaptation Gama et al. 2014); (6) Historical Recency-Aware with Shrinkage (HRAS) uses per-signature means with EB shrinkage stabilize predictions for rare user- group-resource patterns, All calibrators fit exclusively on validation data following honest prediction principles (Wager& Athey 2018), with LightGBM (Ke al. 2017) as primary regressor.
Multi-stage predictor. Baselines without task-signature features reached only 19.5-20.2% Cov @25% (Table 3). motivating our calibration-centtr designs. We therefore evaluate all meth- ods within one leakage-free framework in Algorithm I cooperating MI. prediction methods above.
Algorithm Multi-Method Job Duration Prediction
I: Input: time-ordered splits by submit time ry: Divain 70%, D vellddation 15%, Drest 15%
2: Features: X; = =[xr, Xt, Xh, xo]
3: x, job totals from task table (CPU/GPU per-inst%o> counts log-totals, per-inst ratios
4: Xy: sin (2thh/244) cos(2th//24) day-of-week
5: X user/group histories with within-group shi (1); time-sincellaat-submit
6: N.: categorical (user, group, workload, gpu type spec) encoded from Dir only
7: Target: U, = log(1 + pf), where pj- = max, Ct min, 8; (excludes queuing)
8: Stage I (train on D,,):
9: Quantiles: Qx < LGBM (quantile), a E (0.1,
10: Regressons: {M } LGBM with {lz, Huber, regularized]
11: Two-Stage: classifier C on three duration classes via 20 expert E, per class
12:
Recency: Riull (time-decay), Rso, R20 (most recent 50%, 20%)
13:
Signatures: wrain-only stats (mediaanqqannillssccomm)) EB shrinkage H. + AApo (A=5)
no+A
14: Stage 2 (calibrate Doa):
15:
CQR: on log- target let = max (Qo.1 - y. y- Qo.)). i = (maxmr, 0));
16:
set bounds L, Qa. k U Qo.9 (used as features; clip Qo.5 to UT) Blender: 3 Ridge [., Qa.s, U, cl.,(Q0.s), Me, priors))
17:
Isotonic: @ monotone fit of (Qo.5, y) (log domain, clipped)
19:
Meta: [MM dispersion(Ms, context]; & LGBM on fmeta
20:
Recency: Ww x (MAE,a
+ c)- normalize Σn ww 1.
21: Stage 3 (predict on Die):
22:
CQR: 1 = ([.., Qo.5, U, clip(Q0.5) Me, priors)): also report [I, U]
23:
Isotonic: ÿ = (Q.s)))
Meta: ý v (imea))
24:
Two-Stage: 1 = De Te Ex()., with म. = P(c| x) from C
25:
Recency: y 2wne (f,)) War R.r
26:
HRAS: ÿ Ja else group EB prior; else global mean
27:
Output: p3 = exp(ū;) for all methods
3.3
SCHEDULING TASK
Implementation Setup. LASched evaluates objectives under following settings: for total com- pletion time (E, C2), single machine with online arrivals and preemptive scheduling (jobs J, released at times ry); for makespan, m parallel machines with batch release (all jobs at time 0) and non-preemptive; and for max-stretch, a single machine with online arrivals and preemption to cap- ture fairness and prevent starvation of large jobs. Unlike prior work on the original dataset (Weng et al. 2022), which exploits recurring laskl patterns, we study job-level scheduling with imper- fect predictions across all job types, scaling from single-machine to chousand-maching clusters.
Baseline Algorithms. For non-clairvoyant baselines, we use FIFO as the online default (Weng et al. 2022), RR, which shares capacity equally among active jobs (Motwani et al. 1994), and LAS Seaast-Attiee--eriiiee which prioritizes the job that has received the least service so far (Nuyens) & Wierman 2008). For clairvoyant baselines, which serve as offline performance bounds, we em- ploy SRPT, the optimal for minimizing total completion time (Schrage. 1968), and SJF (Shortest.
8

---

Published as a conference paper at ICLR 2026
Job First). For the multi maclinemachie makespan objective, evaluate LPT (Longest Processing Time), which balances load by assigning the largest job to the least-loaded machine (Della Croce & Scata- macchia 2020), alongside SPT (Shortest Processing Time) and Random assignment baseline.
Scheduling Algorithms. These algorithms integrate predictions P generated by our prediction benchmark models (e.g.. CQR, TwoSt) into online aeriiiionmmaking For total completion time, we evaluate SPJF and PRR (Preferential Round-Robin). PRR is a robust mechanism that reserves processor share À for the job with the smallest P while distributing the remaining rate A) equally among all jobs (Kumar et al. 2018). For max-stretch, we evaluate SPRPT (SRPT using predicted remaining work) and EDF-P, an predicted deadlines dj = ry + Sat- PJ.. For makespan, we substitute true sizes with predictions to create L.PTT (Longest Predicted Processing Time) and SPPT (Shortest Predicted Processing Time), prediction variant of SPT, evaluating how prediction errors impact scheduling policies.
policy that schedules based on
3.4 EVALUATION
Prediction error. From theoretical study, 7 = maxisisn max "i P)) and I1 = =E== l8,
P) 1p.
Př (Kumar et al., 2018 Zhao et al. 2022) are often reported. The community moves toward build- ing portfolio of metrics rather than single number (Ahmed et al.
2022). and propose di- verse empirical error metrics for the prediction task. We present Root Mean Squared Logarithmic
(m=( - In(1 + +)))² which de-emphasizes large outliers in heavy-tailed job distributions (Soysal & Streit 2021). Operational tolerance is captured by Cov- erage at t, the fraction of jobs predicted within a relative error т; we report 7 E (0.25, 0.50]:
Error RMSLE V
Covêr 100 n Σラ= 1p-pp] <T)
(Minku & Yao 2013). Finally, to assess ranking quality, we report Spearman's 's rank correlation p between Pj and pj (Pearson correlation) (Bedō & Ong 2016).
Scheduling Performance. We evaluate algorithms via empirical competitive ratios against op- timal solutions tight bounds. For total completion time 1/r pmtn| Σ Cj. we normalize by use McNaughton's preemp- tive bound OPT pre = max (E, P2 ///m, max, p2} as baseline: PMS = preemptive makespan is NP-hard, LPT empirically achieves near-optimal performance (PMs z 1) our instances. For max-stretch 1/r,, pryyt max, Sj. we obtain S* by bisection on S with EDF-feasibility (Harchol-Balte) 2013), then run EDF at 5* and normalize by the realized Semp = ma ry)/Pj. reporting ma, Smax/ Semp together with ps,99 and pS,med-
SRPT: PTC Ej CaLo; Ej C.CCPTT For makespan Pl|Cmax.
While non-
4 RESULTS AND DISCUSSION
Table 3 summarizes prediction quality. LGBM-Meta achieves the strongest overall accuracy: 51.2% Cov@25, 70.6% Cov@ 50, and 0=0911 through meta-stacking, while Two-Stage attains the highest recurring-jok coverage (78.0% Cov@50) via classification-firs routing. The consistent advantage over baselines confirms that task-signature features and empirical Bayes shrinkage are essential. Recurring jobs, which constitute 81.3% of the test set, exhibit markedly higher accuracy across all methods, validating the effective utility of historical job execution data. As shown in Table 4 each scheduling objective interacts with prediction quality differently. For total completion time, Meta- SPJF achieves a ratio of 1.106, a 10.5% degradation from clairvoyant SJF, and consistently out- performs PRR (A=0.7), indicating that fully committing to predicted rankings supersedes hedging with
Makespan shows moderate sensitivity: Iso-L.PTT (1.574) and CQR-SPPT (1.701) achieve the best ratios, as this objective depends primarily on identifying the largest jobs. Max- stretch reveals qualitatively different ranking, SPRPT consistently dominates EDF-P, with Meta- SPRPT achieving an 11x x improvement over LAS/FB. EDF-P improves median stretch (pmed R3 4.5 for Meta-EDF. 40x gain over LAS/FB), yet its worst case remains comparable LAS/FB. HRAS drives EDF-P to Pmax s 1700, over 8x worse than Meta-EDF, while SPRPT degrades only from 18 to 51, confirming that seadline-hased scheduling consistently amplifies absolute size emoos.

---

Published as a conference paper at ICLR 2026
Table 3: LASched prediction performance PAI trace. Cov@ fraction of predictions within 1% relative
enror.
Method
Task Signatures
Causal History
Ensemble
Cov@25 (%)
Cov@50 50 (%)
RMSLE
Type
All Rec.
All Rec.
(All)
(AII)
Single-Stage
Rolling
None
19.5
/
1.568
0.690
Beatine
CRR++
Rolling
Cluster
20.2 / 29.2
37.7/53..1
1.621
0.665
Scale-Experts
Rolling
Bucket
19.6 28.0
33.0/53.3
1.611
0.673
Recency- Fixed
Rolling
Temporal
20.2 28.2
50.9
1,578
0.685
CQR-Stack
EWM+EB
Conformal
47.0
67.5/76.8
0.944
0.897
HRAS
None
25.6/ 29.1
/5.9
1.373
0.801
Advancad
QMed+Iso
EWM+EB
Isotonic
45.9 53.4
67.0/76.2
0.975
0.888
LGBM-Meta
EWM+EB
Meta-stack
51.2 58.3
70.6/
0.871
0.912
Two-Stage
EWM+EB
Classify
49.8/ 58.0
68.8 78.0
0,945
0.897
Weighted-Res
EWM+EB
Temporal
39.9 45.5
6 70.9
0.948
0.896
Notes: Rec. task signature seen in training. Baselines: 52.0% recurring (no signature features); Advanced:
81.3% recurring (with signature features and EB shrinkage).
Complete scheduling performance across three objectives; lower better. Max-stretch on ==5,000 jobs; total completion time on n=10,000 jobs; makespan ratios averaged over m E {5, 10, 20, 50, 100).
Table 4:
(A) Max-Streich
(B) Total Compl. Time
(C) Makespan
Algorithm
Pa.max
P5.99
PS.
Algorithm
Ratio
Algorithm
Ratio
OPT (EDF S*)
1.000
1.000
1.000
SRPT
1.000
LPT
1.000
SRPT
1.581
1.304
0,932
SJF
1.001
SPT
1.539
LAS/FB
205.28
134.18
187.31
RR
1.975
Random
1.898
FIFO
5.372
CQR-SPRPT
21.45
5,802
6,219
CQR-SPIF
1.135
CQR-LPPT CQR-SPPT
1.721
CQR-EDF
400.06
75.78
13.30
CQR-PRR
1.328
1.701
HRAS SPRPP
51.10
7.517
14.05
HRAS-SPIR
1.686
HRAS-LPPT
1.937
HRAS-EDF
1700.5
514.83
405.33
HRAS-PRR
1.887
HRAS-SPPI
1.753
Iso-SPRPT
21.66
5.162
6.327
Iso-SPJF
1.161
Iso-LPPT
1.574
Iso-EDF
400.36
86.11
22.63
Iso-PRR
1.354
Iso-SPPT
1.717
Meta-SPRPT
18.13
4.643
4.718
Meta-SPJF
1.106
Meta-LPPT
1.689
Meta-EDF
204.66
34.13
4.504
Meta-PRR
1.292
Meta-SPPT
1.731
TwoSt-SPRPY
20.77
4.469
5,469
TwoSt-SPJF
1.124
TwoSt-LPPT
1.643
TwoSt-EDF
411.24
88.62
9,436
TwoSt-PRR
1.310
TwoSt-SPPT
1.722
Rec-SPRPT
31.11
5.571
8,471
Rec-SPJF
1.172
Rec-LPPT
1.842
Rec-EDF
572.26
136.03
77.85
Rec-PRR
1.369
Rec-SPPT
1.943
CONCLUSION
We introduce ATLAS, research-ready dataset comprising over 730k+ cluster jobs with compre- hensive features and ground-truth sizes, alongside LASched, a standardized benchmark and imple- mentation framework for leaming-augmentted scheduling. We establish rigorous prediction base- lines evaluated via coverage, RMSLE, and rank correlation, and benchmark a suite of scheduling algorithms. Serving as a community reference point, our empirical evaluation demonstrates that systems utilizing calibrated predictors achieve near-optimal total completion times and competitive makespan ratios. For tail-sensitive objectives, algorithmic structure is as vital as predictive accuracy: while EDF with predicted deadlines substantially outperforms non-clairvoyant baselines, the worst- case stretch remains an order of magnitude above optimal, exposing a fundamental gap between aggregate fidelity and worst-case guarantees. The findings motivate key directions: (1) asymmetric or sair-wware loss functions to explicitly align model training with scheduling metrics, (2) distribu- tionally robust optimization (e.g., CVaR) to minimize worst-case stretch, and (3) the co-design of prediction and deadline assignment mechanisms to fully exploit the structural advantages of EDF. style policies. Future work will extend the cross-dataset validation and optimization objectives.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
Zhiyun Jiang gratefully acknowledges the support of her husband, Wu. Professor Albert Zomaya and Dr Tianming Zhao would like to acknowledge the support of the Australian Research Council Research Hub for Future Digital Manufacturing (IH230100013).
Reproducibility Statement. We make our work reproducible along three aspects: data, pre- diction, and scheduling. The clear step-by-step user guide, including data downloading notes for different systems, code scripts running suggestions, and detailed evaluation method, is in com/ anonymized link, referred to readme.md file in the following link: Three reproducible infor- zhiyun
mation are listed.
(i) Data. ATLAS is derived from the public Alibaba PAI-2020 trace with novel formalized job schema and label construction; we release an anonymized repository, shown in abstract, with scripts to rebuild the submit-time job table and ground-truth labels from raw datasets, including checks for terminated rows only and exact time semantics (earliest task start, latest task end). Please see dataset link in the abstract and Section 2 Dataset. Users can employ this dataset construct interested columns, such as maximum task duration and instance duration. Also, users can make Python plots see job size distribution, actual resource utilization rate, which could be both at submit-time or post-execution, any workload characterization preferred.
(ii) Prediction. We release code to reproduce the split (70%/15%/155 by submit time), train-only feature engineering (resources, temporal signals, recurrence signatures, strictly causal group/user histories with shi (1), and label encoding), and all six calibrated baselines with validation-only calibration; configuration files and fixed random seeds are provided to regenerate Table Jende-nend-o (Cov@2555) @2550, RMSLE, and Spearman's 's p). Users can use traditional ML models to make job size and task size predictions, which is also provided, and technical details shown in Appendix C
(iii) Scheduling. The benchmark includes an executable simulator with reference implementations of all policies and the exact normalizations used in Table 4 Σ, C, reported relative to SRPT; makespan reported relative to the preemptive lower bound OPT prei and max-stretch computed via an Deasiiibiyyyyty test at the bisection optimum. Scripts are provided to recreate every number from fixed commit.
REFERENCES
Nasim Ahmed, Andre LC Barezak, Mohammad A Rashid, and Teo Susnjak. Runtime prediction of big data jobs: performance comparison of machine learning algorithms and analytical models. Journal of Big Data, 9(1):67, 2022.
Evripidis Bampis, Alexander Kononov, Giorgio Lucarelli, and Fanny Pascual. Non-clairvoyant makespan minimization scheduling with predictions. In 34th International Symposium on Algo- rithms and Computation (ISAAC 2023), pp. 9-1. Schloss Seattuul-Leibiiz-eeee für Infor- matik, 2023.
Justin Bedō and Cheng Soon Ong. Multivariate spearman's for aggregating ranks using copulas. Journal of Machine Learning Research, 17(201):1-30, 2016.
Ziyad Benomar and Vianney Perchet. Non-clairvoyant scheduling with partial predictions. arXiv preprint wxXv:22405.01013 2024.
Robert D Blumofe and Charles E Leiserson. Scheduling multithreaded computations by work steal- ing. Journal of the ACM (JACM), 46(5):20--448 1999.
Eli Cortez, Anand Bonde, Alexandre Muzio, Mark Russinovich, Marcus Fontoura, and Ricardo Bianchini. Resource central: Understanding and predicting workloads for improved resource management in large cloud platforms. In Proceedings of the 26th Symposium Operating Sys- tems Principles, pp. 167, 2017.
Federico Della Croce and Rosario Scatamacchia. The longest processing time rule for identical parallel machines revisited. Journal of Scheduling, 23(2):163-77) 2020.

---

Published as a conference paper at ICLR 2026
Yuping Fan, Zhiling Lan, Paul Rich, William Allcock, and Michael E Papka. Hybrid workload scheduling on hpe systems. In 2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pp. 470-480. IEEE, 2022.
João Gama, Indrè Zliobaité, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey concept drift adaptation. ACM computing surveys (CSUR), 46(4):1-37, 2014.
Mor Harchol-Balter. Performance modeling and design of computer systems: queueing theory in action. Cambridge University Press, 2013.
Sungjin Im, Ravi Kumar, Mahshid Montazer Qaem, and Manish Purohit. Non-clairvoyant schedul- ing with predictions. ACM Transactions on Purallel Computing, 10(4))1-26, 2023.
Mycongjac Jeon, Shivaram Venkataraman, Amar Phanishayee, Junjie Qian, Wencong Xiao, and Fan Yang. Analysis of (Large-Scale Sange-Scale) (Multi-Tenant) (GPU) clusters for (DNN) training workloads. In 20/9 USENIX Annual Technical Conference (USENIX ATC 19), pp. 947-9607 2019.
Wenhua Jiang and Cun-Hui Zhang. General maximum likelihood empirical bayes estimation of normal means. Journal of the American Statistical Association, 2009.
Michael Jordan and Robert A Jacobs. Hierarchical mixtures of experts and the em algorithm.
Neural computation, 6(2):181-214, 1994.
Sayash Kapoor and Arvind Narayanan. Leakage and the reproducibility crisis in machine-learning- based science. Patterns, 4(9), 2023.
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie- Yan Liu. Lightgbm: highly efficient gradient boosting decision tree. Advances neural information processing systems, 30, 2017.
Sung-Seok Ko and Richard F Serfozo. Response times in m/m/s fork-join networks. Advances in Applied Probability, 36(3):854-871, 2004.
Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon. Accurate uncertainties for deep learning using calibrated regression. In International conference machine learning, pp. 2796-2804. PMLR, 2018.
Ravi Kumar, Manish Purohit, and Zoya Svitkina. Improving online algorithms via ml predictions. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 9684 9693, 2018.
Silvio Lattanzi, Thomas Lavastida, Benjamin Moseley, and Sergei Vassilvitskii. Online scheduling via learned weights. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1859-1877. SIAM, 2020.
Alexander Lindermayr and Nicole Megow. Algorithms with predictions. https: //
algorithms www.thhwwwwwsihihhhhehhcee ion.iiiiiiiiiiiioooooiiooomiiiie ions github. 1o/ 2022. Accessed: 2024-01-24.
Chengzhi Lu, Kejiang Ye, Guoyao Xu, Cheng-Zhong Xu, and Tongxin Bai. Imbalance in the cloud: An analysis on alibaba cluster trace. In 2017 IEEE International Conference on Big Data (Big Data), pp. 2884-2892. IEEE, 2017.
Leandro L Minku and Xin Yao. Software effort estimation multiobjective learning problem. ACM Transactions on Software Engineering and Methodology (TOSEM), 22(4):1-32, 2013.
Rajeev Motwani, Steven Phillips, and Eric Torng. Nonclairvoyant scheduling. Theoretical computer science, 130(1):17-47, 1994.
Misja Nuyens and Adam Wierman. The breground-bbackground queue: a survey. Performance evaluation, 65(3-4):286-377 2008
Fabian Pedregosa, Gaël Varoquaux. Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Seikit-leam: Machine learning in python. the Journal of machine Learning research, 12:2825-2830. 2011.
12

---

Published as a conference paper at ICLR 2026
Charles Reiss, John Wilkes, and Joseph L Hellerstein. Google cluster-usage traces: format+ schema. Google Inc., White Paper, 1:1-14, 2011.
Yaniv Romano, Evan Patterson, and Emmanuel Candes. Conformalized quantile regression. Ad- vances in neural information processing systems, 32, 2019.
Linus Schrage. A proof of the optimality of the shortest remaining processing time discipline. Operations Research, 16(3):687-690, 1968.
Mchmet Soysal and Achim Streit. Collection of job scheduling prediction methods. In Job Schedul- ing Strategies for Parallel Processing: 24th International Workshop, JSSPP 2021, Virtual Event, May 21, 2021, Revised Selected Papers 24, pp. 35 42. Springer, 2021.
Muhammad Tirmazi, Adam Barker, Nan Deng, Md E Haque, Zhijing Gene Qin, Steven Hand, Mor Harchol-Baiee Baateer and John Wilkes. Borg: the next generation. In Proceedings of the fifteenth European conference on computer systems, pp. 1-14, 2020.
Abhishek Verma, Luis Pedrosa, Madhukar Korupolu, David Oppenheimer, Eric Tune, and John Wilkes. Large-scale cluster management at google with borg. In Proceedings of the tenth euro- pean conference on computer systems, pp. 1-17, 2015.
Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523):1228-12444 2018.
Qizhen Weng. Wencong Xiao, Yinghao Yu, Wei Wang, Cheng Wang, Jian He, Yong Li, Liping Zhang, Wei Lin, and Yu Ding. {MLaaS] in the wild: Workload analysis and scheduling in (Large-Scabe Scale- heterogeneous {GPU) clusters. In 19th USENIX Symposium on Networked Sys- tems Design and Implementation (NSDI 22), pp. 945-960, 2022.
David H Wolpert. Stacked generalization. Neural networks, 5(2):241-259, 1992.
Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multiclass proba- bility estimates. Proceedings of the eighth ACM SIGKDD international conference on Knowl- edge discovery and data mining, pp. 694-699, 2002.
Matei Zaharia, Andy Konwinski, Anthony Joseph, Randy Katz, and Ion Stoica. Improving mapreduce performance in heterogencous environments, In Osdi, volume 8, pp. 7, 2008.
Tianming Zhao, Wei Li, and Albert Y Zomaya. Uniform machine scheduling with predictions. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 32, pp. 413-422, 2022.
Tianming Zhao, Xiaomin Chang, Chunhao Li, Wei Li, Albert Zomaya, al. Competitive fair scheduling with predictions. In The Thirleenth International Conference on Learning Represen- fations, 2024.
Pengfei Zheng, Rui Pan, Tarannum Khan, Shivaram Venkataraman, and Aditya Akella. Shockwave: Fair and efficient cluster scheduling for dynamic adaptation in machine learning. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), pp. 703-723, 2023.
13

---

Published as a conference paper at ICLR 2026
A NOTATIONS
A.1
BASIC ENTITIES AND TIME VARIABLES
Symbol
Name
Meaning
Example
Jj
Job
A job submitted by user role within job
Training job for classifier
Task
Worker task in distributed training
Instance
One copy task running Physical server in cluster
Worker instance #3 out 10 Server with 8 V100 GPUs
T7
Machine
Tj
Arrival time
When job was submitted Any point in time When task begins running When first task starts Time spent waiting
Submitted at 10:00 AM
Time variable
Checking availability 10:15 AM Task starts 10:30 AM
8t
Task time
8y
Job start time
Job starts with carliest task
Qucuing delay
Waited 30 minutes for resources
A.2
SETS AND COLLECTIONS
Symbol
Name
Meaning
Example
T(j)
Task set
All tasks belonging to job j All instances of task t
{PS, Worker, Evaluator Worker-l, Worker-10) (V100, P100} but not T4
I(!)
Instance set
T,
GPU type
Compatible GPU types
A.3 RESOURCE VECTORS AND DEMANDS
Symbol
Name
Meaning
Example
Resource request
Resources per instance
GPUs, 16 CPUs, 64GB RAM]
Cm
Machine capacity
Total machine resources
GPUs, 96 CPUs, 512GB RAM] 10 worker instances
Ri
Instance count
Number of task copies
A.4 RESOURCE UTILIZATION
A.5
JOB PROCESSING TIME CALCULATION
We define job processing time based on a hicrarchical Fork-Join model.
Instance Duration For an instance running on the interval [ss, 6:):
d: e, 8;
(1)
Task Span (Fork-Join for Instances) A task completes only when its last instance finishes.
S; min Si et max e; sEI())
aEi(()
d St max G
min 8;
(3)
EEl(1)
sE((E)
Note: For gang-scheduled tasks (gi 1). 8; st for all so de max; d.
3. Jobb Processing Time (Fork-Join for Tasks) A job J, completes only when its last task finishes.
S = min
St: TETU)
E
max E tET()
(4)
pj
E
S
(5)
This definition correctly captures the true resource occupancy window, including barriers from stag- gered lask starts, unlike mean-based aggregations.
14

---

Published as a conference paper at ICLR 2026
Symbol
Name
Meaning
Example
R.
Total task request
R.
Total resources for all workers Sum of tasks' requests
R
Total job request
R ElETo) R.
Instance utilization
Average usage of resource & by instance AiA Wi,k Cmi(t). d;
85% GPU usage over execution
Resource-time
Total GPU-seconds consumed
A.6
KEY SCHEDULING CONSTRAINTS
1. Resource Capacity Constraint
(m) Te()) S Cm
(6)
Total resources used by all instances on machine cannot exceed that machine's capacity
2. Gang Scheduling Constraint (when gt =)
Σ
E
Fi = 142
(7)
REI(E)
All n, instances of the task must be placed at the same time
3. Locality Constraint (when
Σ
Di,me (T) = T4t
(8)
iEI(t)
All instances must be the same machine m4
4. GPU Type Constraint
Fi,m (Ti =
when g(m) $ I,
(9)
Cannot place instances on machines with incompatible GPU types
B DATASET CHARACTERISTIC, AND JOB EXAMPLES
To illustrate the diversity of workloads the ATLAS dataset and clarify the nature of the Processing Time label, 5 three representative jobs drawn directly from the trace. These examples showcase different scales of operation, from small inference tasks to large-scale distributed training.
Crucially, the workloads captured in the Alibaba PAI trace are non-preemptible. Once job begins execution, runs to completion without interruption by the scheduler. Therefore, the Processing Time reported in the final column of Table 5 represents the actual, continuous duration of the job from start to finish. This single value accurately reflects the job's size for prediction tasks, as there preemption
resumption dynamics prediction, such as the number of tasks and instances, requested resources (CPU, GPU, memory), and workload type. The wide range of processing times, from just over two minutes more than 12 hours, demonstrates the challenge of the prediction task.
model. The table highlights key features used for
Table 5: Three representative joh examples from the ATLAS submit-time dataset, illustrating different job types, scales, and their corresponding processing times. The Processing Time serves as the prediction label and represents the uninterrupted execution duration, as jobs in this trace are not preempted.
Jah Type
pei job
jable
pel.gren Jalle table
Pricesing Time (labll
Joh
subell
Instanene
(530
vorklaad
sTeb422b....
261245.5....
0922115
6o247247
136
Scate
x6:454bb3..
45289962...
1412-45
105
$5,000
4.00
2.011
#Ddic184..
44,333
15

---

Published as a conference paper at ICLR 2026
C TRADITIONAL ML PREDICTION MODELS TRIED
Our preliminary methods explored four baselines: (1) single-stage gradient-boosted trees model on the log target with monotone constraints on obvious scale features (instances, GPUs), to en- code weak priors and reduce pathological splits in sparse regions Ke al. (2017). (2) CRR++ "cluster--route->refff variant that forms train-only k-means families, learns router, and applies per-family experts with a global fallback-usefut when coarse workload types exist but are unla- beled Pedregosa al. (011). (3) scale-bucker experts using train- train-quantile edges on instance/GPU counts to fit per-bucket regressor, again with global fallback; and (4) a simple recency ensem- ble mixing& full-history model with a recent-window model to hedge concept drift (aaaa et al. (2014). These baselines improved over naive single models but exposed core gaps: poor uncertainty calibration and tail handling (heavy-tailed durations), weak rank fidelity in some regimes, brittle behavior for signatures, and limited drift adaptation from fixed mixtures. So, bad performances motivated our current seven-method toolkit that adds calibrated quantile intervals and blending, ex- plicit monotone safe calibration (isotonic), diversity via stacking, a principled gated-exper expperss split, stronger recency weighting with validation- basedd ensembling, and empirical -Bayes priors for sparse signatures-
calibration, ranking, sparsity, and drift more systematically than the four preliminaries could alone.
Algorithm Common preprocessing (used by MI-M4)
1: Input: raw tables (job, task, tag), submit time ry. lahel pi. features x;
Split by time: sort by ry: pick cut times fuain < bva;; define Dt. = {j : r. < feain), Don = { furinn S < 6.a). Dte = [i:r; T. }-
3: Core features (submit-time only): logs of totals (log(1 + x)) for CPU/GPU/EEWinssaaeettsss per-instance ratios, cyclic time (nour, wday, sin / cos of 24h and 168h).
4: Causal histories (group/user): means and counts with a one-step shi ft (1); time since previous submit; small shifted rolling means; an empirical-Ban Bayes (EB) group mcan RB using only D.r to set the global prior /oo
the log target Vj log(1 + P2), compute within-group expanding
5: Train-only encodings: map aese///roop//oooloaaa www.orkoooaaagpuooooo typ. spec to integer codes using the vo- cabulary in Dr.; umseen OTHER.
6: Sanity: replace oo and NaNs with 0 for numeric features; never touch labels in Dua; Die beyond metrics.
7: Return: design matrices X X Xxe and vectors loog(1 + +p"), Bear = p"
Algorithm MI Single-Stare Gradient Boosting (log target) with causal histories
1: Input: Xte. X from Alg. 2 Vte, you; Vite
(optional) Monotone constraints: choose feature subset M expected be non-decreasing (c.g.. log(1 + instances), log(1 + GPU)) and pass a monotonicity vector to the booster.
3: Train a gradient--oosteed trees regressor on Ver with early stopping on (Xve, Fva) (all log-domain).
4: Predict: Ôte <- model( X return fr. exp( -1
5: Metrics: report Cov @5v505 RMSLE, MAF, and Spearman (pre, yte).
Algorithm 4 M2 - CRR++ (Cluster + Route + Refit) with per-family experts
1: Input: Xtr. Xva: Xte: yer, Yea: Ute: clusters K; min-support Timin
2: Standardize (train-only): fit a scaler on X transform to Ztr, Z Zse-
3: Unsupervised families (train-only): fit K -mmans on Zer; obtain family IDs fer: assign feus Sie by predict.
Router (train-only): train multi-class classifier to map + f using (Ztr. Str).
5: Fallback regressor: train a global log-target booster on (Xtr, yer) with carly stopping on (Xyas Voa).
6: Per-family experts: for each lo E (l...., K} with #(j E Der (j) = k} 2 Nanin train a log-target
booster on the subset (j (j) k);; optionally use (Xealfon k), If k)) for carly stopping.
7: Predict: for each test sample set l +- router(z); if expert & = lc exists use it, else use the fallback;
return I exp(ū) 1.
8: Metries: as in Alg.
16

---

Published as a conference paper at ICLR 2026
Algorithm 5 M3 Scale-Bucket Experts (train train--uanttils - per-bucket models)
1: Input: Xtr, va. Xte: yer, Moa: Ste: train-only features = instances, b = GPU; min-bucket Amin
2: Train-only bucket edges: compute quantiles Qa and Qs on (a, b) over Der (e.g.. {0.5, 0.9, 0.99)); define
bucketizers bin., bins.
Assign buckets: U, max max(bin. (a,) bin)(b,)} } for all j train/val/tsst.
4: Global fallback: train a log log-target booster on all of Dt. with early stopping on Pra-
5: Per-bucket experts: for each bucket u with # # E Dtr : Uj = 4) 2 Tamin. train log-target booster on
{j :1 w): optionally early-stop on {j Dea My u..
6: Predict: for each sample with bucket u, use expert(n)) if available; else the global fallback; retum
I = exp(ū)
7: Metrics: as in Alg.
Algorithm 6 M4 Recency Ensemble (full vs. recent window; train-only gates)
1: Input: Ste,Xpoo Xtei Ute, Vva, Mtei training times {r, : E De->: window quantile mixture weight a
2: Full model: train log-target booster on (Xir, Ver): early-stop on (Xva, Vea).
3: Recent cut (train-only): set lgg + Quantile, : Der define Discom [j E Der :i tq)-
4: Recent model (if enough support): train log-target booster on Direm, early-stopped on (X Vra); otherwise skip.
5: Predict & mix: let strnmt) = exp(i(futt) 1, pfree) = exp(i(ree)) (if present); output ý
(1 - (Fll)) + a pl g(ree) recent model exists, else p(rwe)
6: Metries: as in Alg.
D PREDICTION MODEL DESCRIPTION
We model yy log(1 1 + pt) from submit-time features x; using gradient boosting with validation- based calibration. Our six methods address different challenges in job duration prediction:
(1) Conformal Quantile Regression (CQR) (Romano al. 2019) addresses prediction uncertainty by learning the conditional distribution rather than just point estimates. We train three LightGBM models with quantile loss at a E {0.1, 0.5,0.9) 0.9) predict the 10th, 50th, and 90th percentiles of job duration. On the validation set, we compute the nonconformity score r = max (Qo.1 y. -- Qu.9) for each example and obtain the conformal correction k Qo.e (max(r, 0)), which captures the typical prediction interval miscalibration. The calibrated bounds are I, - Qo. - and U = Qo expanding the quantile interval to achieve the desired coverage. The final prediction blends Qn.5+ the calibrated bounds [I, U). and contextual features like group means using Ridge regression. This approach particularly effective for jobs with high uncertainty- instance, experimental ML training jobs where duration depends convergence criteria, data processing jobs where input size varies significantly. The method provides both accurate point estimates and reliable confidence intervals.
(2) Isotonic Calibration Zadrrzzyy & Elkan 2002: Kuleshov et al. 2018) corrects systematic pre- diction biases while preserving ranking order. The method fits monotonic non-decreasing function é R - TR mapping raw predictions to calibrated values, ensuring ((x1) S Φ(I2) whenever 21 S x2. This is crucial when the model consistently over-predicts short jobs (c.g., quick validation scripts that always take 5 seconds but are predicted as 30 seconds) or under-predicts long jobs (c.g.. full model training that takes 10 hours but is predicted as 2 hours). The isotonic regression finds the optimal step function that minimizes squared eтror on the validation set while maintaining mono- tonicity. This property essential for scheduling decisions where relative job ordering matters job A is predicted to be shorter than job B, this relationship is preserved after calibration.
(3) Meta-Stacking (Wolprtt 1992 leverages model diversity to improve robustness. We train four base models with different loss functions: (i) L. loss for standard regression, (ii) Huber loss (6 = 0.9) for robustness to outliers like crashed jobs or anomalously long runs, (iii) quantile loss for median prediction, and (iv) heavily regularized L2 (a 0.5, X 1.0) to prevent overfitting. Each model captures different aspects: L.2 minimizes average error, Huber handles extreme cases, quantile focuses the median behavior, and regularized models provide stable baselines. The meta-learner (another LGBM) takes these base predictions, their standard deviation (measuring dis- agreement), and contextual features (group history, signature statistics) input. It learns non-linear
17

---

Published as a conference paper at ICLR 2026
combinations for example, trusting the Huber model more when base predictions diverge sig- nificantly (indicating potential outliers), or weighting the regularized model higher for users with limited history.
(4) Two-Stage Gated Experts (Jordan & Jacobs 1994) recognizes that different job types require different prediction strategies. The gating network (classifier) first categorizes jobs into three dura- tion classes based on the 30th and 70th percentiles of training durations: (i) short jobs (<30th per- centile, typically <100s): quick validation runs, status checks, small data samples; (ii) medium jobs (30th-70th percentile, 100s=1000s): regular training epochs, moderate data processing; (iii) long jobs (>70th percentile, > 1000ss: full model training, large-scale data processing, or hyper- parameter sweeps. Each category gets a specialized expert model with appropriate complexity simple models predictable short jobs, complex deep trees for variable long jobs. The final predic- tion aggregates expert outputs weighted by soft probabilities: ÿ = Eee (short, med, long) P(c[x) Ex(x). This prevents short jobs from being influenced by patterns from marathon training runs and vice versa.
(5) Weighted Recency (Gama et al. 2014) adapts to temporal drift in workload patterns. Computing clusters exhibit temporal patterns: new framework releases change typical training times, approach- ing deadlines increase job submissions, and hardware upgrades affect processing speeds. We train three models on progressively recent data windows: (i) full history for stable long-term patterns, (ii) recent 50% for medium-term trends, (iii) recent 20% for immediate patterns. Each training sample receives weight Wa = exp(-A(T'- where I is the current time and À controls decay rate. Mod- els are combined using validation performance weights-if recent models show lower validation error, they receive higher weight in the ensemble, automatically adapting to drift. For example, after PyTorch version upgrade that speeds up training by 30%, the recent-20% model would quickly adapt while the full-history model provides stability.
(6) Historical Recency-Aware with Shrinkage (HRAS) addresses the cold-start problem for rare job signatures. A signature combines user, group, workload type, and resource requirements (buck- eted into deciles). Rare signatures include: (i) new users or groups with no history, (ii) unusual resource combinations (e.g., user alice from vision.grous
suddenly requesting 8 GPUs when historically using only CPUs), (iii) infrequent workload types (e.g., monthly financial reports primarily ML-focused cluster). For signature with n, historical observations and mean duration He. we apply Empirical Bayes shrinkage: Ae = (Nafi + \uo)/(n, + A), where Hg is the global mean and À 5 is the shrinkage strength. When n, (completely new pattern), the prediction equals the global mean, As n. grows, the prediction gradually shifts toward the mean. The exponentially weighted component gives more weight to recent instances of the signa- ture, capturing evolution in user behavior.
18


Reviews:
[{"review_id": "EZHws0CIv1", "reviewer": "Reviewer_Lijb", "summary": "The paper introduces ATLAS, a dataset derived from Alibaba's artificial intelligence platform cluster trace that processes many thousands of machine learning jobs per day. The dataset contains about 730K jobs with actual processing times in addition to many other metrics along with LASched, a benchmark for learning augmented scheduling. The work specifically addresses the gap between theoretical advances in learning augmented scheduling with the evaluation of those algorithms on real workloads. Specifically, they develop a procedure to reproducibly and comprehensively evaluate scheduling algorithms with clear train test splits. across three commonly used scheduling metrics including total completion time, max stretch and makespan.", "questions": "- A major suggestion I have for the authors would be to simplify and/or reduce the amount of mathematical notation used here. While I understand that a major audience is the learning augmented scheduling researchers who tend to be more theoretical, in my personal opinion the notation here reduces accessibility to broader audiences (those implementing these systems in industry for instance) that this paper could impact.\n- Another suggestion would be to really explain and elaborate why certain algorithms and job duration methods seem to perform better than others. There seems to be a disconnect in the writing between actually predicting the job duration versus the performance of the algorithms (competitive ratios). I could imagine that small changes in prediction could have large algorithmic impacts or even vice versa. Clearly characterizing this with a more in depth analysis connecting these two aspects would seriously improve the impact of the paper both for algorithm developers and those actually implementing schedulers in industry.\n- Maybe I missed this but I don't think I see results showing that you used the various job duration prediction methods on all of the scheduling algorithms listed. Adding these results, even if it is only to the appendix would really help out both algorithm developers and those in industry.\n- Given the scale and value of the dataset, I think it is reasonable for the authors to clearly explain what the holes are with current methods and suggest avenues/algorithmic directions for improvement. This seems low effort but would positively impact the community and improve uptake of this work.\n\nIf my concerns are addressed, I'm very inclined to increase my score. I think this paper adds substantial value to the community and if some of my questions are answered then it could have greater impact.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The scale of the dataset (730K) jobs and the vast metadata about the details of each job and resource it runs on make it a valuable contribution to the practical evaluation of learning augmented scheduling algorithms.\n- Moreover, to the best of my knowledge, they seem to be the first to enforce leakage safe construction, that is only information that an actual scheduler would have, for the evaluation of their algorithms which is critically important for rigorous evaluation.\n- Multiple scheduling objectives that are commonly used in the literature are evaluated on.\n- Provide an easy to use simulator to evaluate various learning augmented scheduling algorithms.", "weaknesses": "- Excessive mathematical formalism and notation that is not used much after introduction. On page 4 and 5 for example, a fair bit of mathematical notation is introduced that would not look out of place in a scheduling paper with theoretical results. However, the notation is only used sparingly in pages 4, 5 and Algorithm 1 and there are no proofs in the paper that would benefit from such formal notation. If the notation was simplified the readability of the paper would significantly improve.\n- The authors introduce a really powerful scheduling benchmark and evaluate on various algorithms but do not really discuss their performance in detail and why certain algorithms perform better than others.\n- The authors do not really comprehensively evaluate the connection between prediction of the job duration and the actual scheduling algorithm used.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "08jUcE8WrC", "reviewer": "Reviewer_aucU", "summary": "This paper introduces Alibaba Trace for Learning-Augmented Scheduling (ATLAS), a research-ready dataset derived from Alibaba’s Platform of Artificial Intelligence (PAI) cluster trace—a production system that processes hundreds of thousands of ML jobs per day. The ATLAS dataset has been cleaned and features engineered to represent the inputs and constraints of non-clairvoyant scheduling, including user tags, resource requests (CPU/GPU/memory), and job structures with ground-truth processing times. This paper develops a prediction benchmark reporting prediction error metrics, along with feature importance analysis, and introduces a novel multiple-stage ML model.", "questions": "1.\tPlease clarify whether each job exclusively occupies its allocated resources or whether multiple jobs can coexist on the same resource group (e.g., node or GPU). \n\n2.\tPlease clarify what scheduler was used during data collection, and discuss whether this underlying policy could bias the dataset or affect the quality of downstream learned schedulers trained on this trace.\n\n3.\tAre both preemptive and non-preemptive execution cases included? If so, how is preemption recorded and labeled?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper provides a real-world dataset and a standardized benchmark, which are valuable for studying learning-augmented scheduling in practical environments.", "weaknesses": "1.\tPlease clarify whether each job exclusively occupies its allocated resources or whether multiple jobs can coexist on the same resource group (e.g., node or GPU). This is important for understanding the dataset’s applicability to scheduling policies that handle interference and resource sharing.\n\n2.\tThe trace seems to be collected under a specific scheduling policy. Please clarify what scheduler was used during data collection, and discuss whether this underlying policy could bias the dataset or affect the quality of downstream learned schedulers trained on this trace.\n\n3.\tThe paper should elaborate on how preemptive scheduling is represented in the dataset. Are both preemptive and non-preemptive execution cases included? If so, how is preemption recorded and labeled?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aEQ3KPRuLO", "reviewer": "Reviewer_ufpH", "summary": "This paper publishes a “plug-n-play” data set for training job size predictors in the context of non-clairvoyant scheduling, a practically-relevant problem that has been extensively studied in the literature on learning-augmented algorithms.  The authors create ATLAS, a data set of real workloads distilled from Alibaba’s PAI trace.  Furthermore, they provide a benchmark environment called LASched that provides a common environment for evaluating learning-augmented schedulers, and give example prediction methods that perform well in this environment.  The paper is released alongside an anonymized repo that contains the data set, simulator code, and code for the example ML models.", "questions": "Can you speak more to the decision to provide the predictor with information about the user and group?  I would typically expect that including these features would bias the model towards overfitting to the data in a way that may not be generalizable (e.g., to a different scheduling context with similar workloads but completely different users).  How many users/groups exist in the underlying trace, and are these literally tied to individual human beings, or are they tied to frameworks such as TensorFlow?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The underlying problem (non-clairvoyant scheduling with job size predictions) is highly relevant in practice.   Providing a benchmark tool (LASched) alongside the data set will be appreciated by the community working on this problem as a way to speed up evaluation of new techniques and establish a common environment in which proposed scheduling algorithms can be directly compared. \n\nIn developing their own models for predicting job sizes, the authors additionally develop a feature engineering schema that is “ready to go” for other researchers developing new model architectures for the job size prediction task — they also show that these engineered features are effective for the ATLAS data set by proposing and evaluating their own predictors that perform well.", "weaknesses": "The Alibaba PAI data set that provides all of the underlying data for ATLAS has been publicly available since 2022 — ATLAS is simply a sensible cleaned and processed version of this data set.  This weakness is a mild one since the paper also provides a new benchmark and simulator environment that is of equal value to the community working on this problem.\n\nWhile not necessarily expected from a paper in the datasets track, the example models trained using the data set are only evaluated with a fixed set of features and parameters — it would be good to see, for example, an ablation study of including/excluding certain features.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hSfpc0lZ26", "reviewer": "Reviewer_tfJj", "summary": "This paper introduces ATLAS, a large-scale dataset of over 730,000 jobs from Alibaba's PAI production cluster, which addresses the lack of public scheduling datasets containing ground-truth processing times. The dataset is specifically designed for non-clairvoyant scheduling research by strictly using only features available at submission time, and the authors build the LASched benchmark on top of it. LASched provides implementations and evaluations for both job size prediction tasks (using baselines like CQR and Meta-stacking) and scheduling tasks (using algorithms like SPJF and LPPT), serving as a reproducible foundation for comparing algorithms on real-world workloads.", "questions": "* Since some metrics are measured on a single machine, how do you envision the benchmark evolving to incorporate real-world complexities, and what new challenges might arise when evaluating learning-augmented algorithms in that more realistic setting?\n\n* Do you have plans to release an extended version of ATLAS, perhaps covering a full year, or how would you recommend researchers use this dataset to develop models that are robust to such long-term changes?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* The paper convincingly argues that existing datasets for scheduling research are insufficient, as they either lack ground-truth processing times or are not publicly available. ATLAS directly fills this gap with a large-scale, public trace from a real-world production system.\n\n* The dataset provides complete ground-truth job sizes, a critical feature missing from other traces. It is rigorously prepared as a non-clairvoyant dataset, strictly excluding post-execution metrics to prevent information leakage and ensure it is research-ready.\n\n* The authors provide LASched, a full benchmark built on ATLAS that defines both prediction and scheduling tasks. This includes implementations and evaluations for multiple prediction models and scheduling algorithms , establishing a strong baseline for future work.", "weaknesses": "* Maybe I'm wrong but there seems to be a discrepancy between the complexity of the data source and the simplicity of the scheduling benchmark. The ATLAS dataset is sourced from a large scale, heterogeneous production environment with over 1,800 machines and complex constraints like gang scheduling. However, the LASched benchmark evaluates key metrics, such as total completion time and max-stretch, using a simplified single machine setting. This simplification might obscure the real world impact of prediction errors, which could be amplified in a complex, multi machine environment with heterogeneous resources and stricter constraints.\n\n* The dataset's two month time span limits its ability to capture long term temporal dynamics. While the trace includes over 730,000 jobs and exhibits clear patterns, a two month window is likely insufficient to model longer cycles, such as quarterly workload peaks or concept drift caused by major software framework updates. The paper does propose models to address drift, but the dataset itself may not provide enough data for researchers to fully validate the robustness of scheduling algorithms against these important long term workload evolutions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OJUJYU6I8L", "reviewer": "Reviewer_shox", "summary": "The paper presents a new real-world dataset for evaluating algorithms with predictions. In particular, the paper addresses non-clairvoyant scheduling, where job sizes are unknown to a scheduler. Instead, a scheduler has access to predictions about the job, for example, the job's processing times. Non-clairvoyant scheduling belongs to one of the most studied online problems within the area of algorithms with predictions. However, many results are only of theoretical nature, and it is unclear whether these algorithm bring a benefit in practice.\n\nThe paper proposes a dataset based on real world data of Alibaba's datacenter and preprocesses that for evaluating such algorithms. There are two main components:\n\n1. The main component is the dataset itself. It is very diverse and comes with different labels.\n2. The second component is a ready-to-use benchmark that already implements baselines such as simple learning-augmented algorithms and optimum solutions. It also comes with benchmark prediction models that can be used for generating e.g. job size predictions. Considered objectives are total completion time, maximum stretch and makespan\n\nThe paper gives characterizations of the dataset, a detailed explanation of the benchmark suite, and an evaluation of multiple learning-augmented and classic algorithms.", "questions": "- L25 mutli-stage?\n- L33: whitespace missing\n- L35: \"non-clairvoyant scheduling has a larger total completion time\" what do you mean here?\n- L105 \"ALTAS\" -> \"ATLAS\"", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- To the best of my knowledge, this is the first attempt to introduce a benchmark for learning-augmented scheduling algorithms, which seems to be overdue. In the past, there often has been critique that learning-augmented algorithms are evaluated on synthetic data. Thus, I think this benchmark suite is a good contribution for the field and creates a bridge to practical applications.\n- The benchmark considers different objectives to capture a large variety of problems.\n- The dataset and benchmark are well-structured: a proper feature engineering pipeline, empirical-Bayes shrinkage, and conformal calibration.\n- The authors show attention to reproducibility.", "weaknesses": "- The introduction is slightly dense, it assumes that readers are familiar with learning-augmented algorithms; also the statements about the different algorithm are a bit confusing in my opinion. I would like to see a revised introduction in the camera-ready version.\n- One could argue that the contribution of a dataset is not sufficient for a ML theory conference. However, since benchmarks are rather novel for learning-augmented algorithms, I would not value this too much.\n\nI would be happy to see the paper accepted, but would not fight for it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)