title: Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð¾Ð² Ð¸Ð½ÑÐµÐ»Ð»ÐµÐºÑÑÐ°Ð»ÑÐ½Ð¾Ð³Ð¾ Ð¿Ð»Ð°Ð½Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ñ Ð²ÑÑÐ¸ÑÐ»Ð¸ÑÐµÐ»ÑÐ½ÑÑ ÑÐµÑÑÑÑÐ¾Ð² Ð² ÑÑÐ¿ÐµÑÐºÐ¾Ð¼Ð¿ÑÑÑÐµÑÐ½ÑÑ ÑÐ¸ÑÑÐµÐ¼Ð°Ñ
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð¼Ð°Ð³Ð¸ÑÑÑÐ°
titleType: Subtitle
title: Development of algorithms for intelligent planning of computing resources in supercomputer systems
Creators: name: Ð¡ÐºÐ¾ÑÐ¾Ð´ÑÐ¼Ð¾Ð², ÐÐ¸ÐºÐ¸ÑÐ°
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2022
Subjects: ÐÑÐºÑÑÑÑÐ²ÐµÐ½Ð½ÑÐ¹ Ð¸Ð½ÑÐµÐ»Ð»ÐµÐºÑ
Ð²ÑÑÐ¾ÐºÐ¾Ð¿ÑÐ¾Ð¸Ð·Ð²Ð¾Ð´Ð¸ÑÐµÐ»ÑÐ½ÑÐµ Ð²ÑÑÐ¸ÑÐ»ÐµÐ½Ð¸Ñ
Ð¿Ð»Ð°Ð½Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ðµ ÑÑÐ¿ÐµÑÐºÐ¾Ð¼Ð¿ÑÑÑÐµÑÐ½ÑÑ Ð·Ð°Ð´Ð°Ð½Ð¸Ð¹
Ð¾ÑÐµÐ½ÐºÐ° Ð²ÑÐµÐ¼ÐµÐ½Ð¸ Ð²ÑÐ¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ñ Ð·Ð°Ð´Ð°Ð½Ð¸Ñ
high-performance computing
supercomputer task scheduling
task execution time estimation

description: Ð Ð°Ð±Ð¾ÑÐ° Ð¿Ð¾ÑÐ²ÑÑÐµÐ½Ð° ÑÐ»ÑÑÑÐµÐ½Ð¸Ñ Ð¿ÑÐ¾ÑÐµÑÑÐ° Ð¿Ð»Ð°Ð½Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ñ Ð·Ð°Ð´Ð°Ð½Ð¸Ð¹ Ð½Ð° ÑÑÐ¿ÐµÑÐºÐ¾Ð¼Ð¿ÑÑÑÐµÑÐµ Ð·Ð° ÑÑÐµÑ Ð±Ð¾Ð»ÐµÐµ ÑÐ¾ÑÐ½Ð¾Ð³Ð¾ Ð¿ÑÐ¾Ð³Ð½Ð¾Ð·Ð° Ð²ÑÐµÐ¼ÐµÐ½Ð¸ Ð²ÑÐ¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ñ Ñ Ð¿Ð¾Ð¼Ð¾ÑÑÑ Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ.Â Ð ÑÐ°Ð±Ð¾ÑÐµ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð»ÑÑ Ð½Ð°Ð±Ð¾Ñ Ð´Ð°Ð½Ð½ÑÑ Ð¾ ÑÐ°Ð±Ð¾ÑÐµÐ¹ Ð½Ð°Ð³ÑÑÐ·ÐºÐµ, Ð²Ð·ÑÑÑÐ¹ Ñ ÐºÐ»Ð°ÑÑÐµÑÐ° ÑÑÐ¿ÐµÑÐºÐ¾Ð¼Ð¿ÑÑÑÐµÑÐ½Ð¾Ð³Ð¾ ÑÐµÐ½ÑÑÐ° (Ð¡ÐÐ¦) Â«ÐÐ¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹Â» Ð½Ð° Ð±Ð°Ð·Ðµ Ð¡ÐÐ±ÐÐ£, Ð·Ð° 2021 Ð³Ð¾Ð´. ÐÑÑÐ¾Ð´Ð½ÑÐµ Ð´Ð°Ð½Ð½ÑÐµ Ð¸Ð¼ÐµÐ»Ð¸ ÐºÑÐ°Ð¹Ð½Ðµ Ð·Ð°Ð²ÑÑÐµÐ½Ð½ÑÐµ Ð¾ÑÐµÐ½ÐºÐ¸ Ð²ÑÐµÐ¼ÐµÐ½Ð¸ Ð²ÑÐ¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ñ Ð·Ð°Ð´Ð°Ð½Ð¸Ð¹, ÑÑÐ¾ Ð½Ðµ Ð¿Ð¾Ð·Ð²Ð¾Ð»ÑÐ»Ð¾ Ð¿Ð»Ð°Ð½Ð¸ÑÐ¾Ð²ÑÐ¸ÐºÑ Ð²ÑÑÑÑÐ°Ð¸Ð²Ð°ÑÑ Ð¸Ñ ÑÐµÐ°Ð»Ð¸ÑÑÐ¸ÑÐ½ÑÐ¹ Ð¿Ð¾ÑÑÐ´Ð¾Ðº ÑÐ°Ð±Ð¾ÑÑ. Ð Ð¿ÑÐ¾ÑÐµÑÑÐµ ÑÐ°Ð±Ð¾ÑÑ Ð±ÑÐ»Ð¸ Ð¾Ð¿ÑÐ¾Ð±Ð¾Ð²Ð°Ð½Ñ 8 Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð´Ð»Ñ Ð¿ÑÐ¾Ð³Ð½Ð¾Ð·Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ñ Ð²ÑÐµÐ¼ÐµÐ½Ð¸ Ð²ÑÐ¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ñ Ð·Ð°Ð´Ð°Ð½Ð¸Ñ: ÐºÐ»Ð°ÑÑÐµÑÐ¸Ð·Ð°ÑÐ¸Ñ, Ð³ÑÐ°Ð´Ð¸ÐµÐ½ÑÐ½ÑÐ¹ Ð±ÑÑÑÐ¸Ð½Ð³ Ð´Ð»Ñ ÑÐµÐ³ÑÐµÑÑÐ¸Ð¸ Ð¸ ÐºÐ»Ð°ÑÑÐ¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¸, ÑÐ»ÑÑÐ°Ð¹Ð½ÑÐ¹ Ð»ÐµÑ Ð´Ð»Ñ ÑÐµÐ³ÑÐµÑÑÐ¸Ð¸ Ð¸ ÐºÐ»Ð°ÑÑÐ¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¸, Ð¼ÐµÑÐ¾Ð´ Ð¾Ð¿Ð¾ÑÐ½ÑÑ Ð²ÐµÐºÑÐ¾ÑÐ¾Ð², Ð¼ÐµÐ´Ð¸Ð°Ð½Ð½Ð°Ñ ÑÐµÐ³ÑÐµÑÑÐ¸Ñ, Ð»Ð¾Ð³Ð¸ÑÑÐ¸ÑÐµÑÐºÐ°Ñ ÑÐµÐ³ÑÐµÑÑÐ¸Ñ. ÐÑÑÑÐ¸Ðµ Ð¸Ð· Ð¿Ð¾Ð»ÑÑÐµÐ½Ð½ÑÑ Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹ Ð¾ÑÐµÐ½ÐºÐ¸ Ð²ÑÐµÐ¼ÐµÐ½Ð¸ Ð±ÑÐ»Ð¸ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ñ Ð² ÑÐ¾Ð²Ð¾ÐºÑÐ¿Ð½Ð¾ÑÑÐ¸ Ñ Ð¸ÑÑÐ¾Ð´Ð½ÑÐ¼Ð¸ Ð´Ð°Ð½Ð½ÑÐ¼Ð¸ Ð´Ð»Ñ Ð¼Ð¾Ð´ÐµÐ»Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ñ ÑÐ°Ð±Ð¾ÑÐµÐ¹ Ð½Ð°Ð³ÑÑÐ·ÐºÐ¸ Ð² ÑÐ¸Ð¼ÑÐ»ÑÑÐ¾ÑÐµ SLURM Ñ ÐºÐ¾Ð½ÑÐ¸Ð³ÑÑÐ°ÑÐ¸ÐµÐ¹, Ð±Ð»Ð¸Ð·ÐºÐ¾Ð¹ Ðº Ð¸ÑÐ¿Ð¾Ð»ÑÐ·ÑÐµÐ¼Ð¾Ð¹ Ð² ÑÑÐ¿ÐµÑÐºÐ¾Ð¼Ð¿ÑÑÑÐµÑÐ½Ð¾Ð¼ ÑÐµÐ½ÑÑÐµ. ÐÐ° Ð¾ÑÐ½Ð¾Ð²Ðµ ÑÐ¼Ð¾Ð´ÐµÐ»Ð¸ÑÐ¾Ð²Ð°Ð½Ð½Ð¾Ð¹ ÑÐ°Ð±Ð¾ÑÑ Ð¿Ð»Ð°Ð½Ð¸ÑÐ¾Ð²ÑÐ¸ÐºÐ° Ð±ÑÐ»Ð¸ Ð¸Ð·Ð¼ÐµÑÐµÐ½Ñ ÑÑÐµÐ´Ð½ÐµÐµ Ð²ÑÐµÐ¼Ñ Ð¾Ð¶Ð¸Ð´Ð°Ð½Ð¸Ñ Ð·Ð°Ð´Ð°Ð½Ð¸Ð¹ Ð² Ð¾ÑÐµÑÐµÐ´Ð¸ Ð¸ ÑÑÐ¸Ð»Ð¸Ð·Ð°ÑÐ¸Ñ ÑÐµÑÑÑÑÐ¾Ð². Ð ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐµ ÑÐµÐ³Ð¾ Ð±ÑÐ»Ð¾ Ð·Ð°ÑÐ¸ÐºÑÐ¸ÑÐ¾Ð²Ð°Ð½Ð¾ ÑÐ½Ð¸Ð¶ÐµÐ½Ð¸Ñ Ð²ÑÐµÐ¼ÐµÐ½Ð¸ Ð¾Ð¶Ð¸Ð´Ð°Ð½Ð¸Ñ Ð¾ÑÐ½Ð¾ÑÐ¸ÑÐµÐ»ÑÐ½Ð¾ ÑÐ°Ð·Ð½Ð¸ÑÑ ÑÑÐ¾Ð³Ð¾ Ð¿Ð¾ÐºÐ°Ð·Ð°ÑÐµÐ»Ñ Ð¿Ð¾ ÑÐ¼Ð¾Ð»ÑÐ°Ð½Ð¸Ñ Ð¸ Ð¿ÑÐ¸ Ð¸Ð´ÐµÐ°Ð»ÑÐ½Ð¾Ð¹ Ð¾ÑÐµÐ½ÐºÐµ Ð´Ð»Ñ Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹ Ð³ÑÐ°Ð´Ð¸ÐµÐ½ÑÐ½Ð¾Ð³Ð¾ ÑÐ¿ÑÑÐºÐ° Ð¸ Ð¼ÐµÐ´Ð¸Ð°Ð½Ð½Ð¾Ð¹ ÑÐµÐ³ÑÐµÑÑÐ¸Ð¸ Ð½Ð° 28 Ð¸ 19 Ð¿ÑÐ¾ÑÐµÐ½ÑÐ¾Ð² ÑÐ¾Ð¾ÑÐ²ÐµÑÑÑÐ²ÐµÐ½Ð½Ð¾.
description: The work is devoted to improving the task scheduling process on a supercomputer due to a more accurate prediction of execution time using machine learning methods. In the work, we used a dataset taken from the cluster of the supercomputer center (SCC) "Polytechnic" based on SPbPU for 2021. The initial data had extremely high estimates of the execution time of tasks, which did not allow the scheduler to build their realistic order of work. In the course of the work, 8 methods were tested to predict the task completion time: clustering, gradient boosting for regression and classification, random forest for regression and classification, support vector machine, median regression, logistic regression. The best time estimation models obtained were used in conjunction with the initial data to simulate the workload in the SLURM simulator with a configuration close to that used in the supercomputing center. Based on the simulated work of the scheduler, the average waiting time for tasks in the queue and resource utilization were measured. As a result, waiting times were recorded to be 28 percent and 19 percent lower than the default for gradient descent and median regression, respectively.