Patent Description:
In recent years, deep learning technologies have been more widely used in various industries. Major public cloud service providers at home and abroad have launched deep learning cloud services. Such cloud services are an inevitable choice for an enterprise to lower requirements for using the technologies and reduce costs of software and hardware deployment. When providing a deep learning service, a cloud service provider needs to consider many indicators, such as costs, performance, resource utilization, reliability, scalability, and maintainability, and these indicators are largely determined based on performance of a scheduling system. This is because an on-demand and flexible cloud service needs to be implemented through intensive resource reuse. The scheduling system that aims to optimally match a job with a resource is exactly a component implementing this process.

A working mode of application software determines design of the scheduling system. In the fields of high-performance computing, cloud computing, and big data processing, common scheduler software can be roughly classified into two types: (<NUM>) batch processing job scheduler; (<NUM>) service scheduler. Table <NUM> compares features of the two types of typical schedulers and their jobs.

In the deep learning field, main application software used by a user is a deep learning library, and a typical instance of the deep learning library includes a TensorFlow, an MXNet, and the like. The user may use the deep learning library in a free way to develop various deep learning application scenarios. This enables the deep learning library to have rich working modes and diversify deep learning job types. <FIG> shows a classification method for deep learning jobs. It can be learned that a typical deep learning job includes a training job, an inference job, a commissioning job, a visualization job, and the like. The training job may be single-node or distributed, and there may be a plurality of distributed implementations. The inference job may be online or offline.

A working mode of the deep learning library is different from a working mode of a conventional batch processing job in a high-performance cluster and a working mode of a long-period service in a cloud environment. A general batch processing job scheduler and service scheduler cannot meet a scheduling requirement of the deep learning library. The differences are as follows:.

Neither of the two types of conventional schedulers can fully meet complex and diverse scheduling requirements of a plurality of deep learning libraries and a plurality of types of deep learning jobs. This is a major obstacle to providing a deep learning service in a public cloud. By simply using an original batch processing job scheduler or service scheduler, a dedicated scheduling policy of deep learning cannot be implemented, which compromises user experience and increases operation and maintenance complexity; and hardware resource utilization is potentially affected, which increases operation costs of the public cloud.

<CIT> discloses: A task definition is received. The task definition indicates at least a location from which one or more software image can be obtained and information usable to determine an amount of resources to allocate to one or more software containers for the one or more software image. A set of virtual machine instances in which to launch the one or more software containers is determined, the one or more software image is obtained from the location included in the task definition and is launched as the one or more of software containers within the set of virtual machine instances.

Embodiments of this application provide a deep learning job scheduling method and system, and a related device, thereby improving compatibility of deep learning job scheduling.

According to a first aspect, a deep learning job scheduling method is provided, including:.

With reference to the first aspect, when the deep learning job includes at least one task, the job request includes at least two implementations.

In a first implementation, the job request further includes at least one piece of the following information: a job name, a deep learning program storage location, an application boot file, a dataset storage location, a type of the at least one task, a quantity of each type of task in the at least one task, a job command line parameter, and a resource requirement of each of the at least one task.

In a second implementation, the job request further includes at least one piece of the following information: a job name, a deep learning program, an application boot file, a dataset storage location, a type of the at least one task, a quantity of each type of task in the at least one task, a job command line parameter, and a resource requirement of each of the at least one task.

In any one of the two implementations, the target job description file may be generated based on the job request, the target job description file template, and the identifier of the target job basic image. Specifically, the target job description file template may be filled with the information included in the job request and the identifier of the target job basic image, to obtain the target job description file.

With reference to the first aspect, the job basic images include at least the following two implementations.

In a first implementation, the job basic images include an image of a deep learning library and an image of a dependency library, and the job basic images do not include an image of the deep learning program.

In a second implementation, the job basic images include an image of a deep learning library, an image of a dependency library, and an image of the deep learning program. The dependency library is a library required when the deep learning job is executed, and an instantiation of the deep learning program is the deep learning job.

With reference to the first aspect, the plurality of pre-stored job description file templates and the plurality of pre-stored job basic images may be generated at least in the following manner:.

With reference to the first aspect, after the sending the target job description file to a container scheduler, the method further includes:.

With reference to the first aspect, the method further includes: monitoring a status change of the at least one container created by the container scheduler; and obtaining a job status of the job request based on the status change of the at least one container.

With reference to the first aspect, the deep learning library type is a type of a computer function library designed for development of the deep learning program, and the job type includes at least one of a training job, an inference job, a commissioning job, and a visualization job.

With reference to the first aspect, each of the plurality of pre-stored job description file templates is used to define an organization structure of common information in a corresponding job description file. Each of the plurality of pre-stored job description file templates includes common information of a same type of deep learning job in a non-variable form and specific information of different types of deep learning jobs in a variable form.

With reference to the first aspect, the target job description file template complies with a specification of the container scheduler.

According to a second aspect, a deep learning job scheduling system is provided. The system includes a job scheduler and a container scheduler.

The job scheduler is configured to obtain a job request of a deep learning job. The job request carries a deep learning library type and a job type.

The job scheduler is configured to determine a target job description file template from a plurality of pre-stored job description file templates based on the deep learning library type and the job type, and determine an identifier of a target job basic image from identifiers of a plurality of pre-stored job basic images based on the deep learning library type and the job type.

The job scheduler is configured to generate a target job description file based on the target job description file template and the identifier of the target job basic image. The job scheduler is configured to send the target job description file to the container scheduler.

The container scheduler is configured to select the target job basic image from the pre-stored job basic images based on the target job description file, and create at least one container for executing the job request,.

With reference to the second aspect, when the deep learning job includes at least one task, the job request includes at least the following two implementations.

In a second implementation, the job request further includes at least one piece of the following information: a job name, a deep learning program, an application boot file, a dataset storage location, a type of the at least one task, a quantity of each type of task in the at least one task, a job command line parameter, and a resource requirement of 0each of the at least one task.

In any one of the two implementations, the job scheduler is further configured to generate the target job description file based on the job request, the target job description file template, and the identifier of the target job basic image. Specifically, the job scheduler is configured to fill the target job description file template with the information included in the job request and the identifier of the target job basic image, to obtain the target job description file.

With reference to the second aspect, the job basic images include at least the following two possible implementations.

With reference to the second aspect, the plurality of pre-stored job description file templates and the plurality of pre-stored job basic images may be generated at least in the following manner:.

With reference to the second aspect, the container scheduler is further configured to: when the container scheduler fails in scheduling, store, in a queue, a job identifier indicating the job request. The job identifier includes at least one of the job request, the information included in the job request, the target job description file, a pointer, and a data structure. The pointer points to at least one of the job request, the information carried in the job request, and the target job description file. The data structure points to at least one of the job request, the information carried in the job request, and the target job description file. The job scheduler is further configured to determine that the container scheduler has a condition for resubmitting a job request, and extract the job identifier from the queue and resubmit the job request to the container scheduler based on the job identifier.

With reference to the second aspect, the job scheduler is further configured to: monitor a status change of the at least one container created by the container scheduler; and obtain a job status of the job request based on the status change of the at least one container.

With reference to the second aspect, the deep learning library type is a type of a computer function library designed for development of the deep learning program, and the job type includes at least one of a training job, an inference job, a commissioning job, and a visualization job.

With reference to the second aspect, each of the plurality of pre-stored job description file templates is used to define an organization structure of common information in a corresponding job description file. Each of the plurality of pre-stored job description file templates includes common information of a same type of deep learning job in a non-variable form and specific information of different types of deep learning jobs in a variable form.

With reference to the second aspect, the target job description file template complies with a specification of the container scheduler.

According to a third aspect, a cloud service cluster is provided, including at least one management node and at least one compute node. The at least one management node communicates with the at least one compute node through a communications network. The at least one management node is configured to perform the method according to any one of the implementations of the first aspect.

In the foregoing technical solutions, different types of deep learning jobs may generate, based on different job description file templates and different job basic images, job description files that comply with the specification of the container scheduler, so that the different types of deep learning jobs can be processed by the container scheduler. This improves compatibility of deep learning job scheduling.

To describe the technical solutions in the embodiments of this application or in the background more clearly, the following briefly describes the accompanying drawings for describing the embodiments of this application or the background.

For ease of understanding, deep learning, a deep learning program, a deep learning library, and a deep learning job are first described separately.

The deep learning is a machine learning technology based on a deep neural network algorithm, and is mainly applied to scenarios such as perception and decision-making in the artificial intelligence field, for example, image and speech recognition, natural language translation, and computer game. The deep learning program is software, developed by a user, that is about a deep learning computing service. In a running process of the deep learning program, the deep learning library usually needs to be called. The deep learning library is a computer function library designed for development of the deep learning program, and is a computer program file including elements such as a data structure, an algorithm, a model, a subprogram, and a constant that are commonly used in the deep learning program, or a set of computer program files. In other words, the deep learning library is application software encapsulated with a bottom-layer operation of the deep learning job. Therefore, when developing deep learning programs for various deep learning application scenarios by using the deep learning library, the user may focus on content related to the deep learning application scenarios, and does not need to pay too much attention to content of the bottom-layer operation. This greatly improves development efficiency. The deep learning library may include a TensorFlow, an MXNet, and the like. It should be understood that the foregoing examples are merely used for description, and should not constitute a specific limitation. An instantiation of the deep learning program is the deep learning job. A computing service mainly applicable to a deep learning application scenario includes training, inference, commissioning, visualization, and the like. Therefore, a job type of the deep learning job mainly includes a training job, an inference job, a commissioning job, a visualization job, and the like. The training job may be single-node or distributed, and there may be a plurality of manners to implement a distributed training job. The inference job may be online or offline. It may be understood that the foregoing examples of the job type are merely used for description, and should not constitute a specific limitation.

To better understand the embodiments of the present invention, the following first separately describes a deep learning job scheduling system in the prior art and a deep learning job scheduling system in this application, so that a reader can understand a difference and a relationship between the two systems.

Refer to <FIG> is a schematic diagram of a deep learning job scheduling system according to the prior art. A user submits a job request to a container scheduler <NUM> through a user interface <NUM>. Correspondingly, the container scheduler <NUM> receives the job request sent by the user through the user interface <NUM>. The job request includes a job description file. After obtaining the job description file, the container scheduler <NUM> performs scheduling based on the job description file. When the container scheduler <NUM> succeeds in scheduling, the container scheduler <NUM> requests, from an image server <NUM> based on the job description file, a job image of which a type is corresponding to a type of the job description file. The type of the job image includes a training job image, an inference image, a commissioning image, and the like. The job image includes a deep learning library, a dependency library, a deep learning program, and the like. Correspondingly, the image server <NUM> sends the job image corresponding to the job description file to the container scheduler <NUM>. The container scheduler <NUM> creates at least one container (namely, a task container set) on a compute node <NUM> to execute, based on the job image, a deep learning job corresponding to the job request.

Refer to <FIG> is a schematic diagram of a deep learning job scheduling system according to an embodiment of this application. A user submits a job request to a job scheduler <NUM> through a user interface <NUM>. The job request carries a deep learning library type and a job type. The job scheduler <NUM> determines a target job description file template from a plurality of pre-stored job description file templates in a storage system <NUM> based on the deep learning library type and the job type. The job scheduler <NUM> determines an identifier of a target job basic image from identifiers of a plurality of pre-stored job basic images in the storage system <NUM> based on the deep learning library type and the job type. The job scheduler <NUM> generates a target job description file based on the target job description file template and the identifier of the target job basic image. The job scheduler <NUM> sends the target job description file to a container scheduler <NUM>. The container scheduler <NUM> selects the target job basic image from the plurality of pre-stored job basic images in the storage system <NUM> based on the target job description file, and creates at least one container (namely, a task container set) on a compute node <NUM> for executing the job request. In addition, the deep learning job scheduling system further includes a job monitor <NUM>, configured to monitor a status change of the at least one container created by the container scheduler <NUM>, and obtain a job status of the job request based on the status change of the at least one container.

Compared with the deep learning job scheduling system shown in <FIG>, the deep learning job scheduling system shown in <FIG> has at least the following improvements:.

In a specific embodiment of this application, the job description file complies with a specification of the container scheduler and is a file used to express the job request. One job description file is corresponding to one deep learning job. Different container schedulers may have different requirements for a format of the job description file.

In a specific embodiment of this application, the job description file is generated through rendering based on the job request, the job description file template, and a job basic image name.

In a specific embodiment of this application, the job request includes the deep learning library type and the job type. In addition, when the deep learning job includes at least one task, the job request includes at least the following two possible implementations. In a first implementation, the job request further includes at least one piece of the following information: a job name, a deep learning program storage location, an application boot file, a dataset storage location, a type of the at least one task, a quantity of each type of task in the at least one task, a job command line parameter, and a resource requirement of each of the at least one task.

The job name is an identifier of the deep learning job. The deep learning program storage location is used by a compute node to read the deep learning program based on a storage location of an application. The deep learning program is software, developed by a user, that is about a deep learning computing service. The application boot file is a file required for starting the deep learning program. The dataset storage location is used by the compute node to read a dataset based on a storage location of the dataset when the deep learning job is executed. The dataset is a set of data required when the deep learning job is executed, for example, historical data used for training a data model when a training job is executed. The type of the at least one task, the quantity of each type of task in the at least one task, and the resource requirement of each of the at least one task are used by the container scheduler to determine a quantity of containers and a resource that needs to be occupied by each container during scheduling.

In a specific embodiment, in the job request, the job name (job_name), the application storage location (app_url and boot_file), and the dataset storage location (data_url) are string fields; the deep learning library type (engine_type) and the job type (job_type) are enumerated fields; the type of the at least one task, the quantity of each type of task in the at least one task, and the resource requirement (attribute_description) of each of the at least one task are key-value pair fields.

In a specific embodiment of this application, the job description file template is a template file used to define an organization structure of common information in the job description file. The job description file template compiles common information of a same type of deep learning job into the template in a non-variable form, and specific information of different types of deep learning jobs is displayed in the template in a variable form. The job description file template may be compiled in a dedicated programming language, for example, a Jinja programming language, or may be a simple text file with a replaceable field. This is not specifically limited herein. <FIG> shows an example of the job description file template, which is a segment of a job description file template for TensorFlow distributed training.

In a specific embodiment of this application, the job basic image name is an identifier of the job basic image. Generally, an image is a file set used to create a container runtime file system. An image file set includes an executable file, a library file, and a configuration file of an operating system, an executable file, a library file, and a configuration file of an application, and a necessary script file and data file. For the deep learning job, the image includes a deep learning application, the deep learning library, and another necessary dependency library. The deep learning application is uploaded by the user and may be different for each job. The deep learning library and the another dependency library are provided by a deep learning service provider and are the same for each job. The job basic image includes at least two possible implementations. In a first implementation, the job basic image includes an image of the deep learning library image and an image of the dependency library, and the job basic image does not include an image of the deep learning program. In a second implementation, the job basic image includes an image of the deep learning library, an image of the dependency library, and an image of the deep learning program.

In a specific embodiment, the job basic image includes the executable file, the library file, the configuration file of the operating system, the deep learning library, a job boot script, and the like. As shown in <FIG>, a TensorFlow training job is used as an example, and a directory structure of a job basic image of the TensorFlow training job is provided. bin and sbin directories store an executable file of a Linux OS. lib and lib64 directories store a library file of the Linux OS. etc and var directories store a configuration file of the Linux OS. usr/local/lib/python2. <NUM>/site-package/tensorflow directory stores a deep learning library--TensorFlow. A home/mind/run_train. sh file is the job boot script. More specifically, <FIG> shows a segment of a Dockerfile used to generate a TensorFlow job basic image. Line <NUM> of the Dockerfile indicates that the job basic image is created based on an image of a Ubuntu Linux OS, to obtain a related file of the operating system. Line <NUM> to line <NUM> indicate that TensorFlow is downloaded and installed. Line <NUM> indicates that the job boot script is replicated. In this way, the job basic image includes all necessary files.

It may be understood that different container technical solutions have different image formats. In the Docker container technical solution used in this embodiment of the present invention, an image is stored in a format of a stacked file system. When creating an image for Docker, the user needs to use a text file named Dockerfile to describe steps for creating the image. Based on the steps, a Docker service program creates a file system structure of the image and stores the file system structure to a local file system. The Dockerfile may be viewed to roughly understand file content included in the image. In a specific embodiment of this application, the job description file template is similar to a blank table. Information carried in the job request and the job basic image name may be filled in the job description file template in a rendering manner, to obtain a complete table. The job description file is this complete table. More specifically, The job description file template is like a blank table with only a field name pre-printed but no specific information filled. A Jinja language is used as an example. In the job description file template, the field name and a placeholder of a field value are usually used to indicate a field. In the Jinja language, the field name is a string ended with a colon (for example, "name: "), and is equivalent to the field name preprinted in the table; the placeholder of the field value is a string enclosed by two braces (for example, "{ {name} } "), and is equivalent to a blank square in the table. For example, line <NUM> in <FIG> indicates that the job description file needs a field named "type". This field indicates a "job type", and a placeholder of the field is "{ {type}}". When the job description file is generated, the placeholder is replaced by a real job type string. If the job description file is corresponding to a table, this line is equivalent to a pair of squares such as ".

The job description file is like a table filled with specific information. Based on the job description file template, the job description file uses a real metainformation field of a specific job instance to replace the placeholder in the job description file template. For example, line <NUM> in <FIG> indicates that a value of a "type" field of the specific job instance is "train". In other words, the job type is "train (train)". If the job description file template is corresponding to a filled table, this line is equivalent to a square, filled with ".

", in the table corresponding to the job description file template. The real metainformation field has two sources: (<NUM>) the job request and the job basic image name; (<NUM>) information automatically generated by the job scheduler.

Two sources of job metainformation are briefly described as follows:.

In a specific embodiment of this application, a basic principle of rendering is to replace a variable with a real value. For example, in the Jinja language, rendering is to replace a placeholder of a field value of a corresponding field name with a real metainformation field. To improve rendering efficiency, in addition to the two semantics equivalent to traditional tables: the field name and the placeholder of the field value, the Jinja language provides some programming statements. These statements are included in a string like {%-. -%}, and provide basic functions of a programming language such as variable, judgment, and loop, and can provide a plurality of programmable and automated capabilities for text processing of a template file. These statements are equivalent to a secretary who can handle many complex issues for the user in a table filling process. For example, some fields in a template need to repeat for a plurality of times, and an organization structure is the same each time, but field values filled in are different. In this case, a loop statement {%- for. -%} may be used to simplify template compilation. In this embodiment of this application, a job description file needs to provide metainformation of a plurality of containers, and the containers have a same organization structure but different specific field values. Therefore, a loop statement is used in the job description file template to process this case.

In a specific embodiment of this application, a job description file defines an organization structure of common information of a type of deep learning job. A job basic image includes a file system required for running a container of a type of deep learning job. Therefore, a deep learning job is corresponding to a job description file and a job basic image. In other words, a deep learning job may be defined through a corresponding job description file and job basic image. A type of job refers to a specific type of job using a specific deep learning library. For example, a distributed training job using a TensorFlow library is a type of job. An online inference job using an MXNet library is another type of job. Theoretically, if the deep learning job scheduling system wants to support a quantity of combinations of a deep learning library type and a job type, the same quantity of job description files need to be created in this step. In other words, a quantity of job description file templates may be a product of a quantity of deep learning library types and a quantity of job types. For example, the training job using the TensorFlow library has a job description file. The inference job using the MXNet library has another job description file. However, content of job description files required for some combinations of a deep learning library type and a job type is the same and may be reused. Theoretically, if the deep learning job scheduling system wants to support a quantity of combinations of a deep learning library type and a job type, the same quantity of job basic images need to be created in this step. In other words, the quantity of job basic images may be a product of a quantity of deep learning library types and a quantity of job types. For example, the training job using the TensorFlow library has a job basic image. The inference job using the MXNet library has another job basic image. However, content of job basic images required for a combination is the same and may be reused. The deep learning library type may include a plurality of deep learning libraries such as the TensorFlow and the MXNet. The job type may include a plurality of job types such as single-node training, distributed training, online prediction, and commissioning.

It may be understood that a deep learning job Ais generated based on a job description file and a job basic image corresponding to the deep learning job A, and a deep learning job B is generated based on a job description file and a job basic image corresponding to the deep learning job B. "Specifications" of deep learning jobs generated based on different job description files and basic job images are the same. Therefore, a "specification" of the deep learning job A is the same as a "specification" of the deep learning job B. Herein, the "specification" refers to a feature, of the deep learning job, that can be accepted by the container scheduler. Therefore, although a deep learning library type and a job type of the deep learning job A are different from a deep learning library type and a job type of the deep learning job B, both the deep learning job A and the deep learning job B can be accepted by the container scheduler.

In a specific embodiment of this application, the storage system may be any type of storage system, including but not limited to a local file system, a network file system, a database system, an object storage system, and the like.

It should be noted that the job description file template and the job basic image are equivalent to two "models". If different "materials" are filled in the job description file template and the job basic image, deep learning jobs with different "materials" but a same "specification" may be generated. In this way, a plurality of deep learning libraries and a plurality of job types are uniformly abstracted and encapsulated, so that the container scheduler can uniformly manage various deep learning jobs.

In the deep learning job scheduling system, a core component includes the job scheduler <NUM> and the job monitor <NUM>. The following separately describes the job scheduler and job monitor in terms of structure.

The job scheduler is a component configured to schedule and execute a deep learning job, and includes a queue component configured to implement batch processing job scheduling. The job scheduler needs to use two types of files stored in the storage system: the job description file template and the job basic image. The job scheduler needs to communicate with the container scheduler to create and manage a task container. The job scheduler also has a user interface program that matches the job scheduler, for the user to perform an operation. When the user submits a job to the job scheduler through the user interface program, an entity that carries information is the job request. When the job scheduler delivers a job to the container scheduler, an entity that carries information is the job description file. To implement a function of deep learning job scheduling, the job scheduler needs to have a reasonable internal structure design. A feasible internal structure design solution of the job scheduler is shown in <FIG>. It should be noted that the internal structure design solution is merely used for description, and should not constitute a specific limitation.

In the internal structure design solution of the job scheduler shown in <FIG>, the job scheduler mainly includes internal components such as a network service <NUM>, an event engine <NUM>, a container scheduler client <NUM>, a scheduling algorithm <NUM>, a queue <NUM>, a template rendering mechanism <NUM>, and a storage system client <NUM>. The network service <NUM> is used to receive messages from a user interface program and a job monitor, for example, a job request submitted by a user through the user interface program, and job status information sent by the job monitor. The event engine <NUM> is configured to process various asynchronous events generated by the network service, and convert the events into function calls to another internal component, to drive overall orderly operation of the job scheduler. The container scheduler client <NUM> is configured to send a request to a container scheduler, to start a container and execute a job on a compute node. The scheduling algorithm <NUM> is used to implement algorithm logic of batch processing job scheduling, for example, a first in first out scheduling algorithm or a priority scheduling algorithm. The queue <NUM> is used to store a job request that has not been successfully scheduled. The template rendering mechanism <NUM> is used to generate, based on a job description file template and a job request, a job description file for use by the container scheduler client. The storage system client <NUM> is configured to access a storage system, to read metainformation such as the job description file template and a job basic image name. Optionally, the network service <NUM> is an RPC service; a client scheduler client <NUM> is a Kubernetes client; a template rendering machine <NUM> is a Jinja language rendering mechanism; and the storage system client <NUM> is an NFS client. The job monitor is a component configured to perceive a running status of a deep learning job. The job monitor needs to communicate with the container scheduler, to obtain a running status of a task container. The job monitor also needs to communicate with the job scheduler, to feed back status information to the job scheduler. To implement a function of deep learning job monitoring, the job monitor needs to have a reasonable internal structure design. A feasible internal structure design solution of the job monitor is shown in <FIG>. It should be noted that the internal structure design solution is merely used for description, and should not constitute a specific limitation. In the internal structure design solution of the job monitor shown in <FIG>, the job monitor mainly includes internal components such as an event engine <NUM>, a status mapping mechanism <NUM>, a container scheduler client <NUM>, and a job scheduler client <NUM>. The event engine <NUM> is configured to process various asynchronous events monitored by the container scheduler client, and convert the events into function calls to another internal component, to drive overall orderly operation of the job monitor. The status mapping mechanism <NUM> is used to map a status of all container sets of a job to an overall status of the job. The container scheduler client <NUM> is configured to send a request to a container scheduler, to start a container and execute the job on a compute node. The job scheduler client <NUM> is configured to communicate with a job scheduler, to feed back a change of the job status to the job scheduler. Optionally, a client scheduler client <NUM> is a Kubernetes client, and the job scheduler client <NUM> is an RPC client.

It should be understood that division of the two components, the job scheduler and the job monitor, indicates only logical division of the components and does not impose a constraint on physical implementation. In the physical implementation, the two components may be implemented in different programs and run in different processes, or may be implemented in a same program and run in a same process, or may even be separately implemented in a distributed manner. In other words, the two components may be located in a plurality of programs and run in a plurality of processes.

The deep learning job scheduling system shown in <FIG> may be implemented on a server, or may be implemented on a cloud computing infrastructure. This is not specifically limited herein. The following focuses on how to implement the deep learning job scheduling system shown in <FIG> on the cloud computing infrastructure. The cloud computing infrastructure may be a cloud service cluster <NUM>. As shown in <FIG>, the cloud service cluster <NUM> includes nodes and a communications network between the nodes. The nodes may be classified into two types by function: management nodes <NUM> and compute nodes <NUM>. The management node <NUM> is configured to run service programs <NUM> of a cloud service provider. The compute node <NUM> is configured to run applications <NUM> of a user. The cloud service cluster <NUM> further provides two external interface pages: a management interface <NUM> oriented to the cloud service provider and a user interface <NUM> oriented to the user. The nodes may be physical servers, or may be virtual machines. A form of the service program <NUM> or the application <NUM> on the nodes is a process. These processes may directly run on an operating system, or may be encapsulated by using containers <NUM>. The container is a virtualization technology. The technology enables a process to run in a relatively independent and isolated environment (including an independent file system, a namespace, a resource view, and the like), thereby simplifying a software deployment process, enhancing software portability and security, and improving system resource utilization. The interface pages may have various forms, such as a web interface, a command line tool, and a REST interface. It should be noted that, for a deep learning job such as an online inference job, the user may also use a client program <NUM> to access the application <NUM> on the compute node <NUM>.

When the deep learning job scheduling system shown in <FIG> is implemented by using the cloud service cluster shown in <FIG>, the cloud service cluster may be a cloud service cluster shown in <FIG>. As shown in <FIG>, the user interface <NUM> in <FIG> is a user interface <NUM> in <FIG>; the job scheduler <NUM>, the container scheduler <NUM>, and the job monitor <NUM> in <FIG> are disposed in a service program <NUM> of a management node <NUM> in <FIG>; the compute node <NUM> in <FIG> is a compute node <NUM> in <FIG>; and the storage system <NUM> in <FIG> is a storage system <NUM> in <FIG>. Optionally, the storage system <NUM> may be disposed outside the cloud service cluster, or may be integrated into the cloud service cluster. This is not specifically limited herein. It should be understood that the cloud service cluster <NUM> is merely an example provided in the embodiments of this application. In addition, the cloud service cluster <NUM> may include more or fewer components than shown components, or may combine two or more components, or may have different component configurations.

Refer to <FIG> is a schematic diagram of a structure of a deep learning job scheduling system according to an implementation of this application. The system includes a computing device cluster. The computing device cluster includes at least one management node <NUM> and at least one compute node <NUM>.

The management node <NUM> includes one or more processors <NUM>, a communications interface <NUM>, and a memory <NUM>. The processor <NUM>, the communications interface <NUM>, and the memory <NUM> may be connected by using a bus <NUM>.

The processor <NUM> includes one or more general-purpose processors. The general-purpose processor may be any type of device that can process an electronic instruction, including a central processing unit (CPU), a microprocessor, a microcontroller, a main processor, a controller, an application specific integrated circuit (ASIC), and the like. The processor <NUM> can be a dedicated processor configured only for the management node <NUM> or can be shared with another management node <NUM> or the compute node <NUM>. The processor <NUM> executes various types of digital storage instructions, for example, software or firmware programs stored in the memory <NUM>, so that the processor <NUM> can enable the management node <NUM> to provide various relatively wide services. For example, the processor <NUM> can execute a program or process data, to execute at least a part of the method discussed in this specification.

The communications interface <NUM> may be a wired interface (for example, an ethernet interface) or a wireless interface (for example, a cellular network interface or a wireless local area network interface), and is configured to communicate with another computing device or a user. When the communications interface <NUM> is a wired interface, the communications interface <NUM> may use a TCP/IP protocol suite, such as, an RAAS protocol, a remote function call (RFC) protocol, a simple object access protocol (SOAP), a simple network management protocol (SNMP), a common object request broker architecture (CORBA) protocol, and a distributed protocol. When the communications interface <NUM> is a wireless interface, cellular communication may be used according to a global system for mobile communications (GSM) or code division multiple access (CDMA) standard. Therefore, the communications interface <NUM> includes a wireless modem, an electronic processing device, one or more digital memory devices, and a dual antenna that are used for data transmission. It should be understood that the modem can be implemented as software stored in the management node and executed by the processor <NUM>, or the modem can be a separate hardware component located inside or outside the management node <NUM>. The modem can operate with any quantity of different standards or protocols (for example, EVDO (CDMA2000 1xEV-DO, EVDO), CDMA, a general packet radio service (GPRS) technology, and an enhanced data rates GSM evolution (EDGE) technology).

The memory <NUM> may include a volatile memory, for example, a random access memory (RAM). Alternatively, the memory may include a non-volatile memory, for example, a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD). Alternatively, the memory may include a combination of the foregoing types of memories. The memory <NUM> may store a service program <NUM>, used to provide a service for the compute node <NUM>. The service program <NUM> may include a job scheduler <NUM>, a job monitor <NUM>, and a container scheduler <NUM>. The job scheduler <NUM> is a component configured to schedule and execute a deep learning job, and includes a queue component configured to implement batch processing job scheduling. The job monitor <NUM> is a component configured to perceive a running status of the deep learning job. The job monitor <NUM> needs to communicate with the container scheduler <NUM> to learn a running status of a task container. The job monitor <NUM> further needs to communicate with the job scheduler <NUM> to feed back status information to the job scheduler <NUM>. The container scheduler <NUM> is configured to, based on a request of the job scheduler <NUM> and the job monitor <NUM>, start a container and execute a job on the compute node <NUM>. In a specific implementation, for a specific implementation of the job scheduler <NUM>, refer to <FIG> and related descriptions. In a specific implementation, for a specific implementation of the job monitor <NUM>, refer to <FIG> and related descriptions.

The compute node <NUM> includes one or more processors <NUM>, a communications interface <NUM>, and a memory <NUM>. The processor <NUM>, the communications interface <NUM>, and the memory <NUM> may be connected by using a bus <NUM>.

The processor <NUM> includes one or more general-purpose processors. The general-purpose processor may be any type of device that can process an electronic instruction, including a CPU, a microprocessor, a microcontroller, a main processor, a controller, an ASIC, and the like. The processor <NUM> can be a dedicated processor configured only for the compute node <NUM> or can be shared with the management node <NUM> or another compute node <NUM>. The processor <NUM> executes various types of digital storage instructions, for example, software or firmware programs stored in the memory <NUM>, so that the processor <NUM> can enable the compute node <NUM> to provide various relatively wide services. For example, the processor <NUM> can execute a program or process data, to execute at least a part of the method discussed in this specification.

The communications interface <NUM> may be a wired interface (for example, an ethernet interface) or a wireless interface (for example, a cellular network interface or a wireless local area network interface), and is configured to communicate with another computing device or a user. When the communications interface <NUM> is a wired interface, the communications interface <NUM> may use a TCP/IP protocol suite, such as, an RAAS protocol, an RFC protocol, a SOAP protocol, an SNMP protocol, a CORBA protocol, and a distributed protocol. When the communications interface <NUM> is a wireless interface, cellular communication may be used according to a GSM or CDMA standard. Therefore, the communications interface <NUM> includes a wireless modem, an electronic processing device, one or more digital memory devices, and a dual antenna that are used for data transmission. It should be understood that the modem can be implemented as software stored in the management node and executed by the processor <NUM>, or the modem can be a separate hardware component located inside or outside the compute node <NUM>. The modem can operate with any quantity of different standards or protocols. The memory <NUM> may include a volatile memory, for example, a RAM. Alternatively, the memory may include a non-volatile memory, for example, a ROM, a flash memory, an HDD, or an SSD. Alternatively, the memory may include a combination of the foregoing types of memories. The memory <NUM> may store program code <NUM> and a database <NUM>. The program code <NUM> may include a deep learning program <NUM>. The database <NUM> may include a deep learning library <NUM> and a dependency library <NUM>. The deep learning program <NUM> is software, developed by the user, that is about a deep learning computing service. An instantiation of the deep learning program is a deep learning job. A computing service mainly applicable to a deep learning application scenario includes training, inference, commissioning, visualization, and the like. Therefore, a job type of the deep learning job includes a training job, an inference job, a commissioning job, a visualization job, and the like. The deep learning library <NUM> is a computer function library designed for development of the deep learning program, and is a computer program file including elements such as a data structure, an algorithm, a model, a subprogram, and a constant that are commonly used in the deep learning program, or a set of computer program files. In other words, the deep learning library is application software encapsulated with a bottom-layer operation of the deep learning job. Therefore, when developing deep learning programs for various deep learning application scenarios by using the deep learning library, the user may focus on content related to the deep learning application scenarios, and does not need to pay too much attention to content of the bottom-layer operation. This greatly improves development efficiency. The deep learning library may include a TensorFlow, an MXNet, and the like. It should be understood that the foregoing examples are merely used for description, and should not constitute a specific limitation. The dependency library <NUM> is a database required when the deep learning job is run.

It should be noted that a job description file template and a job basic image may be stored in some management nodes <NUM> and/or some compute nodes <NUM> in the computing device cluster. In other words, the storage system <NUM> may include storage resources inside the management node <NUM> and/or the compute node <NUM>. For example, the storage system <NUM> is a distributed storage pool. Alternatively, the job description file template and the job basic image may be stored outside the management node <NUM> and the compute node <NUM>. In other words, the storage system <NUM> does not include the storage resources inside the management node <NUM> and/or the compute node <NUM>.

The management node <NUM> is configured to run the program stored in the memory <NUM>, to execute the following instructions:.

Optionally, the deep learning job includes at least one task.

The job request further includes at least one piece of the following information: a job name, a deep learning program storage location, an application boot file, a dataset storage location, a type of the at least one task, a quantity of each type of task in the at least one task, a job command line parameter, and a resource requirement of each of the at least one task.

Alternatively, the job request further includes at least one piece of the following information: a job name, a deep learning program, an application boot file, a dataset storage location, a type of the at least one task, a quantity of each type of task in the at least one task, a job command line parameter, and a resource requirement of each of the at least one task.

The management node <NUM> is further configured to generate the target job description file based on the job request, the target job description file template, and the identifier of the target job basic image.

Optionally, the management node <NUM> is further configured to:
fill the target job description file template with the information included in the job request and the identifier of the target job basic image, to obtain the target job description file.

Optionally, the job basic images include an image of a deep learning library and an image of a dependency library, and the basic job images do not include an image of the deep learning program.

Alternatively, the job basic images include an image of a deep learning library, an image of a dependency library, and an image of the deep learning program.

The dependency library is a library required when the deep learning job is executed, and an instantiation of the deep learning program is the deep learning job.

Optionally, the plurality of pre-stored job description file templates are generated based on deep learning library types and job types, and each of the plurality of pre-stored job description file templates is corresponding to one deep learning library type and one job type.

The plurality of pre-stored job basic images are generated based on the deep learning library types and the job types, and each of the plurality of pre-stored job basic images is corresponding to one deep learning library type and one job type.

Optionally, the management node <NUM> is further configured to: when the container scheduler fails in scheduling, store, in a queue, a job identifier indicating the job request. The job identifier includes at least one of the job request, the information included in the job request, the target job description file, a pointer, and a data structure. The pointer points to at least one of the job request, the information carried in the job request, and the target job description file. The data structure points to at least one of the job request, the information carried in the job request, and the target job description file.

Optionally, the management node <NUM> is further configured to determine that the container scheduler has a condition for resubmitting a job request, and extract the job identifier from the queue and resubmit the job request to the container scheduler based on the job identifier.

For brevity, the cloud service cluster shown in <FIG> and the computing device cluster shown in <FIG> are not described in detail. For details, refer to <FIG> and related descriptions.

Refer to <FIG> is a schematic diagram of a deep learning job scheduling method according to this application. The deep learning job scheduling method in this embodiment of this application includes the following steps.

S101: A job scheduler obtains a job request submitted by a user through a user interface program.

In a specific embodiment of this application, when the user submits a job, the user needs to notify the user interface program of description information (including but not limited to information such as an application, a command parameter, and a resource requirement) of the job, and the user interface program transfers the information to the job scheduler, to implement a submission process of the job request.

S102: The job scheduler generates a job description file based on a job basic image name, a job description file template, and the job request.

In a specific embodiment of this application, the job scheduler loads a corresponding job description file template and a corresponding job basic image name from a storage system based on a deep learning library type and a job type that are specified in the job request. Then, through template rendering or a filling mechanism, the job description file is created by using the job basic image name and information in the job request as input of the job description file template.

S103: The job scheduler submits the job to a container scheduler.

In a specific embodiment of this application, the job scheduler calls an interface of the container scheduler to transfer the job description file to the container scheduler, so as to request the container scheduler to create and run a container corresponding to each task of the deep learning job. If the container scheduler successfully receives the request, step S1041 is performed. If the container scheduler fails to receive the request, step S1051 is performed. The container scheduler may fail to receive the job request due to insufficient hardware resources or an occasional network fault.

S1041: The container scheduler creates the container for each task based on a job basic image and the job description file.

In a specific embodiment of this application, the container scheduler loads a corresponding job basic image from the storage system based on the deep learning library type and the job type that are specified in the job description file. Then, based on a task specification specified in the job description file, a basic image is used to create a corresponding quantity of containers. In addition, a process of the task is run in the container based on the application, the command parameter, and other information specified in the job description file. The resource requirement information specified in the job description file is used by the container scheduler to select a host environment (namely, a compute node) for running the container.

S1042: The job scheduler monitors a status change of the container, and maps a container set status to a job status.

In a specific embodiment of this application, the job monitor monitors, in real time through the interface of the container scheduler, running statuses of containers corresponding to all tasks of the deep learning job, and perceives a change of the statuses. When a status of a container changes, the job monitor obtains an overall status of the deep learning job through a status mapping mechanism. The status may be fed back to the job scheduler and may be queried by the user through a user interface. As the status of the container changes, this step may be repeated for a plurality of times until the job is complete.

S1051: The job scheduler adds the job request to a queue.

In a specific embodiment of this application, a queue in the job scheduler is used to store a job request that fails in the submission process. It should be noted that the job scheduler storing the job request is merely an example. In actual application, the job scheduler may further store another identifier of the deep learning job, for example, the information in the job request submitted by the user interface program, the job description file generated by the job scheduler, a pointer or another data structure to these objects, or the like.

S1052: The job scheduler checks the container scheduler and the queue, to search for a job request that meets a re-submission condition.

In a specific embodiment of this application, the job scheduler calls the interface of the container scheduler to check whether the container scheduler has a condition for accepting submission of a new job. Then, the job scheduler reads the queue of the job scheduler, to check whether there is a job request that failed to be submitted. If the container scheduler meets the condition, and there is such a job request in the queue, the job scheduler extracts, from the queue based on a job selection logic, the job request that meets the re-submission condition, and the step S103 is performed again to submit the job to the container scheduler. If the container scheduler does not meet the condition, and there is no such a job request in the queue, this step is repeated to continue checking a status of the container scheduler and a status of the queue.

For brevity, for definitions of the job description file, the job request, the job description file template, and the job basic image name, and relationships between the job description file, the job request, the job description file template, and the job basic image name, refer to the foregoing embodiment. The following uses open-source container orchestration software --Kubernetes as an example to describe the deep learning job scheduling method provided in the embodiments of this application. As shown in <FIG>, when the open-source container orchestration software --Kubernetes is used, components in the deep learning job scheduling system in this application have the following features.

In this embodiment, a user interface program is a web application, and a user needs to operate the user interface program by using a web browser. In this embodiment, a job request sent by the user interface program to a job scheduler is an RPC message.

In this embodiment, the job scheduler and a job monitor are two independent computer programs that run on a management node in a public cloud service environment. The job scheduler communicates with the job monitor through a remote procedure call (RPC) protocol, because this protocol has advantages of cross-platform and cross-language and facilitates construction of a distributed cloud service environment. A queue of the job scheduler is implemented as a queue data structure in a programming language, and a pointer to the job request is stored in the queue.

In this embodiment, a Docker container is used as a task container, and a Docker image is used as a job basic image. The Kubernetes uses open-source container management software Docker as platform software for creating and managing a container. Therefore, in this embodiment, the Docker container is used as the task container, and the Docker image is used as the job basic image. The Docker container is run on a compute node. A plurality of containers may be run on one compute node. An application corresponding to a user job runs in the Docker container. The user may access a resource in the container through application code or an interface provided by the application. In this embodiment, a format of a job description file is a YAML (YAML Ain't Markup Language, YAML markup language) format. The Kubernetes accepts the YAML format. Therefore, the job description file in this embodiment uses the YAML format. The job scheduler submits the job description file to the Kubernetes through a REST API of the Kubernetes. In this embodiment, a Jinja format is selected as a format of a job description file template, because a file in the Jinja format can be rendered by a corresponding compiler, with simple syntax, to generate a text configuration file including YAML.

In this embodiment, a storage system is a network file system (NFS). The storage system stores the job description file template and the job basic image, because the NFS has single storage space that can be globally accessed in a distributed system, and facilitates technical implementation of the scheduling system. When reading the two types of files, the job scheduler directly uses a file system interface. The Kubernetes also reads the job basic image through the file system interface to create a container based on the image.

In this embodiment, the job scheduler and the job monitor are two independent computer programs that run on the management node in the public cloud service environment. The job scheduler communicates with the job monitor through the RPC (remote procedure call) protocol, because this protocol has the advantages of cross-platform and cross-language and facilitates construction of the distributed cloud service environment. The queue of the job scheduler is implemented as the queue data structure in the programming language, and the pointer to the job request is stored in the queue. When the open-source container orchestration software --Kubernetes is used, the deep learning job scheduling method shown in <FIG> may be specifically as follows: S101: A job scheduler obtains a job request submitted by a user through a user interface program.

In a specific embodiment of this application, the user needs to use a web browser to access a web service-based user interface program. When submitting a job, information that the user needs to provide includes: a job name, a deep learning library type, a job type, an application storage location, a dataset storage location, a task type and quantity, a resource requirement of each task, and the like. The user interface program encodes the information, generates a job request in an RPC format, and sends the job request to the job scheduler. In this way, the job scheduler obtains the job request of the user.

In this embodiment, the job name (job_name), the application storage location (app_url and boot_file), and the dataset storage location (data _url) are string fields; the deep learning library type (engine_type) and the job type (job_type) are enumerated fields; the task type and quantity (scale_description) and the resource requirement (attribute_description) of each task are key-value pair fields.

In a specific embodiment of this application, the job scheduler loads a corresponding job description file template from an NFS based on a deep learning library type and a job type that are specified in a job request RPC message. Then, the job scheduler calls a built-in Jinja template rendering mechanism, and uses information in the job request as an input variable during rendering, to create a job description file in a YAML format. The job description file generated by the job scheduler may be parsed by the Kubernetes to create a task container.

In a specific embodiment of this application, the job scheduler calls a REST API of the Kubernetes to transfer the job description file in the YAML format to the Kubernetes, so as to request the Kubernetes to create and run a container corresponding to each task of the job. If the Kubernetes successfully receives the request, step S1041 is performed. If the Kubernetes fails to receive the request, step S1051 is performed. The Kubernetes may fail to receive the job request due to insufficient hardware resources or an occasional network fault.

In a specific embodiment of this application, the Kubernetes loads a corresponding job basic image from the NFS based on the deep learning library type and the job type that are specified in the job description file. In this embodiment, the loaded job basic image is a TensorFlow image. Then, the Kubernetes uses the job basic image to create a corresponding quantity of Pod objects and Service objects based on a task specification (namely, a total number of Pods running PS and worker tasks) specified in the job description file. In this way, each Pod is corresponding to one container, and each Service is corresponding to one network port. Next, a process of each task is run in the container based on an application, a command parameter, and other information specified in the job description file. A TensorFlow distributed training job is used as an example. These processes include one PS task process and two worker task processes of the TensorFlow distributed training job. Application code required to start a task process is specified by a command field in the job description file. In this embodiment, a run_train. sh program specified by the command field is used to start a TensorFlow training task. The program first downloads an application code directory specified by the app_url field in the job request, and then executes an application specified by the boot_file field. Command parameters of the process are also from the job description file, and main parameters include a task _index parameter used to specify a task sequence number and a data_url parameter used to specify the dataset storage location. The resource requirement information specified in the job description file is used by the Kubernetes to select a host environment (namely, a compute node) for running the container. For example, each Pod (namely, a container) declares that <NUM> GiB of memory and four CPU cores are required. Therefore, the Kubernetes searches for a compute node with more idle resources than declared resources. Each container may be run on a same compute node, or may be run on different compute nodes.

In a specific embodiment of this application, the job monitor monitors, in real time through a Watch interface of the Kubernetes, running statuses of containers corresponding to all tasks of the job, and perceives a change of the statuses. When a status of a container changes, the job monitor obtains a latest status of the container in real time through an event stream read from the Watch interface. Then, the job monitor obtains an overall status of the job through a built-in status mapping mechanism. Finally, the job monitor calls an RPC interface provided by the job scheduler to feed back the status to the job scheduler. The user may query the job status through the user interface program. As the status of the container changes, this step may be repeated for a plurality of times until the job is complete.

In this embodiment, main mapping rules of the status mapping mechanism include: a) If all the containers enter a running state, it is considered that the job enters a running state; b) If all worker task containers enter a complete state, it is considered that the job enters a complete state; c) If one or more containers enter a failed state, it is considered that the job enters a failed state.

In a specific embodiment of this application, a queue in the job scheduler is used to store a job request that fails in a submission process. In this embodiment, the queue of the job scheduler is implemented as a queue data structure in a programming language, and a pointer to the job request is stored in the queue. Therefore, adding a failed job request to the queue is actually adding, to the queue, a pointer to the job request.

In a specific embodiment of this application, the job scheduler calls an interface of the Kubernetes to check whether the Kubernetes has a condition for accepting submission of a new job. In this embodiment, the condition mainly means that a compute node has sufficient resources. Then, the job scheduler reads the queue of the job scheduler, to check whether there is a job request that failed to be submitted. If resources on the compute node managed by the Kubernetes meets the condition, and there is such a job request in the queue of the job scheduler, the job scheduler extracts, from the queue based on a job selection logic, the job request that meets the re-submission condition, and the step S103 is performed again to submit the job to the Kubernetes. Otherwise, this step is repeated to continue checking a status of the Kubernetes and a status of the queue. In this embodiment, the job selection logic uses a simple "first in first out (FIFO)" rule. To be specific, a job request with earliest enqueue time is selected from the queue.

In the foregoing technical solutions, different types of deep learning jobs may generate, based on different job description file templates and different job basic images, job description files that comply with a specification of the container scheduler, so that the different types of deep learning jobs can be processed by the container scheduler. This improves compatibility of deep learning job scheduling.

Claim 1:
A deep learning job scheduling method, comprising:
obtaining (S101), by a job scheduler (<NUM>, <NUM>), a job request of a deep learning job, wherein the job request carries a deep learning library type and a job type;
determining, by the job scheduler (<NUM>, <NUM>), a target job description file template from a plurality of pre-stored job description file templates based on the deep learning library type and the job type;
determining, by the job scheduler (<NUM>, <NUM>), an identifier of a target job basic image from identifiers of a plurality of pre-stored job basic images based on the deep learning library type and the job type;
generating (S102) , by the job scheduler (<NUM>, <NUM>), a target job description file based on the target job description file template and the identifier of the target job basic image;
sending (S103) , by the job scheduler (<NUM>, <NUM>), the target job description file to a container scheduler (<NUM>, <NUM>); and
selecting (S1041), by the container scheduler (<NUM>, <NUM>), the target job basic image from the pre-stored job basic images based on the target job description file, and creating at least one container (<NUM>) for executing the job request,
wherein the job basic image includes a file system required for running a container of a type of deep learning job, and
wherein the content of job basic images required for a combination of the deep learning library type and the job type is the same and is reusable.