Mapping resources to each layer of a neural network model based computing task

A method for processing a computing task comprises: determining parameter data of multiple layers associated with a neural network model in response to receiving a computing task based on the neural network model; ranking at least one part of the multiple layers on the basis of the parameter data so as to obtain a layer list; ranking multiple computing resources on the basis of status information of the multiple computing resources so as to obtain a resource list; and determining a mapping between a corresponding layer among the multiple layers and a corresponding computing resource among the multiple computing resources on the basis of the layer list and the resource list, where the mapping indicates one computing resource among the multiple computing resources is to process parameters associated with one layer among the multiple layers.

The present application claims priority to Chinese Patent Application No. 201810618733.X, filed Jun. 15, 2018, and entitled “Method, Apparatus, and Computer Program Product for Processing Computing Task,” which is incorporated by reference herein in its entirety.

FIELD

Embodiments of the present disclosure generally relate to a computing system including dedicated computing resources, and more specifically, to a method, an apparatus and a computer program product for processing a computing task.

BACKGROUND

Applications on the client can be designed to complete various processing or analyzing tasks using computing resources, such as processing and storage resources. As the requirement and complexity of the tasks such as machine learning, deep learning, and data mining continuously grow, a large amount of and/or variable computing resources are required to satisfy operations of the corresponding application. This can be fulfilled by a machine or system having multiple dedicated computing resources, where the application can be scheduled to operate on one or more dedicated computing resources of the machine or system. For example, a cloud-based computing system has already been developed, and such a computing system includes a machine having one or more dedicated computing resources. Different clients can rent computing resources (for example, dedicated computing resources) of the system as required to operate their own applications.

With the development of computer technologies, there is an increasing variety of computing resources, and computing resources are no longer limited to traditional ones like central processing units. For example, the computing capability of graphics processing units (GPUs) has increased significantly. GPUs are particularly suitable to execute computing tasks in deep learning, high performance computing, and machine learning applications and the like by virtue of their distinctive properties. However, for ordinary client devices and conventional cloud computing devices, their GPUs are rather limited in performance and lack a high-performance processing capability. Recently, the challenge of how to process a computing task using the computing capability of a GPU of another device (i.e., remotely processing the computing task) has become a focus of research.

However, some existing technical solutions fail to make sufficient and effective use of the processing capability of remote computing resources (e.g., computing resources in a computing resource pool), whereby idleness and/or workload imbalance arises in the resource pool. Therefore, it is desirable to provide a technical solution for easily and effectively processing a computing task using multiple computing resources in a resource pool.

SUMMARY

Embodiments of the present disclosure provide a method, an apparatus and a computer program product for processing a computing task.

According to a first aspect of the present disclosure, there is provided a method for processing a computing task. The method comprises: determining parameter data of multiple layers associated with a neural network model in response to receiving a computing task based on the neural network model; ranking at least one part of the multiple layers on the basis of the parameter data so as to obtain a layer list; ranking multiple computing resources on the basis of status information of the multiple computing resources so as to obtain a resource list; and determining a mapping between a corresponding layer among the multiple layers and a corresponding computing resource among the multiple computing resources on the basis of the layer list and the resource list, wherein the mapping indicates one computing resource among the multiple computing resources is to process parameters associated with one layer among the multiple layers.

According to a second aspect of the present disclosure, there is provided an apparatus for processing a computing task, comprising: at least one processor; a volatile memory; and a memory coupled to the at least one processor, the memory having instructions stored thereon, the instructions, when executed by the at least one processor, causing the apparatus to perform acts. The acts include: determining parameter data of multiple layers associated with a neural network model in response to receiving a computing task based on the neural network model; ranking at least one part of the multiple layers on the basis of the parameter data so as to obtain a layer list; ranking multiple computing resources on the basis of status information of the multiple computing resources so as to obtain a resource list; and determining a mapping between a corresponding layer among the multiple layers and a corresponding computing resource among the multiple computing resources on the basis of the layer list and the resource list, here the mapping indicates one computing resource among the multiple computing resources is to process parameters associated with one layer among the multiple layers.

According to a third aspect of the present disclosure, there is provided a computer program product. The computer program product is tangibly stored on a computer readable medium and comprises machine executable instructions which, when executed, cause the machine to implement a method according to the first aspect.

DETAILED DESCRIPTION

Preferred embodiments of the present disclosure will be described in more detail with reference to the drawings. Although the drawings illustrate preferred embodiments of the present disclosure, it should be appreciated that the present disclosure can be implemented in various manners and should not be limited to the embodiments explained herein. On the contrary, the embodiments are provided to make the present disclosure more thorough and complete and to fully convey the scope of the present disclosure to those skilled in the art.

As used herein, the term “includes” and its variants are to be read as open-ended terms that mean “includes, but is not limited to.” The term “or” is to be read as “and/or” unless the context clearly indicates otherwise. The term “based on” is to be read as “based at least in part on.” The terms “one example embodiment” and “one embodiment” are to be read as “at least one example embodiment.” The term “a further embodiment” is to be read as “at least a further embodiment.” The terms “first”, “second” and so on can refer to same or different objects.

The following text also can comprise other explicit and implicit definitions. As described above, dedicated computing resources can be provided locally at the client or by a remote machine or system. In some examples, a cloud-based computing system may be deployed, which includes a plurality of machines having one or more dedicated computing resources. The dedicated computing resources of the computing system can be utilized by different clients based on their needs to dispatch the corresponding applications to operate on available dedicated computing resources.

FIG. 1is a schematic diagram illustrating an example computing system100in which embodiments of the present disclosure can be implemented. A plurality of servers for running applications, including a server110-1, a server110-2, a server110-3, . . . , a server110-U (hereinafter collectively or individually referred to as a server110, where U is a natural number greater than 1), are deployed in the computing system100. The computing system100further includes a dedicated computing resource160-1, a dedicated computing resource160-2, a dedicated computing resource160-3, . . . , a dedicated computing resource160-V (hereinafter collectively or individually referred to as a dedicated computing resource160, where V is a natural number greater than 1). Each server110has one or more dedicated computing resources160.

In the example ofFIG. 1, the server110-1has the dedicated computing resource160-1, the server110-2has the dedicated computing resource160-2, the server110-3has the dedicated computing resource160-3, and the server110-U has the dedicated computing resource160-V. It will be understood that each server is not limited to have only one computing resource, rather one server may have one or more computing resources. Therefore, here the value of U may not equal that of V.

In the context of the present disclosure, examples of the dedicated computing resource160may include, but not limited to, a GPU, a Field Programmable Gate Array (FPGA), or the like. For the sake of discussion, some embodiments will use a GPU as the example of the dedicated computing resources. In addition to the dedicated computing resource160, the server110may further include, for example, one or more general processing units such as a central processing unit or CPU (not shown).

FIG. 1further illustrates a plurality of clients120-1,120-2, . . . ,120-P (hereinafter collectively or individually referred to as a client120, where P is a natural number greater than 1) respectively having applications150-1,150-2, . . . ,150-Q (hereinafter collectively or individually referred to as an application150, where Q is a natural number greater than 1) to run. The application150may be any application running on the machine, and the application can be designed to perform corresponding data processing or analyzing tasks. As an example, the application150can perform data processing or analyzing tasks associated with a neural network. It will be appreciated that each client is not limited to having only one application, but one client may have one or more applications. Therefore, here the value of P may not equal that of Q.

To quickly and efficiently run these applications and/or save local computing resources, the client120may request the dedicated computing resource160of the server110to run these applications150. In such an embodiment, the client120may be connected via an interconnected network130to one or more servers110and hand over the application150to be run by one or more dedicated computing resources160of the server110. The interconnected network130can support different types of wired or wireless connections based on various network transmission techniques, for example, remote direct memory access (RDMA), transmission control protocol (TCP) or the like, depending on interfaces supported by the client120, the server110and/or the dedicated computing resource160.

It should be understood that the device and/or arrangement as shown inFIG. 1is provided as an example only. In other examples, the computing system100can include any suitable number of servers110and clients120. Each server110can be mounted with any suitable number of dedicated computing resources160and each client120can have a plurality of applications150to run. In addition, a scheduler140, although separately shown, can be implemented by other devices independent of the server110in the practical application, or can be implemented at least in part on one or more servers110.

To describe in a clear and concise manner, example embodiments of the present disclosure will be described in detail by mainly using the GPU kernel as an example. It is known that a GPU, as a dedicated processor, has strong computing capability due to a large number of kernels and high-bandwidth memory. In the GPU hardware architecture, one GPU usually has a large number of GPU kernels, such as 5120 or up to 10000 kernels. The GPU kernel, as a dedicated computing resource, is the most basic processing unit, which is also known as a stream processor (SP). Instructions and tasks are eventually processed on the GPU kernel. A plurality of GPU kernels simultaneously execute the instructions to implement parallel computing of the GPU. A plurality of SPs, in addition to other resources such as a register and shared memory, can compose one stream multiprocessor (SM).

However, it should be understood that a GPU is only an example of a dedicated computing resource, and shall not limit the scope of the present disclosure. Spirits and principles described herein can be applied to other dedicated computing resources, for example, computing resources in an accelerator such as an FPGA currently known or to be developed in the future, without limiting to the GPU kernel only.

With the development of cloud computing, cloud architecture based technical solutions for processing a computing task have been proposed. For example, the application120at the client150may request the computing resource160in the server110. It is noteworthy due to the complexity of a computing task, usually multiple computing resources need to be called. More details of the embodiment of the present disclosure will be described by using a neural network model-based computing task as a specific example.FIG. 2schematically shows a block diagram200of the process for processing a neural network model-based computing task210according to one technical solution. As depicted, the computing task210may be a computing task based on a neural network model, which neural network may involve multiple layers, e.g., layer 1, layer 2, layer N denoted by reference numerals212,214, . . . ,216, respectively. It will be understood that each layer among layer 1 to layer N involves a large number of parameters defining the neural network model, e.g., gradient, weight, bias and other parameters. For different layers, the data number of parameters being involved will differ significantly, e.g., the number of parameters may range between tens and millions or even larger. Therefore, it is a difficult issue regarding how to process the computing task210by multiple computing resources (e.g., the computing resources160-1to160-V) in as much of a balanced way as possible.

It will be understood that technical solutions for processing a neural network model-based computing task have been provided. In one technical solution, a mapping relationship between computing resources and layers may be built at random, and parameters for a corresponding layer may be processed by a corresponding computing resource on the basis of this mapping. In another technical solution, for one of multiple layers, status of multiple computing resources that are currently available may be detected dynamically, and a computing resource with the best status may be selected to process parameters for this layer. Since there is a great difference between workloads for processing parameters of different layers, existing technical solutions fail to effectively utilize the computing performance of multiple computing resources in the resource pool. In view of the drawbacks in the prior art, the present disclosure proposes a method for processing a computing task.

FIG. 3schematically shows a block diagram300for processing a computing task according to example embodiments of the present disclosure. As depicted, status information330of multiple computing resources160-1to160-V in a resource pool320may be obtained. Based on the status information330, the multiple computing resources160-1to160-V may be ranked to obtain a resource list332. For example, these computing resources may be ranked by status in best-to-worst order or may be ranked by status in worst-to-best order.

Where the neural network model-based computing task210is received, configuration information of multiple layers (e.g., layer 1, layer 2, . . . , layer N denoted by reference numerals212,214216respectively inFIG. 2) associated with the neural network model may be determined. Parameter data310associated with the multiple layers may be obtained on the basis of the configuration information. It will be understood that in the context of the present disclosure, the multiple layers may be all or part of layers associated with the neural network model. Based on the parameter data310, the multiple layers212,214, . . . ,216may be ranked to obtain a layer list312. For example, these layers may be ranked in increasing order or decreasing order by the number of parameters indicated by the parameter data310. Subsequently, a mapping340between a corresponding layer among the multiple layers and a corresponding computing resource among the multiple computing resources160may be determined on the basis of the resource list332and the layer list312.

FIG. 4schematically shows a flowchart of a method400for processing a computing task210according to example embodiments of the present disclosure. At block410, it is determined whether a neural network model-based computing task210is received. If yes, then the operational flow proceeds to block420. At420, parameter data of multiple layers associated with the neural network model are determined. According to example embodiments of the present disclosure, the multiple layers and associated configuration information may be determined based on the definition of the neural network model. At block430, the multiple layers are ranked based on the parameter data so as to obtain a layer list. More details of obtaining the configuration information and ranking will be described below with reference toFIG. 5.

At block440, multiple computing resources are ranked based on status information330of the multiple computing resources so as to obtain a resource list332. The status information330here may involve indicators of various aspects of the computing resources160. According to example embodiments of the present disclosure, the status information of the multiple computing resources160comprises at least any one indicator of processing capacity information, memory resource information and bandwidth resource information of the multiple computing resources160.

At block450, a mapping between a corresponding layer among the multiple layers and a corresponding computing resource among the multiple computing resources is determined based on the layer list and the resource list, the mapping indicating that one computing resource among the multiple computing resources will process parameters associated with one layer among the multiple layers.

According to example embodiments of the present disclosure, the memory resource information may comprise, for example, the size of an available storage space of a GPU. The indicator may be measured by an absolute value, for example, the memory of a GPU may comprise an 8 GB (Giga Bytes) storage space, while the memory of another GPU may comprise a 5 GB storage space. When the measurement of an absolute value is adopted, in order to conveniently compare memory resource information of the respective computing resources160, a normalization rule may be specified to unify memory resource information of the multiple computing resources160to the same standard.

For example, suppose a memory with the largest capacity among the multiple computing resources160comprises 10 GB, then memory resource information of this computing resource may be set to 1, and memory resource information of other computing resource may be determined based on a ratio of memory capacity. For example, memory resource information of a computing resource comprising a 5 GB storage space may be represented as 5 GB/10 GB=0.5. Alternatively or additionally, the indicator may further be measured by a relative value. For example, suppose a GPU comprises an 8 GB storage space among which 7 GB has been used, then at this point memory resource information of this GPU may be represented as (8 GB−7 GB)/8 GB=0.125.

According to example embodiments of the present disclosure, the processing capacity information may comprise, for example, a processing capacity indicator of a GPU, which may be measured by an absolute value, such as processing frequency, the number of processor cores, etc. When the measurement of an absolute value is adopted, in order to conveniently compare processing capacity information of the respective computing resources160, a normalization rule may be specified to unify processing capacity information of the multiple computing resources160to the same standard. For example, processing capacity information of each computing resource160may be determined based on the proportion of processing frequency of the computing resource160to the highest processing frequency. Alternatively or additionally, the indicator may further be measured by a relative value. For example, suppose a theoretical processing capacity of a GPU is 1 among which 50% of the processing capacity has been used by other computing tasks, then at this point an available processing capacity may be represented as 0.5.

According to example embodiments of the present disclosure, the bandwidth resource information may represent a communication bandwidth of a GPU, which may be measured for an absolute value, for example. When the measurement of an absolute value is adopted, in order to conveniently compare bandwidth resource information of the respective computing resources160, a normalization rule may be specified to unify bandwidth resource information of the multiple computing resources160to the same standard. For example, bandwidth resource information of each computing resource160may be determined based on the proportion of a communication bandwidth of the computing resource160to the highest communication bandwidth. Alternatively or additionally, the indicator may further be measured by a relative value. For example, suppose a theoretical bandwidth of a GPU is 4 GB/s (per second) among which 2 GB/s of the communication bandwidth has been used by other computing task, then at this point the bandwidth resource information may be represented as 2/4=0.5.

According to example embodiments of the present disclosure, for a given computing resource among the multiple computing resources160, importance of a corresponding indicator among the multiple indicators for the given computing resource may be determined based on the computing task210. For example, if it is found that the computing task210involves a higher computation load, then higher importance may be set for the processing capacity information. If it is found that the computing task210involves a large amount of data, then higher importance may be set for the memory resource information. If it is found that the computing task210involves a large amount of data communication, then higher importance may be set for the bandwidth resource. Subsequently, status information330for the given computing resource160may be determined based on importance of a corresponding indicator and the corresponding indicator. For example, the status information330may be determined from Equation 1 below:

In Equation 1, Status (i) represents status information of the ithcomputing resource in the resource pool320, ProcessingCapacity represents processing capacity information, Weightprocessingcapacity represents importance of the processing capacity information, MemoryCapacity represents memory resource information, Weightmemory capacityrepresents importance of the memory resource information, BandWidth represents bandwidth resource information, and Weightbandwidthrepresents importance of the bandwidth resource information. According to example embodiments of the present disclosure, the status information330may be represented as a vector, where each dimension in the vector represents status information of a corresponding computing resource. Still with reference toFIG. 4, at block450, a mapping between a corresponding layer among the multiple layers and a corresponding computing resource among the multiple computing resources160is determined based on the layer list and the resource list, the mapping indicating that one computing resource among the multiple computing resources160will process parameters associated with one layer among the multiple layers. In brief, if the parameter data310indicates a certain layer in the computing task210involves a higher workload, then a computing resource with better status may be selected from the resource list to process parameters associated with this layer. If the parameter data310indicate a certain layer in the computing task210involves a lower workload, then a computing resource160with ordinary and even worse status may be selected from the resource list to process parameters associated with this layer. In this way, the multiple computing resources160in the resource pool320may be utilized more effectively.

According to example embodiments of the present disclosure, for each layer, a computing resource for processing parameters associated with the layer may be selected. It is not limited here that all layers among the multiple layers are processed using the embodiments of the present disclosure, but only at least one part of the multiple layers may be processed using the embodiments of the present disclosure. For a further layer among the multiple layers, a computing resource for processing parameters associated with the further layer may be selected in another way. In this embodiment, a computing resource suitable to the number of parameters on a layer may be allocated for each layer involved in the neural network model by performing processing to each layer one by one.

According to example embodiments of the present disclosure, the corresponding number of parameters included in a corresponding layer among the multiple layers may be obtained based on the parameter data310; and the corresponding layers are ranked by the corresponding number so as to obtain the layer list312. With the example embodiments, the intensity for processing a workload required by each layer may be measured by the number of parameters included in the layer, and further a corresponding computing resource may be selected based on the workload of each layer in a simpler and more effective way.

FIG. 5shows a block diagram500for obtaining parameter data associated with a neural network according to example embodiments of the present disclosure. As depicted, reference numeral510schematically shows configuration information of a neural network model according to example embodiments. Here, the configuration information510defines multiple layers comprised in the neural network model and parameters involved in each layer. Parameter data310associated with the neural network model may be obtained by parsing the configuration information510.

According to example embodiments of the present disclosure, first statistics may be generated on the number of parameters involved in each layer, so that a computing resource may first be allocated for a layer with a larger number of parameters. For example, based on the parameter data310as shown inFIG. 5, corresponding numbers of parameters associated with at least one part of layers may be determined. In this example, by extracting values in the field Param-size in the parameter data310, the number of parameters associated with each layer may be represented as: [23232, 64, 307200, 192, 663552, 384, 1327104, 384, 884736, 256, 37748736, 4096, 16777216, 4096, 4100096, 1001]. Subsequently, the at least one part of layers are ranked based on the corresponding numbers, and a first layer is selected based on the ranking. In this embodiment, the ranking may follow a decreasing order. More details will be described below with reference toFIG. 6.

According to example embodiments of the present disclosure, for a first layer among the multiple layers, a first computing resource matching with a first number of parameters of the first layer is selected from the multiple computing resources160based on the resource list322, so as to process parameters associated with the first layer; and the mapping is determined based on the first layer and the first computing resource.

Here, a first computing resource matching with a first number of parameters of the first layer is selected from the multiple computing resources160based on the status information330of the multiple computing resources160, so as to process parameters associated with the first layer. Each layer in the layer list312may be processed one by one.FIG. 6schematically shows a block diagram600for determining a mapping between a layer and the computing resource160based on the layer list312and the resource list332according to example embodiments of the present disclosure. As depicted, the layer list312is formed by ranking various layers in decreasing order by the number of parameters, and the resource list332is formed by ranking the multiple computing resources160by the status information330in a best-to-worst order.

As shown inFIG. 6, the resource list332may be stored as a vector, where the ithdimension stores status information330of the ithcomputing resource determined with the above method. In this example, only status information for the computing resources 1, 2, 3, 4 and the like as determined with the above method is shown. It will be understood although the status information is normalized to a value range between 0 and 100 in the example ofFIG. 6, in other example the status information may further be normalized to other value range. Alternatively or additionally, the status information may be measured using an absolute value. At the header of the layer list312inFIG. 6, for a layer612involving 37748736 parameters, since the layer612involves a larger workload, the computing resource 1 with the best status may first be selected from the multiple computing resources160in the resource pool320.

According to example embodiments of the present disclosure, a first resource allocation required for processing parameters associated with the first layer is determined; the status information is updated based on the first resource allocation; and the multiple computing resources are ranked based on the updated status information so as to obtain an updated resource list. With the example embodiments, the first resource allocation that has been allocated for processing the first layer may be deducted from a corresponding computing resource so as to determine a currently available portion in the corresponding computing resource. In this way, it can be guaranteed that the status information330is updated in time, and further it can be guaranteed that the status information330can accurately reflect the latest status of the multiple computing resources160in the resource pool210. More details of the updating will be described below with reference toFIG. 7.

As shown inFIG. 6, the number of parameters involved in the layer612is 37748736. Suppose to process the layer612will occupy 50% of the resource allocation in the computing resource 1, then at this point relevant status information of the computing resource 1 in the status information330may be updated based on the resource allocation. For example, the relevant status information of the computing resource 1 may be updated as 100−100*50%=50. At this point, the relevant status information of the computing resource 1 has been updated to 50, and status information of the computing resources 2 and 3 in the existing resource list332is 90 and 70 respectively, higher than 50. Then, the resource list332may be updated, and the computing resource 1 is updated to the third place in the resource list332.FIG. 7schematically shows a block diagram700for determining a mapping between a layer and a computing resource based on the layer list and the updated resource list according to example embodiments of the present disclosure. As depicted inFIG. 7, the updated resource list is as shown by a reference numeral720.

According to example embodiments of the present disclosure, other layers may be processed in a similar way of processing the first layer. For example, regarding a second layer neighboring the first layer among the multiple layers, a second computing resource matching with a second number of parameters of the second layer is selected from the multiple computing resources160based on the updated resource list332, so as to process parameters associated with the second layer; and the mapping is determined based on the second layer and the second computing resource.

Still with reference toFIG. 7, the computing resource160may be selected for parameters of a layer710ranked as the second one. As depicted, the computing resource 2 may be selected to process parameters involved in the layer710. The number of parameters involved in the layer710is 16777216. Suppose to process the layer710will occupy 60% of the resource allocation in the computing resource 2, at this point relevant status information of the computing resource 2 in the status information330may be updated based on the resource allocation. For example, the relevant status information of the computing resource 2 may be updated as 90-100*60%=30. Subsequently, the relevant status information of the computing resource 2 as denoted by a reference numeral722may be updated to 30 (FIG. 7only shows status information “90” that has not been updated, but does not show the updated status information “30”). Like the above description, after the relevant status information of the computing resource 2 is updated from 90 to 30, a resource list720needs to re-ranked. At this point, since status information of the computing resources 3, 1 and 4 is 70, 50 and 40 respectively, more than 30, the computing resource 2 ranks the 4thin the updated status list.

According to example embodiments, the layer list312is ranked in order by the corresponding number, the resource list332is ranked in order of status information of the multiple computing resources, and selecting the second computing resource comprises: selecting a computing resource at an endpoint position in the updated resource list332as the second computing resource.

FIG. 8schematically shows a block diagram800for determining a layer and a computing resource based on the updated layer list and the updated resource list according to example embodiments of the present disclosure. The above example only describes the circumstance where the resource list332is updated. According to example embodiments of the present disclosure, the layer list312may further be updated. In the embodiments, a layer that has been processed may be deleted from the layer list (e.g., a layer at the header position in the layer list312inFIG. 7). At this point, as shown inFIG. 8, a layer and a computing resource may be selected from the header of a layer list810and a resource list respectively, and a mapping relationship may be built between the selected layer and computing resource.

In this way, for the layer list312, a processed layer may be deleted from the layer list312upon completion of each round. For the resource list332, the ranking of computing resources may be updated based on an allocated resource upon completion of each round, and a computing resource with the strongest computing capability may be arranged at the header of the resource list332. In this way, it may be determined conveniently and quickly which computing resource is used to process parameters of which layer.

According to example embodiments of the present disclosure, the number of computing resources required for processing the multiple layers may be predetermined, and a corresponding computing resource may be selected from the resource pool320with the above method. When the number of computing resources being selected reaches a desired number, no further computing resource is selected from the resource pool320, but when selecting a computing resource160for a further layer subsequently, the computing resource for processing parameters associated with the further layer in the at least one part of layers may be selected from the computing resources160that have been selected. In this way, it can be guaranteed that the total number of selected computing resources matches with the computing task.

For example, suppose it is determined that 5 computing resources need to be called for executing the computing task210. Regarding a ranked layer list shown at block312inFIG. 7, the first 5 computing resources with the best status may be selected from the resource pool320to process parameters associated with layers ranking 1stto 5th. Subsequently, for layers ranking 6thand below, a corresponding computing resource is still selected from the selected 5 computing resources.

It will be understood although description has been presented to the circumstances where the two lists are ranked in decreasing order of the number of parameters in a layer and in decreasing order of the computing capability of a computing resource, in other example embodiments the two lists may further be ranked in increasing order, or one list is ranked in increasing order while the other list is ranked in decreasing order.

As examples of the method according to the present disclosure have been described in detail with reference toFIGS. 2 to 8, the method may be implemented in an apparatus. According to example embodiments of the present disclosure, provided is an apparatus for processing a computing task. The apparatus comprises: a determining module configured to determine parameter data of multiple layers associated with a neural network model in response to receiving a computing task that is based on the neural network model; a layer list determining module configured to rank at least one part of the multiple layers on the basis of the parameter data so as to obtain a layer list; a resource list determining module configured to rank multiple computing resources on the basis of status information of the multiple computing resources so as to obtain a resource list; and a mapping determining module configured to determine a mapping between a corresponding layer among the multiple layers and a corresponding computing resource among the multiple computing resources on the basis of the layer list and the resource list, where the mapping indicates one computing resource among the multiple computing resources is to process parameters associated with one layer among the multiple layers.

FIG. 9schematically shows a block diagram of an apparatus900for processing a computing task according to example embodiments of the present disclosure. As depicted, the apparatus900includes a central process unit (CPU)901, which can execute various suitable actions and processing based on the computer program instructions stored in the read-only memory (ROM)902or computer program instructions loaded in the random-access memory (RAM)903from a storage unit908. The RAM903can also store all kinds of programs and data required by the operations of the apparatus900. CPU901, ROM902and RAM903are connected to each other via a bus904. The input/output (I/O) interface905is also connected to the bus904.

A plurality of components in the apparatus900is connected to the I/O interface905, including: an input unit906, such as keyboard, mouse and the like; an output unit907, e.g., various kinds of display and loudspeakers etc.; a storage unit908, such as magnetic disk and optical disk etc.; and a communication unit909, such as network card, modem, wireless transceiver and the like. The communication unit909allows the apparatus900to exchange information/data with other devices via the computer network, such as Internet, and/or various telecommunication networks.

The above described each procedure and processing, such as the method400, can also be executed by the processing unit901. For example, in some embodiments, the method400can be implemented as a computer software program tangibly included in the machine-readable medium, e.g., the storage unit908. In some embodiments, the computer program can be partially or fully loaded and/or mounted to the apparatus900via ROM902and/or the communication unit909. When the computer program is loaded to the RAM903and executed by the CPU901, one or more steps of the above described method400can be implemented. Alternatively, in other embodiments, the CPU901also can be configured in other suitable manners to realize the above procedure/method.

According to example embodiments of the present disclosure, there is provided an apparatus for processing a computing task, comprising: at least one processor; a volatile memory; and a memory coupled to the at least one processor, the memory having instructions stored thereon, the instructions, when executed by the at least one processor, causing the apparatus to perform acts. The acts include: determining parameter data of multiple layers associated with a neural network model in response to receiving a computing task based on the neural network model; ranking at least one part of the multiple layers on the basis of the parameter data so as to obtain a layer list; ranking multiple computing resources on the basis of status information of the multiple computing resources so as to obtain a resource list; and determining a mapping between a corresponding layer among the multiple layers and a corresponding computing resource among the multiple computing resources on the basis of the layer list and the resource list, where the mapping indicates one computing resource among the multiple computing resources is to process parameters associated with one layer among the multiple layers.

According to example embodiments of the present disclosure, obtaining the layer list comprises: obtaining a corresponding number of parameters comprised by a corresponding layer among the multiple layers on the basis of the parameter data; and ranking the multiple layers on the basis of the corresponding number so as to obtain the layer list.

According to example embodiments of the present disclosure, determining the mapping comprises: for a first layer among the multiple layers, selecting, on the basis of the resource list, a first computing resource matching with a first number of parameters of the first layer from the multiple computing resources so as to process parameters associated with the first layer; and determining the mapping on the basis of the first layer and the first computing resource.

According to example embodiments of the present disclosure, the acts further comprise: determining a first resource allocation required for processing parameters associated with the first layer; updating the status information on the basis of the first resource allocation; and ranking the multiple computing resources on the basis of the updated status information so as to obtain an updated resource list.

According to example embodiments of the present disclosure, the acts further comprise: for a second layer neighboring the first layer among the multiple layers; selecting, on the basis of the updated resource list, a second computing resource matching with a second number of parameters of the second layer from the multiple computing resources so as to process parameters associated with the second layer; and determining the mapping on the basis of the second layer and the second computing resource.

According to example embodiments of the present disclosure, the layer list is ranked in order of the corresponding numbers, the resource list is ranked in order of status information of the multiple computing resources, and selecting the second computing resource comprises: selecting a computing resource at an endpoint position of the updated resource list as the second computing resource.

According to example embodiments of the present disclosure, determining status information of the multiple computing resources comprises: monitoring resource information of the multiple computing resources; and determining status information of the multiple computing resources on the basis of the resource information.

According to example embodiments of the present disclosure, resource information of the multiple computing resources comprises at least any one indicator of: processing capacity information, memory resource information and bandwidth resource information of a corresponding computing resource among the multiple computing resources.

According to example embodiments of the present disclosure, determining status information of the multiple computing resources on the basis of the resource information further comprises: for a given computing resource among the multiple computing resources, determining importance of a corresponding indicator among the multiple indicators for the given computing resource on the basis of the computing task; and determining status information for the given computing resource on the basis of the importance of the corresponding indicator and the corresponding indicator.

According to example embodiments of the present disclosure, the multiple computing resources are multiple graphics processing units.

According to example embodiments of the present disclosure, there is provided a computer program product. The computer program product is tangibly stored on a computer readable medium and comprises machine executable instructions which, when executed, cause the machine to implement the method according to the present disclosure.

According to example embodiments of the present disclosure, there is provided a computer readable medium. The computer readable medium is stored with machine executable instructions thereon, the machine executable instructions, when executed by at least one processor, causing the at least one processor to implement the method according to the present disclosure. The present disclosure can be method, device, system and/or computer program product. The computer program product can include a computer-readable storage medium, on which the computer-readable program instructions for executing various aspects of the present disclosure are loaded.

The computer-readable storage medium can be a tangible apparatus that maintains and stores instructions utilized by the instruction executing apparatuses. The computer-readable storage medium can be, but not limited to, such as electrical storage device, magnetic storage device, optical storage device, electromagnetic storage device, semiconductor storage device or any appropriate combinations of the above. More concrete examples of the computer-readable storage medium (non-exhaustive list) include: portable computer disk, hard disk, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash), static random-access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical coding devices, punched card stored with instructions thereon, or a projection in a slot, and any appropriate combinations of the above. The computer-readable storage medium utilized here is not interpreted as transient signals per se, such as radio waves or freely propagated electromagnetic waves, electromagnetic waves propagated via waveguide or other transmission media (such as optical pulses via fiber-optic cables), or electric signals propagated via electric wires.

The described computer-readable program instruction can be downloaded from the computer-readable storage medium to each computing/processing device, or to an external computer or external storage via Internet, local area network, wide area network and/or wireless network. The network can include copper-transmitted cable, optical fiber transmission, wireless transmission, router, firewall, switch, network gate computer and/or edge server. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium of each computing/processing device.

The computer program instructions for executing operations of the present disclosure can be assembly instructions, instructions of instruction set architecture (ISA), machine instructions, machine-related instructions, microcodes, firmware instructions, state setting data, or source codes or target codes written in any combinations of one or more programming languages, where the programming languages consist of object-oriented programming languages, e.g., Smalltalk, C++ and so on, and traditional procedural programming languages, such as “C” language or similar programming languages. The computer-readable program instructions can be implemented fully on the user computer, partially on the user computer, as an independent software package, partially on the user computer and partially on the remote computer, or completely on the remote computer or server. In the case where remote computer is involved, the remote computer can be connected to the user computer via any type of networks, including local area network (LAN) and wide area network (WAN), or to the external computer (e.g., connected via Internet using the Internet service provider). In some embodiments, state information of the computer-readable program instructions is used to customize an electronic circuit, e.g., programmable logic circuit, field programmable gate array (FPGA) or programmable logic array (PLA). The electronic circuit can execute computer-readable program instructions to implement various aspects of the present disclosure.

Various aspects of the present disclosure are described here with reference to flow chart and/or block diagram of method, apparatus (system) and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flow chart and/or block diagram and the combination of various blocks in the flow chart and/or block diagram can be implemented by computer-readable program instructions.

The computer-readable program instructions can be provided to the processing unit of general-purpose computer, dedicated computer or other programmable data processing apparatuses to manufacture a machine, such that the instructions that, when executed by the processing unit of the computer or other programmable data processing apparatuses, generate an apparatus for implementing functions/actions stipulated in one or more blocks in the flow chart and/or block diagram. The computer-readable program instructions can also be stored in the computer-readable storage medium and cause the computer, programmable data processing apparatus and/or other devices to work in a particular manner, such that the computer-readable medium stored with instructions contains an article of manufacture, including instructions for implementing various aspects of the functions/actions stipulated in one or more blocks of the flow chart and/or block diagram.

The computer-readable program instructions can also be loaded into computer, other programmable data processing apparatuses or other devices, so as to execute a series of operation steps on the computer, other programmable data processing apparatuses or other devices to generate a computer-implemented procedure. Therefore, the instructions executed on the computer, other programmable data processing apparatuses or other devices implement functions/actions stipulated in one or more blocks of the flow chart and/or block diagram.

The flow chart and block diagram in the drawings illustrate system architecture, functions and operations that may be implemented by system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each block in the flow chart or block diagram can represent a module, a part of program segment or code, wherein the module and the part of program segment or code include one or more executable instructions for performing stipulated logic functions. In some alternative embodiments, it should be noted that the functions indicated in the block can also take place in an order different from the one indicated in the drawings. For example, two successive blocks can be in fact executed in parallel or sometimes in a reverse order dependent on the involved functions. It should also be noted that each block in the block diagram and/or flow chart and combinations of the blocks in the block diagram and/or flow chart can be implemented by a hardware-based system exclusive for executing stipulated functions or actions, or by a combination of dedicated hardware and computer instructions.

Various embodiments of the present disclosure have been described above and the above description is only exemplary rather than exhaustive and is not limited to the embodiments of the present disclosure. Many modifications and alterations, without deviating from the scope and spirit of the explained various embodiments, are contemplated. The selection of terms in the text aims to best explain principles and actual applications of each embodiment and technical improvements made in the market by each embodiment, or enable other ordinary skilled in the art to understand embodiments of the present disclosure.