Patent Publication Number: US-2023145437-A1

Title: Execution prediction for compute clusters with multiple cores

Description:
BACKGROUND 
     Portions of some workloads and applications may be processed in parallel and independent of one another, while other workloads and applications may be processed sequentially. Large-scale machine learning applications may be distributed among numerous compute nodes for parallel processing, while single threaded applications may not be easily divisible. 
     Some computer systems may include a single core for sequential processing, while others computer systems may include multiple cores for parallel processing. The time required to execute a given application may depend on the way the application is written, the processor speed of the executing computer system, and the number of cores in the computing system. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The written disclosure herein describes illustrative examples that are nonlimiting and non-exhaustive. Reference is made to certain of such illustrative examples that are depicted in the figures described below. 
         FIG.  1    illustrates an example block diagram of a workload orchestration system to orchestrate machine learning workloads among heterogeneous compute clusters with different numbers of cores. 
         FIG.  2 A  illustrates a specific example of cluster management of an artificial intelligence application via a workload orchestration system in a heterogeneous network of on-premises compute devices. 
         FIG.  2 B  illustrates a block diagram of example subsystems of the artificial intelligence scheduler of  FIG.  2 A . 
         FIG.  3    illustrates a block diagram of an example workload orchestration system implemented in a computer system. 
         FIG.  4    illustrates a flowchart of a method to allocate workloads of a machine learning application to heterogeneous compute clusters based on discovered resources, estimated thread scaling ratios, and estimated resource demands. 
         FIG.  5 A  illustrates an example block diagram of a multithreaded application with full upscaling with additional cores. 
         FIG.  5 B  illustrates an example block diagram of a multithreaded application without any upscaling with additional cores. 
         FIG.  6 A  illustrates an example block diagram of a multithreaded application that only uses some cores and does not upscale with additional cores. 
         FIG.  6 B  illustrates an example block diagram of a multithreaded application that only uses some cores and upscales with a constant number of reserved cores. 
         FIG.  6 C  illustrates an example block diagram of a multithreaded application that only uses some cores and upscales with a constant ratio of utilized cores to reserved cores. 
         FIG.  7    illustrates an example block diagram of a multithreaded application that downscales with all cores utilized. 
         FIG.  8    illustrates an example block diagram of a multithreaded application that only uses some cores and downscales with all cores utilized. 
         FIG.  9    illustrates an example block diagram of a multithreaded application that only uses some cores and downscales with a constant number of reserved cores. 
         FIG.  10    illustrates an example block diagram of a multithreaded application that only uses some cores and downscales with a constant ratio of utilized cores to reserved cores. 
         FIG.  11    illustrates a flow chart of an example approach to calculate minimum and maximum thread scaling ratios. 
     
    
    
     DETAILED DESCRIPTION 
     As used herein, a compute cluster may be defined in terms of the collective available compute resources of multiple compute nodes, each of which may have a different number of cores for executing multithreaded applications. A compute cluster includes at least one compute node but may include multiple compute nodes. In a cloud computing environment, the various compute nodes can be selected to have homogeneous or near homogeneous compute resources. Some tasks, such as large-scale data processing, can be divided into any number of increasingly smaller and simpler workflows for parallel processing based on the number of discrete compute nodes and available compute resources. In contrast, single-threaded applications may benefit more from increased process speed instead of an increased number of processing cores. 
     As used herein, the term workload includes specific compute jobs or tasks and an associated or underlying dataset. For example, a workload may include the execution of a particular application or portion of an application. Each compute job or task may include instructions for actions to be taken with respect to the associated dataset. Thus, each compute job or task may include the execution of an application with respect to an associated dataset. 
     In some instances, it may be desirable to execute artificial intelligence or other machine learning applications using on-premises compute devices and/or edge devices. Unlike in a cloud computing environment, on-premises compute devices and edge devices may each have widely different compute resources. Thus, while a cloud computing environment can be modeled as a plurality of distributed compute clusters with homogeneous compute resources, on-premises and/or edge device networks are more accurately described as a plurality of distributed compute clusters with heterogeneous compute resources. 
     Examples of compute resources include, without limitation, central processing unit (CPU) resources, GPU resources, volatile memory resources, network communication capabilities, persistent storage resources, and the like. For example, compute resources of homogeneous or heterogeneous compute resources may be quantified and compared in terms of a function of one or more of available volatile memory, available persistent storage, processor clock speed, floating-point operations per second (“flops”), number of compute cores or processors, memory speed, network bandwidth, and the like. The assignment of machine learning workloads in a heterogeneous compute network can result in underutilization of some compute clusters and/or asynchronous learning parameter exchanges between compute clusters. 
     Some applications may be executable or at least partially executable by multicore processors in parallel, while others may not. For instance, a fully multithreaded application may be executed by a compute node having four cores much faster than if the same application is executed by a compute node having a single core. The performance difference between various compute node configurations may depend partially on the number of cores in each compute node, but only to the extent that the application being executed can take advantage of additional cores. Thus, a compute node with 12 cores may not necessarily execute an application faster than an otherwise similar compute node that only has 6 cores. 
     As used herein, the term “core” or “cores” may refer to physical or virtual cores (threads) implemented by processors and/or virtual machines. In many examples, the evaluation of what constitutes a “core” may be based on what is exposed to the operating system or operating systems of the machine (physical or virtual) on which the application is executed. 
     The presently described systems and methods may estimate the amount of time it will take to execute an application or other task on a given compute node. For example, some tasks or applications may not scale at all when additional cores are available. Such tasks or applications may have a thread scaling ratio of 1, indicating that the presence of additional cores will not result in a faster execution time of the task or application. In contrast, some tasks or applications may be fully scalable to utilize the full number of cores available. As described herein, some tasks or applications may partially scale to include a constant number or fixed ratio of utilized cores and reserved cores. The hyperparameters associated with each workload may be defined or adjusted to compensate for the different processing speeds and number of cores in each compute cluster to which it is assigned. 
     In one example, a workload orchestration system includes a discovery subsystem to identify the specific compute resources of each compute cluster in a network of computer clusters (e.g., heterogeneous compute clusters). The system may determine the number of cores available in the various compute clusters. A manifest subsystem may receive or generate a manifest that describes the resource demands for each workload associated with an application, such as a single-threaded application, an artificial intelligence application, a machine learning application, or other application. The system may estimate an execution time for the various workloads based on a scaling ratio associated with the workload and the number of cores in each compute cluster. A placement subsystem may assign each workload to one of the compute clusters by matching or mapping the resource demands of each workload and the compute resources of each compute cluster. The placement subsystem may assign each workload in further consideration of affinity, anti-affinity, and/or co-locality constraints and policies. An adaptive modeling subsystem may dynamically define (e.g., establish, set, or modify) hyperparameters for each workload as a function of the identified compute resources of the compute cluster to which each respective workload is assigned and the dataset. 
     For example, a discovery subsystem or module may determine a measured execution time for a first compute cluster with a first number of cores to execute a task. A manifest subsystem or module may identify resource demands for each workload. A scaling subsystem or module may calculate thread scaling ratios (e.g., minimum and maximum thread scaling ratios) for the application indicative of the scalability of the application across the compute clusters having variations in the number of cores. For example, the thread scaling ratios may be calculated based on a measured execution time for a first compute cluster with a first number of cores to execute the application. 
     A placement subsystem may assign each workload to one of the compute clusters by matching the identified resource demands of each respective workload using the calculated thread scaling ratios for the application and the identified compute resources of each compute cluster (including the number of cores in each respective compute cluster). 
     An adaptive modeling subsystem may define hyperparameters of each workload based, at least in part, on the calculated thread scaling ratio and the number of cores in each respective compute cluster to which each respective workload is assigned. 
     Various modules, systems, and subsystems are described herein as implementing functions and/or as performing various actions. In many instances, modules, systems, and subsystems may be divided into sub-modules, subsystems, or even as sub-portions of subsystems. Modules, systems, and subsystems may be implemented in hardware or as processor-executable instructions stored in, for example, a non-transitory computer-readable medium. Some examples may be embodied as a computer program product, including a non-transitory computer and/or machine-readable medium having stored thereon instructions that may be used to program a computer (or another electronic device) to perform processes described herein. 
     The examples of the disclosure may be further understood by reference to the drawings. It is readily understood that the components of the disclosed examples, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the examples of the systems and methods of the disclosure is not intended to limit the scope of the disclosure, as claimed, but is merely representative of possible examples of the disclosure. In addition, the elements of a method do not necessarily need to be executed in any specific order, or even sequentially, nor do the elements need to be executed only once, unless otherwise specified. In some cases, well-known features, structures, or operations are not shown or described in detail. 
       FIG.  1    illustrates an example block diagram  100  of a workload orchestration system  150  to orchestrate machine learning workloads among heterogeneous compute clusters  130 . An application service  105 , such as an artificial intelligence application or other machine learning application, may communicate with the workload orchestration system  150  for distributed execution. The workload orchestration system  150  includes a discovery subsystem  152 , an adaptive modeling subsystem  154 , a placement subsystem  156 , and a manifest subsystem  158 . 
     The discovery subsystem  152  identifies or otherwise profiles the individual compute resources of each of the compute clusters A, B, C, and Z ( 131 ,  132 ,  133 , and  139 ) that are part of the heterogeneous compute clusters  130 . The network of heterogeneous compute clusters  130  may include any number of compute clusters, but only four are illustrated in the example. The compute clusters A, B, C, and Z ( 131 ,  132 ,  133 , and  139 ) may have various combinations of compute resources, graphically illustrated by bar charts in the lower right corner of each compute cluster  131 - 139 . For instance, each compute cluster  131 - 139  may have different CPU resources, GPU resources, volatile memory resources, network bandwidth, network latency characteristics, persistent storage resources, or the like. Each compute cluster  131 - 139  may have a different number of cores, illustrated as M, N, P, and X integer values that differ from one another. 
     In some examples, the discovery subsystem  152  may identify the total (e.g., theoretical) compute resources of each compute cluster  131 - 139 , currently available compute resources of each compute cluster  131 - 139 , and/or expected or scheduled availability of compute resources of each compute cluster  131 - 139  during a future time window. Some compute clusters may have the same compute resources as one another, while others may be heterogeneous. 
     A scaling subsystem  153  may calculate thread scaling ratios for the application indicative of the scalability of each of a plurality of workloads for the various compute clusters. The thread scaling ratios may be calculated based on a measured execution time for a first computer cluster with a first number of cores. For example, the thread scaling ratios may be calculated based on a measured execution time for a first compute cluster with a first number of cores to execute the application. The system may calculate minimum and maximum estimated execution times for each respective compute cluster that has a different number of cores based on the thread scaling ratios. As described herein, the minimum and maximum thread scaling ratios may, in some instances, be the same value. 
     The manifest subsystem  158  may maintain a manifest that describes or specifies the resource demands for each of a plurality of workloads that implement the application service  105 . In some examples, the manifest subsystem  158  divides the application service  105  into the plurality of workloads based on the compute resources of the compute clusters  131 - 139  identified by the discovery subsystem  152 . For example, the workloads may be intentionally selected to have resource demands that correspond to the identified compute resources of the compute clusters  131 - 139 . In other embodiments, the manifest subsystem  158  may receive a manifest that describes the resource demands of predefined workloads that are, for example, specified by the application service  105  independent of the identified compute resources of the compute clusters  131 - 139 . 
     A placement subsystem  156  allocates each workload to one of the compute clusters  131 - 139 . For example, the placement subsystem  156  may match workloads of the application service  105  as defined in the manifest subsystem  158  with compute clusters  131 - 139  by matching the resource demands of each respective workload with the identified compute resources of each of the compute clusters  131 - 139 . The placement subsystem  156  may allocate workloads with compute clusters  131 - 139  based, at least in part, on the number of cores in each compute cluster and the calculated thread scaling ratio. 
     In some examples, the placement subsystem  156  implements resources-aware workload scheduling in which the resource demands comprise metrics corresponding to the identified compute resources of the compute clusters  131 - 139  (e.g., CPU resources, GPU resources, vCPUs, volatile memory resources, network bandwidth, network latency characteristics, persistent storage resources, or the like). In other examples, the placement system  156  additionally implements models-aware workload scheduling in which the allocation of workloads to the compute clusters  131 - 139  is further based on co-locality, anti-affinity and/or affinity constraints and policies of workloads. 
     In some examples, the placement subsystem  156  may relax or ignore locality constraints based on the available compute resources. The resulting fragmentation may be resolved through migration and defragmentation, as described herein. That is, while the placement subsystem  156  may implement resources-aware and models-aware workload assignments, sometimes the models-aware constraints may be relaxed or even ignored based on limitations of the available compute resources. The workload orchestration system  150  may migrate workloads from one compute cluster  131 - 139  to another compute cluster  131 - 139  after partial or complete execution to satisfy affinity and/or co-locality constraints. The workload orchestration system  150  may implement a checkpoint approach that allows for the restoration of machine learning models to avoid rerunning, for example, training from the beginning. 
     In some examples, the placement subsystem  156  may assign each workload to one of the compute clusters based on a matching of (i) the identified resource demands of each respective workload, (ii) the calculated thread scaling ratios for the application, and (iii) the identified compute resources of each compute cluster, including the number of cores in each respective compute cluster. 
     An adaptive modeling subsystem  154  sets, adjusts or otherwise defines the hyperparameters of each workload as a function of the identified compute resources of the compute cluster  131 - 139  to which each respective workload is assigned. The hyperparameters of a workload assigned to the compute cluster A  131  may be different than the hyperparameters of a workload assigned to the compute cluster B  132  due to the different compute resources of compute clusters A and B ( 131  and  132 , respectively). For instance, different batch sizes, number of epochs, learning rates, and/or other hyperparameters may be assigned to the workload(s) assigned to compute cluster A  131  than to the workload(s) assigned to compute clusters B-Z ( 132 - 139 ). The system may reduce straggler issues by dynamically modifying the hyperparameters of the workloads assigned to the various heterogeneous compute clusters  131 - 139  to synchronize or more closely synchronize execution of the workloads. 
       FIG.  2 A  illustrates a specific example of cluster management of an artificial intelligence application  204  via a workload orchestration system  200  comprising a manifest  202 , an artificial intelligence scheduler  206 , an execution estimator  207 , a cluster manager  210 , and a registry  214 . The workload orchestration system  200  may implement cluster management in the heterogeneous network of on-premises compute devices  212 . The manifest  202  may specify the resource demands for each of a plurality of workloads associated with the artificial intelligence application  204 . The cluster manager  210 , messaging queue  208 , execution estimator  207 , and the artificial intelligence scheduler  206  may be components of a placement subsystem that allocates each workload of the artificial intelligence application  204  to one of the compute devices in the network of on-premises compute devices  212  by, for example, matching the resource demands of each respective workload with identified compute resources of each respective compute device. 
     The registry  214  may be maintained by a discovery subsystem, as previously described. In some examples, the registry  214  may be a dynamic database identifying the compute resources of each of the compute devices in the network of on-premises compute devices  212 . The workload orchestration system  200  may also set, adjust, or otherwise define the hyperparameters of each assigned workload as a function of the identified compute resources of the compute devices in the network of on-premises compute devices  212 . The discovery subsystem may acquire or reserve compute resources as they become available. Accordingly, since compute resource availability is dynamic, the workload orchestration system  200  may use a hysteresis margin to acquire resources with appropriate locality and/or affinity to meet the corresponding resource demands of the workload(s). 
     The workload orchestration system  200  may assign workloads to the compute devices in the network of on-premises compute devices  212  in a resources-aware manner. That is, the workload orchestration system  200  may assign workloads to the compute devices by matching or mapping the compute resources of each compute device with the resource demands of each workload. In some examples, the workload orchestration system  200  may also assign the workloads to the compute devices in a models-aware manner. That is, the workload orchestration system  200  may assign the workloads to the compute devices in further consideration co-locality constraints, anti-affinity, and/or affinity constraints and policies of the workloads. 
     The workload orchestration system  200  may isolate workloads (e.g., machine learning training and interference jobs) of the artificial intelligence application  201 . The workload orchestration system  200  may assign workloads to the compute devices in the network of on-premises compute devices  212  based at least in part on an evaluation of network communication costs associated with assigning workloads to distributed computing devices having the computational benefits of increased processor speeds and/or and increased number of cores. 
     In some examples, the workload orchestration system  200  may implement preemption policies for the compute resources of the compute devices in the network of on-premises compute devices  212  to enforce resource sharing (e.g., the shared file system  216 ). The preemption policies can be set to avoid using the full capacity of any one compute device and/or mitigate head-of-line issues. In some examples, if higher priority jobs arrive in the message queue  208  while there are lower priority jobs running, the system  200  may free up compute resources of the compute devices in the network of on-premises compute devices  212  to focus on the higher priority jobs. For example, the lower priority jobs may be temporarily suspended while the higher priority jobs are executed. 
     The artificial intelligence scheduler  206  may include an execution estimator  207  to calculate minimum and maximum estimated execution times for a compute cluster with a number of cores, N, to execute a task or application. In various examples, the execution estimator may estimate the minimum and maximum execution times for the N-core compute cluster based on a measured execution time on a different compute cluster with kN cores, where k is any number greater than 0. In some instances, the measured compute cluster may have more or fewer cores than the compute cluster for which the execution time is being estimated. 
     A minimum scaling ratio for the compute cluster executing the task or application may be calculated as one (i.e., unity or 1) when N is equal to or exceeds kN. When the number of cores N is less than the number of cores kN, the minimum scaling ratio is equal to the N/kN. In either case, the maximum scaling ratio may be equal to N/kN. As described herein, further refinement in the minimum and maximum scaling ratios may be calculated based on a fixed number or fixed ratio of reserved cores. 
       FIG.  2 B  illustrates a block diagram of example subsystems  230 - 238  of the artificial intelligence scheduler  206  of  FIG.  2 A , which includes an integrated scaling estimator subsystem  231 . The manifest  202 , cluster manager  210 , network of on-premises compute devices  212 , and register  214  are described above in the context of  FIG.  2 A . In the illustrated example, the artificial intelligence scheduler  206  includes a job service subsystem  230 , the integrated scaling estimator subsystem  231 , a resource manager subsystem  232 , an artificial intelligence or machine learning model adjuster subsystem  234 , a placement engine subsystem  236 , and a profiler engine subsystem  238 . The various subsystems  230 - 238  may be part of the discovery, adaptive modeling, and placement subsystems described herein (e.g., in conjunction with  FIG.  1   ). 
       FIG.  3    illustrates a block diagram of an example workload orchestration system  300  implemented in a computer system. As illustrated, a workload orchestration system  300  may include a processor  330 , memory  340 , and a network interface  350  connected to a computer-readable storage medium  370  via a communication bus  320 . The processor  330  may execute or otherwise process instructions stored in the computer-readable storage medium  370 . The instructions stored in the computer-readable storage medium  370  include operational modules  380 - 386  to implement the subsystems described herein. 
     For example, a discovery subsystem module  380  may discover the compute resources of each compute cluster of a plurality of compute clusters. At least some of the compute clusters may have differing compute resources, including differing numbers of cores. An adaptive modeling subsystem module  382  may associate hyperparameters with each workload associated with an application (such as an artificial intelligence or other machine learning application) based at least in part on the compute resources of the compute cluster to which each respective workload is assigned. 
     A placement subsystem module  384  may assign each workload to a compute cluster that has compute resources corresponding to the resource demands of each respective workload. The manifest subsystem module  386  may identify resource demands of predefined workloads of an application. Alternatively, the manifest subsystem module  386  may divide an application, such as a machine learning application, into a plurality of workloads with each workload having resource demands corresponding to the compute resources of the compute clusters. 
     The specific arrangement of subsystems and modules may be modified from the specific examples illustrated. Accordingly, any of a wide variety of computing systems, electronics, computer-executable instructions, network configurations and the like may be adapted to implement the concepts described herein. 
       FIG.  4    illustrates a flowchart  400  of a method to allocate workloads of an application, such as an artificial intelligence application, to heterogeneous compute clusters based on discovered compute resources of individual compute clusters and estimated workload resource demands. The system may identify, at  405 , available compute resources of each compute cluster in a network of compute clusters with heterogeneous compute resources, including different numbers of cores. The system may identify resource demands of predefined workloads of an application. The system may calculate, at  407 , thread scaling ratios for an application for each of the compute clusters. Alternatively, the system may divide, at  410 , an application, such as a machine learning application, into a plurality of workloads, with each workload having resource demands corresponding to the compute resources of the compute clusters. 
     The system may assign, at  415 , each workload to one of the compute clusters. For example, each workload may be assigned to one of the compute clusters based on a matching between the resource demands of each workload and available compute resources of the respective compute cluster to which each workload is assigned. The system may dynamically define, at  420 , hyperparameters for each workload as a function of the identified compute resources of each respective compute cluster and the dataset. 
     When predicting the execution time of a workload with traces collected during a run in a processor with fewer cores than the processor of a prediction target, the performance of the application may be impacted by the fact that the application may or may not upscale. When multithreaded applications are executed in processors with different numbers of cores, their behavior may be unpredictable. The present system may estimate execution times and/or thread scaling ratios for a given application based on information from one or more tests or sample executions of the application in compute nodes having different numbers of cores. 
     As illustrated and described in conjunction with  FIGS.  5 A- 10   , the application may scale to utilize up to a fixed maximum number of cores, scale to utilize any number of available cores, scale to utilize all cores except a fixed number of reserved cores, scale to utilize a fixed ratio of utilized cores and reserved cores. The scaling may be defined in terms of upscaling or downscaling, as described herein. 
     The system may utilize a known thread scaling ratio as a data point for assigning workloads between various compute clusters. The prediction accuracy of the execution time by each compute cluster is improved by using bounded thread scaling ratios. Even outside the context of dividing machine learning tasks among different compute clusters, the presently described systems and methods may be used to make decisions and provide suggestions for which machine or machines to utilize to execute a given application. As a specific example, a user may plan to execute an application on a primary machine that has 4 cores. The system may evaluate minimum and maximum scaling ratios calculated for the application based on measured execution times on other machines having any number of cores. The system may suggest that the user utilize a machine with more cores (e.g., 16 cores) based on a determination that the application would be able to fully utilize (or at least partially utilize) the additional cores. 
       FIG.  5 A  illustrates an example block diagram of a multithreaded application with full upscaling with additional cores. As illustrated, a multithreaded application may be executed on all four cores of the P1 compute node  500 . The system may evaluate (e.g., measure the execution time) the thread scalability of the application by executing the application on a P2 compute node  510  with 6 cores and a P3 compute node  520  with 8 cores. Based on the number of cores utilized by the multithreaded application in each of the compute nodes  500 ,  510 , and  520 . The system may estimate minimum and maximum scaling ratios based on the evaluation. 
     In the illustrated example, the system may determine that the application will fully scale upward to utilize every available core. Thus, the minimum and maximum thread scaling ratios may be equal to one another and based on the ratio of cores in the compute node for which an execution time is being estimated relative to the number of cores in the measured compute node (or one of the measured compute nodes). 
       FIG.  5 B  illustrates an example block diagram of a multithreaded application without any upscaling with additional cores. In the illustrated example, the multithreaded application executed on the P1 compute node  500  utilizes all 4 cores. However, when the multithreaded application is executed on the P2 compute node  530  with 6 cores and on the P3 compute node  540  with 8 cores, only 4 cores are utilized. The system may determine that the application does not upscale with additional core availability. 
       FIG.  6 A  illustrates an example block diagram of a multithreaded application that only uses some cores and does not upscale with additional cores. The multithreaded application only uses 2 of the 4 cores on the P1 compute node  600 . The application may not scale, and so the P2 compute node  610  and the P3 compute note  615  may only see the utilization of 2 cores, even though each of the P2 compute node  610  and the P3 Compute node  615  have 6 and 8 cores, respectively. 
       FIG.  6 B  illustrates an example block diagram of the multithreaded application that only uses some cores and upscales with a constant number of reserved cores. P1 compute node  600  shows 2 cores being utilized to execute the application with two cores “reserved” (e.g., unused or utilized for purposes other than execution of the application). As illustrated, the example application scales with 2 reserved cores, regardless of the number of available cores in a given compute node. Accordingly, the P2 compute node  620  illustrates 4 cores being utilized to execute the application, and the P3 compute node  625  illustrates 6 cores being utilized to execute the application. In each case, 2 cores are reserved as either unused or for execution of instructions other than those of the application. 
       FIG.  6 C  illustrates an example block diagram of a multithreaded application that only uses some cores and upscales with a constant ratio of utilized cores to reserved cores. Again, the P1 compute node  600  shows 2 cores utilized to execute the application and 2 cores that are unused. In this example, the application scales with a constant ratio of reserved cores. The P1 compute node  600  executes with 50% of the cores executing the application and 50% of the cores reserved. Accordingly, the P2 compute node  630  illustrates 3 cores utilized and 3 cores reserved. Similarly, the P3 compute node  635  illustrates 4 cores utilized and 4 cores reserved. 
       FIG.  7    illustrates an example block diagram of a multithreaded application that downscales with all cores utilized. In this example, the Application downscales to continue utilizing all available cores. The P1 compute node  700  utilizes all 6 cores and, when downscaled to the 4-core P2 compute node  710 , uses the available 4 cores. Based on the available data, the system may determine that the downscaled execution of the application on the 4-core P2 compute node  710 , or on a different compute node with fewer than 6 cores, will utilize all available cores. 
       FIG.  8    illustrates an example block diagram of a multithreaded application that only uses some cores and downscales with all cores utilized. The P1 compute node  800  utilizes 5 of the 6 cores to execute the application. When downscaled for execution on the 4-core P2 compute node  810 , all 4 cores are utilized for execution of the application. 
       FIG.  9    illustrates an example block diagram of a multithreaded application that only uses some cores and downscales with a constant number of reserved cores. As illustrated, the P1 compute node  900  utilizes 4 of the 6 cores to execute the application, with 2 cores reserved as unused or for other purposes. The number of reserved cores may be constant. Accordingly, the P2 compute node  910  retains the constant number of 2 reserved cores, leaving 2 cores available for execution of the application. 
       FIG.  10    illustrates an example block diagram of a multithreaded application that only uses some cores and downscales with a constant ratio of utilized cores to reserved cores. In the illustrated example, the P1 compute node  1000  utilizes 75% of the available cores for the execution of the application and reserves 25% of the cores. Accordingly, when downscaled for execution on the 4-core P2 compute node  1010 , 75% of the cores (3) are utilized to execute the application, and 25% (1) are retained or reserved for other purposes. 
       FIG.  11    illustrates a flow chart of an example approach  1100  to calculate minimum and maximum thread scaling ratios when an exact thread scaling ratio is not known. The minimum and maximum thread scaling ratios can provide guidance (e.g., another data point) for calculating minimum and maximum estimated execution times for an application on a compute node and/or for assigning each workload of a plurality of workloads to different compute clusters. 
     As illustrated, the start, at  1101 , of the analysis begins with a comparison of the number of cores, CoresP1, in a first compute node and the number of cores, CoresP2, in a second compute node. The thread scaling ratio calculated via the approach  1100  represents the relative execution times of the first compute node and the second compute node, depending on how the application scales with compute nodes having more nodes (upscaling) or fewer nodes (downscaling), with respect to the first compute node. 
     As illustrated, if the first compute node has the same number of cores as the second compute node, at  1103 , then the scaling ration is 1, at  1190 . If the number of cores in the first compute node is greater than the number of cores in the second compute node, at  1103 , then the analysis is a downscaling analysis. Following the downscaling analysis, if the application uses all of the cores in the first compute node, at  1107 , then the scaling ratio is equal to the number of cores in the second compute node, CoresP2, divided by the number of cores in the first compute node, CoresP1, at  1191 . 
     If, however, only some of the cores are used by the first compute node, at  1107 , then, at  1110 , a constant MulCONST is calculated as the number of cores used to execute the application in the first compute node, BusyCoresP1, divided by the total number of cores in the first compute node, CoresP1. An EstMulCores constant is then calculated as the greater of (a)  1  and (b) the number of cores in the second compute node, CoresP2, multiplied by the MulCONST. The minimum thread scaling ratio is then calculated as the EstMulCores divided by the number of cores used to execute the application in the first compute node, BusyCoresP1, at  1192 . 
     If the number of cores used to execute the application in the first compute node, BusyCoresP1, is greater than the number of cores in the second compute node, CoresP2, at  1115 , then the thread scaling max is equal to, at  1193 , the number of cores in the second compute node, CoresP2, divided by the number of cores used to execute the application in the first compute node, BusyCoresP1. Otherwise, if, at  1115 , the number of cores used to execute the application in the first compute node, BusyCoresP1, is less than or equal to the number of cores in the second compute node, CoresP2, then the thread scaling max is equal to 1, at  1194 . 
     If the number of cores in the first compute node, CoresP1 is less than the number of cores in the second compute node, CoresP2, at  1103 , then the system conducts an upscaling analysis. If the first compute node utilizes all of the cores to execute the application, at  1120 , then the minimum thread scaling ratio is 1 and the maximum thread scaling ratio is the number of cores in the second compute node, CoresP2, divided by the number of cores in the first compute node, CoresP1, at  1195 . 
     If, however, the first compute node utilizes only some of the cores to execute the application, at  1120 , then the minimum thread scaling ratio is 1. A Subconst is calculated as the number of cores in the first compute node, CoresP1, less the number of cores used by the first compute node to execute the application, BusyCoresP1. The maximum thread scaling ratio is calculated as the difference between the number of cores in the second compute node less the SubCONST, divided by the number of cores used by the first compute node to execute the application, BusyCoresP1, at  1196 . 
     According to various examples, the system may assign each workload to one of the compute clusters based on a matching of the identified resource demands of each respective workload, the calculated thread scaling ratios (e.g., minimum and maximum thread scaling ratios) for the application, and the identified compute resources of each compute cluster, including the number of cores in each respective compute cluster. 
     In some examples, a thread scaling ratio may be calculated based on a first measured execution time on a first compute node with a known number of cores and a second measured execution time on a second compute node with a second number of cores. The system may estimate an execution time for a third compute node with a third number of cores based on the calculated thread scaling ratio. The thread scaling ratio may be an exact thread scaling ratio, in which the minimum scaling ratio and the maximum scaling ratio are equal. Alternatively, the thread scaling ratio may comprise a distinct minimum thread scaling ratio and a distinct maximum thread scaling ratio. The system may use the minimum and maximum thread scaling ratios to estimate an execution time range that includes estimates for minimum and maximum execution times. 
     Specific examples of the disclosure are described above and illustrated in the figures. It is, however, appreciated that many adaptations and modifications could be made to the specific configurations and components detailed above. In some cases, well-known features, structures, and/or operations are not shown or described in detail. Furthermore, the described features, structures, or operations may be combined in any suitable manner. It is also appreciated that the components of the examples, as generally described, and as described in conjunction with the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, all feasible permutations and combinations of examples are contemplated. Furthermore, it is appreciated that changes may be made to the details of the above-described examples without departing from the underlying principles thereof. 
     In the description above, various features are sometimes grouped together in a single example, figure, or description thereof for the purpose of streamlining the disclosure. This method of disclosure, however, is not to be interpreted as reflecting an intention that any claim now presented or presented in the future requires more features than those expressly recited in that claim. Rather, it is appreciated that inventive aspects lie in a combination of fewer than all features of any single foregoing disclosed example. The claims are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate example. This disclosure includes all permutations and combinations of the independent claims with their dependent claims.