Patent Description:
In typical parallel processing system, multiple computation nodes each comprising one or more processing elements are connected by a high-speed network. The processing elements of the computation node each have internal memory. The processing elements are connected within their computation node. This on-computation node connectivity might be realized with the technology of the high speed network, a separate on-computation node high speed network or a common memory (as for example in Symmetric-Multiprocessing, SMP, systems). Such an arrangement is illustrated in <FIG>.

<FIG> shows an arrangement of multiple computation nodes CN, with each computation node including a plurality of processing elements, PE, each having a respective memory, MEM. The computation nodes are connected to each other via a high speed network, HSN, with each computation node including a network interface controller, NIC, for connecting to the high speed network. The individual processing elements are connected together and also connect to the network interface controller.

A processing element has a peak performance, PP, which is the upper limit for the number of (floating point) operations the processing element can execute per second, measured as floating point operations per second or in short "flops" (while floating point operations are referred to, the operations may equally be integer operations). The peak performance of a computation node, PPCN, is the sum of the peak performances of its processing elements. A given application A, in general, can realize only a fraction ηA of the peak performance, with <NUM> < η < <NUM>, η being called sustained efficiency. The reason for this is that the data transfer rate, i. e, the memory bandwidth (MBW) between the memory of the processing element to its compute registers is finite and thus will decrease the exploitation of the peak performance to ηA for a given application. A similar argument can be made for the input/output data transfer rate, i.e. the communication bandwidth (CBW) of a processing element to another processing element off-computation node, potentially further decreasing the exploitation of the peak performance.

As a rule of thumb, practitioners of high performance computing consider a ratio R = MBW / PP of <NUM> byte / flop a necessary requirement to achieve η close to <NUM> for a majority of data-intensive applications. Depending on the data rate required by a given application A to exhaust the peak performance, the actual memory bandwidth of the processing element determines the ηA that can be achieved for the application A.

Current high-end CPUs suffer from an R as low as <NUM> to <NUM> byte / flop, a number that was continuously decreasing in the last decade with the number of compute cores of a processing element increasing. Current high-end GPUs achieve an R below <NUM> byte/flop only, mainly designed to meet the data requirements of graphics applications and, just recently, deep learning applications. As a detrimental consequence, the most data intensive applications will achieve an ηA below <NUM> to <NUM> % on current CPUs and around <NUM> % on current GPUs a priori, independent of any further decrease by the algorithm being executed or the parallelism in terms of processing elements required. The more data intensive the application is the lower the efficiency ηA becomes for a given R of the processing element.

This problem has been identified by others, as for example described in an article by <NPL>.

A similar consideration can be made for the communication bandwidth of a given processing element describing the data transfer rate to another processing element on-computation node and off-computation node. Of importance here is the impact of the communication bandwidth on the scalability of the code.

As far as the communication bandwidth on-computation node is concerned, one can distinguish between three cases, computation nodes, where the processing elements are connected via the high speed network, computation nodes where the processing elements are connected to a separate network on the computation node that is again connected to the high speed network, and computation nodes that exchange data on-computation node through a common memory.

Concerning communication off-computation node, the practitioners of high performance computing consider a ratio r = CBW/MBW > <NUM> to <NUM> as appropriate to achieve scalability for a multitude of applications. It is evident that the closer the communication bandwidth is to the memory bandwidth, the better are the conditions for scalability.

The theoretically possible communication bandwidth is determined by the number of serial lanes available from the processing element to the high speed network (this holds for both CPUs and GPUs). This number is limited by the serializer-deserializer implementation that is constrained on current chip technology.

It is important that the network interface controller, NIC, of the computation node is dimensioned appropriately to sustain the data stream from and to the processing elements of the computation node.

<CIT> describes the use of a variable operating frequency of system processors in order to maximize a system memory bandwidth.

<CIT> describes a computing system comprising multiple computing devices each connected to a storage device, such as a hard drive, as opposed to a memory. The aim of the system is to match a data rate of retrieving stored data and a processing speed. The suggestion in this document is to use storage units with a higher data transfer rate, i.e. solid state drive in place of or in addition to a hard disk drive, combined with particular low power processors which operate at a lower clock rate.

<CIT> describes a technique for estimating the scalability of a workload in a computing system. The system has a single multi-core processor.

The present invention provides a computing unit for operating in a parallel computing system, the computing unit comprising a plurality of processing elements and an interface for connecting the computing unit to other components of the computing system wherein each processing element has a nominal maximum processing rate NPR and each processing element includes a respective random access memory unit such that data can be transferred from the memory unit at a predetermined maximum data rate MBW and the interface provides a maximum data transfer rate CBW, wherein in order to provide a predetermined peak calculation performance for the computing unit PP obtainable by a number n processing elements operating at the nominal maximum processing rate such that PP = n x NPR operations per second, the computing unit includes an integer multiple f times n processing elements wherein f is greater than one and each processing element is limited to operate at a processing rate of NPR/f.

In a further aspect, the invention provides a method of operating a computing unit comprising a multicore processor and a plurality of graphics processing units, GPUs, each GPU having a nominal peak performance of PPG operations per second, the method comprising operating the GPUs at a fraction <NUM>/f of their nominal peak performance rate, wherein the computing unit provides a predetermined peak calculation performance of PP operations per second and the computing unit has n times f GPUs such that PP is equal to n times PPG.

The present invention relates to the fact that a reduction of the clock frequency v of a processing element by a factor f can reduce the energy consumption of the processing element by a factor f or more. This process is called "underclocking".

The following approximate formula holds for the power consumption of a device of a processing element, P ∝ CV<NUM>v, with C being the capacitance, V the voltage and P the power consumption. This means, P scales linearly with v and quadratically with V.

Concerning the clock frequency of GPUs as an example, in recent years, a large number of articles on power modelling have been published, which, among other things, strive to allocate the power consumption to individual parts of the processing element. With the latest NVIDIA GPUs, one can change the frequency of the streaming multiprocessors (SMs). This is increasingly designed dynamically and autonomously controlled by the hardware in order to make optimum use of the available power budget. As far as the literature tells, the frequency of the memory subsystem cannot be changed and is autonomously clocked in the current generation. This makes it possible for applications whose performance is limited by memory bandwidth to improve the energy balance by reducing the frequency of the SMs somewhat. One can expect an effect of about <NUM> % in this manner.

The performance of an underclocked machine is often better than expected. With normal desktop use, full processing element performance is rarely required. Even when the system is busy, a lot of time is usually spent waiting for data from memory or other devices.

This fact allows, in principle, the substitution a processing element mounted on a computation node, operating at the frequency v, by a number of f processing elements operating at the frequency v/f, without changing the accumulated compute capability PPCN of the computation node. In addition, the energy consumption of the computation node is maintained or potentially decreased. In practice one would choose f = <NUM> or f = <NUM>.

A key aspect of the invention is that for processing elements like modern CPUs and GPUs, the compute frequency f can be decreased without, at the same time, decreasing the memory bandwidth of the processing element. As a consequence, the ratio R is increased by a factor f under this modification. It is remarked that it is not possible to increase the operating frequency of the compute cores without adapting the memory speed.

Secondly, increasing the number of processing elements of a computation node by a factor f increases the total number of available serial lanes on the computation node by a factor f. Therefore, the ratio r for input/output operations off-computation node is improved by a factor f as well.

These improvements increase the concurrency per computation node by a factor f. This requires tuning of the algorithmic approaches for the diverse highly scalable applications but this in principle poses no problems.

While the energy consumption is expected to stay constant, the increased number of processing elements might increase the investment costs, at first sight. However, a large amount of these costs would be due to the memory, which can be decreased for each processing element by a factor of f, while holding the total amount of memory per computation node constant. Moreover, using high-end processing elements at lower frequency might allow to exploit the much less costly sector of the yield that cannot operate at peak frequency.

As a second measure, a reduction of the operating voltage, V, of the processing element can be carried out and will lead to a further reduction of the energy consumption. The dependency on the voltage can be dramatic as power consumption scales quadratically with the voltage. This "undervolting" can be used in addition to underclocking or separately and is another element of the strategy of this invention to improve the energy consumption of the compute section of the processing element.

This invention provides an apparatus that increases the efficiency of parallel processing systems in terms of performance and energy consumption. Technical modifications are introduced that reduce the operating frequency of the processing elements and increase the number of processing elements accordingly to achieve the same peak performance of the entire system at increased application performance. These modifications affect two system parameters that impact the overall efficiency; the memory to register data bandwidth divided by the peak performance of the processing nodes, and the data bandwidth of the processing nodes into the high-speed network of the parallel system divided by the peak performance of the processing nodes. This allows the parallelism of the nodes to be increased with constant or even lower energy consumption of the node. In this manner, the system can be tuned to an optimum of the application performance. The optimum can be selected for any desired measure, e.g. the average application performance of a certain application portfolio or the optimum performance for a certain application. The overall investment costs are also expected to remain similar, as the processing elements used will operate at a lower operating frequency of the compute cores of their processing units while maintaining memory and input/output performance.

The invention presented allows to choose the factor f, which determines the reduction of the processing element frequency and the corresponding increase of the number of processing elements on a computation node, according to selected desired criteria, e.g. the average maximum power for a certain portfolio of applications or the maximum power for a certain application. In fact, both modifications can also be applied independently, depending on the influence of system-critical parameters such as energy consumption and investment costs as well as optimal performance, especially with regard to the interaction of architecture and application with regard to scalability.

This invention can be realized with technology existing today. To give an example, it can be a method to speed up application performances in a booster module within a modular supercomputing system that is aiming at peak exascale performance by<CIT> and subsequent applications <CIT> and <CIT>.

The goal of the invention is to improve the on-computation node application performance for data-intensive calculations compared to any other architectural design by a factor f, and, in addition, to increase the communication bandwidth in order to keep step with the memory bandwidth for better scaling of many applications with large communication requirements off-computation node.

A realization is given by a set of computation nodes that use Mellanox BlueField (BF) multicore system on chip technology. A BlueField card may comprise multiple graphical processor units, GPUs, a PCIe gen <NUM> switch and one or more high data rate, HDR, switches. Each BlueField card can be equipped with up to four GPUs. The BF cards comprise two Mellanox host channel adaptors, HCAs, each, thus up to two times HDR performance can be achieved off-computation node.

The AMD Radeon Vega <NUM> GPU is considered as concrete example for the processing element, expected for general delivery mid of <NUM>. The Vega-<NUM> GPU can be connected to the PCI-e interface on the BF-computation node by <NUM> PCle gen4 lanes. The GPU is expected to be equipped with <NUM> GigaByte HBM-<NUM> memory, partitioned in four memory banks of <NUM> GigaByte each. <NUM> GB of HBM-<NUM> is also a possibility, organized again in four memory banks of <NUM> GigaByte each. Thus, the memory rate can be the same for both configurations.

With an expected memory bandwidth of <NUM> TeraByte per second and an expected peak performance of <NUM> Teraflop per second (double precision), R = <NUM>. While this is far off by a factor <NUM> from the practitioner's rule of <NUM> byte / flop, it is still one of the best ratios R available.

The communication bandwidth is capped by <NUM> PCle gen4 lanes, each capable of <NUM> Gigabyte per lane and direction. With r = <NUM> Gigabyte / <NUM> Terabyte = <NUM>, one certainly will have to fight severe scalability problems for data intensive applications. Any improvement of R and r will help in this respect.

This is illustrated schematically by <FIG>.

Let the standard configuration comprise two GPUs as processing elements per BF-CN, operated at the peak frequency v achieving peak performance. The initial configuration is depicted in <FIG>. As far as the computation node is concerned, the following system parameters are given or are expected:.

An improved configuration shown in <FIG> comprises four GPUs as processing elements per BF-computation node operated at half the peak frequency v, with f = <NUM>, thereby providing the same computation nominal node peak performance value. In this case, the processing element will operate up to one half of the peak performance of the standard configuration. As to the improved computation node, the following system parameters are given or are expected:.

To the underclocking can be added an undervolting to further decrease the energy consumption. The stability of the processing element under undervolting might be less affected than in the case of the full voltage applied.

Claim 1:
A computing unit for operating in a parallel computing system, the computing unit comprising a plurality of processing elements and an interface for connecting the computing unit to other components of the computing system wherein each processing element has a nominal maximum processing rate NPR and each processing element includes a respective random access memory unit such that data can be transferred from the memory unit at a predetermined maximum data rate MBW and the interface provides a maximum data transfer rate CBW, wherein in order to provide a predetermined peak calculation performance, PP, for the computing unit obtainable by a number n processing elements operating at the nominal maximum processing rate such that PP = n x NPR operations per second, the computing unit includes an integer multiple f times n processing elements wherein f is greater than one and each processing element is limited to operate at a processing rate of NPR/f.