Sharing consumed off-heap for parallel data loading

A method is provided for sharing a global memory by a plurality of threads in a memory management system. The method includes allocating, by a controller of the system, thread-local memory areas in the global memory to a given thread and other threads, from among the plurality of threads. The method further includes gathering, by the controller, fragments of the thread-local memory areas previously allocated to the other threads, responsive to the fragments being scanned. The method also includes allocating, by the controller to the given thread, a requested memory size of the fragments of the thread-local memory areas previously allocated to the other threads, responsive to the fragments not being collectively smaller than the requested memory size. The method additionally includes allocating, by the controller to the given thread, a new memory area from the global memory, responsive to the fragments being collectively smaller than the requested memory size.

BACKGROUND

Technical Field

The present invention generally relates to data processing, and more particularly to sharing consumed off-heap for parallel data loading.

Description of the Related Art

Big-data processing software (e.g., Spark, Hadoop, Parquet) often consumes a large amount of heap memory. Java and Scala are commonly used big data processing software because of their ease of developments.

However, it is difficult to achieve both high performance and low engineering cost. For example, integrated Java heap management reduces engineering cost but sacrifices performance (e.g., garbage collection, lazy free, redundant buffering for file I/O). Also, off-heap memory (allocated via sun.misc.Unsafe Java APIs) increases performance but complex heap management must be re-implemented. Hence, there is a need for an improved approach for heap use that achieves both high performance and low engineering cost.

SUMMARY

According to an aspect of the present invention, a computer-implemented method is provided for sharing a global memory by a plurality of threads in a memory management system. The method includes allocating, by a controller of the memory management system, thread-local memory areas in the global memory to a given thread and other threads, from among the plurality of threads. The method further includes gathering, by the controller, fragments of the thread-local memory areas previously allocated to the other threads, responsive to the fragments being scanned. The method also includes allocating, by the controller to the given thread, a requested memory size of the fragments of the thread-local memory areas previously allocated to the other threads, responsive to the fragments not being collectively smaller than the requested memory size. The method additionally includes allocating, by the controller to the given thread, a new memory area from the global memory, responsive to the fragments being collectively smaller than the requested memory size.

According to another aspect of the present invention, a computer program product is provided for sharing a global memory by a plurality of threads in a memory management system of a computer. The computer program product includes a non-transitory computer readable storage medium having program instructions embodied therewith. The program instructions are executable by the computer to cause the computer to perform a method. The method includes allocating, by a controller of the memory management system, thread-local memory areas in the global memory to a given thread and other threads, from among the plurality of threads. The method further includes gathering, by the controller, fragments of the thread-local memory areas previously allocated to the other threads, responsive to the fragments being scanned. The method also includes allocating, by the controller to the given thread, a requested memory size of the fragments of the thread-local memory areas previously allocated to the other threads, responsive to the fragments not being collectively smaller than the requested memory size. The method additionally includes allocating, by the controller to the given thread, a new memory area from the global memory, responsive to the fragments being collectively smaller than the requested memory size.

According to yet another aspect of the present invention, a computer system is provided. The computer system includes a memory management system, having a controller and a global memory, for sharing the global memory by a plurality of threads. The controller is configured to allocate thread-local memory areas in the global memory to a given thread and other threads, from among the plurality of threads. The controller is further configured to gather fragments of the thread-local memory areas previously allocated to the other threads, responsive to the fragments being scanned. The controller is also configured to allocate, to the given thread, a requested memory size of the fragments of the thread-local memory areas previously allocated to the other threads, responsive to the fragments not being collectively smaller than the requested memory size. The controller is additionally configured to allocate, to the given thread, a new memory area from the global memory, responsive to the fragments being collectively smaller than the requested memory size.

DETAILED DESCRIPTION

The present invention is directed to sharing consumed off-heap for parallel data loading.

In an embodiment, the present invention utilizes off-heap management specialized to parallel data loading. In an embodiment, the present invention leverages the following typical sequence of memory usages in parallel data loading: allocation, a single scan, and discard in many conflict-free threads.

FIG. 1is a block diagram showing an exemplary computer processing system100to which the present invention can be applied, in accordance with an embodiment of the present invention.

The system100includes a processing element110connected to a global memory120and a memory controller130via a system bus140. The system100can further include additional elements such as, for example, but not limited to, a network adapter150, a transceiver160, and other devices170. The network adapter150and other devices170can be connected to the system via system bus140. Moreover, the transceiver160is connected to the network adapter150. The other devices170can include, but are not limited to, additional memories, peripherals (e.g., a mouse, a keyboard, and so forth).

In an embodiment, the global memory120and memory controller130can be considered to form a memory management system (also interchangeably referred to herein as a “heap management system”)190. Various features of memory management system190are further described herein below.

FIG. 2is a block diagram showing another exemplary computer processing system200to which the present invention can be applied, in accordance with an embodiment of the present invention.

The system200includes a set of multiple processing elements (collectively and individually denoted by the figure reference numeral)210connected to a global memory220and a memory controller230via a system bus240. The system200can further include additional elements such as, for example, but not limited to, a network adapter250, a transceiver260, and other devices270. The network adapter250and other devices270can be connected to the system via system bus240. Moreover, the transceiver260is connected to the network adapter250. The other devices270can include, but are not limited to, additional memories, peripherals (e.g., a mouse, a keyboard, and so forth).

In an embodiment, the global memory220and memory controller230can be considered to form a memory management system (also interchangeably referred to herein as a “heap management system”)290. Various features of memory management system290are further described herein below.

Moreover, it is to be appreciated that processing system100and/or processing system200may perform at least part of the method described herein including, for example, at least part of method400ofFIGS. 4-5.

In an embodiment, the present invention can utilize a memory management system (e.g., memory management systems190and290) that can have the following primary characteristics. The memory management system has global memory to allocate thread-local memory. The memory management system allocates the requested size of memory by gathering fragments of other thread's local memory if threads have already scanned the fragments. The memory management system allocates new memory from the global memory if the gathered fragments are smaller than the requested size. Threads must read each address of the memory only once in one direction (e.g., from a low address to a high address).

In an embodiment, the present invention can utilize a memory management system that can have the following minor characteristics. The memory management system can use bypass zero fills, unlike the existing Java DirectByteBuffer. The memory management system uses direct I/O support, unlike the existing Java HeapByteBuffer and a normal (conventional) heap.

In an embodiment, the present invention uses a heap management system with common ByteBuffer interfaces.

When a thread requests a memory allocation, the system calculates the total size of reusable (i.e., consumed) memory in other threads and re-assigns the reusable memory to a returned ByteBuffer with the requested size. Further to the preceding, the system gathers reusable memory fragments in different threads into the ByteBuffer, and provides extended interfaces to pass the fragments to file APIs as a single ByteBuffer.

The system can allocate new Unsafe memory if the reusable memory is not enough. For example, the system can be configured to support the 512-byte address alignment for direct I/O in Linux.

While a thread scans the allocated memory, the scanned position for the reusable memory calculation is automatically tracked via the ByteBuffer's position( ) API. For example, the system tracks duplicate( )'ed/slice( )'ed ByteBuffer to track the scanned position.

If the position reaches the end, the assigned Unsafe memory is automatically released if other threads do not share the memory.

FIG. 3is a block diagram showing an exemplary memory allocation300when gathering two thread-local heaps, in accordance with an embodiment of the present invention.

The memory allocation300involves a thread A local memory310, a thread B local memory320, a thread C local memory330, and a new local memory allocation for a thread340.

Each of the local memories310,320, and330include a respective scanned (reusable) memory portion310A,320A, and330A, and further include a respective current position310B,320B, and330B. The respective current positions310B,320B, and330B correspond to the starting positions of the reusable memory portions310A,320A, and330A, respectively, in the local memories310,320, and330, respectively. Regarding local memories310,320, and330, the respective scanned (reusable) memory portions310A,320A, and330A thereof are shown without hatching, and used memory portions thereof are shown using diagonal hatching.

As shown inFIG. 3, the upper portion340A of the new local memory allocation for a thread340corresponds to the scanned (reusable) memory of thread B local memory320, and the lower portion340B of the new local memory allocation for a thread340corresponds to the scanned (reusable) memory of thread C local memory330.

FIGS. 4-5are flow diagrams showing an exemplary method400for sharing a global memory by a plurality of threads in a memory management system, in accordance with an embodiment of the present invention. In an embodiment, method400can be specifically used for sharing consumed off-heap for parallel data loading.

At block410, allocate thread-local memory areas in the global memory to a given thread and other threads, from among the plurality of threads.

At block420, gather fragments of the thread-local memory areas previously allocated to the other threads, responsive to the fragments being scanned.

In an embodiment, step420can include one or more of blocks420A through420B.

At block420A, read each address of the global memory only once in one direction. For example, in an embodiment, each address of the global memory is read only once in the one direction, from a low address to a high address. In another embodiment, each address of the global memory is read only once in the one direction, from a high address to a low address.

At block420B, gather the fragments into a ByteBuffer. For example, in an embodiment, the fragments are gathered into a single ByteBuffer.

In an embodiment, block420B can include block420B1.

At block420B1, bypass zero fills.

At block420C, track positions of the fragments in the thread-local memory areas using a ByteBuffer position Application Programming Interface.

At block430, allocate, to the given thread, a requested memory size of the fragments of the thread-local memory areas previously allocated to the other threads, responsive to the fragments not being collectively smaller than the requested memory size. As used herein, the term “the fragments not being collectively smaller” refers to a sum of the fragment sizes of all the gathered fragments not being smaller than the requested memory size.

In an embodiment, block430can include block430A.

At block430A, perform the allocation to the given thread using direct input/output support.

At block440, allocate, to the given thread, a new memory area from the global memory, responsive to the fragments being collectively smaller than the requested memory size. As used herein, the term “the fragments being collectively smaller” refers to a sum of the fragment sizes of all the gathered fragments being smaller than the requested memory size.

In an embodiment, block440can include one or more of blocks440A through440C.

At block440A, perform the allocation to the given thread using direct input/output support.

At block440B, allocate the new memory area using unsafe memory.

At block440C, automatically release the unsafe memory, responsive to the unsafe memory being unshared with the other threads.

A description will now be given regarding some of the many attendant advantages of the present invention over the prior art.

For example, as one advantage, the present invention can use a JVM® Garbage Collection (GC) bypass. As another example, the present invention can perform automatic memory release. As yet another example, the present invention can include eager memory release. As still another example, the present invention can employ a JVM®-level memory reuse. As a further example, the present invention has thread awareness. Moreover, as another advantage, the present invention can perform a zero fill skip. Also, as another advantage, the present invention can use direct input/output support. These and other advantages of the present invention are readily contemplated by one of ordinary skill in the art, given the teachings of the present invention provided herein, while maintaining the spirit of the present invention.