Remote direct memory access (RDMA) optimized high availability for in-memory data storage

A method for RDMA optimized high availability for in-memory storing of data includes receiving RDMA key-value store write requests in a network adapter of a primary computing server directed to writing data to an in-memory key-value store of the primary computing server and performing RDMA write operations of the data by the network adapter of the primary computing server responsive to the RDMA key-value store write requests. The method also includes replicating the RDMA key-value store write requests to a network adapter of a secondary computing server, by the network adapter of the primary computing server. Finally, the method includes providing address translation data for the in-memory key-value store of the primary computing server from the network adapter of the primary computing server to the network adapter of the secondary computing server.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to the in-memory storage of data in an application server and more particularly to supporting high availability while storing application related data in-memory in an application server.

Description of the Related Art

An application server is a software system executing upon a hardware platform that exposes business logic to client applications through various protocols such as the hypertext transfer protocol (HTTP). While a Web server mainly support the transmitting of hypertext markup language (HTML) defined pages to requesting browser clients for display in a browser, an application server provides access to business logic for use by client application programs. In this regard, the application program can use supplied business logic just as it would call a method on an object internally disposed on a supporting client device.

In most cases, an application server exposes its business logic through a component application programming interface (API) and the application server manages its own resources. Therefore, the application server also provides gate-keeping services including security, transaction processing, resource pooling, and messaging. The application server also must provide performance-enhancing services such as an in-memory store. Finally, like a Web server, an application server may also support scalability and fault-tolerance including high availability.

Advanced application servers provides for in-memory storing of data to support lightning fast data access. In-memory storing can be provided globally to all logic resources of the application server, or at the container level so as to support only a subset of the instances of logic resources in the application server. Recent advances in in-memory stores for application servers utilize direct memory access (DMA) techniques. One such technique includes remote DMA. In computing, remote direct memory access (RDMA) is a direct memory access from the memory of one computer into that of another without involving either one's operating system.

Thus, RDMA permits high-throughput, low-latency networking, which is especially useful in massively parallel computer clusters. RDMA supports zero-copy networking by enabling the network adapter to transfer data directly to or from application memory, eliminating the need to copy data between application memory and the data buffers in the operating system. Such transfers require no work to be done by the central processing units (CPUs), key-value stores, or context switches, and transfers continue in parallel with other system operations. When an application performs an RDMA Read or Write request, the application data is delivered directly to the network, reducing latency and enabling fast message transfer.

Distributed key/value pair store systems that exploit one-sided RDMA such as those found in an in-memory store engine of an application server can directly read from and write to the server's memory. This direct memory access is performed by utilizing RDMA between the network adapter of the server and the memory of the server without involving the CPU or CPUs of the server. Consequently, ultra high throughput and ultra low latency results. However, high availability remains an important problem for one-sided RDMA access. Because the server processor is not involved there are no software-level hooks for high availability replication, which leaves the key-value store vulnerable to hardware failures. Further, involving the server processor even minimally can cause dramatic performance degradation, measured in millions of requests per second. Therefore software-based high availability schemes are undesirable when seeking performance through RDMA supported in-memory stores.

BRIEF SUMMARY OF THE INVENTION

Embodiments of the present invention address deficiencies of the art in respect to in-memory stores in an application server and provide a novel and non-obvious method, system and computer program product for RDMA optimized high availability for in-memory storing. In an embodiment of the invention, a method for RDMA optimized high availability for in-memory storing of data is provided. The method includes receiving RDMA key-value store write requests in a network adapter of a primary computing server directed to writing data to an in-memory key-value store of the primary computing server and performing RDMA write operations of the data by the network adapter of the primary computing server responsive to the RDMA key-value store write requests. The method also includes replicating the RDMA key-value store write requests to a network adapter of a secondary computing server, by the network adapter of the primary computing server. Finally, the method includes providing address translation data for the in-memory key-value store of the primary computing server from the network adapter of the primary computing server to the network adapter of the secondary computing server.

In one aspect of the embodiment, in response to a determination that the primary computing server has failed, an RDMA key-value store read request is received from a requesting client in the network adapter of the secondary computing server in respect to data stored in the in-memory key-value store of the primary computing server. Thereafter, because there are no guarantees that the data in the primary and secondary computing servers are written into the same address in both servers, an address for the data stored in the in-memory key-value store of the primary computing server is translated to an address in an in-memory key-value store of the secondary computing server utilizing the address translation data and an RDMA key-value store read operation is performed at the translated address. Finally, the data produced by the RDMA key-value store read operation is returned by the network adapter of the secondary computing server to the requesting client. Optionally, a local address table of the network adapter of the secondary computing server is translated based upon the address translation data all subsequent RDMA key-value store requests for data in the in-memory key-value store of the primary computing server are processed in the network adapter of the secondary computing server utilizing the translated local address table.

In another aspect of the embodiment, an RDMA key-value store update request is received in the network adapter of the primary computing server directed to data stored in the in-memory key-value store of the primary computing server. Thereafter, in response to the RDMA key-value store update request, an RDMA key-value store update operation is performed on the stored data by the network adapter of the primary computing server. As before, the RDMA key-value store update request is replicated to the network adapter of the secondary computing server, by the network adapter of the primary computing server and the addressing for the stored data in the in-memory key-value store of the primary computing server is translated to an address in an in-memory key-value store of the secondary computing server utilizing the address translation data. Finally, the stored data is updated in the in-memory key-value store of the secondary server utilizing the translated addressing.

In another embodiment of the invention, an application server data processing system is configured for RDMA optimized high availability for in-memory storing. The system includes a primary computing server with a corresponding in-memory key-value store and at least one processor and network adapter, and also a secondary computing server with a corresponding in-memory key-value store and at least one processor and a network adapter. The system also includes an RDMA optimized high availability module disposed in each of the network adapters. The module includes program code enabled to receive RDMA key-value store write requests in the network adapter of a primary computing server directed to writing data to the in-memory key-value store of the primary computing server, to perform RDMA write operations of the data by the network adapter of the primary computing server responsive to the RDMA key-value store write requests, to replicate the RDMA key-value store write requests to the network adapter of the secondary computing server, by the network adapter of the primary computing server, and to provide address translation data for the in-memory key-value store of the primary computing server from the network adapter of the primary computing server to the network adapter of the secondary computing server.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the invention provide for RDMA optimized high availability for in-memory storing. In accordance with an embodiment of the invention, each RDMA key-value store write request received in a network adapter of a primary computing server to perform an RDMA write operation of data to an in-memory key-value store defined in the memory of the primary server, can be replicated to a network adapter of a secondary computing server. Further, address translation data for the in-memory key-value store of the primary computing server can be provided to the network adapter of the secondary computing server. In this way, during failover from the primary server to the secondary server, access to the data of the in-memory key-value store in the memory of the secondary server can continue as if it were the in-memory key-value store in the memory of the primary server without requiring intervention of the CPU of the primary server.

In further illustration,FIG. 1pictorially shows a process for RDMA optimized high availability for in-memory storing. As shown inFIG. 1, a primary server110A can implement an in-memory key-value store120A accessible through RDMA operations by a corresponding network adapter130A. Likewise, a secondary server110B can implement an in-memory key-value store120B accessible through RDMA operations by a corresponding network adapter130B. Optimization logic140can execute in the memory of the primary and secondary servers110A,110B. The optimization logic140can process RDMA requests from a coupled client150, such as write, read and update.

Initially, the client150can issue an RDMA write request190A to the primary server110A to write data to the in-memory key-value store120A. The optimization logic140can direct the network adapter130A to perform the RDMA write operation for the data and the network adapter130A can return addressing data160to the client150, including an address of the data in the in-memory key-value store120and an address of the secondary server110B. Further, the optimization logic140can replicate the RDMA write request190A to the network adapter130B along with address translation data170indicating the address in the in-memory key-value store120A at which the data is stored. The optimization logic140in the secondary server110B can process the replicated RDMA write request190A and can determine a delta between the address translation data170and the address at which the data of the replicated RDMA write request190A has been stored in the in-memory key-value store120B.

Thereafter, RDMA update requests replicated by the optimization logic140from the network adapter130A of the primary server110A to the network adapter130B of the secondary server110B can be processed by translating the address of the RDMA update request utilizing the delta before directing an RDMA update for the data in the in-memory key-value store120B. Of note, the client150can store a local hash map that for each key stored therein in respect to data stored in one of the servers110A,110B contains a remote pointer (i.e. address) to the data corresponding to the key in the primary server110A. As such, the client upon failover to the secondary server120may access the in-memory key-value store120B of the secondary server110B directly using the remote pointers/addresses of the local hash map.

To that end, the client150repeatedly can issue to the primary server110A RDMA read requests for data disposed at respective addresses of the primary server110A. However, to the extent that the primary server110A becomes non-responsive, the client150utilizing the address of the secondary server110B can issue a failover RDMA read request190B to the secondary server with the address of the requested data. The optimization logic140in turn can translate the address of the data to a corresponding address in the in-memory key-value store120B utilizing the delta between addresses in the in-memory key-value store120A and the in-memory key-value store120B. Using the translated address the optimization logic140can direct the network adapter130B to perform an RDMA read operation at the translated address and the network adapter130B can return the retrieved data180at the translated address to the client150. Optionally, a local translation table maintained in the client150can be translated in response to a failover with the delta so as to issue subsequent RDMA read requests and update requests to the secondary server110with an already translated address.

The process described in connection withFIG. 1can be implemented in an application server data processing system. In yet further illustration,FIG. 2schematically shows an application server data processing system configured for RDMA optimized high availability for in-memory storing. The system includes primary and secondary application servers210A,210B coupled to one another over a computer communications network250. The primary and secondary application servers210A,210B each include memory and at least one processor and also include respective network adapters220A,220B. The primary and secondary application servers210A,210B also include respective storing engines240A,240B managing correspondingly different in-memory key-value stores230BA,230B into which application server objects are key-value stored.

Importantly, an optimization module300can be included as part of the network adapter firmware of each of the application servers210A,210B. The optimization module300can include program code that when executed in the memory of the application servers210A,210B is enabled to receive an RDMA write request for data from a client260. The program code further can be enabled to direct the network adapter220A to perform an RDMA write operation in the in-memory key-value store230A resulting in an address at which the data is stored in the in-memory key-value store230A. The program code yet further can replicate the RDMA write request for the data to the network adapter220B with the address such that the network adapter220B can perform an RDMA write operation of the data into the in-memory key-value store230resulting in an address at which the data is stored in the in-memory key-value store.

Utilizing the address received from the optimization logic300of the network adapter220A, the optimization logic300of the network adapter220B can determine a delta between the addresses acting as address translation data. Finally, the program code can return not only the address of the data in the in-memory key-value store230A to the client, but also the program code can return to the client260the address of the secondary application server210B in the event of a failure of the primary application server210A. In this regard, in response to a failure of the primary application server210A, the client260can issue an RDMA read request to the secondary application server210B with the address of the sought after data in the in-memory key-value store230A. Using the delta, the network adapter220B can translate the address to a valid address in the in-memory key-value store230B and the network adapter220B can retrieve the sought after data in the in-memory key-value store230B.

In even yet further illustration of the operation of the optimization module300,FIGS. 3A through 3D, taken together, are a flow chart illustrating a process for RDMA optimized high availability for in-memory storing. Beginning in block305, an RDMA write request for data can be received from a client in a network adapter of a primary computing server and in block310, the RDMA write request can be replicated to a network adapter of a secondary computing server. In block315, an RDMA write operation can be performed by the network adapter of the primary computing server on the data resulting in an address of storage of the data in the in-memory key-value store of the primary computing server. In block320, the address can be retrieved and in block325the address can be provided to the network adapter of the secondary computing server with which the network adapter of the secondary computing server can compute a delta between the addresses in the in-memory key-value store of the primary computing server and those of the secondary computing server. Finally, the address of the data and an address of the secondary computing server can be returned to the requesting client.

Turning now toFIG. 3B, an RDMA update request an be received in the network adapter of the primary computing server in block335an RDMA update request specifying an update of data at a specified address can be received in the network adapter of the primary computing server. In block340the update request can be replicated to the network adapter of the secondary computing server in response to which the network adapter of the secondary computing server can update data at the address provided adjusted to account for the delta. Likewise, in block345the network adapter of the primary computing server can perform an RDMA update of the data at the specified address.

In this regard, as shown inFIG. 3C, at block350the replicated RDMA update request can be received in the network adapter of the secondary computing server along with an address in the in-memory key-value store of the primary computing server at which the data to be updated can be located. In block355the translation delta previously computed by the network adapter of the secondary computing server can be retrieved and in block360the received address can be translated to an address of the in-memory key-value store of the secondary computing server. Finally, in block365an RDMA update operation can be performed by the network adapter of the secondary computing server on data at the translated address.

Referring now toFIG. 3D, in block370a failover RDMA read request can be received from the client at the network adapter of the secondary computing server. IN block375, the translation delta previously computed can be retrieved and applied to the address of the RDMA read request at block380. Subsequently, in block385the network adapter can perform an RDMA read operation at the translated address. Finally, data retrieved from the in-memory key-value store of the secondary computing server resulting from the RDMA read operation can be returned to the client in block390.

The present invention may be embodied within a system, a method, a computer program product or any combination thereof. The computer program product may include a computer readable storage medium or media having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention. The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.

A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.