System and method for multi-node buffer transfer

A method, computer program product, and computing system for receiving, at a local node, a request to buffer data on a remote persistent cache memory system of a remote node. A target memory address within the remote persistent cache memory system may be sent from the local node via a remote procedure call (RPC). The data may be sent from the local node to the target memory address within the remote persistent cache memory system via a remote direct memory access (RDMA) command.

BACKGROUND

Storing and safeguarding electronic content may be beneficial in modern business and elsewhere. Accordingly, various methodologies may be employed to protect and distribute such electronic content.

For example, each battery backup RAM system of a multi-node storage system may be defined per node without a mirroring connection between each node. Accordingly, when persisting data within a battery backup RAM system to one node (e.g., a local node), data may not automatically be persisted to the other node (e.g., a remote node). Conventional processes for ensuring consistent data between battery backup RAM systems may require multi-node signaling overhead and/or may require many copies of buffered data within the multi-node storage system.

SUMMARY OF DISCLOSURE

In one example implementation, a computer-implemented method executed on a computing device may include, but is not limited to, receiving, at a local node, a request to buffer data on a remote persistent cache memory system of a remote node. A target memory address within the remote persistent cache memory system may be sent from the local node via a remote procedure call (RPC). The data may be sent from the local node to the target memory address within the remote persistent cache memory system via a remote direct memory access (RDMA) command.

One or more of the following example features may be included. The remote persistent cache memory system may include a battery backup random access memory (RAM) system. At least a portion of the remote persistent cache memory system may be allocated, during initialization of the remote node, for buffering data, thus defining one or more buffer portions. A memory address associated with a first buffer portion of the one or more buffer portions may be sent from the remote node. a list of free buffer portions within the remote persistent cache memory system may be generated. The target memory address within the remote persistent cache memory system for buffering the data may be determined based upon, at least in part, the list of free buffer portions within the remote persistent cache memory system. An updated target memory address within the remote persistent cache memory system may be received, from the remote node, via an RPC response. The list of free buffer portions within the remote persistent cache memory system may be updated based upon, at least in part, the updated target memory address. The data to buffer on the remote persistent cache memory system of the remote node may include a cache status bitmap.

In another example implementation, a computer program product resides on a computer readable medium that has a plurality of instructions stored on it. When executed by a processor, the instructions cause the processor to perform operations that may include, but are not limited to, receiving, at a local node, a request to buffer data on a remote persistent cache memory system of a remote node. A target memory address within the remote persistent cache memory system may be sent from the local node via a remote procedure call (RPC). The data may be sent from the local node to the target memory address within the remote persistent cache memory system via a remote direct memory access (RDMA) command.

One or more of the following example features may be included. The remote persistent cache memory system may include a battery backup random access memory (RAM) system. At least a portion of the remote persistent cache memory system may be allocated, during initialization of the remote node, for buffering data, thus defining one or more buffer portions. A memory address associated with a first buffer portion of the one or more buffer portions may be sent from the remote node. a list of free buffer portions within the remote persistent cache memory system may be generated. The target memory address within the remote persistent cache memory system for buffering the data may be determined based upon, at least in part, the list of free buffer portions within the remote persistent cache memory system. An updated target memory address within the remote persistent cache memory system may be received, from the remote node, via an RPC response. The list of free buffer portions within the remote persistent cache memory system may be updated based upon, at least in part, the updated target memory address. The data to buffer on the remote persistent cache memory system of the remote node may include a cache status bitmap.

In another example implementation, a computing system includes at least one processor and at least one memory architecture coupled with the at least one processor, wherein the at least one processor may be configured to receive, at a local node, a request to buffer data on a remote persistent cache memory system of a remote node. The processor may be further configured to send, from the local node, a target memory address within the remote persistent cache memory system via a remote procedure call (RPC). The processor may be further configured to send, from the local node, the data to the target memory address within the remote persistent cache memory system via a remote direct memory access (RDMA) command.

One or more of the following example features may be included. The remote persistent cache memory system may include a battery backup random access memory (RAM) system. At least a portion of the remote persistent cache memory system may be allocated, during initialization of the remote node, for buffering data, thus defining one or more buffer portions. A memory address associated with a first buffer portion of the one or more buffer portions may be sent from the remote node. a list of free buffer portions within the remote persistent cache memory system may be generated. The target memory address within the remote persistent cache memory system for buffering the data may be determined based upon, at least in part, the list of free buffer portions within the remote persistent cache memory system. An updated target memory address within the remote persistent cache memory system may be received, from the remote node, via an RPC response. The list of free buffer portions within the remote persistent cache memory system may be updated based upon, at least in part, the updated target memory address. The data to buffer on the remote persistent cache memory system of the remote node may include a cache status bitmap.

DETAILED DESCRIPTION

Referring toFIG. 1, there is shown data buffering process10that may reside on and may be executed by storage system12, which may be connected to network14(e.g., the Internet or a local area network). Examples of storage system12may include, but are not limited to: a Network Attached Storage (NAS) system, a Storage Area Network (SAN), a personal computer with a memory system, a server computer with a memory system, and a cloud-based device with a memory system.

As is known in the art, a SAN may include one or more of a personal computer, a server computer, a series of server computers, a mini computer, a mainframe computer, a RAID device and a NAS system. The various components of storage system12may execute one or more operating systems, examples of which may include but are not limited to: Microsoft® Windows®; Mac® OS X®; Red Hat® Linux®, Windows® Mobile, Chrome OS, Blackberry OS, Fire OS, or a custom operating system. (Microsoft and Windows are registered trademarks of Microsoft Corporation in the United States, other countries or both; Mac and OS X are registered trademarks of Apple Inc. in the United States, other countries or both; Red Hat is a registered trademark of Red Hat Corporation in the United States, other countries or both; and Linux is a registered trademark of Linus Torvalds in the United States, other countries or both).

The instruction sets and subroutines of data buffering process10, which may be stored on storage device16included within storage system12, may be executed by one or more processors (not shown) and one or more memory architectures (not shown) included within storage system12. Storage device16may include but is not limited to: a hard disk drive; a tape drive; an optical drive; a RAID device; a random access memory (RAM); a read-only memory (ROM); and all forms of flash memory storage devices. Additionally/alternatively, some portions of the instruction sets and subroutines of data buffering process10may be stored on storage devices (and/or executed by processors and memory architectures) that are external to storage system12.

Various IO requests (e.g. IO request20) may be sent from client applications22,24,26,28to storage system12. Examples of IO request20may include but are not limited to data write requests (e.g., a request that content be written to storage system12) and data read requests (e.g., a request that content be read from storage system12).

The instruction sets and subroutines of client applications22,24,26,28, which may be stored on storage devices30,32,34,36(respectively) coupled to client electronic devices38,40,42,44(respectively), may be executed by one or more processors (not shown) and one or more memory architectures (not shown) incorporated into client electronic devices38,40,42,44(respectively). Storage devices30,32,34,36may include but are not limited to: hard disk drives; tape drives; optical drives; RAID devices; random access memories (RAM); read-only memories (ROM), and all forms of flash memory storage devices. Examples of client electronic devices38,40,42,44may include, but are not limited to, personal computer38, laptop computer40, smartphone42, notebook computer44, a server (not shown), a data-enabled, cellular telephone (not shown), and a dedicated network device (not shown).

Users46,48,50,52may access storage system12directly through network14or through secondary network18. Further, storage system12may be connected to network14through secondary network18, as illustrated with link line54.

Client electronic devices38,40,42,44may each execute an operating system, examples of which may include but are not limited to Microsoft® Windows®; Mac® OS X®; Red Hat® Linux®, Windows® Mobile, Chrome OS, Blackberry OS, Fire OS, or a custom operating system. (Microsoft and Windows are registered trademarks of Microsoft Corporation in the United States, other countries or both; Mac and OS X are registered trademarks of Apple Inc. in the United States, other countries or both; Red Hat is a registered trademark of Red Hat Corporation in the United States, other countries or both; and Linux is a registered trademark of Linus Torvalds in the United States, other countries or both).

In some implementations, as will be discussed below in greater detail, a data buffering process, such as data buffering process10ofFIG. 1, may include but is not limited to, receiving, at a local node, a request to buffer data on a remote persistent cache memory system of a remote node. A target memory address within the remote persistent cache memory system may be sent from the local node via a remote procedure call (RPC). The data may be sent from the local node to the target memory address within the remote persistent cache memory system via a remote direct memory access (RDMA) command.

For example purposes only, storage system12will be described as being a network-based storage system that includes a plurality of electro-mechanical backend storage devices. However, this is for example purposes only and is not intended to be a limitation of this disclosure, as other configurations are possible and are considered to be within the scope of this disclosure.

The Storage System:

Referring also toFIG. 2, storage system12may include storage processor100and a plurality of storage targets T 1-n (e.g., storage targets102,104,106,108). Storage targets102,104,106,108may be configured to provide various levels of performance and/or high availability. For example, one or more of storage targets102,104,106,108may be configured as a RAID 0 array, in which data is striped across storage targets. By striping data across a plurality of storage targets, improved performance may be realized. However, RAID 0 arrays do not provide a level of high availability. Accordingly, one or more of storage targets102,104,106,108may be configured as a RAID 1 array, in which data is mirrored between storage targets. By mirroring data between storage targets, a level of high availability is achieved as multiple copies of the data are stored within storage system12.

While storage targets102,104,106,108are discussed above as being configured in a RAID 0 or RAID 1 array, this is for example purposes only and is not intended to be a limitation of this disclosure, as other configurations are possible. For example, storage targets102,104,106,108may be configured as a RAID 3, RAID 4, RAID 5 or RAID 6 array.

While in this particular example, storage system12is shown to include four storage targets (e.g. storage targets102,104,106,108), this is for example purposes only and is not intended to be a limitation of this disclosure. Specifically, the actual number of storage targets may be increased or decreased depending upon e.g., the level of redundancy/performance/capacity required.

Storage system12may also include one or more coded targets110. As is known in the art, a coded target may be used to store coded data that may allow for the regeneration of data lost/corrupted on one or more of storage targets102,104,106,108. An example of such a coded target may include but is not limited to a hard disk drive that is used to store parity data within a RAID array.

While in this particular example, storage system12is shown to include one coded target (e.g., coded target110), this is for example purposes only and is not intended to be a limitation of this disclosure. Specifically, the actual number of coded targets may be increased or decreased depending upon e.g. the level of redundancy/performance/capacity required.

Examples of storage targets102,104,106,108and coded target110may include one or more electro-mechanical hard disk drives and/or solid-state/flash devices, wherein a combination of storage targets102,104,106,108and coded target110and processing/control systems (not shown) may form data array112.

The manner in which storage system12is implemented may vary depending upon e.g. the level of redundancy/performance/capacity required. For example, storage system12may be a RAID device in which storage processor100is a RAID controller card and storage targets102,104,106,108and/or coded target110are individual “hot-swappable” hard disk drives. Another example of such a RAID device may include but is not limited to an NAS device. Alternatively, storage system12may be configured as a SAN, in which storage processor100may be e.g., a server computer and each of storage targets102,104,106,108and/or coded target110may be a RAID device and/or computer-based hard disk drives. Further still, one or more of storage targets102,104,106,108and/or coded target110may be a SAN.

In the event that storage system12is configured as a SAN, the various components of storage system12(e.g. storage processor100, storage targets102,104,106,108, and coded target110) may be coupled using network infrastructure114, examples of which may include but are not limited to an Ethernet (e.g., Layer 2 or Layer 3) network, a fiber channel network, an InfiniB and network, or any other circuit switched/packet switched network.

Storage system12may execute all or a portion of data buffering process10. The instruction sets and subroutines of data buffering process10, which may be stored on a storage device (e.g., storage device16) coupled to storage processor100, may be executed by one or more processors (not shown) and one or more memory architectures (not shown) included within storage processor100. Storage device16may include but is not limited to: a hard disk drive; a tape drive; an optical drive; a RAID device; a random access memory (RAM); a read-only memory (ROM); and all forms of flash memory storage devices. As discussed above, some portions of the instruction sets and subroutines of data buffering process10may be stored on storage devices (and/or executed by processors and memory architectures) that are external to storage system12.

As discussed above, various IO requests or commands (e.g. IO request20) may be generated. For example, these IO requests may be sent from client applications22,24,26,28to storage system12. Additionally/alternatively and when storage processor100is configured as an application server, these IO requests may be internally generated within storage processor100. Examples of IO request20may include but are not limited to data write request116(e.g., a request that content118be written to storage system12) and data read request120(i.e. a request that content118be read from storage system12).

During operation of storage processor100, content118to be written to storage system12may be processed by storage processor100. Additionally/alternatively and when storage processor100is configured as an application server, content118to be written to storage system12may be internally generated by storage processor100.

Storage processor100may include frontend cache memory system122. Examples of frontend cache memory system122may include but are not limited to a volatile, solid-state, cache memory system (e.g., a dynamic RAM cache memory system) and/or a non-volatile, solid-state, cache memory system (e.g., a flash-based, cache memory system).

Storage processor100may initially store content118within frontend cache memory system122. Depending upon the manner in which frontend cache memory system122is configured, storage processor100may immediately write content118to data array112(if frontend cache memory system122is configured as a write-through cache) or may subsequently write content118to data array112(if frontend cache memory system122is configured as a write-back cache).

Data array112may include backend cache memory system124. Examples of backend cache memory system124may include but are not limited to a volatile, solid-state, cache memory system (e.g., a dynamic RAM cache memory system) and/or a non-volatile, solid-state, cache memory system (e.g., a flash-based, cache memory system). During operation of data array112, content118to be written to data array112may be received from storage processor100. Data array112may initially store content118within backend cache memory system124prior to being stored on e.g. one or more of storage targets102,104,106,108, and coded target110.

As discussed above, the instruction sets and subroutines of data buffering process10, which may be stored on storage device16included within storage system12, may be executed by one or more processors (not shown) and one or more memory architectures (not shown) included within storage system12. Accordingly, in addition to being executed on storage processor100, some or all of the instruction sets and subroutines of data buffering process10may be executed by one or more processors (not shown) and one or more memory architectures (not shown) included within data array112.

Further and as discussed above, during the operation of data array112, content (e.g., content118) to be written to data array112may be received from storage processor100and initially stored within backend cache memory system124prior to being stored on e.g. one or more of storage targets102,104,106,108,110. Accordingly, during use of data array112, backend cache memory system124may be populated (e.g., warmed) and, therefore, subsequent read requests may be satisfied by backend cache memory system124(e.g., if the content requested in the read request is present within backend cache memory system124), thus avoiding the need to obtain the content from storage targets102,104,106,108,110(which would typically be slower).

In some implementations, storage system12may include multi-node active/active storage clusters configured to provide high availability to a user. As is known in the art, the term “high availability” may generally refer to systems or components that are durable and likely to operate continuously without failure for a long time. For example, an active/active storage cluster may be made up of at least two nodes (e.g., storage processors100,126), both actively running the same kind of service(s) simultaneously. One purpose of an active-active cluster may be to achieve load balancing. Load balancing may distribute workloads across all nodes in order to prevent any single node from getting overloaded. Because there are more nodes available to serve, there will also be a marked improvement in throughput and response times. Another purpose of an active-active cluster may be to provide at least one active node in the event that one of the nodes in the active-active cluster fails.

In some implementations, storage processor126may function like storage processor100. For example, during operation of storage processor126, content118to be written to storage system12may be processed by storage processor126. Additionally/alternatively and when storage processor126is configured as an application server, content118to be written to storage system12may be internally generated by storage processor126.

Storage processor126may include frontend cache memory system128. Examples of frontend cache memory system128may include but are not limited to a volatile, solid-state, cache memory system (e.g., a dynamic RAM cache memory system) and/or a non-volatile, solid-state, cache memory system (e.g., a flash-based, cache memory system).

Storage processor126may initially store content118within frontend cache memory system126. Depending upon the manner in which frontend cache memory system128is configured, storage processor126may immediately write content118to data array112(if frontend cache memory system128is configured as a write-through cache) or may subsequently write content118to data array112(if frontend cache memory system128is configured as a write-back cache).

In some implementations, the instruction sets and subroutines of data buffering process10, which may be stored on storage device16included within storage system12, may be executed by one or more processors (not shown) and one or more memory architectures (not shown) included within storage system12. Accordingly, in addition to being executed on storage processor126, some or all of the instruction sets and subroutines of data buffering process10may be executed by one or more processors (not shown) and one or more memory architectures (not shown) included within data array112.

Further and as discussed above, during the operation of data array112, content (e.g., content118) to be written to data array112may be received from storage processor126and initially stored within backend cache memory system124prior to being stored on e.g. one or more of storage targets102,104,106,108,110. Accordingly, during use of data array112, backend cache memory system124may be populated (e.g., warmed) and, therefore, subsequent read requests may be satisfied by backend cache memory system124(e.g., if the content requested in the read request is present within backend cache memory system124), thus avoiding the need to obtain the content from storage targets102,104,106,108,110(which would typically be slower).

As discussed above, storage processor100and storage processor126may be configured in an active/active configuration where processing of data by one storage processor may be synchronized to the other storage processor. For example, data may be synchronized between each storage processor via a separate link or connection (e.g., connection130). In some implementations, one of the storage processors may fail which may cause a significant amount of desynchronization between the storage processors.

The Data Buffering Process:

Referring also toFIGS. 3-7and in some implementations, data buffering process10may receive300, at a local node, a request to buffer data on a remote persistent cache memory system of a remote node. A target memory address within the remote persistent cache memory system may be sent302from the local node via a remote procedure call (RPC). The data may be sent304from the local node to the target memory address within the remote persistent cache memory system via a remote direct memory access (RDMA) command.

As will be discussed in greater detail below, implementations of the present disclosure may allow for efficient data buffer transfer in a multi-node, battery backup random access memory (RAM) cache memory system. For example, when working with battery backup RAM, each battery backup RAM system may be defined per node without a mirroring connection between each node. Accordingly, when persisting data within a battery backup RAM cache memory system to one node (e.g., a local node), data may not automatically be persisted to the other node (e.g., a remote node).

As will be discussed in greater detail below, data buffering process10may transfer the data from one node to another node to provide consistent data across both nodes in persistent memory. In one example, transferring data between nodes may utilize a Non-Transparent Bridge (NTB). As is known in the art, NTB is a type of PCI-Express bridge chip that connects the separate memory systems of two or more computers to the same PCI-Express fabric. NTB may transfer data using a remote direct memory access (RDMA) write or RDMA read that includes sending a remote procedure call (RPC) to the other node and then performing an RDMA write, which will result in an extra RPC for every RDMA command. Accordingly, data buffering process10may optimize the transfer of data or data buffer(s) between nodes in a multi-node persistent cache memory system by sending fewer messages between the nodes and by making as few copies of the data as possible. As will be discussed in greater detail below, data buffering process10may transfer data from an initiator node (e.g., a local node) directly to a persistent cache memory system of another node (e.g., a remote node) with an RDMA write command without doing an extra copy of the data buffer(s).

As shown inFIG. 2and in some implementations, a multi-node, battery backup random access memory (RAM) cache memory system may include at least a pair of nodes (e.g., storage processors100,126) communicatively coupled to a data array (e.g., data array112) and communicatively coupled to one another (e.g., via connection130). Referring also toFIG. 4and in some implementations, each node (e.g., storage processors100,126) may include a persistent cache memory system (e.g., persistent cache memory system400,402, respectively). As will be discussed in greater detail below, each node may be configured to communicate with one another to transfer data between each node's persistent cache memory system. Accordingly, a node initiating a transfer of data may be referred to as a “local node” and a node receiving the data may be referred to as a “remote node”. It will be appreciated that the local node (e.g., storage processor100) and the remote node (e.g., storage processor126) may be positioned physically adjacent to one another or may be positioned in physically separate spaces while being communicatively coupled. Accordingly, the terms “local” and “remote” are for explanation purposes only and do not impose any minimum degree of physical separation between nodes, within the scope of the present disclosure.

The persistent cache memory system (e.g., persistent cache memory systems400,402) may include persistent RAM-based storage. As is known in the art, persistent RAM-based storage will not lose its data in the event of e.g., a power failure. Persistent RAM-based storage may be accomplished using various methodologies, such as incorporating an independent battery backup that will maintain the content stored within the RAM-based storage system during a power failure; or utilizing procedures that will rebuild the content stored within the RAM-based storage system after recovery from a power failure. Accordingly and in some implementations, persistent cache memory systems400,402may include a battery backup RAM cache memory system.

In some implementations, data buffering process10may allocate306, during initialization of the remote node, at least a portion of the remote persistent cache memory system for buffering data, thus defining one or more buffer portions. Referring again toFIG. 4and upon initialization of each node (e.g., storage processors100,126), data buffering process10may allocate306at least a portion of the remote persistent cache memory system (e.g., persistent cache memory system400,402, respectively) for buffering data. For example, suppose that persistent cache memory system400of storage processor100has a plurality of portions (e.g., portions404,406,408,410) and that persistent cache memory system402of storage processor100has a plurality of portions (e.g., portions412,414,416,418). In this example, suppose data buffering process10allocates portions404and406of persistent cache memory system400of a local node (e.g., storage processor100) for buffering data (e.g., buffer portions404,406) and portions412and414of persistent cache memory system402of a remote node (e.g., storage processor126) for buffering data (e.g., buffer portions412,414). Accordingly and as will be described in greater detail below, data buffering process10may allocate buffer portions of the persistent cache memory system from each node for persisting data pages and cache status bitmaps from the local node and/or the remote node.

When allocating306the one or more buffer portions from the persistent cache memory system, data buffering process10may identify an initial memory address within the persistent cache memory system and a total allocation size for the one or more buffers. In the example ofFIG. 4, data buffering process10may allocate306buffer portions412and414by identifying an initial memory address within persistent cache memory system402(e.g., memory address420) and a total allocation size for the one or more buffers. While the total allocation size in this example is e.g., two portions of the persistent cache memory system, it will be appreciated that any number of portions, of any size, may be allocated306for buffering data within the scope of the present disclosure.

Allocating306the one or more buffer portions from the persistent cache memory system may include allocating one or more page data buffers from the one or more buffer portions. For example, data buffering process10may allocate306buffer portions412,414of various sizes that may be distinct from page sizes typically utilized by cache memory systems. Suppose that buffer portions412,414each represent an equal-sized portion of persistent cache memory system402. In this example, suppose that buffer portions412,414are sized to include many pages. Accordingly, data buffering process10may generate a mapping of pages to buffer portions412,414to allow for particular pages to be identified within persistent cache memory system402. In one example, data buffering process10may generate a hash table to map pages to buffer portions. As shown in the example ofFIG. 4, data buffering process10may generate hash table422for the mapping of pages to buffer portions404,406of persistent cache memory system400on the local node (e.g., storage processor100) and hash table424for the mapping of pages to buffer portions412,414of persistent cache memory system402on the remote node (e.g., storage processor126).

In some implementations, data buffering process10may send308, from the remote node, a memory address associated with a first buffer portion of the one or more buffer portions. Referring also toFIG. 5and in some implementations, data buffering process10may send308, from the remote node (e.g., storage processor126), a memory address (e.g., memory address420) associated with a first buffer portion (e.g., buffer portion412) of the one or more buffer portions (e.g., buffer portions412,414). In this example, data buffering process10may send308memory address420and/or the total allocation size for the one or more buffers using a message (e.g., message500). In this manner, data buffering process10may provide each node with the buffer locations allocated from its respective persistent cache memory system.

In some implementations, data buffering process10may generate310a list of free buffer portions within the remote persistent cache memory system. Referring again toFIG. 5and in response to receiving message500, data buffering process10may generate a list of free or unused buffer portions within the remote persistent cache memory system (e.g., list of free buffer portions502). In this example, list of free buffer portions502may include the initial memory address (e.g., memory address420) for the first buffer portion (e.g., buffer portion412) of the one or more buffer portions within the remote persistent cache memory system (e.g., persistent cache memory system402of storage processor126). In some implementations, list of free buffer portions502may include the total allocation size for the one or more buffer portions within persistent cache memory system402of storage processor126. As shown inFIG. 5, storage processor126may be configured to generate a similar list of free or unused buffer portions within persistent cache memory system400of storage processor100(e.g., list of free buffer portions504).

Referring also toFIG. 6and in some implementations, data buffering process10may receive300, at a local node, a request to buffer data on a remote persistent cache memory system of a remote node. For example, suppose persistent cache memory system400includes data600within buffer portion404. In this example, data buffering process10may commit data600within persistent cache memory system400for storage in the remote persistent cache memory system (e.g., persistent cache memory system402of storage processor126). Accordingly, data buffering process10may commit data600based upon, at least in part, a threshold amount of data being stored within persistent cache memory system400of storage processor100and/or reaching a threshold amount of data being stored within persistent cache memory system402of storage processor126. In some implementations, the request to buffer data600on the remote persistent cache memory system of the remote node may be received by the local node (e.g., storage processor100) and/or may be generated internally by the local node (e.g., storage processor100).

Data buffering process10may, via the local node, allocate one or more buffer portions from the list of free buffer portions in the size of the number of pages of the commit request. For example and in some implementations, data buffering process10may determine312the target memory address within the remote persistent cache memory system for buffering the data based upon, at least in part, the list of free buffer portions within the remote persistent cache memory system. Data buffering process10may determine312a target memory address within persistent cache memory system402of storage processor126for buffering data using the list of free buffer portions (e.g., list of free buffer portions502). In this example, data buffering process10may determine312that memory address420is the next free or available buffer address for buffering data within persistent cache memory system402of storage processor126and may allocate buffer portion412for buffering data. As will be discussed in greater detail below, buffer portion412may be allocated for persistently caching data600in remote persistent cache memory system402of storage processor126. In this manner, data buffering process10may reduce the number of times data600is copied when persisting to a remote storage node.

In some implementations, data buffering process10may send302, from the local node, a target memory address within the remote persistent cache memory system via a remote procedure call (RPC). As is known in the art, an RPC may generally include a protocol that one program can use to request a service from a program located in another computer on a network without having to understand the network's details. Referring again toFIG. 6, an RPC (e.g., RPC602) may be sent302from the local node (e.g., storage processor100) to a remote node (e.g., storage processor126). For example, RPC602may include target memory address420within persistent cache memory system402of storage processor126and metadata associated with data600. As will be discussed in greater detail below, data buffering process10may utilize a combination of RPC602and an RDMA command to identify a buffer portion in a remote persistent cache memory system and to write the data to the identified buffer portion with minimal signaling overhead between the local node and the remote node.

In some implementations, data buffering process10may send304, from the local node, the data to the target memory address within the remote persistent cache memory system via a remote direct memory access (RDMA) command. As is known in the art, an RDMA command is a command for direct memory access from the memory of one computer into that of another without involving either computer's operating system. With the target memory address of the buffer portion within the remote persistent cache memory system from the first RPC, data buffering process10may utilize the combination of the first RPC and the RDMA command to send data directly to the target memory address. Referring again to the example ofFIG. 6, data buffering process10may send, from the local node (e.g., storage processor100), the data (e.g., data600) to the target memory address (e.g., target memory address420) within the remote persistent cache memory system (e.g., persistent cache memory system402) via an RDMA command (e.g., RDMA command604). In this manner, data buffering process10may send304data600directly to buffer portion412using target memory address420provided in RPC602and data600provided in RDMA604. Accordingly, data buffering process10may buffer data600in buffer portion412without sending additional RPC signals or requests from the local node to determine the target memory address within the remote persistent cache memory system of the remote node.

In some implementations, the data to buffer on the remote persistent cache memory system of the remote node may include a cache status bitmap. For example, the amount of data that may be persisted within a persistent cache memory system may be limited during system failure (e.g., because of a limited battery life in a battery backup RAM cache memory system). Additionally, because it may be undesirable to divide between read and write cache memory, the entire cache may be maintained in the persistent cache memory system. Accordingly, data buffering process10may utilize a cache status bitmap to determine whether a page is “dirty” or “clean.” For example, if a page is dirty, data buffering process10may write the page to persistent memory, while also ensuring that the number of dirty pages does not exceed a threshold amount of data that can be copied on battery to the persistent memory.

When the RPC and the RDMA command are received at the remote node, data buffering process10may use the buffer portion(s) within the remote persistent cache memory system as a cache entry for the data received from the local node. For example, data buffering process10may receive the RPC (e.g., RPC602) with the target memory address (e.g., target memory address420), and may process the target memory address in the hash table to find the page data buffer address and remove it from the hash table. Data buffering process10may persist the buffer portion(s) (e.g., buffer portion412) by setting a persistence bit of the respective data page buffer(s) in the remote persistent cache memory system (e.g., persistent cache memory system402). Buffer portion412may function as a cache entry for data600within persistent cache memory system402. Accordingly, the remote node (e.g., storage processor126) may process IO requests for data600using buffer portion412of persistent cache memory system402as a cache entry. In this manner, data buffering process10may reduce the number of times that data600is copied when persisting data600in a remote node.

In some implementations, data buffering process10may identify, on the remote node, a next free buffer portion from the one or more buffer portions. For example and in response to buffering data600with buffer portion412, data buffering process10may identify buffer portion414as the next free buffer portion in persistent memory cache system402. In some implementations, data buffering process10may determine that no other buffer portions are free. For example, if buffering data600requires all of the allocated buffer portions and/or if data buffering process10reallocates the buffer portions for other purposes, data buffering process10may determine that there are no additional free buffer portions within remote persistent cache memory system. Accordingly, data buffering process10may attempt to allocate one or more new page data buffers in the amount that was sent with RDMA604(e.g., in the commit request). When new page data buffers are allocated, data buffering process10may add the new page data buffers to the hash table (e.g., hash table504) generated for the remote node (e.g., storage processor126).

In some implementations, data buffering process10may receive314, from the remote node, an updated target memory address within the remote persistent cache memory system via an RPC response. For example and as discussed above, data buffering process10may identify a next free buffer portion from within the remote persistent cache memory system and may provide the memory address for the next free buffer portion to the local node. Referring also toFIG. 7, suppose data buffering process10identifies buffer portion414as the next free buffer portion within remote persistent cache memory system402. In this example, data buffering process10may return (e.g., using an RPC response (e.g., RPC response700)) the memory address of the next free buffer portion (e.g., memory address702of buffer portion414) to the local node (e.g., storage processor100). Accordingly, data buffering process10may receive314, from the remote node (e.g., storage processor126) an updated target memory address (e.g., memory address702) within remote persistent cache memory system (e.g., persistent cache memory system402) for buffering additional data within the remote node (e.g., storage processor126).

In some implementations, data buffering process10may update316the list of free buffer portions within the remote persistent cache memory system based upon, at least in part, the updated target memory address. For example and in response to receiving RPC response700, data buffering process10may update the list of free buffer portions (e.g., list of free buffer portions502) with updated target memory address702. In some implementations and as discussed above, when new data buffers are allocated by the remote node, data buffering process10may add these newly allocated data buffer to the list of free buffer portions (e.g., list of free buffer portions502) for future remote data buffering requests (e.g., in response to commit requests received on the local node).