Patent Description:
A memory sub-system can be a memory module, such as a dual in-line memory module (DIMM), a small outline DIMM (SO-DIMM), or a non-volatile dual in-line memory module (NVDIMM). A memory sub-system can be a storage system, such as a solid-state drive (SSD), or a hard disk drive (HDD). A memory sub-system can include one or more memory components that store data. The memory components can be, for example, non-volatile memory components and volatile memory components. Examples of memory components include memory integrated circuits. Some memory integrated circuits are volatile and require power to maintain stored data. Some memory integrated circuits are non-volatile and can retain stored data even when not powered. Examples of non-volatile memory include flash memory, Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM) and Electronically Erasable Programmable Read-Only Memory (EEPROM) memory, etc. Examples of volatile memory include Dynamic Random-Access Memory (DRAM) and Static Random-Access Memory (SRAM). In general, a host system can utilize a memory sub-system to store data at the memory components and to retrieve data from the memory components.

For example, a computer can include a host system and one or more memory sub-systems attached to the host system. The host system can have a central processing unit (CPU) in communication with the one or more memory sub-systems to store and/or retrieve data and instructions. Instructions for a computer can include operating systems, device drivers, and application programs. An operating system manages resources in the computer and provides common services for application programs, such as memory allocation and time sharing of the resources. A device driver operates or controls a particular type of devices in the computer; and the operating system uses the device driver to offer resources and/or services provided by the type of devices. A central processing unit (CPU) of a computer system can run an operating system and device drivers to provide the services and/or resources to application programs. The central processing unit (CPU) can run an application program that uses the services and/or resources. For example, an application program implementing a type of applications of computer systems can instruct the central processing unit (CPU) to store data in the memory components of a memory sub-system and retrieve data from the memory components.

An operating system of a computer system can allow an application program to use virtual addresses of memory to store data in, or retrieve data from, memory components of one or more memory sub-systems of the computer system. The operating system maps the virtual addresses to physical addresses of one or more memory sub-systems connected to the central processing unit (CPU) of the computer system. The operating system implements the memory accesses specified at virtual addresses using the physical addresses of the memory sub-systems.

A virtual address space can be divided into pages. A page of virtual memory can be mapped to a page of physical memory in the memory sub-systems. The operating system can use a paging technique to access a page of memory in a storage device via a page of memory in a memory module. At different time instances, the same page of memory in a memory module can be used as proxy to access different pages of memory in the storage device or another storage device in the computer system.

A computer system can include a hypervisor (or virtual machine monitor) to create or provision virtual machines. A virtual machine is a computing device that is virtually implemented using the resources and services available in the computer system. The hypervisor presents the virtual machine to an operating system as if the components of virtual machine were dedicated physical components. A guest operating system runs in the virtual machine to manage resources and services available in the virtual machine, in a way similar to the host operating system running in the computer system. The hypervisor allows multiple virtual machines to share the resources of the computer system and allows the virtual machines to operate on the computer substantially independently from each other.

Message Passing Interface (MPI) is a communication protocol for programming parallel computers. MPI supports both point-to-point communication and collective communication. MPI provides virtual topology, synchronization, and communication functionality between a set of processes (that have been mapped to nodes/servers/computer instances) in a language-independent way.

A dynamically distributed file system which operates on a computer network and includes a first file server that is operably connected to a network fabric and a second file server that is operably connected to the network fabric is described in the <CIT>. Moreover, US Patent Publication No. <CIT> discloses data access systems including a processor and a final level cache module and US Patent Publication No. <CIT> discloses a cache controller which controls data input/output of the storage device and causes the semiconductor storage device to function as a cache memory of the storage device. Other storage and caching systems are disclosed in <CIT>, <CIT>, <CIT>, <CIT> and <CIT>. Finally, US Patent Publication No. <CIT> discloses a method for data placement in a memory-based file system by copying a user data unit from a second storage type device to a first storage type device based on an access request to the file system.

The invention relates to a memory system, more particularly, to memory access communications implemented using message passing interface in a memory system. The invention is defined as in claim <NUM>.

The embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references indicate similar elements.

At least some aspects of the present disclosure are directed to a data movement technique that uses a message passing interface implemented in a memory system to accelerate data communications among peripheral devices. A memory sub-system is also hereinafter referred to as a "memory device. " An example of a memory sub-system is a memory module that is connected to a central processing unit (CPU) via a memory bus. Examples of memory modules include a dual in-line memory module (DIMM), a small outline DIMM (SO-DIMM), a non-volatile dual in-line memory module (NVDIMM), etc. Another example of a memory sub-system is a storage device that is connected to the central processing unit (CPU) via a peripheral interconnect (e.g., an input/output bus, a storage area network). Examples of storage devices include a solid-state drive (SSD), a flash drive, a universal serial bus (USB) flash drive, and a hard disk drive (HDD). In some embodiments, the memory sub-system is a hybrid memory/storage sub-system that provides both memory functions and storage functions. In general, a host system can utilize a memory sub-system that includes one or more memory components. The host system can provide data to be stored at the memory sub-system and can request data to be retrieved from the memory sub-system.

A conventional system can have a graphics processing unit (GPU) and a solid state drive (SSD) configured on a bus for peripheral devices, such as a peripheral component interconnect express (PCIe) bus. A central processing unit (CPU) of the system can communicate with the GPU and the SSD separately using communication protocols adapted for the GPU and the SSD respectively. Thus, such a system allows the GPU to access the SSD indirectly via a central processing unit (CPU). The CPU bridges the communication gaps between the GPU and the SSD by reorganizing data for communication and/or by performing protocol translation. However, involving CPU in data access communications between the GPU and the SSD is inefficient, which degrades the operating performance of the system.

At least some aspects of the present disclosure address the above and other deficiencies by implementing a communication capability in a memory sub-system that allows the memory sub-system to communicate with other peripheral devices on a bus without CPU involvement. For example, the communication capability can include a message passing interface (MPI) implemented in the memory sub-system. Using such a communication capability, peripheral devices, such as storage devices, SSDs, processing units, GPUs, can perform predictive data movements to distribute and/or redistribute data without assistance from the CPU. Predictive data movements can be carried out on one or more peripheral buses and/or over a computer network to improve the operating performance of the computer system.

Further, a conventional system can have a cache structure where slower memories are accessed through faster memories. When a processor accesses data that is currently in a slower memory, the data is loaded to a faster memory as a proxy of the data in the slower memory. Subsequently, the processor operates on the proxy/cache of the data in the faster memory for improved performance. The faster memory typically has a capacity smaller than the slower memory. Thus, only a portion of the data in the slower memory can be cached concurrently in the faster memory. A cache miss occurs when an item accessed by the processor is not currently in the faster memory. A cache hit occurs when an item accessed by the processor is currently in the faster memory. The percentage of accesses that result in cache hits is a cache hit ratio. Improving the cache hit ratio can improve the operating performance of the computing system. However, it is a challenge to design a cache policy to improve cache hit ratio.

At least some aspects of the present disclosure address the above and other deficiencies by performing predictive data movements across different tiers of memories using a machine learning technique. Memories of different tiers can have different data access speeds For example, to improve operating performance of a computing system, frequently used data can be placed in a faster memory; and less frequently used data can be placed in a slower memory. The faster memory can be optionally configured as a cache memory for the slower memory. In some instances, at least a portion of the slower memory can be accessed directly without going through the faster memory as a cache. Data usage information can be applied in a predictive model, trained using a machine learning technique, to predict workload intend and thus data movements across the memories of different tiers. Data usage information include the history of data accesses and attributes related to data accesses, such as applications or programs that uses the data, user accounts in which the data assesses are made, virtual machines that access the data, objects to which the data belong, mapping between data blocks to objects as organized in applications, relations among objects, etc. The data movements predicted according to the data usage information can be performed preemptively to improve the operating performance of the computing system. The prediction model can be initially trained offline using historic data usage information and historic data movements caused by data accesses associated with the data usage information. The training minimizes the differences between the historic data movements and predictions generated by applying the historic data usage information in the prediction model. Subsequently, the prediction model can be used for real time prediction using the real time data usage information. Performing the predicted data movements can reduce the need to move data in response to data access requests. The data movements caused by the real time data access requests, and/or indications of whether the predicted data movements reduce the need to move data across the tires, can be used to identify desired real time prediction results. The desired results can further train the prediction model using a reinforcement machine learning technique for continued improvement and adaptation of the prediction model. The prediction model can be dynamically adapted to the current workloads in real time usage of the computing system.

<FIG> illustrates an example computing system <NUM> having a memory sub-system <NUM> in accordance with some embodiments of the present disclosure. The memory sub-system <NUM> can include media, such as memory components 109A to 109N. The memory components 109A to 109N can be volatile memory components, non-volatile memory components, or a combination of such. In some embodiments, the memory sub-system <NUM> is a memory module. Examples of a memory module includes a DIMM, NVDIMM, and NVDIMM-P. In some embodiments, the memory sub-system is a storage system. An example of a storage system is an SSD. In some embodiments, the memory sub-system <NUM> is a hybrid memory/storage sub-system. In general, the computing environment can include a host system <NUM> that uses the memory sub-system <NUM>. For example, the host system <NUM> can write data to the memory sub-system <NUM> and read data from the memory sub-system <NUM>.

The host system <NUM> can be a computing device such as a desktop computer, laptop computer, network server, mobile device, or such computing device that includes a memory and a processing device. The host system <NUM> can include or be coupled to the memory sub-system <NUM> so that the host system <NUM> can read data from or write data to the memory sub-system <NUM>. The host system <NUM> can be coupled to the memory sub-system <NUM> via a physical host interface. As used herein, "coupled to" generally refers to a connection between components, which can be an indirect communicative connection or direct communicative connection (e.g., without intervening components), whether wired or wireless, including connections such as electrical, optical, magnetic, etc. Examples of a physical host interface include, but are not limited to, a serial advanced technology attachment (SATA) interface, a peripheral component interconnect express (PCIe) interface, universal serial bus (USB) interface, Fibre Channel, Serial Attached SCSI (SAS), a double data rate (DDR) memory bus, etc. The physical host interface can be used to transmit data between the host system <NUM> and the memory sub-system <NUM>. The host system <NUM> can further utilize an NVM Express (NVMe) interface to access the memory components 109A to 109N when the memory sub-system <NUM> is coupled with the host system <NUM> by the PCIe interface. The physical host interface can provide an interface for passing control, address, data, and other signals between the memory sub-system <NUM> and the host system <NUM>. <FIG> illustrates a memory sub-system <NUM> as an example. In general, the host system <NUM> can access multiple memory sub-systems via a same communication connection, multiple separate communication connections, and/or a combination of communication connections.

The host system <NUM> includes a processing device <NUM> and a controller <NUM>. The processing device <NUM> of the host system <NUM> can be, for example, a microprocessor, a central processing unit (CPU), a processing core of a processor, an execution unit, etc. In some instances, the controller <NUM> can be referred to as a memory controller, a memory management unit, and/or an initiator. In one example, the controller <NUM> controls the communications over a bus coupled between the host system <NUM> and the memory sub-system <NUM>.

In general, the controller <NUM> can send commands or requests to the memory sub-system <NUM> for desired access to memory components 109A to 109N. The controller <NUM> can further include interface circuitry to communicate with the memory sub-system <NUM>. The interface circuitry can convert responses received from memory sub-system <NUM> into information for the host system <NUM>.

The controller <NUM> of the host system <NUM> can communicate with controller <NUM> of the memory sub-system <NUM> to perform operations such as reading data, writing data, or erasing data at the memory components 109A to 109N and other such operations. In some instances, the controller <NUM> is integrated within the same package of the processing device <NUM>. In other instances, the controller <NUM> is separate from the package of the processing device <NUM>. The controller <NUM> and/or the processing device <NUM> can include hardware such as one or more integrated circuits and/or discrete components, a buffer memory, a cache memory, or a combination thereof. The controller <NUM> and/or the processing device <NUM> can be a microcontroller, special purpose logic circuitry (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or another suitable processor.

The memory components 109A to 109N can include any combination of the different types of non-volatile memory components and/or volatile memory components. An example of non-volatile memory components includes a negative-and (NAND) type flash memory. Each of the memory components 109A to 109N can include one or more arrays of memory cells such as single level cells (SLCs) or multi-level cells (MLCs) (e.g., triple level cells (TLCs) or quad-level cells (QLCs)). In some embodiments, a particular memory component can include both an SLC portion and a MLC portion of memory cells. Each of the memory cells can store one or more bits of data (e.g., data blocks) used by the host system <NUM>. Although non-volatile memory components such as NAND type flash memory are described, the memory components 109A to 109N can be based on any other type of memory such as a volatile memory. In some embodiments, the memory components 109A to 109N can be, but are not limited to, random access memory (RAM), read-only memory (ROM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), phase change memory (PCM), magneto random access memory (MRAM), Spin Transfer Torque (STT)-MRAM, ferroelectric random-access memory (FeTRAM), ferroelectric RAM (FeRAM), conductive bridging RAM (CBRAM), resistive random access memory (RRAM), oxide based RRAM (OxRAM), negative-or (NOR) flash memory, electrically erasable programmable read-only memory (EEPROM), nanowire-based non-volatile memory, memory that incorporates memristor technology, and a cross-point array of non-volatile memory cells. A cross-point array of non-volatile memory can perform bit storage based on a change of bulk resistance, in conjunction with a stackable cross-gridded data access array. Additionally, in contrast to many flash-based memories, cross-point non-volatile memory can perform a write in-place operation, where a non-volatile memory cell can be programmed without the non-volatile memory cell being previously erased. Furthermore, the memory cells of the memory components 109A to 109N can be grouped as memory pages or data blocks that can refer to a unit of the memory component used to store data.

The controller <NUM> of the memory sub-system <NUM> can communicate with the memory components 109A to 109N to perform operations such as reading data, writing data, or erasing data at the memory components 109A to 109N and other such operations (e.g., in response to commands scheduled on a command bus by controller <NUM>). The controller <NUM> can include hardware such as one or more integrated circuits and/or discrete components, a buffer memory, or a combination thereof. The controller <NUM> can be a microcontroller, special purpose logic circuitry (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or another suitable processor. The controller <NUM> can include a processing device <NUM> (processor) configured to execute instructions stored in local memory <NUM>. In the illustrated example, the local memory <NUM> of the controller <NUM> includes an embedded memory configured to store instructions for performing various processes, operations, logic flows, and routines that control operation of the memory sub-system <NUM>, including handling communications between the memory sub-system <NUM> and the host system <NUM>. In some embodiments, the local memory <NUM> can include memory registers storing memory pointers, fetched data, etc. The local memory <NUM> can also include read-only memory (ROM) for storing micro-code. While the example memory sub-system <NUM> in <FIG> has been illustrated as including the controller <NUM>, in another embodiment of the present disclosure, a memory sub-system <NUM> may not include a controller <NUM>, and can instead rely upon external control (e.g., provided by an external host, or by a processor or controller separate from the memory sub-system).

In general, the controller <NUM> can receive commands or operations from the host system <NUM> and can convert the commands or operations into instructions or appropriate commands to achieve the desired access to the memory components 109A to 109N. The controller <NUM> can be responsible for other operations such as wear leveling operations, garbage collection operations, error detection and error-correcting code (ECC) operations, encryption operations, caching operations, and address translations between a logical block address and a physical block address that are associated with the memory components 109A to 109N. The controller <NUM> can further include host interface circuitry to communicate with the host system <NUM> via the physical host interface. The host interface circuitry can convert the commands received from the host system into command instructions to access the memory components 109A to 109N as well as convert responses associated with the memory components 109A to 109N into information for the host system <NUM>.

The memory sub-system <NUM> can also include additional circuitry or components that are not illustrated. In some embodiments, the memory sub-system <NUM> can include a cache or buffer (e.g., DRAM) and address circuitry (e.g., a row decoder and a column decoder) that can receive an address from the controller <NUM> and decode the address to access the memory components 109A to 109N.

The computing system <NUM> includes a data orchestrator <NUM> in the memory sub-system <NUM> that can perform predictive data movements between faster memory (e.g., 109A) and slower memory (e.g., 109N) of the memory sub-system <NUM>, and/or perform predictive data movements between peripheral devices. In some embodiments, the controller <NUM> in the memory sub-system <NUM> includes at least a portion of the data orchestrator <NUM>. In other embodiments, or in combination, the controller <NUM> and/or the processing device <NUM> in the host system <NUM> includes at least a portion of the data orchestrator <NUM>. For example, the controller <NUM>, the controller <NUM>, and/or the processing device <NUM> can include logic circuitry implementing the data orchestrator <NUM>. For example, the controller <NUM>, or the processing device <NUM> (processor) of the host system <NUM>, can be configured to execute instructions stored in memory for performing the operations of the data orchestrator <NUM> described herein. In some embodiments, the data orchestrator <NUM> is implemented in an integrated circuit chip disposed in the memory sub-system <NUM>. In other embodiments, the data orchestrator <NUM> is part of an operating system of the host system <NUM>, a device driver, or an application.

The data orchestrator <NUM> can predict data usages and data movements in the computing system <NUM>, including data movements within the memory sub-system <NUM>, data movements between the memory sub-system <NUM> and an optional peripheral device. The optional peripheral device can be connected to the memory sub-system <NUM> via a bus and/or a computer network. Examples of such an optional peripheral device include a memory sub-system similar to, or different from, the sub-system <NUM>, a graphics processing unit, a computer system connected the computing system <NUM> via a local area network, a computer storage device connected to the computing system <NUM> via an InfiniBand (IB) connection, etc. For example, the data orchestrator <NUM> can communicate with the optional peripheral device using a message passing interface (MPI), or another communication protocol that is suitable for communications between processing units, such as a CPU and a GPU. Further details with regards to the operations of the data orchestrator <NUM> are described below.

<FIG> shows a computing system having a data orchestrator <NUM> to facilitate data access communications in accordance with at least some embodiments disclosed herein.

The computing system of <FIG> includes a host system <NUM>, a memory module <NUM> connected to the host system <NUM> via a memory bus <NUM>, and a storage device <NUM> connected to the memory module <NUM> via a peripheral interconnect <NUM>. The storage device <NUM> and/or the memory module <NUM> are examples of the memory sub-system <NUM> illustrated in <FIG>.

The host system <NUM> has a processing device <NUM>, which can be a central processing unit or a microprocessor with one or more processing cores. The host system <NUM> can have a memory management unit <NUM> and cache memory <NUM>. The memory management unit <NUM> and/or at least a portion of the cache memory <NUM> can be optionally integrated within the same integrated circuit package of the processing device <NUM>.

The memory module <NUM> illustrated in <FIG> has multiple types of memory (e.g., <NUM> and <NUM>). For example, memory of type A <NUM> is faster than memory of type B <NUM>.

For example, the memory bus <NUM> can be a double data rate bus; and the peripheral interconnect <NUM> can be a peripheral component interconnect express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a universal serial bus (USB) bus, and/or a storage area network. Memory of type B <NUM> in the memory module <NUM> can be accessed at a speed faster than accessing memory of type B <NUM> in the storage device <NUM>.

The storage device <NUM> illustrated in <FIG> has multiple types of memory (e.g., <NUM> and <NUM>). For example, memory type B <NUM> is faster than memory type C <NUM>.

In general, a plurality of memory modules (e.g., <NUM>) can be coupled to the memory bus <NUM>; and a plurality of storage devices (e.g., <NUM>) can be coupled to the peripheral interconnect <NUM>. In some instances, the peripheral interconnect <NUM> and the storage devices (e.g., <NUM>) are optional and can be absent from the computing system. In other instances, the memory bus <NUM> and the memory modules (e.g., <NUM>) can be optional and can be absent from the computing system.

In a possible configuration when a plurality of memory modules (e.g., <NUM>) are coupled to the memory bus <NUM>, one of the memory modules (e.g., <NUM>) has memory of type A <NUM>; and another of the memory modules has memory of type B <NUM> that is accessible at a speed lower than the memory of type A <NUM> in a separate memory module (e.g., <NUM>).

Similarly, in a possible configuration when a plurality of storage devices (e.g., <NUM>) are coupled to the interconnect <NUM>, one of the storage device (e.g., <NUM>) has memory of type B <NUM>, and another of the storage devices has memory of type C <NUM> that is accessible at a speed lower than the memory of type B <NUM> in a separate storage device (e.g., <NUM>).

The processing device <NUM> and/or the MMU <NUM> are configured via instructions (e.g., an operating system and/or one or more device drivers) to access a portion of memory in the computer system via another portion of memory in the computer system using a paging technique and/or a memory map interface.

For example, memory of type B <NUM> of the memory module <NUM> can be accessed via memory of type A <NUM> of the memory module <NUM> (or another memory module).

For example, memory of type B <NUM> of the storage device <NUM> can be accessed via memory of type A <NUM> of the memory module <NUM> and/or via memory of type B <NUM> of the memory module <NUM>.

For example, memory of type C <NUM> of the storage device <NUM> can be accessed via memory of type A <NUM> of the memory module <NUM>, via memory of type B <NUM> of the memory module <NUM>, and/or via memory of type B <NUM> of the storage device <NUM> (or another storage device).

For example, in some instances, memory of type A <NUM> and memory of type B <NUM> in the same memory module <NUM> (or different memory modules) are addressable directly and separately over the memory bus <NUM> by the memory management unit <NUM> of the processing device <NUM>. However, since the memory of type B <NUM> is slower than memory of type A <NUM>, it is desirable to access the memory type B <NUM> via the memory of type A <NUM>.

In other instances, memory of type B <NUM> of the memory module <NUM> is accessible only through addressing the memory of type A <NUM> of the memory module <NUM> (e.g., due to the size restriction in the address portion of the memory bus <NUM>).

The data orchestrator <NUM> can communicate with one or more peripheral devices <NUM> coupled to the peripheral interconnect <NUM> without going through the host system <NUM>. For example, the data orchestrator <NUM> implements a message passing interface that allows the data orchestrator <NUM> to move data to or from a peripheral device <NUM>. For example, the peripheral device <NUM> can be another storage device that is similar to the storage device <NUM> that also has a data orchestrator. For example, the peripheral device <NUM> can be a graphics processing unit (GPU) that communicate using message passing interface. The data orchestrator <NUM> can communicate with the peripheral device <NUM> over the peripheral interconnect <NUM> without assistance from the host system <NUM>. Thus, the computing resources of the host system <NUM> are not used for the data communications between the data orchestrator <NUM> and the peripheral device <NUM>; and the data communications between the data orchestrator <NUM> and the peripheral device <NUM> can be accelerated by eliminating the delay at the host system <NUM>.

The data orchestrator <NUM> can instruct a controller Y <NUM> in the storage device <NUM> to perform data transfer/movement between the memory of type B <NUM> and the memory of type C <NUM> within the storage device <NUM>.

Further, the data orchestrator <NUM> can instruct a controller Y <NUM> in the storage device <NUM> to communicate with a controller, processing unit, processor or data orchestrator in the peripheral device <NUM> to perform data transfer/movement between storage device <NUM> and the peripheral device <NUM>.

In some instances, the data orchestrator <NUM> can instruct a controller Y <NUM> in the storage device <NUM> to communicate with a controller X <NUM> of the memory module <NUM> to perform data transfer/movement between storage device <NUM> and the memory module <NUM>.

The computing system of <FIG> can be used to implement a desktop computer, laptop computer, network server, mobile device, or such computing device that includes a memory and a processing device. The processing device <NUM> can read data from or write data to the memory sub-systems (e.g., <NUM> and <NUM>).

The processing device <NUM> can be coupled to a memory sub-system (e.g., <NUM>, <NUM>) via one or more physical interface (e.g., <NUM>, <NUM>).

As used herein, "coupled to" generally refers to a connection between components, which can be an indirect communicative connection or direct communicative connection (e.g., without intervening components), whether wired or wireless, including connections such as, electrical, optical, magnetic, etc..

Examples of a physical host interface include, but are not limited to, a serial advanced technology attachment (SATA) interface, a peripheral component interconnect express (PCIe) interface, universal serial bus (USB) interface, Fibre Channel, Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), etc..

The physical host interface can be used to transmit data between the processing device <NUM> and the memory sub-system (e.g., <NUM>). The computer system can further utilize an NVM Express (NVMe) interface to access the memory (e.g., <NUM>,. , <NUM>) when the memory sub-system <NUM> is coupled with the peripheral interconnect <NUM> via the PCIe interface. The physical host interface can provide an interface for passing control, address, data, and other signals between the memory sub-system (e.g., <NUM>) and the processing device <NUM>.

In general, a memory sub-system (e.g., <NUM> and <NUM>) includes a printed circuit board that connects a set of memory devices, such as memory integrated circuits, that provides the memory (e.g., <NUM>,. The memory (e.g., <NUM>,. , <NUM>) on the memory sub-system (e.g., <NUM> and <NUM>) can include any combination of the different types of non-volatile memory devices and/or volatile memory devices.

An example of non-volatile memory devices includes a negative-and (NAND) type flash memory or a negative-or (NOR) type flash memory. A memory integrated circuit can include one or more arrays of memory cells, such as single level cells (SLCs), multi-level cells (MLCs), triple level cells (TLCs), quad-level cells (QLCs), etc. In some implementations, a particular memory device can include both an SLC portion and a MLC (or TLC or QLC) portion of memory cells. Each of the memory cells can store one or more bits of data used by the host system <NUM>. Although non-volatile memory devices such as NAND type flash memory are described, the memory integrated circuits can be based on any other type of memory such as a volatile memory. In some implementations, the memory (e.g., <NUM>,. , <NUM>) can include, but are not limited to, random access memory (RAM), read-only memory (ROM), dynamic random access memory (DRAM), static random access memory (SRAM), synchronous dynamic random access memory (SDRAM), phase change memory (PCM), magneto random access memory (MRAM), negative-or (NOR) flash memory, electrically erasable programmable read-only memory (EEPROM), and/or a cross-point array of non-volatile memory cells. A cross-point array of non-volatile memory can perform bit storage based on a change of bulk resistance, in conjunction with a stackable cross-gridded data access array. Additionally, in contrast to many Flash-based memory, cross point non-volatile memory can perform a write in-place operation, where a non-volatile memory cell can be programmed without the non-volatile memory cell being previously erased. Furthermore, the memory cells of the memory devices can be grouped as memory pages or data blocks that can refer to a unit of the memory device used to store data.

A memory sub-system (e.g., <NUM> or <NUM>) can have a controller (e.g., <NUM> or <NUM>) that communicate with the memory (e.g., <NUM>,. , <NUM>) to perform operations such as reading data, writing data, or erasing data in the memory (e.g., <NUM>,. , <NUM>) and other such operations, in response to requests, commands or instructions from the processing device <NUM> and/or the memory management unit (MMU) <NUM>. The controller (e.g., <NUM> or <NUM>) can include hardware such as one or more integrated circuits and/or discrete components, a buffer memory, or a combination thereof. The controller (e.g., <NUM> or <NUM>) can be a microcontroller, special purpose logic circuitry (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or another suitable processor. The controller (e.g., <NUM> or <NUM>) can include one or more processors (processing devices) configured to execute instructions stored in local memory.

The local memory of the controller (e.g., <NUM> or <NUM>) can include an embedded memory configured to store instructions for performing various processes, operations, logic flows, and routines that control operation of the memory sub-system (e.g., <NUM> or <NUM>), including handling communications between the memory sub-system (e.g., <NUM> or <NUM>) and the processing device <NUM>/MMU <NUM>, and other functions described in greater detail below. The local memory <NUM> of the controller (e.g., <NUM> or <NUM>) can include read-only memory (ROM) for storing micro-code and/or memory registers storing, e.g., memory pointers, fetched data, etc..

While the example memory sub-systems (e.g., <NUM> and <NUM>) in <FIG> have been illustrated as including controllers (e.g., <NUM> and <NUM>), in another embodiment of the present disclosure, a memory sub-system (e.g., <NUM> or <NUM>) may not include a controller (e.g., <NUM> or <NUM>), and can instead rely upon external control (e.g., provided by the MMU <NUM>, or by a processor or controller separate from the memory sub-system (e.g., <NUM> or <NUM>)).

In general, the controller (e.g., <NUM> or <NUM>) can receive commands, requests or instructions from the processing device <NUM> or MMU <NUM> in accordance with a standard communication protocol for the communication channel (e.g., <NUM> or <NUM>) and can convert the commands, requests or instructions in compliance with the standard protocol into detailed instructions or appropriate commands within the memory sub-system (e.g., <NUM> or <NUM>) to achieve the desired access to the memory (e.g., <NUM>,. For example, the controller (e.g., <NUM> or <NUM>) can be responsible for operations such as wear leveling operations, garbage collection operations, error detection and error-correcting code (ECC) operations, encryption operations, caching operations, and address translations between a logical block address and a physical block address that are associated with the memory (e.g., <NUM>,. The controller (e.g., <NUM> or <NUM>) can further include host interface circuitry to communicate with the processing device <NUM> via the physical host interface. The host interface circuitry can convert the commands received from the processing device <NUM> into command instructions to access the memory devices (e.g., <NUM>,. , <NUM>) as well as convert responses associated with the memory devices (e.g., <NUM>,. , <NUM>) into information for the processing device <NUM>.

The memory sub-system (e.g., <NUM> or <NUM>) can also include additional circuitry or components that are not illustrated. In some implementations, the memory sub-system (e.g., <NUM> or <NUM>) can include a cache or buffer (e.g., DRAM) and address circuitry (e.g., a row decoder and a column decoder) that can receive an address from the controller (e.g., <NUM> or <NUM>) or the MMU <NUM> and decode the address to access the memory (e.g., <NUM>,.

In one example, the interconnect <NUM>, or the memory bus <NUM>, has one or more connectors to provide the memory sub-system (e.g., <NUM> or <NUM>) with power and/or communicate with the memory sub-system (e.g., <NUM> or <NUM>) via a predetermined protocol; and the memory sub-system (e.g., <NUM> or <NUM>) has one or more connectors to receive the power, data and commands from the processing device <NUM>. For example, the connection between the connector on the interconnect <NUM> and the connector on a memory sub-system (e.g., <NUM>) can utilize a PCIe bus or a SATA bus.

In some instances, the peripheral interconnect <NUM> can be connected to the host system <NUM> through the memory module <NUM> and/or the memory bus <NUM> (e.g., as illustrated in <FIG>). In such a situation, the data orchestrator <NUM> can be implemented on the memory module <NUM>, as illustrated in <FIG>.

In general, the processing device <NUM>, the controller <NUM> and/or the data orchestrator <NUM> can execute one or more operating systems to provide services, including acceleration of memory access in which a portion of memory in the computer system is accessed via another portion of memory in the computer system using a paging technique and/or a memory map interface, as further discussed below.

<FIG> shows a computing system having data orchestrators for efficient data access communications. For example, the computing system of <FIG> can be implemented using the storage device <NUM> of <FIG> and/or the memory sub-system <NUM> of <FIG>.

In <FIG>, a set of graphics processing units (GPUs) <NUM> and at least one storage device <NUM> are connected via a switched fabric <NUM> to provide peripheral resources for a cluster of host systems <NUM>. The cluster of host systems <NUM> and graphics processing units (GPUs) <NUM> can share a storage device (A or B) <NUM> via the switched fabric <NUM>. The clusters can be connected via a network <NUM> (e.g., using InfiniBand (IB) interfaces) to scale up the system.

The storage devices <NUM> in <FIG> have data orchestrators <NUM>. Message passing interfaces <NUM> are implemented in the data orchestrators <NUM>. The data orchestrators <NUM> can use the message passing interfaces <NUM> to communicate with each other over the switch fabrics <NUM> and/or the network <NUM>, to communicate with graphics processing units (GPUs), and/or to communicate with the host systems <NUM>. The communication capability of the data orchestrators <NUM> can be used to perform predictive data movements within the computing system. For example, data movements can go from a storage device A <NUM>, across the switch fabric <NUM>, to a graphics processing unit (GPU), and/or over the network <NUM> to another storage device B <NUM> or a graphics processing unit (GPU) in another cluster, without going through any of the host systems <NUM>. The data movement can be performed through remote direct memory access (RDMA) without any assistance from any of the host systems <NUM>.

The prediction of the data movement can be made using the techniques discussed below in connection with <FIG>. Some examples and details can be found in <CIT> and entitled "Predictive Data Orchestration in Multi-Tier Memory Systems.

<FIG> illustrates a distributed storage system implemented via data orchestrators. For example, the system of <FIG> can be implemented using the computing systems of <FIG>, <FIG>, and/or <NUM>.

The distributed storage system of <FIG> includes a system A <NUM> and a system B <NUM> that are connected by a computer network <NUM> (and/or a switched fabric <NUM>).

The system A <NUM> includes a storage device A <NUM> having a data orchestrator <NUM>. Similarly, the system B <NUM> includes a storage device B <NUM> having a data orchestrator <NUM>. The data orchestrators <NUM> can communicate with each other over the network <NUM> (and/or a switched fabric <NUM>) without the assistance of a host system <NUM>, as in <FIG>.

The host system <NUM> can run a hypervisor <NUM> to host a virtual machine <NUM>. The hypervisor <NUM> can support the virtual machine <NUM> to have a logic volume <NUM> that is distributed among systems A <NUM> and B <NUM> that are connected by the network <NUM>.

For example, a portion A <NUM> of the logical volume <NUM> in the virtual machine <NUM> is provisioned on a portion A <NUM> in the storage device A <NUM> in the system A <NUM>; and, a portion B <NUM> of the logical volume <NUM> in the virtual machine <NUM> is provisioned on a portion B <NUM> in the storage device B <NUM> in the system B <NUM>.

For example, the hypervisor <NUM> can implement a thin provisioning method of implementing the logical volume <NUM> of the virtual machine <NUM>. During an initial time period, the hypervisor <NUM> allocates the portion A <NUM> of the storage device A <NUM> for the portion <NUM> of the logical volume <NUM> in the virtual machine <NUM>. As the virtual machine <NUM> uses more and more of the logical volume <NUM>, the portion <NUM> can become insufficient; and it becomes necessary to allocation the storage capacity for the portion B <NUM> of the logical volume <NUM>. However, the storage device A <NUM> in the system A <NUM> may run out of capacity at the time to allocate storage capacity for the portion B <NUM> of the logical volume <NUM>, while the storage device B <NUM> in the system B <NUM> has sufficient storage capacity for the portion B <NUM> of the logical volume <NUM>. Thus, the hypervisor <NUM> allocates the portion B <NUM> of the storage device B <NUM> for the portion <NUM> of the logical volume <NUM> in the virtual machine <NUM>.

In general, it is slower for the virtual machine <NUM> running in the host system <NUM> to access the storage device B <NUM> across the network <NUM> than accessing the storage device A <NUM>. Thus, the hypervisor <NUM> can be configured to access the storage device B <NUM> via the data orchestrator <NUM> of the storage device A <NUM>, as if the storage device B <NUM> were part of the storage device A <NUM>. The data orchestrator <NUM> can predictively cache, in the storage device A <NUM>, a portion of the data of the storage device B <NUM>, based on predictions of data usage, such that the operating performance of the system is improved.

For example, the storage device A <NUM> can have faster memory <NUM> and slower memory <NUM>. The faster memory <NUM> can be used to cache data that is predicted to be used by the host system <NUM> in the virtual machine <NUM>.

For example, when the data predicted to be used by the virtual machine <NUM> is in the portion <NUM> of the logical volume <NUM>, the data orchestrator <NUM> caches the data from the slower memory <NUM> into the faster memory <NUM> of the storage device A <NUM> by moving data internally within the storage device A <NUM>.

When the data predicted to be used by the virtual machine <NUM> is in the portion <NUM> of the logical volume <NUM>, the data orchestrator <NUM> of the storage device A communicates with the data orchestrator <NUM> of the storage device B <NUM> to cache the data from the storage device B <NUM> across the network <NUM> (e.g., through remote direct memory access (RDMA)) into the storage device A <NUM>.

Further, the data orchestrator <NUM> of the storage device B <NUM> can predict data movements and/or receive relevant predictions from the data orchestrator <NUM> of the storage device A <NUM>. In response to the predictions, the data orchestrator <NUM> can optionally cache data from the portion B <NUM>. For example, data from slower memory <NUM> of the storage device B <NUM> is cached into faster memory <NUM> of the storage device B <NUM>. The predictive caching operation of the data orchestrator <NUM> of the storage device B <NUM> can accelerate data movement between the system A <NUM> and the system B <NUM>, such as when transmitting data from the slower memory <NUM> of the storage device B <NUM> to the storage device A <NUM> over the network <NUM> is slower than transmitting data from the faster memory <NUM> of the storage device B <NUM> to the storage device A <NUM> over the network <NUM>.

The data orchestrators <NUM> of <FIG> can implement message passing interface (MPI) such that the data movements within the storage devices A and B <NUM> do not go through any of their host systems <NUM>. Alternative communication protocols for data transferring between the storage devices <NUM> over the network <NUM> without the involvement of the host systems <NUM> can also be used.

<FIG> shows a method of data orchestration. The method of <FIG> can be performed by processing logic that can include hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e.g., instructions run or executed on a processing device), or a combination thereof. In some embodiments, the method of <FIG> is performed at least in part by the data orchestrator <NUM> of <FIG>. Although shown in a particular sequence or order, unless otherwise specified, the order of the processes can be modified. Thus, the illustrated embodiments should be understood only as examples, and the illustrated processes can be performed in a different order, and some processes can be performed in parallel. Additionally, one or more processes can be omitted in various embodiments. Thus, not all processes are required in every embodiment. Other process flows are possible.

For example, the method of <FIG> can be implemented in a computing system of <FIG>, <FIG>, <FIG>, or <FIG> for communication with peripheral devices <NUM> and/or host systems <NUM> over a peripheral interconnect <NUM> of <FIG>, a switched fabric <NUM> of <FIG>, and/or a network <NUM> of <FIG>. For example, the data orchestrator <NUM> can be implemented at least in part via the controller <NUM> of a memory sub-system <NUM>, such as the controller <NUM> of a storage device <NUM> of <FIG> or the controller <NUM> of a memory module <NUM> of <FIG>. Optionally, the data orchestrator <NUM> can include the predictive data movement techniques discussed further below in connection with <FIG>.

At block <NUM>, a data orchestrator <NUM> stores data in memory components (e.g., 109A to 109N in <FIG>; <NUM> to <NUM> in <FIG>) of a memory sub-system <NUM> (e.g., storage device <NUM> in <FIG>, <FIG>, or <FIG>; memory module <NUM> in <FIG>).

At block <NUM>, the data orchestrator <NUM> communicates with a host system <NUM> via a bus, such as a peripheral interconnect <NUM>, a memory bus <NUM>, a switch fabric <NUM>, and/or a network <NUM>.

At block <NUM>, the memory sub-system <NUM> services the data to the host system <NUM> via communications over the bus.

At block <NUM>, the data orchestrator <NUM> communicates with a processing device that is separate from the host system <NUM> using a message passing interface <NUM> over the bus. In general, any communication technique/protocol that is suitable for communication among peripheral devices, such as storage devices and/or graphics processing units (GPUs), on a peripheral bus without the assistance from the host system <NUM> can be used.

For example, the memory sub-system <NUM> can be a storage device <NUM>, such as an solid state drive (SSD), configured on the peripheral interconnect <NUM>. The processing device can be a graphics processing unit (GPU) <NUM> connected to the peripheral interconnect <NUM>, or a controller <NUM> of another storage device <NUM> coupled to the peripheral interconnect <NUM>. For example, the storage devices <NUM> can be connected via a peripheral component interconnect express (PCIe) bus, a switched fabric, and/or an InfiniBand (IB) network connection.

At block <NUM>, the data orchestrator <NUM> provides data access to the processing device through communications made using the message passing interface <NUM> over the bus.

For example, the communications made using the message passing interface <NUM> over the bus does not involving the host system <NUM> and/or the central processing unit (CPU) of the host system <NUM>.

Preferably, the memory sub-system <NUM> includes faster memory, such as dynamic random access memory (DRAM), persistent random access memory (PDRAM), static random access memory (SRAM), etc. The memory sub-system <NUM> can slower memory, such as flash memory, cross-point memory.

Optionally, the data orchestrator <NUM> predicts data movements and performs data cache in the faster memory based on the predictions. The communications made using the message passing interface <NUM> over the bus can be used to transport data for the caching operations.

For example, the predictions can be made based on activities related to a logical volume <NUM> configured in a virtual machine <NUM> running in the host system <NUM> of the system A <NUM> illustrated in <FIG>. The logical volume <NUM> has a portion <NUM> implemented on a portion <NUM> of the storage device A in the system A <NUM> and another portion <NUM> implemented on a portion <NUM> of a storage device B in another system B <NUM>. The system A <NUM> and the system B <NUM> are connected via a network <NUM>. When the data orchestrator <NUM> predicts that the data on the portion B <NUM> of the system B <NUM> is to be used, the data orchestrator <NUM> caches the data in its faster memory via communications made using message passing interface (MPI) communications and remote direct memory access (RDMA) operations.

For example, the data orchestrator <NUM> can be implemented using a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The data orchestrator <NUM> uses an artificial neural network implemented in the FPGA or ASIC to predict the data movements. Optionally, the data orchestrator <NUM> trains the artificial neural network as further discussed below.

The data orchestrator <NUM> can predict data usages and movements across different tires of memories, faster memory (e.g., 109A) and slower memory (e.g., 109N). Applications may access certain data in sequences; and certain objects may be used together. Thus, the use of a data item in a user account, in an application, in a virtual machine, as part of an object, can be indication of the subsequent use of another related data item. Before the related data item is accessed, the data orchestrator <NUM> can instruct the controller <NUM> to rearrange the physical storage locations of the data items in the memory sub-system <NUM>, such that at a time when the processing device <NUM> of the host system <NUM> accesses the related data item, the data item is already in the faster memory (e.g., 109A). Thus, the operation performance of the computing system is improved. The predictive model of the data orchestrator <NUM> can be implemented via an artificial neural network, which can be initially trained offline using historic data access records initially and then continuously trained in real time use using the real time data access records.

In one example, the central processing unit (CPU) can access two sets of memory provided in one or more memory systems connected to the CPU. For example, one set of memory can be slower than the other set of memory; and the central processing unit (CPU) can be configured to access the slower set of memory via the faster set of memory using a paging technique. The faster set of memory can be used as the cache memory of the slower set of memory. For example, one set of memory cannot be directly addressable by the CPU and is coupled to the other set of memory that is directly addressable by the CPU; and the central processing unit (CPU) can be configured to access a set of memory that is not directly addressable via the set of memory that is directly addressable in a way similar to the use of the paging technique. The set of memory that can be accessed directly can be used as the cache memory of the set of memory that cannot be assessed directly.

When a faster memory is used as a cache of a slower memory, the data stored in the faster memory has a corresponding copy in the slower memory. When the faster memory is changed, the corresponding copy in the slower memory becomes out of date. The changed content in the faster memory is to be flushed to the slower memory for update.

Alternatively, the content in the slower memory can be accessed without going through the faster memory in some instances; and the content in the faster memory may not have a corresponding copy in the slower memory. The distribution of the content in the slower memory and the faster memory can be dynamically changed to optimize the operating performance for the current workload. In such a situation, the faster memory can still be considered as a cache for tracking cache hit ratio. For example, if a data item being accessed is serviced from the faster memory, a cache hit is counted; and if a data item being accessed is serviced from the slower memory, a cache miss is counted.

In some instances, a memory virtualizer can be implemented in a device driver of a memory component to virtualize memory access to the memories of different tiers to shield the differences in the memory components 109A to 109N from applications and/or virtual machines. The memory virtualizer automatically adjusts data storage locations across the memories of different tiers to optimize the performance of the computing system. Some details and examples of memory virtualizers can be found in <CIT> and entitled "Memory Virtualization for Accessing Heterogeneous Memory Components.

When a data item being accessed is in the slower set of memory but not in the faster set of memory, the data item can be accessed in the slower set of memory directly, or swapped to the faster set of memory for accessing in the faster set of memory, or cached in the faster set of memory. If the workload of accessing the data item is predicted by the data orchestrator <NUM>, the data orchestrator <NUM> instructs the controller <NUM> to swap the data item to the faster set of memory, or cache the data item in the faster set of memory, before the data access. After the data movement performed in accordance with workload prediction, the data access can be served from the faster set of memory when the data item is accessed. Since the data access is serviced from the faster set of memory, the time to complete the data access is shorter than servicing from the slower set of memory, or swapping to the faster set of memory for servicing, or loading the data from the slower set of memory to the faster set of memory for caching and then servicing.

For example, when a page of virtual memory being accessed is currently in the slower set of memory but not in the faster set of memory, a page can be allocated from the faster set of memory to service the page in the slower set of memory; and the data of the page can be fetched from the slower set of memory and stored in the allocated page in the faster set of memory, such that the data access of the page of the virtual memory can be made via accessing the allocated page in the faster set of memory in subsequent operations.

In some instances, swapping a page takes a time longer than simply access a requested data element from the slower memory. Thus, the requested data element is first serviced to the requester, while the page swapping is performed to speed up subsequent access to the data elements in the hot page. Thus, the overall performance is better than holding the request for the data element until the page swap is completed.

Further, information related to the use of the pages in the slower set of memory can be used to train a self-learning prediction engine in predicting the use of the pages. For example, a supervised machine learning technique can be used to train, using the information, an artificial neural network to predict the use of the pages in the slower set of memory by reducing the errors between predictions and the actual use of the pages. After the training of the artificial neural network, the prediction engine can use the current information to predict the next pages to be used. Further, the training, prediction, and feedback from the actual usage following the prediction for further training can be performed in a continuous fashion to adapt the prediction model of the artificial neural network to the most recent usage patterns of memory pages.

In response to the memory usage prediction that a page in the slower set of memory is to be used soon, the data orchestrator <NUM> can instruct the controller <NUM> to proactively swap or cache the page of data from the slower set of memory to the faster set of memory, such that when needed for processing, the page of data is already in the faster set of memory, which arrangement improves the data access speed of the page of data.

The accuracy of the prediction can be measured against the subsequent actual page use; and the prediction and the subsequent actual page use can be used to further train or adjust the artificial neural network to track the most recent usage patterns of memory pages.

As claimed, the machine learning-based prediction can be replaced or augmented with policy based prediction rules. For example, pages storing resident codes (e.g., in lower addresses) can be maintained in the faster set of memory when possible to reduce swapping of frequently used pages. For example, a huge page can be loaded into the faster set of memory when a page that is a portion of the huge page is being accessed. For example, predictions can be made at least in part using heuristic rules, based on indications such as whether the pages are accessed sequentially or randomly, whether the data access is in a steady state mode or in a bursty mode, and/or the logical relations between pages (and pages of different sizes).

Some details and examples regarding the prediction techniques can be found in <CIT> and entitled "Predictive Paging to Accelerate Memory Access".

<FIG> shows a computing system having different tiers of memory and a data orchestrator to optimize data locations in accordance with at least some embodiments disclosed herein.

The computing system of <FIG> includes a host system <NUM>, a memory module <NUM> connected to the host system <NUM> via a memory bus <NUM>, and a storage device <NUM> connected to the memory module <NUM> via a interconnect <NUM>. The storage device <NUM> and/or the memory module <NUM> are examples of the memory sub-system <NUM> illustrated in <FIG>.

For example, the memory bus <NUM> can be a double data rate bus; and the interconnect <NUM> can be a peripheral component interconnect express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a universal serial bus (USB) bus, and/or a storage area network. Memory of type B <NUM> in the memory module <NUM> can be accessed at a speed faster than accessing memory of type B <NUM> in the storage device <NUM>.

The data orchestrator <NUM> can instruct a controller X <NUM> in the memory module <NUM> to perform data transfer/movement between the memory of type A <NUM> and the memory of type B <NUM> within the memory module <NUM>, especially when the memory of type B <NUM> of the memory module <NUM> is not directly addressable using the memory bus <NUM>.

Further, the data orchestrator <NUM> can instruct a controller X <NUM> in the memory module <NUM> to communicate with a controller Y <NUM> in the storage device <NUM> to perform data transfer/movement between memories <NUM> to <NUM> in the storage device <NUM>, and/or between the storage device <NUM> and the memory module <NUM>.

In one variation, the memory (e.g., <NUM> and <NUM>) of the memory module <NUM> can have the same performance individually within the memory module <NUM>; however, the memory management unit <NUM> and/or the processing device <NUM> are restricted to access via the memory <NUM> via the memory <NUM> (e.g., due to the size restriction in the address portion of the memory bus <NUM>). Thus, the memory <NUM> appears to be slower than the memory <NUM> to the processing device <NUM>.

In general, the memory sub-systems (e.g., <NUM> and <NUM>) can include media, such as memory (e.g., <NUM>,. The memory (e.g., <NUM>,. , <NUM>) can includes volatile memory, non-volatile memory (NVM), and/or a combination of such. In some embodiments, the computer system includes at least one memory sub-system that is a storage device <NUM>. An example of a storage device <NUM> is a solid-state drive (SSD). In some embodiments, the computer system includes at least one memory sub-system that is a hybrid memory/storage system configured as a memory module <NUM>. The processing device <NUM> can write data to each of the memory sub-systems (e.g., <NUM> and <NUM>) and read data from the memory sub-systems (e.g., <NUM> and <NUM>) directly or indirectly.

In some instances, the interconnect <NUM> is connected to the host system <NUM> without going through the memory module <NUM> and/or the memory bus <NUM>. When the storage device <NUM> is coupled to the host system <NUM> without going through the memory module <NUM> (e.g., as illustrated in <FIG>), a data orchestrator <NUM> can be implemented in the storage device <NUM> in a way similar to the data orchestrator <NUM> in the memory module <NUM>.

In some instances, the data orchestrator <NUM> can be implemented at least in part in the host system <NUM>.

<FIG> shows a system having a data orchestrator <NUM>. For example, the system of <FIG> can be implemented in a computer system of <FIG>, <FIG>, <FIG>, <FIG>, or <FIG>.

The system of <FIG> includes a host operating system <NUM> that can run in the processing device <NUM> of the computer system of <FIG>, <FIG>, <FIG>, <FIG>, or <FIG>. The host operating system <NUM> includes one or more device drives that provides memory services using the memory (e.g., <NUM>,. , <NUM>) of memory sub-systems, such as the memory module <NUM> and/or the storage device <NUM>.

The host operating system <NUM> includes a hypervisor <NUM> that provisions a virtual machine <NUM>. The virtual machine <NUM> has virtual hardware implemented via the resources and services provided by the host operating system <NUM> using the hardware of the computing system of <FIG>, <FIG>, <FIG>, <FIG>, or <FIG>. For example, the hypervisor <NUM> can provision virtual memory as part of the virtual machine <NUM> using a portion of the memory (e.g., <NUM>,. , <NUM>) of memory sub-systems, such as the memory module <NUM> and/or the storage device <NUM>.

The virtual machine <NUM> allows a guest operating system <NUM> to provide resources and/or services to applications (e.g., <NUM>,. , <NUM>) running in the guest operating system <NUM>, in a way as the operating system <NUM> running on a physical computing machine that has the same or similar set of hardware as provisioning in the virtual machine. The hypervisor <NUM> manages the mapping between the virtual hardware provisioned in the virtual machine and the services of hardware in the computing system managed by the host operating system <NUM>.

<FIG> illustrates an instance in which a virtual machine <NUM> is provisioned by the hypervisor <NUM>. In general, the hypervisor <NUM> can provision a plurality of virtual machines (e.g., <NUM>) that can run the same guest operating system <NUM>, or different guest operating systems (e.g., <NUM>). Different sets of users and/or application programs can be assigned to use different virtual machines.

In some instances, the host operating system <NUM> is specialized to provide services for the provisioning of virtual machines and does not run other application programs. Alternatively, the host operating system <NUM> can provide additional services to support other application programs, such as applications (e.g., <NUM>,.

In <FIG>, the hypervisor <NUM> is configured to use a single-root I/O Virtualization to organize data streams of different characteristics/attributes. For example, the memory module <NUM> has a physical function <NUM> that can implement a plurality of virtual functions (e.g., <NUM>). A virtual function <NUM> provides the service of the memory module <NUM> via the physical function <NUM>. The hypervisor <NUM> allocates and reserves the virtual function <NUM> for memory access by a particular virtual machine <NUM>, a particular application (e.g., <NUM> or <NUM>), a particular user account, etc. Thus, the identify of the virtual function <NUM> used to access the memory module <NUM> can be used to infer the data usage information of the data access, such as the identities of the virtual machine <NUM>, the application <NUM> and/or the user account that are associated with and/or responsible for the data access made using the virtual function <NUM>. Such information can be used in the data orchestrator <NUM> in machine learning to predict data workload and/or movements and in making real time predictions.

For example, the data orchestrator <NUM> can be trained to predict the use of a data item in a slower memory and load the data item into a faster memory before the data item actually requested for use by the virtual machine <NUM>, the application <NUM> running in the virtual machine, and/or a user account operating the application <NUM>. The prediction reduces the time between a request to use the data item and the availability of the item in the faster memory by loading, transferring, and/or, caching the item into the faster memory before the request to use the item reaches the memory module <NUM>, which accelerates the data access of the page.

For example, the slower memory can be the memory <NUM> in the memory module <NUM> and the faster memory be the memory <NUM> in the same memory module <NUM> (or another memory module connected to the same memory bus <NUM> as the memory module <NUM>).

For example, the slower memory can be the memory <NUM> in the storage device <NUM>; and the faster memory can be the memory <NUM> of the same type in the memory module <NUM>, or the memory <NUM> in the memory module <NUM>.

For example, the slower memory can be the memory <NUM> in the storage device <NUM>; and the faster memory can be the memory <NUM> in the same storage device <NUM> or another storage device connected to the interconnect <NUM>, or memory (e.g., <NUM> or <NUM>) in the memory module <NUM>.

Preferably, the predictive data movement is performed within a same memory sub-system, such as within the same memory module <NUM>, the same storage device <NUM>, or the same combination of the memory module <NUM> and the storage device <NUM>, to avoid or reduce congestion in communication channels connected to the processing device <NUM>, such as the memory bus <NUM> and/or the interconnect <NUM>. For example, the predictive data movement can be performed to copy data from the slower memory <NUM> in the memory module <NUM> to the faster memory <NUM> in the memory module <NUM>, under the control of a controller <NUM> in the memory module <NUM> in response to one or more command, request, or instruction from the data orchestrator <NUM>. For example, the predictive data movement can be performed to copy data from the slower memory <NUM> in the storage device <NUM> to the faster memory <NUM> in the storage device <NUM>, under the control of a controller <NUM> in the storage device <NUM> in response to one or more command, request, or instruction from the data orchestrator <NUM>. For example, the predictive data movement can be performed to copy data from the storage device <NUM> to the memory module <NUM>, under the control of the controller <NUM> and the controller <NUM> in the storage device <NUM>, in response to one or more command, request, or instruction from the data orchestrator <NUM>.

In one embodiment, the hypervisor <NUM> not only requests the device driver to access a memory (e.g., <NUM>,. , or <NUM>) in a memory sub-system (e.g., memory module <NUM> or storage device <NUM>) but also provides the device driver with information that can be used in making predictions of which data items in the memory (e.g., <NUM>,. , or <NUM>) are likely to be used in a subsequent time period and which data items in the memory (e.g., memory (e.g., <NUM>,. , or <NUM>) are unlikely to be used in the subsequent time period. The information can be provided at least in part via the use of virtual functions (e.g., <NUM>) that are pre-associated with certain data usage attributes, such as virtual machine <NUM>, application <NUM>, user account, etc..

For example, a page that is likely to be used can be referred to as a hot page; and a page that is unlikely to be used can be referred to as a cold page. The likelihood of a page being used in the subsequent time period can be referred to as the temperature of the page. The data orchestrator <NUM> uses the information provided/identified by the hypervisor <NUM> to predict the temperatures of the pages, moves cold pages from faster memory to slower memory, and moves hot pages from slower memory to faster memory to optimize the distribution of the pages in the memory (e.g., <NUM>,. , or <NUM>) and accelerate data access.

Examples of information provided by the hypervisor <NUM> and used by the data orchestrator <NUM> to make the predictions include: sequences of pages being used in a prior time period, instances of requests to load pages from the slower memory to the faster memory, content attributes of the pages, ownership attributes of the pages, identifications of users or applications of the pages, an indication of whether pages are accessed in a sequential mode in a virtual machine and/or in a user account, an indication of whether page accesses are in a steady state, an indication whether a page used is associated with a huge page, mapping between data blocks and objects, etc..

<FIG> illustrates an implementation of a data orchestrator <NUM>.

In <FIG>, the data orchestrator <NUM> includes a cache controller <NUM> and a workload recognizer <NUM>. The workload recognizer <NUM> includes a prediction model <NUM> that can be implemented using an artificial neural network.

The cache controller <NUM> processes data access requests <NUM> from the host system <NUM>. The cache controller <NUM> monitors a higher performance memory used as a cache relative to a lower performance memory, analyzes the usage of the cache, optimizes the usage of the cache, and manages the use of the cache. Conventional cache techniques can be implemented in the cache controller <NUM>.

In response to the data access requests <NUM>, the cache controller <NUM> determines whether the data targeted by the requests <NUM> are in the higher performance memory at the time of the requests <NUM>. If so, the cache controller <NUM> counts the corresponding data access requests <NUM> as cache hits; and otherwise, the cache controller <NUM> counts the corresponding data access requests <NUM> as cache misses. Thus, the cache controller <NUM> can generate the measurement of cache hit ratio <NUM> for the data distribution at the time of the data access requests <NUM>.

Optionally, the cache controller <NUM> may service a portion of data access requests <NUM> directly from the lower performance memory without caching/loading the corresponding data into the higher performance memory.

The cache policy used the cache controller <NUM> can be used to identify data movements <NUM> that are implemented by the cache controller <NUM>.

The data usage information <NUM> corresponding to the data access requests <NUM> is collected for an initial time period of the operation of the computing system for the training of the prediction model <NUM>. For example, a supervised machine learning technique can be used to train the artificial neural network of the prediction model <NUM> to minimize the different between the data movements <NUM> implemented by the cache controller <NUM> responsive to the data access requests <NUM> and the data movement <NUM> predicted using the prediction model <NUM> using the data usage information <NUM> corresponding to the data access requests <NUM>. The machine learning can be performed offline on another computing device to establish the initial prediction model <NUM>.

Subsequently, the prediction module <NUM> can be used in the workload recognizer <NUM> to make real time predictions of data movements <NUM> based on real time data usage information <NUM> and real time data access requests <NUM>. The workload recognizer <NUM> instructs the cache controller <NUM> to perform the predicted data measurements, which can cause changes in the cache hit ratio <NUM>. The prediction model <NUM> is adjusted and/or trained in real time using a hybrid reinforcement machine learning technique to continuously drive up the cache hit ratio <NUM>. Thus, the prediction model <NUM> can automatically adapt to the current workload of the computing system and implement predicted data movements <NUM> to achieve a cache hit ratio <NUM> higher than that can be achieved via the cache controller <NUM> alone.

Preferably, the predictions made by the workload recognizer <NUM> are based at least in part on a block to object map <NUM>. For a statistical analysis of the data usage information <NUM>, the data orchestrator <NUM> can identify the underlying relations among data blocks. For example, some data blocks represent parts of a same data object in an application; parts of a data objects are accessed together; some data objects have a pattern of being accessed in a particular order; the access to one data object in a user account running an application on a virtual machine can have a high probability of leading to the access to another data object. The block to object map <NUM> identifies the relations that improve the prediction accuracy of the workload recognizer <NUM>.

<FIG> shows a method of predictive data orchestration. The method of <FIG> can be performed by processing logic that can include hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e.g., instructions run or executed on a processing device), or a combination thereof. In some embodiments, the method of <FIG> is performed at least in part by the data orchestrator <NUM> of <FIG>. Although shown in a particular sequence or order, unless otherwise specified, the order of the processes can be modified. Thus, the illustrated embodiments should be understood only as examples, and the illustrated processes can be performed in a different order, and some processes can be performed in parallel. Additionally, one or more processes can be omitted in various embodiments. Thus, not all processes are required in every embodiment. Other process flows are possible.

For example, the method of <FIG> can be implemented in a computing system of <FIG>, <FIG>, <FIG>, <FIG> or <FIG> with a host operating system <NUM> of <FIG> and a prediction model <NUM> of <FIG>. For example, the data orchestrator <NUM> can be implemented at least in part via the cache controller <NUM> and the workload recognizer <NUM> of <FIG> and/or the virtual function <NUM> of <FIG>.

At block <NUM>, the data orchestrator <NUM> receives, from a processing device <NUM>, first data access requests that cause first data movements across tiers of.

For example, the memory components of different tiers (e.g., 109A to 109N in <FIG>, <NUM> to <NUM> and/or to <NUM> in <FIG>) can have first memory and second memory, where the first memory functions as cache of the second memory. For example, the first memory can be volatile dynamic random-access memory; and the second memory can be non-volatile cross-point memory. In some instances, the first memory and the second memory are housed in a same memory sub-system, such as a memory module <NUM> or a storage device <NUM>. In other instances, the first memory and the second memory can be housed in separate same memory sub-systems that can communicate with each other without involving the host system <NUM> and/or the memory bus <NUM>.

When the processing device <NUM> accesses the second memory, the access requests causes caching, in the first memory, the portion of the second memory that is being access. In other instances, the first memory does not function as cache of the second memory; and in response to a request to access a data item that is in the second memory, the data orchestrator <NUM> determines, based on a set of policies or rules, whether or not to change the storage location of the data item from the second memory to the first memory; and if so, the data orchestrator <NUM> can swap the data item from the second memory to the first memory.

At block <NUM>, the data orchestrator <NUM> performs the first data movements responsive to the first data access requests.

For example, the first data movements performed/implemented in response to the first data access requests can be recorded in connection with data usage information associated with the first data access requests. For example, the data usage information can identify a sequence of data blocks being used in a period of time, instances of requests to load data blocks from the second memory to the first memory, content attributes of data blocks loaded from the second memory to the first memory, ownership attributes of data blocks loaded from the second memory to the first memory, identifications of users of data blocks loaded from the second memory to the first memory, identifications of applications that cause data blocks being loaded from the second memory to the first memory, an identification of data blocks that are accessed in a sequential mode in a virtual machine, an identification of data blocks that are accessed in a sequential mode in a user account, and/or an identification of data accesses that are in a steady state.

The first data movements can be used as desired prediction results of a prediction model <NUM> that makes predictions using the data usage information associated with the first data access requests. For example, the prediction model <NUM> has an artificial neural network that can be trained using a supervised machine learning technique to reduce the different between the first data movements and the predictions made using the data usage information associated with the first data access requests. An initial training of the artificial neural network can be optionally performed offline using a separate computer and the recorded information about the first data access requests, the first data movements caused by the first data access requests, and the data usage information before the first data access requests. For example, the data orchestrator <NUM> can store the recorded information in a portion of memory controlled by the data orchestrator <NUM>; and another processing device <NUM> can access the portion of the memory to perform the initial training for the data orchestrator <NUM>. Alternatively, the initial training of the artificial neural network can be performed in the data orchestrator <NUM> until the prediction accuracy of the prediction model <NUM> reaches a threshold level.

At block <NUM>, a memory sub-system <NUM> (e.g., memory module <NUM> and/or storage device <NUM>) services the first data access requests after the first data movements. The performance of the computing system can be improved by predicting data movements and performing the predicted data movements before the corresponding data access requests.

At block <NUM>, the data orchestrator <NUM> receives data usage information <NUM> from the processing device <NUM>.

At block <NUM>, the data orchestrator <NUM> predicts, based on the data usage information <NUM> and the prediction model <NUM> trained via machine learning, second data movements <NUM> across the tiers in the memory components.

At block <NUM>, the data orchestrator <NUM> performs the second data movements <NUM> before receiving second data access requests <NUM>. The second data movements <NUM> reduce data movements across the tiers caused by the second data access requests.

The data orchestrator <NUM> can optionally further train the prediction model based on a performance measurement of the plurality of memory components in servicing the second data access requests from the processing device <NUM> and/or the data movements caused by the second data access requests.

For example, the performance measurement can be a cache hit ratio of second data access requests measured by the data orchestrator <NUM>. For example, requests of the processing device <NUM> for data in the second memory can cause movements of the requested data from the second memory to the first memory; such movements can be counted as cache misses; and data access requests that do not cause such movements can be counted as cache hits. The data orchestrator <NUM> can train the prediction model <NUM> using a hybrid reinforcement learning technique to drive up the cache hit ratio, reduce the count of cache misses, and/or match predictions with the desired data movements identified from the data access requests.

For example, the data orchestrator <NUM> can be implemented as a controller in an integrated circuit chip disposed on a memory module or a storage device, in the form of a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The data orchestrator <NUM> obtains the data usage information <NUM> based at least in part on the identities of virtual functions (e.g., <NUM>) in which the data access requests are used. For example, different virtual functions (e.g., <NUM>) can be used to represent different combinations of data usage information for a period of time, such as virtual machines, applications, user accounts, data access modes, etc..

Optionally, the data orchestrator <NUM> further perform a statistical analysis of the data access requests <NUM> and data usage information <NUM> to identify a mapping between data blocks in the plurality of memory components and data objects as organized in applications running in the processing device. The use of the mapping with the prediction model <NUM> can improve the prediction accuracy of the data orchestrator <NUM>.

Preferably, the predicted data movements <NUM> are performed without going through the bus (e.g., <NUM>) that is used by the data orchestrator <NUM> to communicate with the host system <NUM>.

In some implementations, a communication channel between the processing device <NUM> and a memory sub-system includes a computer network, such as a local area network, a wireless local area network, a wireless personal area network, a cellular communications network, a broadband high-speed always-connected wireless communication connection (e.g., a current or future generation of mobile network link); and the processing device <NUM> and the memory sub-system can be configured to communicate with each other using data storage management and usage commands similar to those in NVMe protocol.

A memory sub-system in general can have non-volatile storage media. Examples of non-volatile storage media include memory cells formed in an integrated circuit and magnetic material coated on rigid disks. Non-volatile storage media can maintain the data/information stored therein without consuming power. Memory cells can be implemented using various memory/storage technologies, such as NAND logic gate, NOR logic gate, phase-change memory (PCM), magnetic memory (MRAM), resistive random-access memory, cross point storage and memory devices (e.g., 3D XPoint memory). A cross point memory device uses transistor-less memory elements, each of which has a memory cell and a selector that are stacked together as a column. Memory element columns are connected via two perpendicular lays of wires, where one lay is above the memory element columns and the other lay below the memory element columns. Each memory element can be individually selected at a cross point of one wire on each of the two layers. Cross point memory devices are fast and non-volatile and can be used as a unified memory pool for processing and storage.

The controller (e.g., <NUM>, or <NUM>) of a memory sub-system (e.g., <NUM> or <NUM>) can run firmware to perform operations responsive to the communications from the processing device <NUM>. Firmware in general is a type of computer program that provides control, monitoring and data manipulation of engineered computing devices.

Some embodiments involving the operation of the controller <NUM> can be implemented using computer instructions executed by the controller <NUM>, such as the firmware of the controller <NUM>. In some instances, hardware circuits can be used to implement at least some of the functions. The firmware can be initially stored in the non-volatile storage media, or another non-volatile device, and loaded into the volatile DRAM and/or the in-processor cache memory for execution by the controller <NUM>.

A non-transitory computer storage medium can be used to store instructions of the firmware of a memory sub-system (e.g., <NUM> or <NUM>) and/or the instructions of the operating system (e.g., <NUM>, <NUM>) in general and the device driver and the hypervisor <NUM> in particular. When the instructions are executed by the controller <NUM> and/or the processing device <NUM>, the instructions cause the controller <NUM> and/or the processing device <NUM> to perform a method discussed above.

<FIG> illustrates an example machine of a computer system <NUM> within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, can be executed. In some embodiments, the computer system <NUM> can correspond to a host system (e.g., the host system <NUM> of <FIG>) that includes, is coupled to, or utilizes a memory sub-system (e.g., the memory sub-system <NUM> of <FIG>) or can be used to perform the operations of a data orchestrator <NUM> (e.g., to execute instructions to perform operations corresponding to the data orchestrator <NUM> described with reference to <FIG>). In alternative embodiments, the machine can be connected (e.g., networked) to other machines in a LAN, an intranet, an extranet, and/or the Internet. The machine can operate in the capacity of a server or a client machine in client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.

The machine can be a personal computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a cellular telephone, a web appliance, a server, a network router, a switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while a single machine is illustrated, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

The example computer system <NUM> includes a processing device <NUM>, a main memory <NUM> (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), static random access memory (SRAM), etc.), and a data storage system <NUM>, which communicate with each other via a bus <NUM> (which can include multiple buses).

Processing device <NUM> represents one or more general-purpose processing devices such as a microprocessor, a central processing unit, or the like. More particularly, the processing device can be a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets, or processors implementing a combination of instruction sets. Processing device <NUM> can also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like. The processing device <NUM> is configured to execute instructions <NUM> for performing the operations and steps discussed herein. The computer system <NUM> can further include a network interface device <NUM> to communicate over the network <NUM>.

The data storage system <NUM> can include a machine-readable storage medium <NUM> (also known as a computer-readable medium) on which is stored one or more sets of instructions <NUM> or software embodying any one or more of the methodologies or functions described herein. The instructions <NUM> can also reside, completely or at least partially, within the main memory <NUM> and/or within the processing device <NUM> during execution thereof by the computer system <NUM>, the main memory <NUM> and the processing device <NUM> also constituting machine-readable storage media. The machine-readable storage medium <NUM>, data storage system <NUM>, and/or main memory <NUM> can correspond to the memory sub-system <NUM> of <FIG>.

In one embodiment, the instructions <NUM> include instructions to implement functionality corresponding to a data orchestrator <NUM> (e.g., the data orchestrator <NUM> described with reference to <FIG>). While the machine-readable storage medium <NUM> is shown in an example embodiment to be a single medium, the term "machine-readable storage medium" should be taken to include a single medium or multiple media that store the one or more sets of instructions. The term "machine-readable storage medium" shall also be taken to include any medium that is capable of storing or encoding a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure. The term "machine-readable storage medium" shall accordingly be taken to include, but not be limited to, solid-state memories, optical media, and magnetic media.

The present disclosure also relates to an apparatus for performing the operations herein. This apparatus can be specially constructed for the intended purposes, or it can include a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program can be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus.

Various general purpose systems can be used with programs in accordance with the teachings herein, or it can prove convenient to construct a more specialized apparatus to perform the method. The structure for a variety of these systems will appear as set forth in the description below. In addition, the present disclosure is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the teachings of the disclosure as described herein.

The present disclosure can be provided as a computer program product, or software, that can include a machine-readable medium having stored thereon instructions, which can be used to program a computer system (or other electronic devices) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). In some embodiments, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., a computer) readable storage medium such as a read only memory ("ROM"), random access memory ("RAM"), magnetic disk storage media, optical storage media, flash memory components, etc..

In this description, various functions and operations are described as being performed by or caused by computer instructions to simplify description. However, those skilled in the art will recognize what is meant by such expressions is that the functions result from execution of the computer instructions by one or more controllers or processors, such as a microprocessor. Alternatively, or in combination, the functions and operations can be implemented using special purpose circuitry, with or without software instructions, such as using Application-Specific Integrated Circuit (ASIC) or Field-Programmable Gate Array (FPGA). Embodiments can be implemented using hardwired circuitry without software instructions, or in combination with software instructions. Thus, the techniques are limited neither to any specific combination of hardware circuitry and software, nor to any particular source for the instructions executed by the data processing system.

In the foregoing specification, embodiments of the disclosure have been described with reference to specific example embodiments thereof.

Claim 1:
A memory system (<NUM>, <NUM>) , comprising:
a plurality of memory components (109A; ..., 109N) configured to store data; and
a controller (<NUM>), operatively coupled to the plurality of memory components (109A; ..., 109N), to at least:
communicate with a host system (<NUM>) via a bus; and
service the data to the host system (<NUM>) via communications over the bus;
characterized by the controller (<NUM>) configured to:
communicate with a processing device (<NUM>) that is separate from the host system (<NUM>) using a Message Passing Interface, MPI, (<NUM>) over the bus;
predict a data movement over the bus using a machine learning model, the machine learning model bring trained using data usage information corresponding to historical data access requests transferring data between the memory components; and
provide data access to the processing device (<NUM>) through communications made using the Message Passing Interface, MPI, (<NUM>) over the bus.