Patent Description:
In computing devices, memory can include both cache memory and main memory. Cache memory can be a very high-speed memory that acts as a buffer between the main memory and a CPU to hold frequently used data and instructions for immediate availability to the CPU. For example, certain computers can include Double Data Rate (DDR) Synchronous Dynamic Random-Access Memory (SDRAM) packaged with a CPU as cache memory for the CPU. Such cache memory is sometimes referred to as "near memory" for being proximate to the CPU. In addition to the near memory, the CPU can also interface with the main memory via Compute Express Link (CXL) or other suitable types of interface protocols. The main memory can sometimes be referred to as "far memory" due to being at farther distances from the CPU than the near memory.

Using DDR SDRAM as cache memory for a CPU can have certain drawbacks. For example, the near memory is typically more expensive than the far memory and not available to be accessed by or even visible to an Operating System (OS) on a computing device. Instead, the CPU has exclusive control over the near memory. In addition, near memory devices, such as DDR SDRAM, can be very expensive. In some datacenter servers, costs of DDR SDRAM used as near memory can be up to about fifty percent of the total costs of the datacenter servers. Thus, if the near memory can be visible to and accessible by the OS, capital investments for the datacenter servers and associated costs for providing various computing services from the datacenter servers can be significantly reduced.

Several embodiments of the disclosed technology are directed to implementing memory tiering according to which near memory is used as a swap buffer for far memory instead of being dedicated cache memory for a CPU in a computing device. As such, the CPU can continue caching data in the near memory while the near memory and the far memory are exposed to the OS as addressable and allocatable system memory. In certain implementations, a hardware memory controller can be configured to control swapping operations at a cache-line granularity (e.g., <NUM> bytes). As such, the computing device would not need any software intervention or cause software impact. In other implementations, a memory controller with both hardware and software components may be used for controlling such swapping operations.

The ratio of storage spaces between near memory and far memory can be flexible. For instance, the ratio between near memory and far memory can be one to any integer greater than or equal to one. In an illustrative example, a range of system memory addresses can be covered by a combination of near memory and far memory in a ratio of one to three. As such, the range of system memory can be divided into four sections, e.g., A, B, C, and D. Each section can include a data portion (e.g., <NUM> bits) and a metadata portion (e.g., <NUM> bits). The data portion can be configured to contain data representing user data or instructions executable by the CPU in the computing device. The metadata portion can include data representing various attributes of the data in the data portion. For instance, the metadata portion can include Error Checking and Correction (ECC) bits or other suitable types of information.

In accordance with several embodiments of the disclosed technology, several bits (e.g., ECC bits) in the metadata portion in the near memory can be configured to indicate (<NUM>) which section of the range of system memory the near memory current holds; and (<NUM>) locations of additional sections of the range of system memory in the far memory. For instance, in the example above with four sections of system memory, eight bits in the metadata portion in the near memory can be configured to indicate the foregoing information. For instance, a first pair of first two bits can be configured to indicate which section is currently held in the near memory as follows:.

As such, a memory controller can readily determine that the near memory contains data from section A of the system memory when Bit <NUM> and Bit <NUM> contains zero and zero, respectively.

In the illustrated example above, while the first two bits correspond to the near memory, the additional six bits can be subdivided into three pairs individually corresponding to a location in the far memory mapped to corresponding sections of the range of system memory. For instance, the second, third, and four pairs can each correspond to a first, second, or third location in the far memory, as follows:.

As such, the memory controller can readily determine where data from a section of the system memory is in the far memory even though the data is not currently in the near memory. For instance, when the second pair (i.e., Bit <NUM> and Bit <NUM>) contains (<NUM>, <NUM>), the memory controller can be configured to determine that data corresponding to Section A of the system memory is in first location in the far memory. Though the foregoing example uses eight bits in the metadata portion to encode locations of the individual sections of the range of system memory, in other implementations, other suitable numbers of bits in the metadata portion may be used to encode the same information. For instance, in the illustrated examples above with four sections, five, six, or seven bits may be used to encode location information of the sections.

Using the data from the metadata portion in the near memory, the memory controller can be configured to manage swap operations between the near and far memory in order to use the near memory as a swap buffer. For instance, during a read operation, the memory controller can be configured to read from the near memory to retrieve data from both the data portion and the metadata portion of the near memory. The memory controller can then be configured to determine which section of the system memory the retrieved data corresponds to using the tables above, and whether the determined section matches a target section to be read. For instance, when the target section is section A, and the first two bits from the metadata portion contains (<NUM>, <NUM>), then the memory controller can be configured to determine that the retrieved data is from section A. Thus, the memory controller can forward the retrieved data from Section A to a requesting entity, such as an application or OS executed on the computing device.

On the other hand, when the first two bits from the metadata portion contains (<NUM>, <NUM>) instead of (<NUM>, <NUM>), for example, the memory controller can be configured to determine that the retrieved data belongs to section B (referred to as "B data"), not section A (referred to as "A data"). The memory controller can then continue to examine the additional bits in the metadata portion to determine which pair of bits contains (<NUM>, <NUM>). For example, when the second pair (Bit <NUM> and Bit <NUM>) from the metadata portion contains (<NUM>, <NUM>), then the memory controller can be configured to determine that A data is located at the first location in the far memory. In response, the memory controller can be configured to read A data from the first location in the far memory and provide the A data to the requesting entity. The memory controller can then be configured to write the retrieved A data into the near memory and the previously retrieved B data to the first section in the far memory. The memory controller can also be configured to modify the bits in the metadata portion in the near memory to reflect the swapping of data between section A and section B in the near memory.

During a write operation, the memory controller can be configured to first read the data from the metadata portion in the near memory. The memory controller can be configured to then determine data from which section of the system memory is currently held in the near memory, and whether the determined section matches a target section to be written. For instance, when the target section for the write operation is section A, and the first two bits from the metadata portion contains (<NUM>, <NUM>), then the memory controller can be configured to determine that A data is currently in the near memory. In response, the memory controller can be configured to overwrite the data in the data portion of the near memory and report a completion of the write operation.

On the other hand, when the first two bits from the metadata portion contains (<NUM>, <NUM>), then the memory controller can be configured to determine that data B is currently in the near memory. In response, the memory controller can be configured to refrain from writing to the near memory and instead continue examining the additional bits of the metadata portion to determine which pair of bits contains (<NUM>, <NUM>). For example, when the second pair (Bit <NUM> and Bit <NUM>) from the metadata portion contains (<NUM>, <NUM>), then the memory controller can be configured to determine that A data is currently located at the first location in the far memory. In response, the memory controller can be configured to write to the first location in the far memory instead of the near memory. Upon completion, the memory controller can be configured to report a completion of the write operation.

Several embodiments of the disclosed technology can improve operations and performance of a computing device by allowing memory previously used as cache memory and invisible to an OS to be configured as system memory addressable by the OS. For instance, instead of using the near memory as dedicated cache memory for the CPU, the near memory can be used as allocatable system memory while continue to provide caching functionality to the CPU via the swapping operations described above. By increasing the amount of addressable system memory, computing or other suitable types of latency can be decreased in the computing device.

Certain embodiments of systems, devices, components, modules, routines, data structures, and processes for memory operations management are described below. In the following description, specific details of components are included to provide a thorough understanding of certain embodiments of the disclosed technology. A person skilled in the relevant art will also understand that the technology can have additional embodiments. The technology can also be practiced without several of the details of the embodiments described below with reference to <FIG>. For example, instead of being implemented in datacenters or other suitable distributed computing systems, aspects of the memory operations management technique disclosed herein can also be implemented on personal computers, smartphones, tablets, or other suitable types of computing devices.

As used herein, the term "distributed computing system" generally refers to an interconnected computer system having multiple network nodes that interconnect a plurality of servers or hosts to one another and/or to external networks (e.g., the Internet). The term "network node" generally refers to a physical network device. Example network nodes include routers, switches, hubs, bridges, load balancers, security gateways, or firewalls. A "host" generally refers to a physical computing device. In certain embodiments, a host can be configured to implement, for instance, one or more virtual machines, virtual switches, or other suitable virtualized components. For example, a host can include a server having a hypervisor configured to support one or more virtual machines, virtual switches, or other suitable types of virtual components. In other embodiments, a host can be configured to execute suitable applications directly on top of an operating system.

A computer network can be conceptually divided into an overlay network implemented over an underlay network in certain implementations. An "overlay network" generally refers to an abstracted network implemented over and operating on top of an underlay network. The underlay network can include multiple physical network nodes interconnected with one another. An overlay network can include one or more virtual networks. A "virtual network" generally refers to an abstraction of a portion of the underlay network in the overlay network. A virtual network can include one or more virtual end points referred to as "tenant sites" individually used by a user or "tenant" to access the virtual network and associated computing, storage, or other suitable resources. A tenant site can host one or more tenant end points ("TEPs"), for example, virtual machines. The virtual networks can interconnect multiple TEPs on different hosts. Virtual network nodes in the overlay network can be connected to one another by virtual links individually corresponding to one or more network routes along one or more physical network nodes in the underlay network. In other implementations, a computer network can only include the underlay network.

Also used herein, the term "near memory" generally refers to memory that is physically proximate to a processor (e.g., a CPU) than other "far memory" at a distance from the processor. For example, near memory can include one or more DDR SDRAM dies that is incorporated into an Integrated Circuit (IC) component package with one or more CPU dies via an interposer and/or through silicon vias. In contrast, far memory can include additional memory on accelerators, memory buffers, or smart I/O devices that the CPU can interface with via CXL or other suitable types of protocols. For instance, in datacenters, multiple memory devices on multiple servers/server blades may be pooled to be allocatable to a single CPU on one of the servers/server blades. The CPU can access the allocated far memory via a computer network in datacenters.

<FIG> is a schematic diagram illustrating a distributed computing system <NUM> implementing memory operations management in accordance with embodiments of the disclosed technology. As shown in <FIG>, the distributed computing system <NUM> can include an underlay network <NUM> interconnecting a plurality of hosts <NUM>, a plurality of client devices <NUM> associated with corresponding users <NUM>, and a platform controller <NUM> operatively coupled to one another. The platform controller <NUM> can be a cluster controller, a fabric controller, a database controller, and/or other suitable types of controller configured to monitor and manage resources and operations of the servers <NUM> and/or other components in the distributed computing system <NUM>. Even though components of the distributed computing system <NUM> are shown in <FIG>, in other embodiments, the distributed computing system <NUM> can also include additional and/or different components or arrangements. For example, in certain embodiments, the distributed computing system <NUM> can also include network storage devices, additional hosts, and/or other suitable components (not shown) in other suitable configurations.

As shown in <FIG>, the underlay network <NUM> can include one or more network nodes <NUM> that interconnect the multiple hosts <NUM> and the client device <NUM> of the users <NUM>. In certain embodiments, the hosts <NUM> can be organized into racks, action zones, groups, sets, or other suitable divisions. For example, in the illustrated embodiment, the hosts <NUM> are grouped into three host sets identified individually as first, second, and third host sets 107a-107c. Each of the host sets 107a-107c is operatively coupled to a corresponding network nodes 112a-112c, respectively, which are commonly referred to as "top-of-rack" network nodes or "TORs. " The TORs 112a-112c can then be operatively coupled to additional network nodes <NUM> to form a computer network in a hierarchical, flat, mesh, or other suitable types of topology. The underlay network can allow communications among hosts <NUM>, the platform controller <NUM>, and the users <NUM>. In other embodiments, the multiple host sets 107a-107c may share a single network node <NUM> or can have other suitable arrangements.

The hosts <NUM> can individually be configured to provide computing, storage, and/or other suitable cloud or other suitable types of computing services to the users <NUM>. For example, as described in more detail below with reference to <FIG>, one of the hosts <NUM> can initiate and maintain one or more virtual machines <NUM> (shown in <FIG>) or containers (not shown) upon requests from the users <NUM>. The users <NUM> can then utilize the provided virtual machines <NUM> or containers to perform database, computation, communications, and/or other suitable tasks. In certain embodiments, one of the hosts <NUM> can provide virtual machines <NUM> for multiple users <NUM>. For example, the host 106a can host three virtual machines <NUM> individually corresponding to each of the users 101a-101c. In other embodiments, multiple hosts <NUM> can host virtual machines <NUM> for the users 101a-101c.

The client devices <NUM> can each include a computing device that facilitates the users <NUM> to access computing services provided by the hosts <NUM> via the underlay network <NUM>. In the illustrated embodiment, the client devices <NUM> individually include a desktop computer. In other embodiments, the client devices <NUM> can also include laptop computers, tablet computers, smartphones, or other suitable computing devices. Though three users <NUM> are shown in <FIG> for illustration purposes, in other embodiments, the distributed computing system <NUM> can facilitate any suitable numbers of users <NUM> to access cloud or other suitable types of computing services provided by the hosts <NUM> in the distributed computing system <NUM>.

<FIG> is a schematic diagram illustrating certain hardware/software components of the distributed computing system <NUM> in accordance with embodiments of the disclosed technology. <FIG> illustrates an overlay network <NUM>' that can be implemented on the underlay network <NUM> in <FIG>. Though particular configuration of the overlay network <NUM>' is shown in <FIG>, In other embodiments, the overlay network <NUM>' can also be configured in other suitable ways. In <FIG>, only certain components of the underlay network <NUM> of <FIG> are shown for clarity.

In <FIG> and in other Figures herein, individual software components, objects, classes, modules, and routines may be a computer program, procedure, or process written as source code in C, C++, C#, Java, and/or other suitable programming languages. A component may include, without limitation, one or more modules, objects, classes, routines, properties, processes, threads, executables, libraries, or other components. Components may be in source or binary form. Components may include aspects of source code before compilation (e.g., classes, properties, procedures, routines), compiled binary units (e.g., libraries, executables), or artifacts instantiated and used at runtime (e.g., objects, processes, threads).

Components within a system may take different forms within the system. As one example, a system comprising a first component, a second component and a third component can, without limitation, encompass a system that has the first component being a property in source code, the second component being a binary compiled library, and the third component being a thread created at runtime. The computer program, procedure, or process may be compiled into object, intermediate, or machine code and presented for execution by one or more processors of a personal computer, a network server, a laptop computer, a smartphone, and/or other suitable computing devices.

Equally, components may include hardware circuitry. A person of ordinary skill in the art would recognize that hardware may be considered fossilized software, and software may be considered liquefied hardware. As just one example, software instructions in a component may be burned to a Programmable Logic Array circuit or may be designed as a hardware circuit with appropriate integrated circuits. Equally, hardware may be emulated by software. Various implementations of source, intermediate, and/or object code and associated data may be stored in a computer memory that includes read-only memory, random-access memory, magnetic disk storage media, optical storage media, flash memory devices, and/or other suitable computer readable storage media excluding propagated signals.

As shown in <FIG>, the source host 106a and the destination hosts 106b and 106b' (only the destination hosts 106b is shown with detail components) can each include a processor <NUM>, a memory <NUM>, a network interface card <NUM>, and a packet processor <NUM> operatively coupled to one another. In other embodiments, the hosts <NUM> can also include input/output devices configured to accept input from and provide output to an operator and/or an automated software controller (not shown), or other suitable types of hardware components.

The processor <NUM> can include a microprocessor, caches, and/or other suitable logic devices. The memory <NUM> can include volatile and/or nonvolatile media (e.g., ROM; RAM, magnetic disk storage media; optical storage media; flash memory devices, and/or other suitable storage media) and/or other types of computer-readable storage media configured to store data received from, as well as instructions for, the processor <NUM> (e.g., instructions for performing the methods discussed below with reference to Figures 5A-5D). Though only one processor <NUM> and one memory <NUM> are shown in the individual hosts <NUM> for illustration in <FIG>, in other embodiments, the individual hosts <NUM> can include two, six, eight, or any other suitable number of processors <NUM> and/or memories <NUM>.

The source host 106a and the destination host 106b can individually contain instructions in the memory <NUM> executable by the processors <NUM> to cause the individual processors <NUM> to provide a hypervisor <NUM> (identified individually as first and second hypervisors 140a and 140b) and an operating system <NUM> (identified individually as first and second operating systems 141a and 141b). Even though the hypervisor <NUM> and the operating system <NUM> are shown as separate components, in other embodiments, the hypervisor <NUM> can operate on top of the operating system <NUM> executing on the hosts <NUM> or a firmware component of the hosts <NUM>.

The hypervisors <NUM> can individually be configured to generate, monitor, terminate, and/or otherwise manage one or more virtual machines <NUM> organized into tenant sites <NUM>. For example, as shown in <FIG>, the source host 106a can provide a first hypervisor 140a that manages first and second tenant sites 142a and 142b, respectively. The destination host 106b can provide a second hypervisor 140b that manages first and second tenant sites 142a' and 142b', respectively. The hypervisors <NUM> are individually shown in <FIG> as a software component. However, in other embodiments, the hypervisors <NUM> can be firmware and/or hardware components. The tenant sites <NUM> can each include multiple virtual machines <NUM> for a particular tenant (not shown). For example, the source host 106a and the destination host 106b can both host the tenant site 142a and 142a' for a first tenant 101a (<FIG>). The source host 106a and the destination host 106b can both host the tenant site 142b and 142b' for a second tenant 101b (<FIG>). Each virtual machine <NUM> can be executing a corresponding operating system, middleware, and/or applications.

Also shown in <FIG>, the distributed computing system <NUM> can include an overlay network <NUM>' having one or more virtual networks <NUM> that interconnect the tenant sites 142a and 142b across multiple hosts <NUM>. For example, a first virtual network 142a interconnects the first tenant sites 142a and 142a' at the source host 106a and the destination host 106b. A second virtual network 146b interconnects the second tenant sites 142b and 142b' at the source host 106a and the destination host 106b. Even though a single virtual network <NUM> is shown as corresponding to one tenant site <NUM>, in other embodiments, multiple virtual networks <NUM> (not shown) may be configured to correspond to a single tenant site <NUM>.

The virtual machines <NUM> can be configured to execute one or more applications <NUM> to provide suitable cloud or other suitable types of computing services to the users <NUM> (<FIG>). For example, the source host 106a can execute an application <NUM> that is configured to provide a computing service that monitors online trading and distribute price data to multiple users <NUM> subscribing to the computing service. The virtual machines <NUM> on the virtual networks <NUM> can also communicate with one another via the underlay network <NUM> (<FIG>) even though the virtual machines <NUM> are located on different hosts <NUM>.

Communications of each of the virtual networks <NUM> can be isolated from other virtual networks <NUM>. In certain embodiments, communications can be allowed to cross from one virtual network <NUM> to another through a security gateway or otherwise in a controlled fashion. A virtual network address can correspond to one of the virtual machines <NUM> in a particular virtual network <NUM>. Thus, different virtual networks <NUM> can use one or more virtual network addresses that are the same. Example virtual network addresses can include IP addresses, MAC addresses, and/or other suitable addresses. To facilitate communications among the virtual machines <NUM>, virtual switches (not shown) can be configured to switch or filter packets directed to different virtual machines <NUM> via the network interface card <NUM> and facilitated by the packet processor <NUM>.

As shown in <FIG>, to facilitate communications with one another or with external devices, the individual hosts <NUM> can also include a network interface card ("NIC") <NUM> for interfacing with a computer network (e.g., the underlay network <NUM> of <FIG>). A NIC <NUM> can include a network adapter, a LAN adapter, a physical network interface, or other suitable hardware circuitry and/or firmware to enable communications between hosts <NUM> by transmitting/receiving data (e.g., as packets) via a network medium (e.g., fiber optic) according to Ethernet, Fibre Channel, Wi-Fi, or other suitable physical and/or data link layer standards. During operation, the NIC <NUM> can facilitate communications to/from suitable software components executing on the hosts <NUM>. Example software components can include the virtual switches <NUM>, the virtual machines <NUM>, applications <NUM> executing on the virtual machines <NUM>, the hypervisors <NUM>, or other suitable types of components.

In certain implementations, a packet processor <NUM> can be interconnected to and/or integrated with the NIC <NUM> to facilitate network traffic operations for enforcing communications security, performing network virtualization, translating network addresses, maintaining/limiting a communication flow state, or performing other suitable functions. In certain implementations, the packet processor <NUM> can include a Field-Programmable Gate Array ("FPGA") integrated with the NIC <NUM>.

An FPGA can include an array of logic circuits and a hierarchy of reconfigurable interconnects that allow the logic circuits to be "wired together" like logic gates by a user after manufacturing. As such, a user <NUM> can configure logic blocks in FPGAs to perform complex combinational functions, or merely simple logic operations to synthetize equivalent functionality executable in hardware at much faster speeds than in software. In the illustrated embodiment, the packet processor <NUM> has one interface communicatively coupled to the NIC <NUM> and another coupled to a network switch (e.g., a Top-of-Rack or "TOR" switch) at the other. In other embodiments, the packet processor <NUM> can also include an Application Specific Integrated Circuit ("ASIC"), a microprocessor, or other suitable hardware circuitry. In any of the foregoing embodiments, the packet processor <NUM> can be programmed by the processor <NUM> (or suitable software components associated therewith) to route packets inside the packet processor <NUM> to achieve various aspects of time-sensitive data delivery, as described in more detail below with reference to <FIG>.

In operation, the processor <NUM> and/or a user <NUM> (<FIG>) can configure logic circuits in the packet processor <NUM> to perform complex combinational functions or simple logic operations to synthetize equivalent functionality executable in hardware at much faster speeds than in software. For example, the packet processor <NUM> can be configured to process inbound/outbound packets for individual flows according to configured policies or rules contained in a flow table such as a MAT. The flow table can contain data representing processing actions corresponding to each flow for enabling private virtual networks with customer supplied address spaces, scalable load balancers, security groups and Access Control Lists ("ACLs"), virtual routing tables, bandwidth metering, Quality of Service ("QoS"), etc..

As such, once the packet processor <NUM> identifies an inbound/outbound packet as belonging to a particular flow, the packet processor <NUM> can apply one or more corresponding policies in the flow table before forwarding the processed packet to the NIC <NUM> or TOR <NUM>. For example, as shown in <FIG>, the application <NUM>, the virtual machine <NUM>, and/or other suitable software components on the source host 106a can generate an outbound packet destined to, for instance, other applications <NUM> at the destination hosts 106b and 106b'. The NIC <NUM> at the source host 106a can forward the generated packet to the packet processor <NUM> for processing according to certain policies in a flow table. Once processed, the packet processor <NUM> can forward the outbound packet to the first TOR 112a, which in turn forwards the packet to the second TOR 112b via the overlay/underlay network <NUM> and <NUM>'.

The second TOR 112b can then forward the packet to the packet processor <NUM> at the destination hosts 106b and 106b' to be processed according to other policies in another flow table at the destination hosts 106b and 106b'. If the packet processor <NUM> cannot identify a packet as belonging to any flow, the packet processor <NUM> can forward the packet to the processor <NUM> via the NIC <NUM> for exception processing. In another example, when the first TOR 112a receives an inbound packet , for instance, from the destination host 106b via the second TOR 112b, the first TOR 112a can forward the packet to the packet processor <NUM> to be processed according to a policy associated with a flow of the packet. The packet processor <NUM> can then forward the processed packet to the NIC <NUM> to be forwarded to, for instance, the application <NUM> or the virtual machine <NUM>.

In certain embodiments, the memory <NUM> can include both cache memory and main memory (not shown). Cache memory can be a very high-speed memory that acts as a buffer between the main memory and the processor <NUM> to hold frequently used data and instructions for immediate availability to the processor <NUM>. For example, certain computers can include Double Data Rate (DDR) Synchronous Dynamic Random-Access Memory (SDRAM) packaged with a processor <NUM> as cache memory for the processor <NUM>. Such cache memory is sometimes referred to as "near memory" for being proximate to the processor <NUM>. In addition to the near memory, the processor 132can also interface with the main memory via Compute Express Link (CXL) or other suitable types of interface protocols. The main memory can sometimes be referred to as "far memory" due to father distances from the processor <NUM> than the near memory.

The use of DDR SDRAM as cache memory for a processor <NUM> can have certain drawbacks. For example, the near memory is typically more expensive than the far memory and not available to be accessed by or even visible to an operating system (OS) on a computing device. Instead, the processor <NUM> has exclusive control over the near memory. In addition, near memory devices, such as DDR SDRAM, can be very expensive. In some datacenter servers, costs of DDR SDRAM as near memory can be up to about fifty percent of the total costs of the servers. Thus, if the near memory can be visible to and accessible by the operating system <NUM>, capital investments for the servers and associated costs for providing various computing services from the hosts <NUM> can be significantly reduced.

Several embodiments of the disclosed technology are directed to implementing memory tiering according to which near memory is used as a swap buffer for far memory instead of being used as dedicated cache memory for the processor <NUM>. As such, the processor <NUM> can continue caching data in the near memory while the near memory and the far memory are exposed to the operating system <NUM> as addressable system memory. In certain implementations, a hardware memory controller (not shown) can be configured to control swapping operations at a cache-line granularity (e.g., <NUM> bytes). As such, the host <NUM> would not experience any software intervention or impact. In other implementations, a memory controller with both hardware and software components may be used for controlling such swapping operations.

A ratio of storage space between near memory and far memory can be flexible. For instance, the ratio between near memory and far memory can be one to any integer greater than or equal to one. In an illustrative example shown in <FIG>, a range of system memory address <NUM> is covered by a combination of near memory <NUM> and far memory <NUM> in a ratio of one to three. As such, the range of system memory <NUM> can be divided into four sections <NUM>, e.g., A, B, C, and D (identified in <FIG> as 152A-152D, respectively). Each section can include a data portion <NUM> (e.g., <NUM> bits) and a metadata portion <NUM> (e.g., <NUM> bits). The data portion <NUM> can be configured to contain data representing user data or instructions executed in the host <NUM> (<FIG>). The metadata portion <NUM> can include data representing various attributes of the data in the data portion <NUM>. For instance, the metadata portion <NUM> can include error checking and correction bits or other suitable types of information.

In accordance with several embodiments of the disclosed technology, several bits in the metadata portion <NUM> in the near memory <NUM> can be configured to indicate (<NUM>) which section of the range of system memory the near memory <NUM> current holds; and (<NUM>) locations of additional sections of the range of system memory in the far memory. In the example with four sections of system memory <NUM>, eight bits in the metadata portion <NUM> in the near memory <NUM> can be configured to indicate the foregoing information. For instance, a first pair of first two bits can be configured to indicate which section <NUM> is currently held in the near memory <NUM> as follows:.

As such, a memory controller <NUM> can readily determine that the near memory <NUM> contains data from section A of the system memory when the Bit <NUM> and Bit <NUM> contains zero and zero, respective, as illustrated in <FIG>.

While the first two bits correspond to the near memory <NUM>, the additional six bits can be subdivided into three pairs individually corresponding to a location in the far memory <NUM>, as illustrated in <FIG>. For instance, the second, third, and four pairs can each correspond to a first, second, or third location in the far memory <NUM>, as follows:.

As such, the memory controller <NUM> can readily determine where data from a particular section of the system memory <NUM> is in the far memory <NUM> even though the data is not currently in the near memory <NUM>. For instance, when the second pair (i.e., Bit <NUM> and Bit <NUM>) contains (<NUM>, <NUM>), the memory controller <NUM> can be configured to determine that data corresponding to Section D of the system memory <NUM> is in first location 158A in the far memory <NUM>. When the third pair (i.e., Bit <NUM> and Bit <NUM>) contains (<NUM>, <NUM>), the memory controller <NUM> can be configured to determine that data corresponding to Section C of the system memory <NUM> is in second location 158B in the far memory <NUM>. When the fourth pair (i.e., Bit <NUM> and Bit <NUM>) contains (<NUM>, <NUM>), the memory controller <NUM> can be configured to determine that data corresponding to Section B of the system memory <NUM> is in third location 158C in the far memory <NUM>, as illustrated in <FIG>.

Using the data from the metadata portion <NUM> in the near memory <NUM>, the memory controller <NUM> can be configured to manage swap operations between the near memory <NUM> and the far memory <NUM> using the near memory <NUM> as a swap buffer. For example, as shown in <FIG>, during a read operation, the CPU can issue a command to the memory controller <NUM> to read data corresponding to section A when such data is not currently residing in a last level cache of the CPU. In response, the memory controller <NUM> can be configured to read from the near memory to retrieve data from both the data portion and the metadata portion of the near memory. The memory controller <NUM> can then be configured to determine which section of the system memory the retrieved data corresponds to using the tables above, and whether the determined section matches a target section to be read. For instance, as shown in <FIG>, when the target section is section A, and the first two bits from the metadata portion contains (<NUM>, <NUM>), then the memory controller <NUM> can be configured to determine that the retrieved data is from section A (i.e., "A data"). Thus, the memory controller <NUM> can forward the retrieved A data from Section A to a requesting entity, such as an application executed by the CPU on the computing device.

On the other hand, as shown in <FIG>, when the first two bits from the metadata portion contains (<NUM>, <NUM>) instead of (<NUM>, <NUM>), the memory controller <NUM> can be configured to determine that the retrieved data belongs to section B (referred to as "B data"), not section A data. The memory controller <NUM> can then continue to examine the additional bits in the metadata portion to determine which pair of bits contains (<NUM>, <NUM>). For example, when the second pair (Bit <NUM> and Bit <NUM>) from the metadata portion contains (<NUM>, <NUM>), then the memory controller <NUM> can be configured to determine that A data is located at the first location in the far memory. In response, the memory controller <NUM> can be configured to read A data from the first location in the far memory and provide the A data to the requesting entity. The memory controller <NUM> can then be configured to write the retrieved A data into the near memory and the previously retrieved B data to the first section in the far memory. The memory controller <NUM> can also be configured to modify the bits in the metadata portion in the near memory to reflect the swapping between section A and section B.

During a write operation, as shown in <FIG>, the memory controller <NUM> can be configured to first read the data from the metadata portion in the near memory. The memory controller <NUM> can then determine data from which section of the system memory is currently held in the near memory, and whether the determined section matches a target section to be written. For instance, when the target section is section A, and the first two bits from the metadata portion contains (<NUM>, <NUM>), then the memory controller <NUM> can be configured to determine that A data is currently in the near memory. Thus, the memory controller <NUM> can overwrite the data in the data portion of the near memory and report a completion of the write operation.

On the other hand, when the first two bits from the metadata portion contains (<NUM>, <NUM>), then the memory controller <NUM> can be configured to determine that data B is currently in the near memory. In response, the memory controller <NUM> can be configured to refrain from writing to the near memory and instead continue examining the additional bits of the metadata portion to determine which pair of bits contains (<NUM>, <NUM>). For example, when the second pair (Bit <NUM> and Bit <NUM>) from the metadata portion contains (<NUM>, <NUM>), then the memory controller <NUM> can be configured to determine that A data is located at the first location in the far memory. In response, the memory controller <NUM> can be configured to write to the first location in the far memory instead of the near memory and report a completion of the write operation.

<FIG> is a computing device <NUM> suitable for certain components of the distributed computing system <NUM> in <FIG>. For example, the computing device <NUM> can be suitable for the hosts <NUM>, the client devices <NUM>, or the platform controller <NUM> of <FIG>. In a very basic configuration <NUM>, the computing device <NUM> can include one or more processors <NUM> and a system memory <NUM>. A memory bus <NUM> can be used for communicating between processor <NUM> and system memory <NUM>.

Depending on the desired configuration, the processor <NUM> can be of any type including but not limited to a microprocessor (µP), a microcontroller (µC), a digital signal processor (DSP), or any combination thereof. The processor <NUM> can include one more level of caching, such as a level-one cache <NUM> and a level-two cache <NUM>, a processor core <NUM>, and registers <NUM>. An example processor core <NUM> can include an arithmetic logic unit (ALU), a floating-point unit (FPU), a digital signal processing core (DSP Core), or any combination thereof. An example memory controller <NUM> can also be used with processor <NUM>, or in some implementations memory controller <NUM> can be an internal part of processor <NUM>.

Depending on the desired configuration, the system memory <NUM> can be of any type including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.) or any combination thereof. The system memory <NUM> can include an operating system <NUM>, one or more applications <NUM>, and program data <NUM>. As shown in Figure <NUM>, the operating system <NUM> can include a hypervisor <NUM> for managing one or more virtual machines <NUM>. This described basic configuration <NUM> is illustrated in Figure <NUM> by those components within the inner dashed line.

The computing device <NUM> can have additional features or functionality, and additional interfaces to facilitate communications between basic configuration <NUM> and any other devices and interfaces. For example, a bus/interface controller <NUM> can be used to facilitate communications between the basic configuration <NUM> and one or more data storage devices <NUM> via a storage interface bus <NUM>. The data storage devices <NUM> can be removable storage devices <NUM>, non-removable storage devices <NUM>, or a combination thereof. Examples of removable storage and non-removable storage devices include magnetic disk devices such as flexible disk drives and hard-disk drives (HDD), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid state drives (SSD), and tape drives to name a few. Example computer storage media can include volatile and nonvolatile, removable, and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. The term "computer readable storage media" or "computer readable storage device" excludes propagated signals and communication media.

The system memory <NUM>, removable storage devices <NUM>, and non-removable storage devices <NUM> are examples of computer readable storage media. Computer readable storage media include, but not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other media which can be used to store the desired information, and which can be accessed by computing device <NUM>. Any such computer readable storage media can be a part of computing device <NUM>. The term "computer readable storage medium" excludes propagated signals and communication media.

The computing device <NUM> can also include an interface bus <NUM> for facilitating communication from various interface devices (e.g., output devices <NUM>, peripheral interfaces <NUM>, and communication devices <NUM>) to the basic configuration <NUM> via bus/interface controller <NUM>. Example output devices <NUM> include a graphics processing unit <NUM> and an audio processing unit <NUM>, which can be configured to communicate to various external devices such as a display or speakers via one or more A/V ports <NUM>. Example peripheral interfaces <NUM> include a serial interface controller <NUM> or a parallel interface controller <NUM>, which can be configured to communicate with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripheral devices (e.g., printer, scanner, etc.) via one or more I/O ports <NUM>. An example communication device <NUM> includes a network controller <NUM>, which can be arranged to facilitate communications with one or more other computing devices <NUM> over a network communication link via one or more communication ports <NUM>.

The network communication link can be one example of a communication media. Communication media can typically be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and can include any information delivery media. A "modulated data signal" can be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media can include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), microwave, infrared (IR) and other wireless media. The term computer readable media as used herein can include both storage media and communication media.

The computing device <NUM> can be implemented as a portion of a small-form factor portable (or mobile) electronic device such as a cell phone, a personal data assistant (PDA), a personal media player device, a wireless web-watch device, a personal headset device, an application specific device, or a hybrid device that include any of the above functions. The computing device <NUM> can also be implemented as a personal computer including both laptop computer and non-laptop computer configurations.

Claim 1:
A method of memory management in a computing device (<NUM>) having a processor (<NUM>), a first memory (<NUM>) proximate to and configured as a cache for the processor (<NUM>), a second memory (<NUM>) separate from and interfaced with the processor (<NUM>), and a memory controller (<NUM>) configured to manage operations of the first and second memory (<NUM>, <NUM>), the method comprising:
receiving, at the memory controller (<NUM>), a request from the processor (<NUM>) to read data corresponding to a system memory section (<NUM>) from the cache of the processor (<NUM>); and
in response to receiving the request to read, with the memory controller (<NUM>),
retrieving, from the first memory (<NUM>), data from a data portion and metadata from a metadata portion of the first memory (<NUM>), the metadata from the metadata portion encoding data location information of multiple system memory section (<NUM>) in the first memory (<NUM>) and the second memory (<NUM>);
analyzing the data location information in the retrieved metadata from the metadata portion of the first memory (<NUM>) to determine whether the first memory (<NUM>) currently contains data corresponding to the system memory section (<NUM>) in the received request; and
in response to determining that the first memory (<NUM>) currently contains data corresponding to the system memory section (<NUM>) in the received request, transmitting the retrieved data from the data portion of the first memory (<NUM>) to the processor (<NUM>) in response to the received request.