Method of allocating referenced memory pages from a free list

Memory pages that are allocated to a memory consumer and continue to be accessed by the memory consumer are included in a free list, so that they may be immediately allocated to another memory consumer as needed during the course of normal operation without preserving the original contents of the memory page. When a memory page in the free list is accessed to perform a read, a generation number associated with the memory page is compared with a stored copy. If the two match, the read is performed on the memory page. If the two do not match, the read is not performed on the memory page.

BACKGROUND

Computing systems typically allocate memory using a regime that guarantees the continued availability of the allocated memory. If a memory consumer requests an allocation of memory and there is sufficient free memory to satisfy the request, then the request is granted. The memory consumer may subsequently use the allocated memory until the memory consumer process terminates or explicitly releases the allocated memory. If sufficient free memory is not available to accommodate the memory allocation request, then the request is denied. Certain memory consumers are tolerant of being denied a memory allocation request, but some memory consumers fail catastrophically if they are denied a memory allocation request. Some kernel space memory consumers are intolerant of being denied memory allocation requests as they are critical to the proper operation of their respective systems services. To avoid catastrophic failure of these critical memory consumers, many computing systems are configured to operate with a significant reserve of memory that purposefully remains idle.

For example, in a conventional virtual machine (VM) setting, proper execution of each VM depends on an associated virtual machine monitor (VMM) having sufficient memory. The VMM may request a memory allocation during normal operation. If a host system has insufficient memory for the VMM at some point during VM execution, then the VMM is forced to kill its VM. If the average amount of idle memory falls below a predetermined threshold, then the host system may be reconfigured to reestablish a certain minimum amount of idle memory. Process migration may be used to rebalance VM system loads among host systems, thereby increasing idle memory on the host system encountering memory pressure. While maintaining an idle memory reserve serves to avoid catastrophic failures, this idle memory represents a potentially expensive unused resource.

SUMMARY

One or more embodiments of the present invention provide techniques for managing memory in a way that better utilizes idle memory resources. These techniques allow memory pages that are allocated to a memory consumer and continue to be accessed by the memory consumer to be included in a free list, so that they may be immediately allocated to another memory consumer as needed during the course of normal operation without preserving the original contents of the memory page. In one embodiment, such techniques allow all memory resources to be used without jeopardizing system stability.

A method of managing memory using a free list of memory pages, according to an embodiment of the present invention, includes the steps of receiving a memory allocation request from a memory consumer, selecting a memory page from the free list, and allocating the memory page to the memory consumer. The free list includes the memory page even though the memory page has been allocated to the memory consumer, and the memory consumer retains a reference to the allocated memory page. In one embodiment, the memory page is removed from the free list upon allocation and then returned to a tail of the free list to be made available for allocation to other memory consumers.

A method of performing a read on a memory page that is in a free list of memory pages, according to an embodiment of the present invention, includes the steps of performing a read on a memory page that is in the free list, and retrieving after the read a current generation number associated with the memory page. If the current generation number retrieved after the read matches the stored copy of the generation number, the read data is considered valid. On the other hand, if the current generation number retrieved after the read does not match the stored copy of the generation number, then the read data is considered invalid.

Further embodiments of the present invention include, without limitation, a non-transitory computer-readable medium that includes instructions that enable a processing unit to implement one or more aspects of the above methods as well as a computer system configured to implement one or more aspects of the above methods.

DETAILED DESCRIPTION

A. System Architecture

FIG. 1Adepicts a block diagram of a computer system100configured to implement one or more aspects of the present invention. The computer system100may be constructed as a desktop, laptop or server grade hardware platform102, including different variations of the x86 architecture platform or any other technically feasible architecture platform. Such a hardware platform102may include a central processing unit (CPU)104, random access memory (RAM)110, mass storage (such as a hard disk drive)106and other I/O devices such as a mouse and keyboard (not shown). RAM110is organized as pages of memory. Each page of memory typically comprises a fixed amount of storage.

A virtualization layer120is installed on top of hardware platform102to support a virtual machine execution space, within which at least one virtual machine (VM)130-0is instantiated for execution. Additional VM instances130may coexist under control of the virtualization layer120, which is configured to map the physical resources of hardware platform102(e.g., CPU104, RAM110, etc.) to a set of corresponding “virtual” (emulated) resources for each virtual machine130. The virtual resources are provided by a corresponding virtual machine monitor124, residing within the virtualization layer120. The virtual resources may function as the equivalent of a standard x86 hardware architecture, such that any x86 supported operating system, e.g., Microsoft Windows, Linux, Solaris x86, NetWare, FreeBSD, etc., may be installed as a guest operating system132. The guest operating system132facilitates application execution within an application space134. In one embodiment, the virtualization layer120comprises a hypervisor within VMware® vSphere® virtualization product, available from VMware, Inc. of Palo Alto, Calif. In an alternative embodiment, a host operating system is installed between the virtualization layer120and hardware platform102. In such an embodiment, the virtualization layer120operates above an abstraction level provided by the host operating system. It should be recognized that the various terms, layers and categorizations used to describe the components inFIG. 1Amay be referred to differently without departing from their functionality or the spirit or scope of the invention.

The virtualization layer120includes a memory manager122configured to allocate pages of memory residing within RAM110. The pages of memory are generally organized as allocated pages114and free pages112. The allocated pages114are pages of memory that are reserved for use by a memory consumer126. The free pages112represent a free page pool comprising pages of memory that are free for allocation to a memory consumer126. A memory consumer126may reside within a VMM124or any other portion of the virtualization layer120.

FIG. 1Bdepicts a block diagram of a computer system150configured to implement one or more aspects of the present invention. The computer system150includes a system platform152, and an operating system170configured to provide an application space180for execution of an application182. In one embodiment, system platform152is constructed as a physical desktop, a physical laptop, an embedded processor, a mobile processing platform, a physical server system, or any other computing platform comprising a processor and random access memory. In an alternative embodiment, the system platform152comprises a VM with operating system170installed as a guest operating system. In either embodiment, the system platform152may include different variations of the x86 architecture platform or any other technically feasible architecture platform. Such a system platform152may include a CPU154, RAM160, mass storage156and other I/O devices such as a mouse and keyboard (not shown). Each system platform component may be a physical subsystem or a virtual subsystem. Each page of memory typically comprises a fixed amount of storage.

The operating system170includes a memory manager122configured to allocate pages of memory residing within RAM160. The pages of memory are generally organized as a set of allocated pages114and a set of free pages112. The allocated pages114are pages of memory that are reserved for use by a memory consumer176. The free pages112are pages of memory that are free for allocation to a memory consumer176. In one embodiment, the memory consumer176resides within any technically feasible portion of the operating system170, such as a kernel space of the operating system170. In alternative embodiments, the memory consumer176resides within a non-kernel execution space, such as in a user space. In one embodiment, the memory manager122resides within the kernel of the operating system170.

In a conventional memory management regime, pages categorized as free pages are not in use by any memory consumer. In such a regime, each free page is presumed to not store presently valid data, and each allocated page is reserved by a memory consumer regardless of whether the page presently stores valid data. Furthermore, in a conventional buffer cache, associated pages are allocated exclusively to the buffer cache, and only trusted users, such as the core of the operating system, may access these pages of the buffer cache. By contrast, pages of yankable memory are not allocated exclusively to a memory consumer, and yankable memory pages may be accessed by any appropriately configured memory consumer, even user-level applications and third party drivers, affording a yankable memory regime greater flexibility and greater usability by more classes of consumers.

In addition, the conventional buffer cache maintains its own list of buffered, freeable memory pages or has to be at least notified when a memory page is not free anymore. The memory management techniques according to one or more embodiments of the present invention may be implemented without any such overhead.

A memory management regime is disclosed herein, whereby a page of memory may be characterized as allocated, idle, or yankable. Allocated pages114are reserved in a conventional manner. However, free pages112are categorized as being either idle or yankable. Idle pages have been released from an associated memory consumer with no expectation of further use by the memory consumer. Yankable pages have been returned to the set of free pages112from an associated memory consumer, with an expectation that the yankable pages may be beneficially accessed at a later time. Both idle and yankable pages are available to be allocated and become allocated pages114.

A yankable memory consumer may, at any time, lose access to data stored within one or more pages of yankable memory. Therefore, a yankable memory consumer needs to be able to restore or retrieve data associated with lost pages of memory from a reliable source, or continue properly without the lost pages. One example of a yankable memory consumer that is able to operate correctly despite potential loss of pages of memory is a read cache. The read cache retrieves data from a backing store that forms a reliable source of data and stores the data in yankable pages. If certain yankable pages remain available, then the read cache performance is improved. If certain yankable pages are lost due to reallocation by the memory manager122, then the read cache performance is diminished. Regardless of whether the yankable pages remain available, the read cache can continue to operate correctly.

In one embodiment, yankable pages are allocated according to conventional techniques but are returned to the free page pool. In one example, a memory consumer requests an allocation of at least one page of memory from the memory manager122. The memory manager122then grants the allocation request. At this point, the memory consumer may fill the page of memory, mark the page as read-only, and subsequently return the page to the memory manager122as a yankable page of memory. The memory manager122then adds the yankable page of memory back to the free page pool. The memory consumer retains a reference to the yankable page of memory for later access. If the yankable page of memory, now among the free pages112, has not been reallocated, then the memory consumer may access the yankable page of memory on a read-only basis. In certain embodiments, the memory consumer may request that the memory manager122reallocate the yankable page of memory to the memory consumer, for example, if the memory consumer needs to write new data into the yankable page of memory. After the write, the memory consumer may release the yankable page of memory back to the free page pool.

In one embodiment, a public generation number is applied to each page of memory being managed by the memory manager122. Public generation numbers may be initialized to zero during an initial boot process. The public generation number of a page of memory is incremented prior to the page of memory being allocated to a particular memory consumer. If any generation number overflows as a result of being incremented, then all memory consumers of yankable memory should be informed that all of their yankable memory pages are invalid and no longer available. Any technically feasible technique may be used to invalidate yankable memory pages. After an overflow event, generation numbers may be cleared and new pages of yankable memory may be allocated again. Upon successful allocation, the memory consumer retains a private copy of each generation number for each yankable page of memory. The memory manager122separately maintains the public generation number for each page of memory, regardless of whether a particular memory consumer treats the page of memory as yankable. When a specific memory consumer reads a particular yankable page of memory, if the private generation number matches the public generation number associated with the page of memory, then it is determined that the page of memory was not allocated to another memory consumer and data stored within the page of memory is valid for the specific memory consumer. However, if the private generation number does not match the public generation number, then it is determined that the page of memory was allocated to another memory consumer and the data stored within the page of memory is not valid for the specific memory consumer. This embodiment is described in greater detail below inFIGS. 2A-2F.

FIG. 2Adepicts a memory consumer220requesting pages of memory212-0,212-1. Upon receiving an allocation request from the memory consumer220, the memory manager122ofFIGS. 1A-1Ballocates pages of memory212-0and212-1to the memory consumer220. Public generation numbers for each page of memory212may be initialized to zero once at boot time. The two pages of memory212-0,212-1are allocated from a free list210. In one embodiment, the free list210is an ordered list of pages associated with the free page pool. The free list210may be implemented using any technically feasible list data structure. In one embodiment, the free list210implements a data structure that preserves list order. A list head211indicates which page212should be allocated from the free list210first by the memory manager122. Upon allocation of the pages212-0and212-1, the public generation numbers for pages of memory212-0and212-1are incremented to one (1), as shown below inFIG. 2B. The memory consumer220receives references to pages of memory212-0,212-1to access the pages of memory212. Each reference may include a pointer to a location in memory for the corresponding page of memory212. Each reference may also include the public generation number for the page of memory212.

FIG. 2Bdepicts the memory consumer220having pages of allocated memory212-0,212-1. As shown, each public generation number is set to one (1). Subsequent to successful allocation of the pages212-0,212-1, the memory consumer copies the public generation numbers to corresponding private generation numbers. As shown, each private generation number associated with a page212-0,212-1is a copy of the corresponding public generation number.

FIG. 2Cdepicts the memory consumer220releasing one page of memory212-0to the head211of a free list210and another page of memory212-1to the back of the free list210. A traditional release is applied to page212-0because the memory consumer220needs no further access to the page212-0. The page212-0should be added to the head211of the free list and may be first in line to be reallocated by the memory manager122. A yankable release is applied to page212-1because the memory consumer220may benefit from further access to the page212-1. The page212-1is added to a tail213(seeFIG. 2A) of the free list210because reallocating the page212-1should be avoided if possible, and the tail213of the free list210represents the last possible page212that may be allocated from the free list210.

Other yankable pages212may be added behind page212-1over time, thereby increasing the likelihood that the page212-1will be reallocated. However, placing yankable pages at the tail213of the free list establishes a low priority for reallocation of the yankable pages, thereby increasing the likelihood that a given yankable page will be available for use when an associated memory consumer attempts access. Prior to the yankable release of page212-1, the memory consumer220copies the public generation number for page212-1and stores the copy as a private generation number for page212-1. In future attempts to access page212-1, the memory consumer compares the private generation number to a then prevailing public generation number. If the private generation number matches the public generation number, then the memory consumer220may access data within the page212-1. The memory consumer220does not need to store a private generation number for page212-0because page212-0is released according to a traditional (non-yankable) release.

FIG. 2Ddepicts the memory consumer220retaining a reference to the page of yankable memory. The reference is depicted as a dashed outline for page212-1within memory consumer220. Immediately subsequent to returning the page212-1to the free list210, the public generation number is one (1) for page212-1, and the private generation number stored within the memory consumer220for page212-1is also one, meaning the page is currently valid for access by memory consumer220.

FIG. 2Edepicts the yankable page212-1in a valid state with respect to the memory consumer220. After some arbitrary amount of time, the memory manager122has allocated pages212-3,212-0, and212-2, as indicated by increasing public generation numbers for the pages. These pages have also been released during that time, as indicated by their presence in the free list210. However, the memory manager122has not yet allocated page212-1, as indicated by a public generation number matching the private generation number for page212-1. In this example, the public and private generation numbers are both equal to one (1) and the memory consumer220may access the page212-1.

FIG. 2Fdepicts the yankable page212-1in an invalid state with respect to the memory consumer220. After some arbitrary amount of time, the memory manager122has allocated pages212-3,212-0,212-2, and212-1to other memory consumers at some point. These pages have also been released during that time, as indicated by their presence in the free list210. In this example, the public and private generation numbers are not equal for page212-1the memory consumer220may not access the page212-1. Alternatively, page212-1may not even be in the free list210if the page212-1is in the set of allocated pages114. Regardless of the disposition of the page212-1, the public generation number will not match the private generation number stored within the memory consumer220if the page212-1was allocated to another memory consumer by the memory manager122subsequent to the yankable release.

In one embodiment, memory consumer220is provided with a memory manager application programming interface (API) comprising at least a yankable memory declaration API call and a yankable memory read API call. The yankable memory declaration API call causes the memory manager122to return a declared page to the free list210, as described previously inFIG. 2C. The yankable memory read API call compares public and private generation numbers for a requested page and either returns data if the public and private generation numbers match, or returns a read error if the generation numbers do not match. One optional yankable memory API comprises a reallocate request that pulls a requested page from the free list210if the public and private generation numbers match and allocates the page to the requesting memory consumer. Persons skilled in the art will understand that any technically feasible technique may be used to uniquely associate a page of memory with a memory consumer and that the regime of generation numbers is one exemplary technique. Another exemplary technique involves using a unique process identification number in place of a generation number, and so forth.

In an alternative embodiment, a yankable memory API based on a try-catch semantic is implemented to provide yankable memory management. The try-catch API includes a yankable allocation API call and a yankable declaration API call. The yankable allocation API call allocates a page and maps the page as a read-write page in a special section of virtual memory that is accessible to the memory consumer220. The memory consumer220may then write to the page to fill the page with any appropriate data. The memory consumer then calls the yankable declaration API call with the page, causing the page mapping to be switched to read-only status. If the page subsequently needs to be yanked away and allocated to a different memory consumer, the mapping is cleared. To later access the yankable page, a try API call is executed on the page. If the try call passes, then the page may be read. Otherwise, a catch method is executed by the memory consumer220. The try-catch semantic avoids a page fault being triggered by the memory consumer220. On what would otherwise be a page fault condition, the memory consumer220is able to execute a catch method instead.

C. Free List Allocation Priority

FIG. 3illustrates reordering pages in a free list210based on access to the pages, according to one embodiment of the invention. The free list210includes idle pages320that are not referenced by any memory consumers, and yankable pages330that are each referenced by at least one memory consumer, such as memory consumer220, as described previously. A memory allocation operation342selects a page324from the head211of the free list210. If there are one or more pages within the set of idle pages320, then page324is allocated from the set of idle pages. Otherwise, page324is allocated from the set of yankable pages330. A traditional release operation340releases a page322permanently from the memory consumer220. No further reference to page322is available unless page322is allocated to the memory consumer220in a separate allocation operation.

In one embodiment, a yankable page declaration operation344places an associated yankable page332at the tail213of the free list210. The page332remains accessible to an associated memory consumer until page332is allocated to a different memory consumer. A yankable page334may be read via a read operation345. As described previously, the read operation345may comprise checking a page attribute, such as a generation number or mapping disposition. The fact that an associated memory consumer read the yankable page334indicates the page is likely of some future value to the memory consumer. To increase the likelihood that the yankable page334will be available to the associated memory consumer, the yankable page334is repositioned at the tail213of the free list210via a repositioning operation346. In one or more embodiments of the invention, the process of reading and repositioning yankable pages implements a least recently used (LRU) allocation policy for yankable pages within the free list210. In certain embodiments involving hardware assisted address mapping, repositioning of a page to the tail213of the free list210may be indicated via a page table entry access bit (or simply ‘A’ bit).

Persons skilled in the art will recognize that the techniques disclosed herein for implementing a yankable memory regime are exemplary and that other implementations of a yankable memory regime are within the scope and spirit of the present invention.

FIG. 4Ais a flow diagram of method steps400, performed by a memory manager, for allocating a page of memory. Although the method steps are described in conjunction with the system ofFIGS. 1-3, it should be understood that there are other systems in which the method steps may be carried out without departing the scope and spirit of the present invention.

The method begins in step410, where the memory manager, such as memory manager122ofFIGS. 1A-1B, receives an allocation request for a page of memory from a memory consumer, such as consumer220ofFIGS. 2A-2F. In step412, the memory manager122selects a page from the free list210. In one embodiment, the memory manager122selects a page from the head211of the free list210. In step414the memory manager increments a public generation number associated with the selected page. Any technically feasible technique or data structure may be implemented to store public generation numbers for pages of memory. For example, an entry field in a page table entry may be used to store public generation numbers. In step416, the memory manager122returns a reference for the selected page to the memory consumer220. The method terminates in step416.

FIG. 4Bis a flow diagram of method steps402, performed by a memory manager, for declaring a page of memory as being yankable. Although the method steps are described in conjunction with the system ofFIGS. 1-3, it should be understood that there are other systems in which the method steps may be carried out without departing the scope and spirit of the present invention.

The method begins in step420, where the memory manager receives a yankable declaration, designating a page as yankable, from a memory consumer. In step422, the memory manager places the page at the tail of the free list. The method terminates in step422.

FIG. 4Cis a flow diagram of method steps404, performed by a memory manager, for reading a page of memory. Although the method steps are described in conjunction with the system ofFIGS. 1-3, it should be understood that there are other systems in which the method steps may be carried out without departing the scope and spirit of the present invention.

The method begins in step430, where the memory manager receives a request for a yankable page generation number for a specified page from the memory consumer. In step432, the memory manager returns a public generation number for the requested yankable page to the memory consumer. If, in step440, data associated with the specified page is requested, then the method proceeds to step442, where the memory manager returns requested data from the specified page. In step444, the memory manager repositions the specified yankable page to the tail of the free list. The method terminates in step450.

Returning to step440, if data associated with the specified page is not requested, then the method terminates in step450.

FIG. 5Ais a flow diagram of method steps500, performed by a memory consumer, for allocating a page of memory. Although the method steps are described in conjunction with the system ofFIGS. 1-3, it should be understood that there are other systems in which the method steps may be carried out without departing the scope and spirit of the present invention.

In one or more embodiments, a usage model for a data object implemented by a memory consumer is evaluated. If the data object is configured to store information that is available from a reliable second source, then the data object may be implemented using pages of yankable memory. If the data object is configured to store uniquely available information, then the data object should not be implemented using yankable memory.

The method begins in step512, where the memory consumer requests a new page allocation from the memory manager. In step514, the memory consumer receives a reference to the new page from the memory manager. In step516, the memory consumer fills the page with data for subsequent read-only access. In step518, the memory consumer creates a private copy of a generation number associated with the new page. In step520, the memory consumer declares the page to be yankable to the memory manager, which places the page on free list210ofFIGS. 2A-2F. The method terminates in step520.

FIG. 5Bis a flow diagram of method steps502, performed by a memory consumer, for reading a page of memory. Although the method steps are described in conjunction with the system ofFIGS. 1-3, it should be understood that there are other systems in which the method steps may be carried out without departing the scope and spirit of the present invention.

The method begins in step550, where the memory consumer retrieves a public generation number for a page the memory consumer is preparing to read. If, in step560, the public generation number matches a private generation number maintained by the memory consumer, then the method proceeds to step562, where the memory consumer performs a read-only request to the page. If the public generation number does not match the private generation number maintained by the memory consumer, then the method proceeds to step568, where the memory consumer retrieves a copy of the data from a reliable source.

After the read is carried out in step562, the memory consumer retrieves the public generation number once more in step563to confirm in step566that it has not changed. It should be recognized that the public generation number would have changed if the memory page is allocated to another memory consumer while steps560and562are being carried out, and in such cases, the read carried out in step562would be invalid and step568would be carried out after step566. If, in step566, the public generation number still matches the private generation number, then the read data is considered valid and the method terminates in step590.

FIG. 6is a flow diagram of method steps600, performed by a memory consumer, during live migration of a virtual machine from a source computer system to a destination computer system. As known in the art, during live migration, memory pages representing the working state of the virtual machine are transmitted from the source to the destination over several pre-copy iterations. Memory pages that are marked “dirty” during any pre-copy iteration are copied to the destination in the next pre-copy iteration. When the set of dirty memory pages is sufficiently small after a pre-copy iteration, the virtual machine is stunned at the source and, after the dirty memory pages have been copied to the destination, resumes execution at the destination.

Sometimes, the memory pages that are marked dirty during a pre-copy iteration are not modified at all (e.g., rewritten with the same data), which means that such memory pages need not be copied to the destination. As a way to determine whether dirty memory pages have indeed been modified during a pre-copy iteration, memory pages representing the working state of the virtual machine that are copied to the destination during any pre-copy iteration are stored in pages of yankable memory.FIG. 6is a flow diagram of a method that is carried out by a live migration module to determine whether a memory page that has been marked dirty during a pre-copy iteration has been modified or not.

The method begins in step610, where the live migration module retrieves a public generation number for the page in yankable memory containing the previously transmitted contents of the dirty memory page. If, in step620, the public generation number matches a private generation number maintained by the live migration module for that page in yankable memory, the method proceeds to step630, where the live migration module performs a read-only request to the page. If the public generation number does not match the private generation number, the method proceeds to step625, where the live migration module transmits the current contents of the dirty memory page to the destination.

After the read is carried out in step630, the live migration module retrieves the public generation number once more in step640to confirm in step650that it has not changed. It should be recognized that the public generation number would have changed if the page of yankable memory containing the previously transmitted contents of the dirty memory page is allocated to another memory consumer while steps620and630are being carried out, and in such cases, the read carried out in step630would be invalid and step625would be carried out after step650. If, in step650, the public generation number still matches the private generation number, the read data is considered valid and is compared against the current contents of the dirty memory page in step660. If the contents do not match, it is determined that the dirty memory page has indeed been modified, and step625is carried out to transmit the current contents of the dirty memory page to the destination. If the contents match, it is determined that the dirty memory page has not been modified, and the method ends in step680without the contents of the dirty memory page being transmitted to the destination.

In sum, a technique is disclosed for managing memory in scenarios where a memory consumer can continue to operate correctly given a loss of a page of allocated memory. Two categories of pages may exist within this memory management regime. One category is identical to traditional pages of memory, which are allocated and kept by a memory consumer until released, after which no further accesses are permitted. A second category allows pages associated with a memory consumer to be yanked from the memory consumer at any time. This second category of page is applicable to certain common applications including, without limitation, caching. In one embodiment, pages of memory are marked with a public generation number, which increments every time the page is allocated. Each memory consumer maintains a private copy of the public generation number for each page. Prior to accessing data within a page of yankable memory, the memory consumer compares public and private generation numbers. If the generation numbers match, then the page may be accessed, if the generation numbers do not match, then the memory consumer must either proceed without the requested data, or the memory consumer must retrieve a copy of the requested data form a reliable source. In an alternative embodiment, yankable pages are mapped via a virtual mapping scheme. A yankable page is mapped during allocation and de-mapped if the memory manager yanks the page back. A try-catch scheme is implemented to avoid memory faults that may arise when the memory consumer attempts to access a yanked page.

According to another embodiment of the present invention, certain contents may be stored in yankable memory and discarded if the public generation number does not match the stored private generation number. These examples include: (1) storing a real, continuous signal from a signal processing card in yankable memory, such that even if some contents are lost as a result of the page being allocated to another memory consumer, quantization can continue, albeit with a slight loss in accuracy; (2) storing a large set of data points (e.g., disk latency values) in yankable memory to perform data fitting to a curve, such that even if some contents are lost as a result of the page being allocated to another memory consumer, curve fitting can continue, albeit with a slight loss in accuracy; (3) storing crypto-secure signatures of previously migrated pages in yankable memory using which comparisons can be done with newer version of those pages and differences can be sent in the form of diff compression, such that even if some contents are lost, this process can still provide better network bandwidth utilization for those pages that still have crypto-secure signatures stored in yankable memory; and (4) storing a cache of information about pages previously transmitted to a destination in yankable memory and sending unique identifiers of previously transmitted pages to the destination instead of the page data, such that even if some contents are lost, this process can still prevent repeat data transmissions to the destination for those pages that still have their information stored in yankable memory.

In addition, while described virtualization methods have generally assumed that virtual machines present interfaces consistent with a particular hardware system, persons of ordinary skill in the art will recognize that the methods described may be used in conjunction with virtualizations that do not correspond directly to any particular hardware system. Virtualization systems in accordance with the various embodiments, implemented as hosted embodiments, non-hosted embodiments, or as embodiments that tend to blur distinctions between the two, are all envisioned. Furthermore, various virtualization operations may be wholly or partially implemented in hardware. For example, a hardware implementation may employ a look-up table for modification of storage access requests to secure non-disk data.