Patent Publication Number: US-8983988-B2

Title: Centralized management of virtual machines

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is a continuation of U.S. patent application Ser. No. 12/274,303, filed on Nov. 19, 2008, which has issued as U.S. Pat. No. 8,392,361 and entitled “Centralized Management of Virtual Machines,” which is a continuation-in-part of U.S. patent application Ser. No. 12/189,737, filed Aug. 11, 2008, which has issued as U.S. Pat. No. 8,171,278 and a continuation-in-part of U.S. patent application Ser. No. 12/246,284, filed Oct. 6, 2008, which has issued as U.S. Pat. No. 8,209,343, all of which are being incorporated herein for all purposes. 
    
    
     BACKGROUND 
     Conventional virtual machines (VMs) employ virtual disks as its primary form of storage. Virtual disks provide fully encapsulated storage for the virtual machines, thereby simplifying management tasks such as versioning and rollback. They also provide other advantages such as mobility (VMs can be easily moved and/or copied) and isolation (storage domains of multiple VMs are isolated from each other). 
     Consequently, in a virtual machine system having multiple VMs, each VM has stored in its virtual disk, operating system files and files that are used in providing management services, such as anti-virus scanning, backup, patching, and versioning, to the VM. When a management service is invoked in response to a file operation issued by a VM, the VM executes the program associated with the invoked management service. For example, in response to an “open file” operation, the VM executes an anti-virus scanning program on the file designated to be opened. 
     The inventors have observed certain inefficiencies with providing management services within the virtual disk framework described above. First, management programs consume both CPU and memory resources allocated to the virtual machines. Second, the installation of management programs (occurring as a result of a software vendor change, for example) and administration of updates to the management programs can be cumbersome because the process has to be repeated for each virtual machine. 
     Software vendors are providing centralized software to automate the process of administering updates, but at best, this is a partial solution because it does not address the inefficiency associated with resource consumption. 
     A virtualization aware file system, known as Ventana, extends a conventional distributed file system to virtual machine environments and thus combines the sharing benefits of a distributed file system with versioning, access control, and disconnected operation features that are available with virtual disks. A detailed description of Ventana is provided in a publication from Stanford University, Department of Computer Science, authored by Ben Pfaff, et al. The publication is entitled “Virtualization Aware File Systems: Getting Beyond the Limitations of Virtual Disks.” Although Ventana provides a file system framework that could be used to address some of the issues with conventional delivery of management services, the publication is silent about how this can be done using Ventana. 
     SUMMARY 
     One or more embodiments of the invention provide a centralized way of managing virtual machines. By centrally managing virtual machines, overall CPU and memory usage is reduced and administration of programs that provide management services to virtual machines is simplified. 
     According to an embodiment, one method enables a central management service to operate on individual files within virtual disks associated with different virtual machines running in one or more host computers that are each networked to a remote storage system. An input/output (IO) request from a virtual machine running on a host computer is received at a virtualization software layer on the host computer, wherein (i) the IO request relates to a file stored in a virtual disk associated with the virtual machine, and (ii) the file is individually stored in the remote storage system in accordance with a file system that governs how data is stored in the remote storage system. The central management service is notified of the received IO request, wherein the central management service uses the file system to access the file in the remote storage system and performs a management task on the file, and the IO request is then performed on the file in the remote storage system by the virtualization software layer upon a notification of a successful completion of the management task by the central management service. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a functional block diagram of a computer system in which one or more embodiments of the invention may be practiced. 
         FIG. 2  illustrates various components of the computer system of  FIG. 1  that support centralized management of virtual machines. 
         FIG. 3  illustrates a process for loading an operating system from a central storage unit. 
         FIG. 4  schematically illustrates a file input/output process after a file system driver and a file system filter driver have been loaded. 
         FIG. 5A  illustrates another process for loading an operating system from a central storage unit. 
         FIG. 5B  illustrates a process for handling I/O requests from local storage or a central storage unit. 
         FIG. 6  schematically illustrates a file input/output process after a file system filter driver, a file system driver, a disk filter driver, a SCSI filter driver, and a SCSI driver have been loaded. 
         FIG. 7  illustrates a process for selectively routing a file IO request to a central storage unit or a switching layer. 
         FIG. 8  illustrates a process for namespace mapping. 
         FIG. 9  is a flow diagram illustrating a method for managing virtual machines according to an embodiment of the invention. 
         FIG. 10  illustrates an alternative embodiment of the invention. 
     
    
    
     DETAILED DESCRIPTION 
       FIG. 1  is a functional block diagram of a computer system  100  in which one or more embodiments of the invention may be practiced. Computer system  100  includes server platforms  110  (also referred to herein as a “host computer”), a local storage unit  120  for each server platform, and a central storage unit  130  that is shared by server platforms  110 . Local storage unit  120  and central storage unit  130  may be implemented as network attached storage (NAS) or storage area network (SAN) arrays. Local storage unit  120  is dedicated to and provides storage for the server platform to which it is connected, and central storage unit  130  provides shared storage to all server platforms  110 . Central storage unit  130  represents one or more storage servers. 
     A server platform is connected to central storage unit  130  through an out-of-band IO path  121  and an in-band IO path that includes IO path  122 , a switching layer computer  125 , a management plug-in framework (MPF) computer  127 , and IO paths  128 . Switching layer computer  125  carries out namespace mapping, as will be described below. MPF computer  127  hosts a management plug-in framework, described in further detail below, and provides a user interface for system administrators for configuration and reporting purposes. IO paths  121 ,  122 ,  128  are communication paths that are based on some file sharing protocol, such as NFS (Network File System) and CIFS (Common Internet File System). 
     Each of server platforms  110  has conventional components of a server computer, and may be implemented as a cluster of multiple server computers. Each server platform has configured therein one or more virtual machines  140  that share hardware resources of the server platform, such as system memory  112 , processor  114  and storage interface  116 . Examples of storage interface  116  are a host bus adapter and a network file system interface. Virtual machines  140  run on top of a virtual machine monitor  150 , which is a software interface layer that enables sharing of the hardware resources of the server platform by virtual machines  140 . Virtual machine monitor  150  may run on top of the server platform&#39;s operating system or directly on hardware components of the server platform. Together, virtual machines  140  and virtual machine monitor  150  create virtualized computer systems that give the appearance of being distinct from the server platform and from each other. Each virtual machine includes a guest operating system and one or more guest applications. The guest operating system is a master control program of the virtual machine and, among other things, the guest operating system forms a software platform on top of which the guest applications run. A virtual disk for each of the virtual machines  140  is maintained within local storage unit  120 . 
       FIG. 2  illustrates various components of the computer system of  FIG. 1  that support centralized management of virtual machines. These components include a host file virtualization layer (HFVL)  210  implemented in each server platform and a remote file virtualization layer (RFVL)  230  implemented in switching layer computer  230 . A cache memory unit  215  is provided to support namespace mapping that is carried out by the server platform and a namespace database  240  is provided to support namespace mapping that is carried out by switching layer computer  220 . Namespace database  240  is used to persistently store the primary namespace map for computer system  100  and contains all of the mapping information needed to correctly map filenames used by VMs  140  to filenames used by central storage unit  130 . Cache memory unit  215  contains only a portion of the primary namespace map. 
     HFVL  210  is a software component that resides on an operating system for the server platform. HFVL  210  acts as a gateway between a file system driver running in the guest operating system of VMs  140  and central storage unit  130 . It also interacts with RFVL  230  to implement the guest namespace virtualization and employs cache memory unit  215  to cache namespace maps as they are resolved by RFVL  230 . 
     RFVL  230  is a software component that with the help of namespace database  240  implements guest namespace virtualization. Guest namespace virtualization is a mechanism to construct and control a virtual tree of files and folders seen by the VM. It comprises of a map between filenames and directory tree structure seen by the VM and their location on central storage  130 . There need not be an exact mapping of a file path that a guest operating system can operate on and the file path on central storage  130 . For example, a guest file “c:/foo/bar.txt” can be mapped to “/server/share/snapshot/1/2/3/xyz.lmnop” on the central storage. RFVL  230  stores this mapping information in namespace database  240  and uses it to resolve file paths referenced by the VM. 
     The guest namespace can be constructed using two types of virtualization techniques, static and dynamic. Static virtualization is that part where the namespace map cannot be altered while the guest operating system of the VM is running This prevents newer versions of files getting introduced in the guest operating system while they are in use and thus breaking runtime dependencies. Dynamic virtualization is a mechanism where names and directory trees can be added or modified in the guest namespace while the guest operating system of the VM is running. This permits applications to be dynamically pushed to the VM or removed from the VM if they are no longer required. Dynamic virtualization is achieved by updating the namespace map in namespace database  240  and invalidating the namespace maps stored in cache memory unit  215 . 
     MPF  250  is a framework where management services are plugged in. MPF  250  exposes a register API that management services utilize to register callbacks on specific events, such as file open, file close, file create, file delete, and file rename. For example, an anti-virus scanning program might register callbacks for file open and file close events. RFVL  230  notifies MPF  250  about various events and MPF  250  invokes callbacks of appropriate services. The callbacks can potentially invoke APIs that are specific to management services to implement the service. MPF  250  interacts with central storage unit  130  to access physical files. 
     MPF  250  also provides a way for system administrators to subscribe to notifications from the registered management services. For this purpose, MPF  250  defines a schema of errors to which the management services conform. For example, in the case where an anti-virus detection service detects an infected file in one of the managed VMs, the service returns an error status to MPF  250  per its schema. MPF  250  then interacts with the system administrator using various alerting mechanisms, such as e-mail or SMS notifications or alerts in the administrative interface. Upon receiving these alerts, the system administrator may take actions using the administrative interface or other such mechanisms. 
     Local MPF (LMPF)  216  is a miniature version of MPF and executes within server platform  110 . It interacts with HFVL  210  to provide management services that can be executed within server platform  110 . 
     In one embodiment of the invention, a guest agent, a software that is running inside the guest OS, interacts with users logged into the guest OS and notifies them about various events occurring in the system. For example, when a virus is detected and the system administrator is alerted as described above, the guest agent pops up a message box informing the users about the virus and the infected file. Optionally, users can act on these notifications. 
     A process of loading an operating system into system memory (known as a boot process) from central storage unit  130  is illustrated in  FIG. 3 . The process begins at step  310  with the system BIOS (Basic Input/Output System) invoking a boot loader from a boot disk or, more generally, a boot volume. At step  312 , the boot loader loads the kernel of the operating system and boot time drivers into system memory from the boot volume. At step  314 , the boot loader loads into system memory a file system driver and a file system filter driver. The file system driver is the driver for the native file system of the operating system, e.g., NTFS for Windows NT operating systems. The file system filter driver is a software component that runs on top of the file system driver and redirects file system calls made to the file system. In the first embodiment of the invention, file system calls made to the file system are redirected to a central storage unit. As a result, when file system calls are made after the file system filter driver has been loaded into system memory, the file system calls are redirected to a central storage unit (step  316 ) and file operations are carried out using the central storage unit (step  318 ). For example, when file system calls are made to load the operating system files that remain unloaded after step  314 , the remaining operating system files are loaded into system memory from the central storage unit instead of the boot volume. Also, file operations that are issued after the boot process has completed are performed using the central storage unit. 
       FIG. 4  schematically illustrates a file input/output process after the file system driver and the file system filter driver have been loaded into system memory. In  FIG. 4 , the file system filter driver is shown as FSFD  410 , the NTFS file system driver as NTFS  420 , and a SCSI driver as SCSI  430 . Before FSFD  410  is loaded into system memory, files are accessed and loaded from a boot storage volume  440 . When a virtual machine is being booted, boot storage volume  440  may be a portion of a virtual disk associated with the virtual machine or may be streamed in using PXE/TFTP protocols. After FSFD  410  is loaded into system memory and runs on top of NTFS  420 , FSFD  410  redirects file accesses to a central storage unit  450 . In the embodiment illustrated in  FIG. 1 , boot storage volume  440  is contained within local storage unit  120  and central storage unit  450  corresponds to central storage unit  130 . 
     Another process for loading an operating system into system memory from a central storage unit is illustrated in  FIG. 5A . The process begins at step  510  with the system BIOS (Basic Input/Output System) invoking a boot loader from a boot disk or, more generally, a boot volume. At step  512 , the boot loader loads the kernel of the operating system and boot time drivers into system memory from the boot volume. At step  514 , the boot loader also loads into system memory a file system filter driver, a file system driver, a disk filter driver, a SCSI filter driver, and a SCSI driver. 
     The file system filter driver is a software component that runs on top of the file system driver and maintains a map between block numbers and files on which input/output is performed. This map is a table that is loaded into system memory with the file system filter driver. This table associates block numbers with a file ID and an offset inside the file, and is modified every time the file layout information changes such as when a file is created, deleted, extended, truncated, etc. The file system driver is the driver for the native file system of the operating system, e.g., NTFS for Windows NT operating systems. The disk filter driver is a software component that runs below the file system driver and tags block input/output requests representing file input/output operations. Block input/output requests for metadata blocks (i.e., “metadata operations”) are not tagged. The disk filter driver only sees block numbers and thus it employs the map between block numbers and files as maintained by the file system filter driver to distinguish between the different types of operations and perform the tagging. Tag information contains a flag indicating a file input/output operation, and the file ID and offset information obtained from the map. The SCSI filter driver examines the tags on the block input/output requests that it receives to differentiate between file input/output and metadata operations. The SCSI driver manages accesses to the local storage unit. 
     After the file system filter driver, the file system driver, the disk filter driver, the SCSI filter driver, and the SCSI driver have been loaded, operating system files that remain unloaded after step  514  are loaded into system memory from the central storage unit in step  515 . 
       FIG. 5B  illustrates a process for handling I/O requests from local storage or a central storage unit after the file system filter driver, the file system driver, the disk filter driver, the SCSI filter driver, and the SCSI driver have been loaded in connection with the loading of an operating system. First, the SCSI filter driver determines whether a block input/output request that it receives is an operation for a file stored in the local storage unit (step  516 ) or is a metadata operation (step  518 ). If either condition is true, the operation is carried out using the local storage unit at step  520 . If both conditions are false, the operation is carried out using the central storage unit at step  522 . 
       FIG. 6  schematically illustrates a file input/output process after the file system filter driver, the file system driver, the disk filter driver, the SCSI filter driver, and the SCSI driver have been loaded into system memory. In  FIG. 6 , the file system filter driver is shown as FSFD  610 , the NTFS file system driver as NTFS  620 , the disk filter driver as Disk FD  630 , a SCSI filter driver as SCSI FD  640 , and a SCSI driver as SCSI  650 . Before Disk FD  630  and SCSI FD  640  are loaded into system memory, files are accessed and loaded from a boot storage volume  660 . When a virtual machine is being booted, boot storage volume  660  may be a portion of a virtual disk associated with the virtual machine or may be streamed in using PXE/TFTP protocols. After Disk FD  630  and SCSI FD  640  are loaded into system memory and runs below NTFS  620 , file input/output operations are directed to a central storage unit  670 , whereas metadata operations and file input/output operations involving files stored in boot storage volume  660  are directed to boot storage volume  660 . In the embodiment illustrated in  FIG. 6 , boot storage volume  660  is contained within local storage unit  120  and central storage unit  670  corresponds to central storage unit  130 . 
     By permitting the booting of the virtual machines from central storage in the manner described above, most of the OS files can be stored centrally and shared by multiple virtual machines. As a result, management services provided on these shared files, such as anti-virus scanning, backup, patching, and versioning, need not be carried out multiple times, thus conserving CPU and memory resources of the individual virtual machines and simplifying administration of these files. 
       FIG. 7  illustrates a process for selectively routing a file IO request to a central storage unit or a switching layer. This process is carried out by the server platform that is hosting one or more virtual machines in response to a file IO request made by an application running in a virtual machine. At step  710 , a filter driver of the virtual machine (either a file system filter driver or a disk filter driver) sends the file IO request to HFVL  210 . HFVL  210  evaluates the file IO request and determines whether the file IO request is a data operation or a metadata operation (step  714 ). A data operation includes a read operation and a write operation. A metadata operation includes file open, file create, file delete, rename, set file attributes, create link, and other file operations known in the art that require a file path. If the file IO request is a data operation, it is routed directly to central storage unit  130  through IO path  121  (step  716 ). If the file IO request is a metadata operation, it is routed to RFVL  230  of switching layer computer  220  through IO path  122  (step  718 ). 
       FIG. 8  illustrates a process for namespace mapping. This process is initiated at HFVL  210  in response to a file IO request, such as file open, file close, and file rename requests (step  810 ). At step  812 , HFVL  210  determines if the namespace mapping information for the filename specified in the file IO request is stored in cache memory unit  215 . If so, the cached mapping information is accessed and used to resolve the filename used by central storage unit  130  (step  814 ). If the filename specified in the file IO request is not stored in cache memory unit  215 , step  816  is carried out by RFVL  230 . At step  816 , RFVL  230  accesses namespace database  240  and maps the filename specified in the file IO request to a filename used by central storage unit  130 . This mapping information is then returned to HFVL  210  and HFVL  210  updates cache memory unit  215  with this information. During dynamic namespace virtualization, when RFVL  230  updates namespace database  240 , RFVL  230  signals each HFVL  210  to invalidate the namespace maps stored in cache memory unit  215 . 
     In the embodiments described above, guest applications and guest operation system of a virtual machine use the file system driver for all of its file access needs. The file system driver forwards these requests to HFVL  210 . The following are some examples of how HFVL handles some of these requests. 
     Open File. HFVL  210  looks into cache memory unit  215  to resolve the VM specific file path to file path of central storage unit  130 . If found, HFVL  210  uses the cached information and interacts with the storage server to open the file and notifies RFVL  230  about the opening of the file. If not, HFVL  210  communicates with RFVL  230  to resolve the file path. It then adds this entry to its cache memory unit  215 . For example, an application executing in a VM tries to open c:\foo\bar.txt. This open call gets routed to HFVL  210  via the file system driver. HFVL  210  examines its cache memory unit  215  to resolve \foo\bar.txt. If this information is not available, it sends the name resolution request to RFVL  230 . RFVL  230  in turn looks into namespace database  240  for the VM specific namespace map and resolves the path to \server1\share3\snapshot7\vm9\foo\bar.txt and returns this path to HFVL  210 . HFVL  210  then forwards the open request to server1 of central storage unit  130  with path \share3\snapshot7\vm9\foo\bar.txt. 
     Create File. HFVL  210  notifies RFVL  230  about a request to create new file/directory. Based on which VM is creating the new file/directory and the configuration policies for that VM, RFVL  230  chooses a file server/share and a system wide unique file/directory name. It then creates a mapping entry between the file/directory name that the VM intends to create and the name RFVL  230  has chosen for that filename in namespace database  240 . RFVL  230  then returns to the requesting HFVL  210  its chosen name. 
     Read/Write. Before a read or write operation can be carried out, a file is opened in the manner described above. This means that the file path to central storage unit  130  has been resolved and stored in-memory or in cache memory unit  215 . This file path is used by HFVL  210  to transfer data to/from central storage unit  130  directly through IO path  121  without involving RFVL  230 . 
     File Close. HFVL  210  notifies RFVL  230  about the closing of the file. 
     File Delete. HFVL  210  notifies RFVL  230  about the deletion of the file. RFVL  230  deletes the mapping between the VM specific file path to file path of central storage unit  130  from namespace database  240 . 
     Namespace mapping, as described herein, allows sharing of files between VMs, whether the VMs are running on the same host computer or different host computers. Also, updates to the primary namespace map maintained in namespace database  240  can be made to reflect changes in file sharing. For example, if two VMs are created from the same template, they begin sharing all of the files. If a VM tries to write to a file, the shared file is copied to a private file (one that is not shared) and the private file is written to. The private file is owned by the VM that is writing to it. 
     The namespace map also supports file deduplication process. This process can be carried out within central storage unit  130  or by any other server that has access to central storage unit  130 , such as server platform  110  or switching layer computer  125  or MPF computer  127 , and entails comparing files stored in central storage unit  130  to determine those files that are identical. Once files are determined to be identical, APIs in RFVL  230  are invoked to change the namespace maps so that multiple VM specific file paths point to each of the files that are found to be identical in central storage unit  130 . 
       FIG. 9  is a flow diagram illustrating a method for managing virtual machines according to an embodiment of the invention. At step  910 , a filter driver of the virtual machine (either a file system filter driver or a disk filter driver) of a virtual machine sends a file operation request to HFVL  210 . HFVL  210  forward this request to RFVL  230  to resolve the filename (step  912 ). Namespace mapping is carried out in the manner described above with reference to  FIG. 8 . At step  920 , RFVL  230  determines if a management program has registered a callback for the file operation that has been requested. If so, at step  922 , RFVL  230  notifies MPF  250  about this file operation with the resolved filename. MPF  250  invokes the callback registered by the management program and the management program accesses the file directly from central storage unit  130  using the resolved filename and executes its program on the file. Then, at step  924 , the requested file operation is executed. If one of the management services has not registered a callback for the file operation that has been requested, step  922  is skipped and the flow jumps to step  924 . 
     An example of the method shown in  FIG. 9  is the anti-virus use case. Anti-virus software generally monitors file IO metadata operations, such as file open, file close, and file rename requests. In response to these operations, the anti-virus software performs anti-virus scanning on the file named in the request. The anti-virus software is running on a computer that is separate (either on a different virtual machine or a different physical server) from the one that issued the request, and has registered itself with MPF  250  for file open, file close, and file rename operations. 
     An application running in a virtual machine tries to open a file, example.exe. The open request is received by the virtual machine. The filter driver of the virtual machine (either the file system filter driver or the disk filter driver) forwards this request to HFVL  210  for additional processing. HFVL  210  forward this request to RFVL  230  to resolve the filename. RFVL  230  is aware that the anti-virus server has registered a callback for file open request and notifies MPF  250  about this open request with the resolved filename. MPF  250  invokes the open callback registered by the anti-virus server. Anti-virus server then opens the file from central storage unit  130  directly using the resolved filename and scans the file. It returns the status of the scan back to MPF  250 , which returns it back to RFVL  230 . If no virus is detected, RFVL  230  returns the resolved file name to HFVL  210  which then goes and opens the file from central storage unit  130  and returns the status back to the filter driver. 
     In case of operations on infected files, the steps remain the same until the anti-virus scan. Once the anti-virus scan determines that the file is infected, it returns the infected status to MPF  250 . MPF  250  notifies RFVL  230  and the system administrator about the infection. RFVL  230 , on seeing the error status, returns to HFVL  210  immediately without returning the resolved filename. HFVL  210  returns the failed open file request to the filter driver and also notifies the guest agent about the infected file. The guest agent pops up a message box informing the user about the file infection. In an alternative embodiment, the anti-virus software disinfects the infected file, in which case, the return path is similar to the normal file operation path. 
     For an on-demand virus scan, the namespace database  240  is accessed to resolve the filenames and the anti-virus software is invoked directly to scan the files from the central storage unit  130  using the resolved names. 
     A computer on which management programs are executed is referred to herein as a “management services computer.” In one embodiment of the invention, MPF computer  127  serves as the management services computer. In other embodiments, management services computer may be a virtual machine or a computer that is physically different from MPF computer  127 . 
       FIG. 10  illustrates an alternative embodiment of the invention. In this embodiment, the functions of HFVL  210  and RFVL  230  are carried out by a single software layer, HFVL/RFVL  1010 , which is implemented in each server platform  110  and communicates with namespace database  240 . 
     The various embodiments described herein may employ various computer-implemented operations involving data stored in computer systems. For example, these operations may require physical manipulation of physical quantities usually, though not necessarily, these quantities may take the form of electrical or magnetic signals where they, or representations of them, are capable of being stored, transferred, combined, compared, or otherwise manipulated. Further, such manipulations are often referred to in terms, such as producing, identifying, determining, or comparing. Any operations described herein that form part of one or more embodiments of the invention may be useful machine operations. In addition, one or more embodiments of the invention also relate to a device or an apparatus for performing these operations. The apparatus may be specially constructed for specific required purposes, or it may be a general purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations. 
     The various embodiments described herein may be practiced with other computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like. 
     One or more embodiments of the present invention may be implemented as one or more computer programs or as one or more computer program modules embodied in one or more computer readable media. The term computer readable medium refers to any data storage device that can store data which can thereafter be input to a computer system computer readable media may be based on any existing or subsequently developed technology for embodying computer programs in a manner that enables them to be read by a computer. Examples of a computer readable medium include a hard drive, network attached storage (NAS), read-only memory, random-access memory (e.g., a flash memory device), a CD (Compact Discs) CD-ROM, a CD-R, or a CD-RW, a DVD (Digital Versatile Disc), a magnetic tape, and other optical and non-optical data storage devices. The computer readable medium can also be distributed over a network coupled computer system so that the computer readable code is stored and executed in a distributed fashion. 
     Although one or more embodiments of the present invention have been described in some detail for clarity of understanding, it will be apparent that certain changes and modifications may be made within the scope of the claims. Accordingly, the described embodiments are to be considered as illustrative and not restrictive, and the scope of the claims is not to be limited to details given herein, but may be modified within the scope and equivalents of the claims. In the claims, elements and/or steps do not imply any particular order of operation, unless explicitly stated in the claims. 
     In addition, while described virtualization methods have generally assumed that virtual machines present interfaces consistent with a particular hardware system, persons of ordinary skill in the art will recognize that the methods described may be used in conjunction with virtualizations that do not correspond directly to any particular hardware system. Virtualization systems in accordance with the various embodiments, implemented as hosted embodiments, non-hosted embodiments, or as embodiments that tend to blur distinctions between the two, are all envisioned. Furthermore, various virtualization operations may be wholly or partially implemented in hardware. For example, a hardware implementation may employ a look-up table for modification of storage access requests to secure non-disk data. 
     Many variations, modifications, additions, and improvements are possible, regardless the degree of virtualization. The virtualization software can therefore include components of a host, console, or guest operating system that performs virtualization functions. Plural instances may be provided for components, operations or structures described herein as a single instance. Finally, boundaries between various components, operations and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the invention(s). In general, structures and functionality presented as separate components in exemplary configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements may fall within the scope of the appended claims(s).