Patent Publication Number: US-2023153106-A1

Title: Techniques for non-disruptive system upgrade

Description:
RELATED APPLICATIONS 
     The present patent is a continuation of, and hereby claims priority under 35 U.S.C. § 120 to pending U.S. patent application Ser. No. 17/336,398, entitled “TECHNIQUES FOR NON-DISRUPTIVE SYSTEM UPGRADE,” by the same inventors, filed on 2 Jun. 2021, which claims the benefit under 35 U.S.C. 119(a)-(d) to Foreign Application Serial No. 202141008544 filed in India entitled “TECHNIQUES FOR NON-DISRUPTIVE SYSTEM UPGRADE”, on Mar. 1, 2021, by the same inventors, the contents of each of which are incorporated herein by reference in their entirety. 
    
    
     BACKGROUND 
     Upgrading an operating system (OS) on a device conventionally leads to significant amounts of downtime for services running on the device. For example, services may need be shut down while the upgraded OS is installed, configured, and booted. Furthermore, device and driver state information is generally lost during the transition to the upgraded OS, leading to further inefficiencies. These drawbacks are particularly problematic in software-defined data centers in which large numbers of services are run and accessed by a plurality of users, and downtimes are not expected. 
     An SDDC may comprise a plurality of hosts in communication over a physical network infrastructure. Each host is a physical computer (machine) that may run one or more virtualized endpoints such as virtual machines (VMs), containers, and/or other virtual computing instances (VCIs). In some cases, VCIs are connected to software-defined networks (SDNs), also referred to herein as logical overlay networks, that may span multiple hosts and are decoupled from the underlying physical network infrastructure. 
     Certain existing techniques for upgrading an OS on a host in an SDDC involve relocating all VCIs from the host to different endpoints in the SDCC for the duration of the upgrade. However, these techniques lead to downtimes while VCIs are being moved and require resources on separate hosts to be available for temporary placements of VCIs. 
     Accordingly, there is a need in the art for improved techniques for upgrading an OS, particularly in SDDCs. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG.  1    depicts example physical and virtual network components with which embodiments of the present disclosure may be implemented. 
         FIGS.  2 A and  2 B  illustrate an existing technique and a new technique for a maintenance mode for OS upgrades. 
         FIGS.  3 A and  3 B  illustrate an existing technique and a new technique for upgrading an OS. 
         FIG.  4    illustrates an existing technique and a new technique for transferring control of resources to an upgraded OS. 
         FIG.  5    depicts example operations related to a non-disruptive system upgrade. 
         FIG.  6    depicts additional example operations related to a non-disruptive system upgrade. 
         FIG.  7    depicts additional example operations related to a non-disruptive system upgrade. 
     
    
    
     To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements disclosed in one embodiment may be beneficially utilized on other embodiments without specific recitation. 
     DETAILED DESCRIPTION 
     The present disclosure provides an approach for non-intrusive system upgrades. In a first aspect, a “lightweight” maintenance mode is utilized for installing and configuring an upgraded version of an OS on a computing system while still allowing some operations to continue uninterrupted on the computing system. As described in more detail below with respect to  FIG.  2   , the lightweight maintenance mode restricts certain types of activity on the computing system, such as the addition of new applications and configuration changes, and may cause some or all storage I/O operations to be serviced externally over the network rather than locally on the computing system. 
     In a second aspect, a reboot-less OS upgrade is performed (e.g., during the maintenance mode). In certain embodiments, as described in more detail below with respect to  FIG.  3   , an upgraded version of an OS is installed and pre-initialized in a paravirtualized environment within the computing system while the existing version of the OS continues to run. The paravirtualized environment may be provided by a guest VCI running on the computing system (e.g., which may be a host computer). The OS may be a hypervisor. As described in more detail below with respect to  FIG.  1   , a hypervisor abstracts processor, memory, storage, and networking resources of a host device for multiple virtual computing instances (VCIs) that run concurrently on the same host device. 
     Once the upgraded OS is ready to replace the existing version of the OS, a third aspect involves a non-disruptive technique for transferring control over devices of the computing system to the upgraded OS without losing cache contents and state information. As described in more detail below with respect to  FIG.  4   , the contents of device caches are maintained and device and driver state information is saved in memory (which may be referred to as a “freeze” operation) prior to transferring control over devices of the computing system to the upgraded OS. The device and driver state information is then restored from memory (which may be referred to as a “thaw” operation) with the upgraded OS as the controlling OS of the computing system. 
     Accordingly, embodiments of the present disclosure constitute an improvement over existing techniques for upgrading an OS by allowing an OS to be upgraded while applications and devices continue to function through the upgrade process with minimal interruption. 
       FIG.  1    depicts example physical and virtual network components with which embodiments of the present disclosure may be implemented. 
     Networking environment  100  includes data center  130  connected to network  110 . Network  110  is generally representative of a network of machines such as a local area network (“LAN”) or a wide area network (“WAN”), a network of networks, such as the Internet, or any connection over which data may be transmitted. 
     Data center  130  generally represents a set of networked machines and may comprise a logical overlay network. Data center  130  includes host(s)  105 , a gateway  134 , a data network  132 , which may be a Layer 3 netwok, and a management network  126 . Host(s)  105  may be an example of machines. Data network  132  and management network  126  may be separate physical networks or different virtual local area networks (VLANs) on the same physical network. 
     It is noted that, while not shown, additional data centers may also be connected to data center  130  via network  110 . Communication between the different data centers may be performed via gateways associated with the different data centers. 
     Each of hosts  105  may include a server grade hardware platform  106 , such as an x86 architecture platform. For example, hosts  105  may be geographically co-located servers on the same rack or on different racks. Host  105  is configured to provide a virtualization layer, also referred to as a hypervisor  116 , that abstracts processor, memory, storage, and networking resources of hardware platform  106  for multiple virtual computing instances (VCIs)  135   1  to  135   n  (collectively referred to as VCIs  135  and individually referred to as VCI  135 ) that run concurrently on the same host. VCIs  135  may include, for instance, VMs, containers, virtual appliances, and/or the like. VCIs  135  may be an example of machines. For example, a containerized microservice may run on a VCI  135 . 
     In certain aspects, hypervisor  116  may run in conjunction with an operating system (not shown) in host  105 . In some embodiments, hypervisor  116  can be installed as system level software directly on hardware platform  106  of host  105  (often referred to as “bare metal” installation) and be conceptually interposed between the physical hardware and the guest operating systems executing in the virtual machines. It is noted that the term “operating system,” as used herein, may refer to a hypervisor. In certain aspects, hypervisor  116  implements one or more logical entities, such as logical switches, routers, etc. as one or more virtual entities such as virtual switches, routers, etc. In some implementations, hypervisor  116  may comprise system level software as well as a “Domain 0” or “Root Partition” virtual machine (not shown) which is a privileged machine that has access to the physical hardware resources of the host. In this implementation, one or more of a virtual switch, virtual router, virtual tunnel endpoint (VTEP), etc., along with hardware drivers, may reside in the privileged virtual machine. 
     Gateway  134  provides VCIs  135  and other components in data center  130  with connectivity to network  110 , and is used to communicate with destinations external to data center  130  (not shown). Gateway  134  may be implemented as one or more VCIs, physical devices, and/or software modules running within one or more hosts  105 . 
     Controller  136  generally represents a control plane that manages configuration of VCIs  135  within data center  130 . Controller  136  may be a computer program that resides and executes in a central server in data center  130  or, alternatively, controller  136  may run as a virtual appliance (e.g., a VM) in one of hosts  105 . Although shown as a single unit, it should be understood that controller  136  may be implemented as a distributed or clustered system. That is, controller  136  may include multiple servers or virtual computing instances that implement controller functions. Controller  136  is associated with one or more virtual and/or physical CPUs (not shown). Processor(s) resources allotted or assigned to controller  136  may be unique to controller  136 , or may be shared with other components of data center  130 . Controller  136  communicates with hosts  105  via management network  126 . 
     Manager  138  represents a management plane comprising one or more computing devices responsible for receiving logical network configuration inputs, such as from a network administrator, defining one or more endpoints (e.g., VCIs and/or containers) and the connections between the endpoints, as well as rules governing communications between various endpoints. In one embodiment, manager  138  is a computer program that executes in a central server in networking environment  100 , or alternatively, manager  138  may run in a VM, e.g. in one of hosts  105 . Manager  138  is configured to receive inputs from an administrator or other entity, e.g., via a web interface or API, and carry out administrative tasks for data center  130 , including centralized network management and providing an aggregated system view for a user. 
     According to embodiments of the present disclosure, hypervisor  116  (or another OS running on a computing device, such as host  105  in data center  130 ) may be upgraded with minimal interruption to applications. For example, a lightweight maintenance mode may be entered on host  105 , an upgraded version of hypervisor  116  may be installed and pre-initialized on host  105 , and control of devices on host  105  may be transferred to the upgraded hypervisor  116  using a freeze-thaw approach in which cache contents and device and driver state are maintained through the upgrade process. 
       FIG.  2 A  depicts an illustration  200 A of an existing maintenance mode technique, which is improved upon by embodiments of the present disclosure described below with respect to  FIG.  2 B . It is noted that while certain embodiments are described with respect to  FIGS.  2 A and  2 B  as involving virtualized components (e.g., VMs,vSANs, and the like), techniques described herein may also be used to upgrade OSes in physical computing environments without virtualized components. 
     At  212 , “entering maintenance mode” (EMM) starts on a computing system. EMM is the process by which maintenance mode is initialized on the computing system, such as to perform an OS upgrade.  214 - 220  represent the EMM process. At  214 , all workloads on the computing system are suspended. At  216 , all agent VMs are powered off. Agent VMs are VMs that provide services related to the virtual infrastructure, such as antivirus services, and support the functioning of other VMs. According to existing techniques, from  216 - 236 , all services on the computing system are unavailable, as all workloads have been suspended and all agent VMs have been powered off. 
     At  218 , the hostd process enters maintenance mode. The hostd process is an application that is responsible for managing many operations on a host computer, such as creation, powering on, and placement of VCIs. Before the hostd process can be placed in maintenance mode at  218 , all network file copy (NFC) sessions of the hostd process must be completed at  242 . NFC is a protocol used to transfer virtual disk data between hosts and management components in a data center. 
     At  220 , the virtual storage area network (vSAN) enters maintenance mode. A vSAN is a logical storage network in which storage resources of host devices are abstracted into logical constructs. Before the vSAN can be placed in maintenance mode at  220 , all component re-syncs and component movements in the vSAN must be completed at  244  and  246 . Component re-syncs in a vSAN are generally triggered by policies, and may involve creating another version of a disk object and synchronizing it with the previous version. Component movements in a vSAN are generally performed to optimize placement of data. 
     At  222 , the EMM process is completed, as maintenance mode has been entered on the computing system. 
     At  224 , the new image is installed. For instance, an image of an upgraded version of the OS is installed on the computing system. At  226 , the new image is configured and/or booted. For instance, code and an initial state of the upgraded OS may be configured, and then the upgraded OS may be booted as the controlling OS of the computing system. 
     At  228 , vSAN log recovery is performed in order to recover the state of the vSAN on the upgraded OS. At  230 , the exit maintenance mode process begins. The exit maintenance mode process comprises steps  232 - 238 . At  232 , the vSAN exits maintenance mode. At  234 , the hostd process exits maintenance mode. At  236 , the system waits for agent VMs to power on. Once agent VMs have powered on, workloads are resumed at  238 . At this point, services on the computing system are once again available. At  240 , the exit maintenance mode process is complete, and the OS has been upgraded. 
     Embodiments of the present disclosure improve upon the existing technique discussed above with respect to illustration  200 A by significantly reducing downtime for services running on the computing system and allowing for a more seamless transition to an upgraded OS. 
       FIG.  2 B  depicts an illustration  200 B of an improved maintenance mode technique according to embodiments of the present disclosure. In particular, illustration  200 B shows the use of a lightweight maintenance mode that allows certain operations to continue with minimal interruption throughout an OS upgrade process. 
     At  250 , a new image is installed on a computing system. For example, an image of an upgraded OS may be installed on the computing system. In one embodiment, as described in more detail below with respect to  FIG.  3 B , the new image may be installed in a paravirtualized environment within the computing system. In some embodiments, the new image may be installed in a separate bootbank from the current version of the OS. Operations on the computing system continue uninterrupted while the new image is installed. 
     At  252  the entering maintenance mode (EMM) process starts on the computing system. The EMM process comprises  254 - 258 . At  254 , any VMs that are intolerant of brief downtimes are evacuated from the computing system. For example, external-facing VMs (e.g., gateways) may not be able to tolerate any downtime at all due to the amount of traffic that they process, and so these VMs may be migrated to other hosts for the duration of the maintenance mode. However, unlike in the existing technique described above with respect to  FIG.  2 A , VMs that can tolerate small amounts of downtime may continue to run on the computing system during the lightweight maintenance mode. 
     At  256 , the hostd process enters maintenance mode. Before the hostd process can be placed in maintenance mode at  256 , all network file copy (NFC) sessions of the hostd process must be completed at  282 . Entering maintenance mode on the hostd process may involve preventing new VCIs from being added to the computing system and/or restricting configuration changes on the computing system for the duration of maintenance mode. 
     At  258 , the virtual storage area network (vSAN) enters maintenance mode. Before the vSAN can be placed in maintenance mode at  258 , all component re-syncs and component movements in the vSAN must be completed at  284  and  286 . Entering maintenance mode for the vSAN may involve changing the handling of storage operations (e.g., reads and writes) such that writes and at least some reads are handled outside of the computing system for the duration of maintenance mode, such as to a mirror copy of data. In some embodiments, reads of data that was written during maintenance mode may need to be handled externally, as the data would have been written externally rather than locally. In certain embodiments, all reads are handled outside of the computing system during maintenance mode, while in other embodiments reads of data that was written prior to entering maintenance mode are handled locally during maintenance mode. 
     At  260 , the EMM process is complete, and maintenance mode has been entered on the computing system. Once the computing system has been placed into maintenance mode, the final stages of the OS upgrade process may be completed. In some embodiments, as described in more detail below with respect to  FIG.  3 B , the upgraded OS may be pre-initialized while the existing version of the OS continues to run without interruption until the time comes to transfer control of the computing system resources from the existing version of the OS to the upgraded OS. 
     At  262 , workloads on the computing system are suspended. Thus, from  264 - 270 , services on the computing system are unavailable. It is noted that this is a significantly shorter period of downtime than  216 - 236  of  FIG.  2 A , during which services were unavailable according to existing techniques. In practical application, the improved process described with respect to illustration  200 B may involve only seconds of downtime as compared to the minutes or even hours of downtime that are common for existing techniques such as that described above with respect to illustration  200 A of  FIG.  2 A . 
     At  264 , agent VMs are powered off. At  266 , the new image is configured and/or booted on the computing system. The process performed at  266  may correspond to the non-disruptive OS upgrade process described below with respect to  FIG.  3 B  or to another process for configuring and/or booting an upgraded OS. 
     At  268 , vSAN log recovery is performed in order to recover the state of the vSAN on the upgraded OS using a memory-transfer filesystem (memxferfs) process. Memxferfs is an in-memory filesystem, the contents of which can be shared between different OS versions. The data in the filesystem is valid until a full system reboot occurs. Any soft reboot mechanism can be used to preserve the contents of the filesystem across an OS reboot. The memxferfs process involves restoring the state of the vSAN based at least in part on data saved in memory, and requires less network traffic in order to restore the state of the vSAN. 
     AT  270 , agent VMs are powered on. At  272 , workloads are resumed on the computing system. From this point on, services are once again available on the computing system. At  274 , the exit maintenance mode process begins. The exit maintenance mode process comprises steps  276 - 278 . At  276 , the hostd process exits maintenance mode. At  278 , the vSAN exits maintenance mode. At  280 , the exit maintenance mode process is complete, and the OS has been upgraded. 
     In some embodiments, the lightweight maintenance mode may be used to install and pre-initialize upgraded OSes on multiple hosts in parallel while services on the hosts continue uninterrupted. 
       FIG.  3 A  depicts an illustration  300 A of an existing technique for upgrading an OS. The existing technique described with respect to illustration  300 A is improved upon by embodiments of the present disclosure, as described below with respect to  FIG.  3 B . 
     At  302 , an OS upgrade process begins, such as by installing an upgraded version of an OS on a computing system. At  304 , the existing version of the OS is shut down, beginning a period of application and VM downtime  330 . 
     At  304 , a hardware reboot is performed. At  308 , boot modules for the upgraded OS are loaded. At  310 , early kernel initialization is performed. At  312 , the boot modules are extracted. At  314 , kernel modules are loaded. At  316 , kernel modules are initialized. At  318 , drivers are initialized. At  320 , OS services are started. Finally, at  322 , applications are restarted, thus ending the period of application and VM downtime and completing the upgrade process. 
     Embodiments of the present disclosure greatly reduce downtime for applications and VCIs by installing and pre-initializing an upgraded version of an OS in a paravirtualized environment while the current version of the OS continues to run. 
       FIG.  3 B  depicts an illustration  300 B of an improved technique for upgrading an OS. It is noted that the improved technique for upgrading an operating system described herein with respect to  FIG.  3 B  may be performed in conjunction with the lightweight maintenance mode technique described above with respect to  FIG.  2 B  or may be performed independently of the lightweight maintenance mode technique. 
     At  350 , a non-disruptive OS upgrade process begins by installing an upgraded version of an OS in a paravirtualized environment within a host computer while a current version of the OS continues to run on the host computer. Paravirtualization is a virtualization technology in which an OS (e.g., an upgraded version of an OS) is modified prior to installation inside a computing device in order to allow the OS within the computing device to share resources and successfully collaborate with other OSes (e.g., an existing version of the OS), rather than attempting to emulate an entire hardware environment. 
     The upgraded version of the OS may be granted supervisor privileges on the host, as it is intended to take over control of the host as the primary OS. In a traditional virtualization context, it would be unacceptable for the guest to be granted privileges outside of its virtualized environment, lest it have free reign over the host. However, in the present case, because the intent is to eventually transfer control of the host to the guest, the guest is assumed to be non-malicious and may be granted higher levels of privileges. The risk of executing compromised code on the guest may further be mitigated by digital signing technologies and other security measures. Whereas hypervisors usually aim for strong isolation guarantees between guests and host, the present solution can take a more cooperative approach to facilitate the eventual handover to the upgraded version of the OS. 
     At  352 , boot modules for the upgraded version of the OS are loaded in the paravirtualized environment, and at  354  the boot modules are extracted. At  356 , kernel modules for the upgraded version of the OS are loaded in the paravirtualized environment. At  358 , the new kernel of the upgraded version of the OS is pre-initialized, at  360  the new kernel modules are pre-initialized, at  362  the drivers are prepared for the upgraded version of the OS (e.g., involving selecting and loading upgraded driver modules), at  364  services of the upgraded version of the OS are prepared, and at  366 , drivers of the upgraded version of the OS are initialized (e.g., involving executing driver initialization code to get drivers into a state where they are ready to accept handed-over state information from the previous drivers). Pre-initializing the kernel and its modules, preparing drivers and services, and initializing drivers for the upgraded OS may involve executing an OS configuration application that uses application programming interface (API) calls to write measured values, represented as hashes of modules and settings, to the system and validates the kernel using a digital certificate. In some embodiments, a new address space is created for the OS configuration application that is separate from the address space of the kernel of the existing version of the OS so that the OS configuration application&#39;s execution is not able to corrupt the existing kernel&#39;s memory. 
     Interrupts may be initially disabled for the OS configuration application, and interrupts may instead be handled using paravirtualization techniques. During a regular boot, the OS configuration application expects to run with interrupts disabled, so the present technique similarly disables interrupts for the OS configuration application execution. However, the CPU executing the OS configuration application might receive an inter-processor interrupt (IPI) from another CPU executing kernel code, as the rest of the system is still operating as normal. Typical IPIs include CPU heartbeats and TLB invalidation requests. These requests cannot simply be ignored or delayed, as this would cause the existing kernel to eventually incorrectly determine that the CPU executing the OS configuration application has locked up. Similarly, the excessive deferral of timer and device interrupts may cause erratic behavior and degraded performance for the existing OS, perhaps even affecting workloads. Therefore, interrupts are forwarded to the kernel of the existing OS. The guest registers its own set of interrupt handlers during the OS configuration application execution, and these handlers perform a hypercall to provide the interrupt&#39;s context and state to the existing kernel, which uses its existing interrupt-handling infrastructure to take an appropriate action. A hypercall is a paravirtualization interface that allow a guest to request something from the running hypervisor, such as to fetch information or perform actions involving the host&#39;s shared resources. Hypercalls are commonly used with hypervisors, and a guest performs a hypercall by executing a special instruction that will trigger a mode switch from the guest to the hypervisor. The hypervisor then retrieves the guest&#39;s request from memory and/or CPU registers, performs the requested action, and returns control to the guest. As such, hypercalls are similar to syscalls, except that they must also cross the virtualization boundary. According to techniques described herein, the OS configuration application process must share host resources with the existing kernel. Instead of a mode switch, the OS configuration application process and the existing kernel perform context switches, saving and restoring their state across transitions. 
     Context switches may be performed manually, much like a kernel&#39;s scheduler would switch between running processes. While the OS configuration application runs with its own page tables, the existing kernel&#39;s page tables must be restored to run the existing kernel&#39;s code. Furthermore, any architectural state information (e.g., a global descriptor table or an interrupt descriptor table) that was modified for the OS configuration application&#39;s execution must be restored. This is necessary both because the existing kernel&#39;s tables are not mapped in the OS configuration application&#39;s address space, and because the OS configuration application requires different interrupt handling behavior. 
     In one example, a context switch between the OS configuration application and the existing kernel may operate as follows:
     1. Save callee-saved registers;   2. Activate new address space&#39;s page tables;   3. Switch to new global descriptor table (GDT) and interrupt descriptor table (IDT);   4. Call target code, respecting typical calling conventions;   5. Return from call;   6. Restore GDT and IDT;   7. Restore address space; and   8. Restore callee-saved registers.   

     Thus, performing a hypercall during the upgrade process described herein includes performing a context switch from the OS configuration application to the existing kernel with the request in a particular CPU register. The existing kernel services the request and then switches back to the OS configuration application. The hypercall interface is easily extended by adding new request identifiers and handlers. 
     Utilizing hypercalls with embodiments of the present disclosure serves two purposes. First, as described above, hypercalls can be used in interrupt and exception handling. Secondly, hypercalls can be used for physical memory allocation. Unlike in a regular boot, the existing OS has already allocated memory for its own use. Thus, techniques described herein involve the guest avoiding conflicts by asking the existing OS to perform the allocation. 
     Control flow returns to the OS configuration application once interrupt handling is complete. The same mechanism used to handle interrupts may be used to handle exceptions, thereby enabling debugger support and the handling of unexpected errors. 
     Once the OS configuration application finishes executing, it will have set up the new kernel&#39;s code and initial state, thus completing steps  358 - 366 . Techniques described herein offer a distinct advantage over existing techniques in the ease with which state can be transferred to the upgraded version of the OS for optimizing its initialization. State which has already been initialized by the existing version of the OS does not need to be recalculated, thus saving precious time. According to certain embodiments, the following state may be copied from the kernel of the existing version of the OS to the OS configuration application&#39;s memory, and then passed on to the upgraded kernel as necessary: hardware timer frequencies; kernel log buffer contents; and retired pages. The symmetry of the pre- and post-upgrade OS may even obviate the need for hypercalls, as the relevant data structures can simply be copied or passed by reference during the upgrade process. 
     Accordingly, after a soft shutdown of the existing version of the OS, resources of the host are handed over to the upgraded version of the OS at  370 , thus beginning a brief period of application and VM downtime  380 . Techniques for handing over control of resources to the upgraded version of the OS are described in more detail below with respect to  FIG.  4   . Finally, the boot process is finished at  372 , and applications are restarted at  374 , thus ending the brief period of application and VM downtime  380 . 
       FIG.  4    depicts an illustration  400  an existing technique  482  and a new technique  484  for transferring control of resources to an upgraded OS. It is noted that while certain embodiments are described with respect to an upgraded version of an existing OS, techniques described with respect to  FIG.  4    may also be employed for transferring control of resources to a rebooted instance of the same version of an OS. Furthermore, techniques described with respect to  FIG.  4    may also be employed independently of techniques described above with respect to  FIGS.  2 A and  3 A . Resources of the computing device include an I/O controller  420  having a device state  422  (representing state information for I/O controller  420  and/or relating to one or more devices connected to I/O controller  420 ) and a driver  426  having a driver state  424  (representing driver state information for one or more drivers). 
     Existing techniques generally involve two phases. First is a shutdown phase of an “old” OS/hypervisor, where the OS/hypervisor notifies device drivers to quiesce the hardware at a minimum, and in some cases to go further and tear down the hardware setup. Next is an initialization phase of a “new” OS/hypervisor, where the same or new device drivers are loaded, wherein the driver goes through an initialization phase for itself and the hardware. 
     In particular, in the existing technique  482 , control is being transferred from an old kernel  402  to either an upgraded version of the kernel or to the same kernel after a restart (same or new kernel  404 ). Prior to transferring control, the old kernel  402  interacts with the driver  426 . First, at  410 , the old kernel  402  interacts with driver  426  to perform a quiesce operation, stopping I/O operations and flushing device caches. Then, at  412 , the old kernel  402  interacts with driver  426  to perform a detach operation, destroying device state  422  and driver state  424 . Device state  422  and driver state  424  can include details such as interrupts allocated to the device, memory for intercommunication between device and driver, and device details gathered by the driver. Thus, in existing technique  482 , device caches and device and driver state information is lost in the transfer process, and must be rebuilt from scratch. 
     At  414 , same or new kernel  404  interacts with driver  426  (which, at this point, may be the same or a new driver after the upgrade process) to perform an attach operation, rebuilding device and driver state. At  416 , same or new kernel  404  repopulates cold caches during I/O operations. This is because the device caches were flushed at  410 , and became cold caches that must be repopulated from scratch as I/O traffic is processed. 
     New technique  484  addresses the issues of losing cache contents, device state, and driver state information that are present in existing technique  482 . According to certain embodiments, during the shutdown phase a device is maintained in as near an initialized state as possible, including any warm device caches in the device and driver. Driver state, which can be related to the device state or unrelated but needed by the driver for its operation, is saved to memory by the old OS/hypervisor using a mechanism such as memxferfs. During the new OS/hypervisor and driver initialization phase, the device is given to the driver in near-initialized state. The new OS/hypervisor reads the old driver&#39;s saved state from memory, enabling the new driver to bring itself and the device to an operational state faster and with less setup work, also avoiding the need to repopulate cold caches in the device or driver. This reduces both shutdown and boot-up work and time, thus reducing overall reboot time for a system. The new OS/hypervisor and driver can be same or different from the old OS/hypervisor and driver. Both the old and new OS/hypervisor and driver need to agree on the format of the saved state in information in memory in order to effect a successful transition. 
     In particular, new technique  484  involves transferring control from an old kernel  432  to either an upgraded version of the same kernel or a different kernel altogether (same or new kernel  434 ). Prior to transferring control, the old kernel  432  interacts with the driver  426 . First, at  450 , the old kernel  432  interacts with driver  426  to perform a quiesce operation, stopping I/O operations but keeping device caches (e.g., not flushing the device caches). Then, at  452 , the old kernel  432  interacts with driver  426  to perform a freeze operation, saving device state  422  and driver state  424  in memory. 
     At  454 , same or new kernel  434  interacts with driver  426  (which, at this point, may be the same or a new driver after the upgrade process) to perform a thaw operation, restoring device state  422  and driver state  424  from memory. Then, at  456 , same or new kernel  434  updates the device caches as needed during I/O, utilizing the existing contents of the caches that were maintained at  450 . Thus, new technique  484  allows device caches as well as device state and driver state information to be maintained through the transfer of control. 
       FIG.  5    depicts example operations  500  related to a non-disruptive system upgrade. For example, operations  500  may be performed by one or more components in networking environment  100  of  FIG.  1   , and may correspond to techniques for entering a lightweight maintenance mode that are described above with respect to  FIG.  2 B . 
     At step  502 , an upgraded version of an OS is installed on a computing system while the current version of the OS continues to run on the computing system. The computing system may, for instance, be a host computer, and the upgraded version of the OS may be an upgraded hypervisor that is installed in a paravirtualized environment within the host computer. 
     At step  504 , a maintenance mode is entered on the computing system. The maintenance mode may be a lightweight maintenance mode, and entering the maintenance mode may include preventing the addition of new applications to the computing system for the duration of the maintenance mode. Entering the maintenance mode may also include modifying the handling of storage operations on the computing system for the duration of the maintenance mode such that the computing system acts like a disk-less system and any storage operations are serviced over the network. In some embodiments, reads of data written to the computing system prior to entering the maintenance mode are still serviced locally during the maintenance mode. Entering the maintenance mode may further include restricting configuration changes on the computing system for the duration of the maintenance mode. In certain embodiments, entering the maintenance mode also includes moving a subset of applications (e.g., VCIs) running on the computing system to a different computing system for the duration of the maintenance mode, such as based on a determination that the subset of applications will not tolerate the amount of downtime required to complete the upgrade process. 
     At step  506 , during the maintenance mode, the upgraded version of the OS is configured. Configuring the upgraded version of the OS may include techniques described above with respect to  FIGS.  3 A and  3 B . 
     At step  508 , after configuring the upgraded version of the OS, a subset of applications running on the computing system is suspended. For instance, the subset of applications may be VCIs, and may include VCIs that were not moved to a different computing system for the duration of the maintenance mode. 
     At step  510 , after suspending the subset of applications, control over resources of the computing system is transferred to the upgraded version of the OS. Transferring control may involve techniques described above with respect to  FIG.  4   . For instance, contents of device caches may be maintained and device and driver state information may be saved in memory during the transfer of control and restored from memory after the transfer is complete. 
     At step  512 , after transferring control, the subset of applications running on the computing system is resumed. Furthermore, any applications moved off of the computing system as part of entering the maintenance mode may be moved back to the computing system. 
     At step  514 , the maintenance mode is exited on the computing system. 
       FIG.  6    depicts additional example operations  600  related to a non-disruptive system upgrade. For example, operations  600  may be performed by one or more components in networking environment  100  of  FIG.  1   , and may correspond to techniques for entering upgrading an OS that are described above with respect to  FIG.  3 B . 
     At step  602 , an upgraded version of an OS is installed in a paravirtualized environment within a computing system while a current version of the OS continues to run on the computing system. 
     At step  604 , the upgraded version of the OS is configured in the paravirtualized environment while the current version of the OS continues to run on the computing system. 
     Step  606  specifies that the upgraded version of the OS is granted a supervisor privilege level on the computing system. 
     Step  608  specifies that the paravirtualized environment requests the current version of the OS on the computing system to allocate memory to the paravirtualized environment for use in configuring the upgraded version of the OS. 
     Step  610  specifies that the paravirtualized environment forwards interrupts and exceptions to the current version of the OS on the computing system while the upgraded version of the OS is being configures. 
     Step  612  specifies that configuring the upgraded version of the OS comprises setting up code and an initial state of the upgraded version of the OS within the paravirtualized environment. 
       FIG.  7    depicts additional example operations  700  related to a non-disruptive system upgrade. For example, operations  700  may be performed by one or more components in networking environment  100  of  FIG.  1   , and may correspond to techniques for transferring control over resources of a computing system to an upgraded version of an OS that are described above with respect to  FIG.  4   . 
     At step  702 , an upgraded version of an OS is installed on a computing system while a current version of the OS continues to run on the computing system. 
     At step  704 , storage operations (e.g., I/O operations) are paused on devices running on the computing system. 
     At step  706 , contents of caches of the devices are maintained. 
     At step  708 , state information for the devices is saved into a memory of the computing system. 
     At step  710 , the upgraded version of the OS is booted. 
     At step  712 , the state information for the devices is restored from the memory of the computing system for use by the upgraded version of the OS. 
     At step  714 , storage operations are resumed on the devices. 
     The various embodiments described herein may employ various computer-implemented operations involving data stored in computer systems. For example, these operations may require physical manipulation of physical quantities—usually, though not necessarily, these quantities may take the form of electrical or magnetic signals, where they or representations of them are capable of being stored, transferred, combined, compared, or otherwise manipulated. Further, such manipulations are often referred to in terms, such as producing, identifying, determining, or comparing. Any operations described herein that form part of one or more embodiments of the invention may be useful machine operations. In addition, one or more embodiments of the invention also relate to a device or an apparatus for performing these operations. The apparatus may be specially constructed for specific required purposes, or it may be a general purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations. 
     The various embodiments described herein may be practiced with other computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and/or the like. 
     One or more embodiments of the present invention may be implemented as one or more computer programs or as one or more computer program modules embodied in one or more computer readable media. The term computer readable medium refers to any data storage device that can store data which can thereafter be input to a computer system—computer readable media may be based on any existing or subsequently developed technology for embodying computer programs in a manner that enables them to be read by a computer. Examples of a computer readable medium include a hard drive, network attached storage (NAS), read-only memory, random-access memory (e.g., a flash memory device), a CD (Compact Discs)—CD-ROM, a CD-R, or a CD-RW, a DVD (Digital Versatile Disc), a magnetic tape, and other optical and non-optical data storage devices. The computer readable medium can also be distributed over a network coupled computer system so that the computer readable code is stored and executed in a distributed fashion. 
     Although one or more embodiments of the present invention have been described in some detail for clarity of understanding, it will be apparent that certain changes and modifications may be made within the scope of the claims. Accordingly, the described embodiments are to be considered as illustrative and not restrictive, and the scope of the claims is not to be limited to details given herein, but may be modified within the scope and equivalents of the claims. In the claims, elements and/or steps do not imply any particular order of operation, unless explicitly stated in the claims. 
     Virtualization systems in accordance with the various embodiments may be implemented as hosted embodiments, non-hosted embodiments or as embodiments that tend to blur distinctions between the two, are all envisioned. Furthermore, various virtualization operations may be wholly or partially implemented in hardware. For example, a hardware implementation may employ a look-up table for modification of storage access requests to secure non-disk data. 
     Certain embodiments as described above involve a hardware abstraction layer on top of a host computer. The hardware abstraction layer allows multiple contexts to share the hardware resource. In one embodiment, these contexts are isolated from each other, each having at least a user application running therein. The hardware abstraction layer thus provides benefits of resource isolation and allocation among the contexts. In the foregoing embodiments, virtual machines are used as an example for the contexts and hypervisors as an example for the hardware abstraction layer. As described above, each virtual machine includes a guest operating system in which at least one application runs. It should be noted that these embodiments may also apply to other examples of contexts, such as containers not including a guest operating system, referred to herein as “OS-less containers” (see, e.g., www.docker.com). OS-less containers implement operating system—level virtualization, wherein an abstraction layer is provided on top of the kernel of an operating system on a host computer. The abstraction layer supports multiple OS-less containers each including an application and its dependencies. Each OS-less container runs as an isolated process in userspace on the host operating system and shares the kernel with other containers. The OS-less container relies on the kernel&#39;s functionality to make use of resource isolation (CPU, memory, block I/O, network, etc.) and separate namespaces and to completely isolate the application&#39;s view of the operating environments. By using OS-less containers, resources can be isolated, services restricted, and processes provisioned to have a private view of the operating system with their own process ID space, file system structure, and network interfaces. Multiple containers can share the same kernel, but each container can be constrained to only use a defined amount of resources such as CPU, memory and I/O. The term “virtualized computing instance” as used herein is meant to encompass both VMs and OS-less containers. 
     Many variations, modifications, additions, and improvements are possible, regardless the degree of virtualization. The virtualization software can therefore include components of a host, console, or guest operating system that performs virtualization functions. Plural instances may be provided for components, operations or structures described herein as a single instance. Boundaries between various components, operations and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the invention(s). In general, structures and functionality presented as separate components in exemplary configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements may fall within the scope of the appended claim(s).