Patent Description:
Unfortunately, because the file system virtualization provided by container virtual computing environments is provided by file system drivers, filter drivers, mini-drivers or other like computer-executable instructions instantiated after the booting of an operating system kernel, processes executing within the container environment cannot change the early stages of the operating system boot within the container environment, rendering such container environments unusable for many functions for which the isolation provided by such container environments would be particularly useful. For example, container environments cannot be utilized to develop, test and evaluate anti-malware software because anti-malware software typically requires the execution of computer-executable instructions during an early stage of an operating system boot. While the anti-malware software executing within the container environment could make such changes, the changes would be persisted within the container file system, which would not be accessible until after the kernel of the operating system had already booted and then instantiated the necessary file system drivers. As such, the changes made by the anti-malware software, as an example, would need to be available at an earlier point in time than existing container file system virtualization environments enable. The inability to persist changes affecting early stages of an operating system boot within container file system virtualization environments negatively impacts the usability of such container file system virtualization environments. <CIT> Al discloses systems and methods for implementing a provisioned machine that persists across a client machine reboot. A bootstrap function executing on a client machine identifies a delta disk stored on a physical disk of the client machine prior to booting up the operating system of the client machine. The bootstrap function establishes the path to the delta disk during the boot up of the operating system of the client machine. <CIT> Al discloses a method for instantiating containers with a unified data volume. Persistent storage is mounted to the host. An image file is copied to the memory of the host as a lower system layer of an isolated guest based on the image file, where the lower system layer is write protected. An upper system layer is constructed in the persistent storage based on the image file. The isolated guest is launched while attached to the lower system layer and the upper system layer. It is therefore the object of the present invention to provide an improved method of booting an operating system in a container providing a file system virtualization environment, as well as a corresponding computing device and corresponding computer-readable storage media.

A layered composite boot device, and a corresponding layered composite file system, can be implemented by computer-executable instructions that can be part of a boot manager, thereby providing access to virtualized container file systems at an earlier stage during an operating system boot in a container file system virtualization environment. Requests directed to the layered composite boot device, and the layered composite file system, can be serviced from a primary device, and primary file system, encapsulated by the layered composite boot device, and layered composite file system, respectively. Such a primary device, and primary file system, can correspond to a virtualized file system within a container environment, thereby enabling changes within the container environment to affect early stages of an operating system boot in such a container environment. Should such requests not be serviceable from the primary layers, the composite device and composite file system can comprise secondary layers, such as a secondary device that can correspond to a container host connection to the host computing environment, and a secondary file system that can correspond to the host file system, providing fallback to existing data if changes within the container environment were not made, and thereby enabling the operating system boot in the container environment to proceed in a traditional manner.

Additional features and advantages will be made apparent from the following detailed description that proceeds with reference to the accompanying drawings.

The following detailed description may be best understood when taken in conjunction with the accompanying drawings, of which:.

The following description relates to a layered composite boot device, and a corresponding layered composite file system, that can be implemented by computer-executable instructions that can be part of a boot manager, thereby providing access to virtualized container file systems at an earlier stage during an operating system boot in a container file system virtualization environment. Requests directed to the layered composite boot device, and the layered composite file system, can be serviced from a primary device, and primary file system, encapsulated by the layered composite boot device, and layered composite file system, respectively. Such a primary device, and primary file system, can correspond to a virtualized file system within a container environment, thereby enabling changes within the container environment to affect early stages of an operating system boot in such a container environment. Should such requests not be serviceable from the primary layers, the composite device and composite file system can comprise secondary layers, such as a secondary device that can correspond to a container host connection to the host computing environment, and a secondary file system that can correspond to the host file system, providing fallback to existing data if changes within the container environment were not made, and thereby enabling the operating system boot in the container environment to proceed in a traditional manner.

Although not required, the description below will be in the general context of computer-executable instructions, such as program modules, being executed by a computing device. More specifically, the description will reference acts and symbolic representations of operations that are performed by one or more computing devices or peripherals, unless indicated otherwise. As such, it will be understood that such acts and operations, which are at times referred to as being computer-executed, include the manipulation by a processing unit of electrical signals representing data in a structured form. This manipulation transforms the data or maintains it at locations in memory, which reconfigures or otherwise alters the operation of the computing device or peripherals in a manner well understood by those skilled in the art. The data structures where data is maintained are physical locations that have particular properties defined by the format of the data.

Generally, program modules include routines, programs, objects, components, data structures, and the like that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the computing devices need not be limited to conventional personal computers, and include other computing configurations, including servers, hand-held devices, multi-processor systems, microprocessor based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, and the like. Similarly, the computing devices need not be limited to stand-alone computing devices, as the mechanisms may also be practiced in distributed computing environment where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

With reference to <FIG>, an exemplary system <NUM> is illustrated, providing context for the descriptions below. The exemplary system <NUM> comprises an exemplary host computing environment <NUM>, which, for purposes of providing context for the descriptions below, can host a file system virtualization environment in the form of the exemplary container <NUM>. As utilized herein, the term "file system virtualization environment" means a virtualized computing environment that does not virtualize the underlying computing hardware or devices, but rather virtualizes the file system, thereby enabling isolation from the host computing device such that file-level changes made within the file system virtualization environment do not impact the host computing device's file system.

More specifically, and as illustrated by the exemplary system <NUM> of <FIG>, the exemplary host computing environment <NUM> can have executing thereon a host operating system, such as in the form of the exemplary host operating system <NUM>. As part of the host operating system <NUM>, one or more file system drivers, filters, mini-filters, and other like file system components, generally illustrated in <FIG> as the host file system drivers <NUM>, can obtain data in structured form from one or more storage media, such as the exemplary storage media <NUM>, and can parse such data into file constructs that can be presented to applications and other processes executing on the host operating system <NUM>. For example, an application <NUM>, executing on the host computing environment <NUM> can be presented with multiple files, such as the exemplary files <NUM>, <NUM> and <NUM>, as part of the host file system <NUM>. Analogously, a container operating system, such as the exemplary container operating system <NUM>, can execute within the exemplary container <NUM>. To provide for file system virtualization, however, the exemplary container operating system <NUM> can present different files then the host file system <NUM>.

According to one aspect, the file system of the exemplary container operating system <NUM> can be a layered file system that can enable applications executing within the container environment <NUM>, such as the exemplary application <NUM>, to access some or all of the same files of the host file system <NUM>, such as, for example, the exemplary files <NUM>, <NUM> and <NUM>, except that any changes or modifications to those files can remain only within the container environment <NUM>. For example, as illustrated by the exemplary system <NUM> of <FIG>, the exemplary container operating system <NUM> can comprise a layered file system in the form of the container file system <NUM>, which can act as a primary layer, or "overlay", in combination with the host file system <NUM>, which can act as a secondary layer, or "underlay".

The file systems referenced herein can be any of the known, existing file systems, such as the NT file system (NTFS), the Apple file system (APFS), the UNIX file system (UFS), and the like, or other file systems. Similarly, the file system drivers can be the corresponding drivers, filters, mini-filters, and other like drivers that can implement such file systems. Thus, for example, if the host file system <NUM> is NTFS, then the host file system drivers <NUM> can be the relevant NTFS drivers. Within the exemplary container environment <NUM>, however, the host file system <NUM> can be implemented in a slightly different manner so as to provide access to the host file system from within a file system virtualization environment. More specifically, according to one aspect, access to the host computing environment <NUM>, from within the container environment <NUM>, can be through a container host connection, such as the exemplary container host connection <NUM>. According to one aspect, the exemplary container host connection <NUM> can be in the form of a virtualized network connection which can simulate the container environment <NUM> being a separate computing device from the host computing environment <NUM>, and the two computing devices being communicationally coupled via a network. Other container host connections can be based on other communication protocols, such as peripheral interconnection protocols and the like. According to one aspect, the container host connection <NUM> can appear as a block data device from within the container environment <NUM>. Accordingly, the host file system <NUM>, from within the container environment <NUM>, can be implemented as a network file system, for example, and, accordingly, the drivers <NUM>, which can include drivers, filters, mini-filters and other like constructs, can implement such a network file system by communicating, through the container host connection <NUM>, with the host computing environment <NUM>. The host computing environment <NUM> can also comprise a file system having a same type as the host file system <NUM>. For example, if the host file system <NUM> is NTFS, the container file system <NUM> can also be NTFS. In such an instance, the container file system drivers <NUM> can comprise analogous drivers, filters, mini-filters and other like constructs to those of the host file system drivers <NUM>. Indeed, the same codebase, or even the same compiled binaries, can be utilized to implement both the container file system drivers <NUM> and the host file system drivers <NUM>.

According to one aspect, changes to files within the file system virtualization environment presented by the container <NUM> can be isolated from other file system virtualization environments and from the host computing environment <NUM> itself. For example, if the exemplary application <NUM>, executing within the container environment <NUM>, were to edit the exemplary file <NUM>, as illustrated by the edit action <NUM>, such a modification can result in a file <NUM>, representing an edited version of the file <NUM>, being part of the container file system <NUM>. The original file <NUM> would be unchanged. However, from within the container environment <NUM>, the layered file system would present the edited file <NUM> instead of the original file <NUM> from the host file system. Colloquially, the edited file <NUM> would "block" or "mask" the presentation of the original file <NUM>. If the exemplary application <NUM> did not edit the files <NUM> or <NUM>, those files would still "pass through" the overlay file system and be presented to applications or processes executing within the container <NUM>, such as the exemplary application <NUM>.

The digital data representing edited file <NUM> can be stored in a sandbox, such as the exemplary sandbox <NUM>, which can be accessed by the container file system drivers <NUM>, described previously, in order to generate the container file system <NUM>. As utilized herein, the term "sandbox" means one or more files, databases, structured storage, or other like digital data repository that can store the relevant data necessary to implement the container file system <NUM>. For example, the sandbox <NUM> can be a file, or a package, within the host file system <NUM>. In such a manner, the host computing environment <NUM> can remain isolated from edits performed within the container environment <NUM>. Thus, for example, if the edit <NUM> was a malicious action, the file <NUM> in the host computing environment <NUM> would remain isolated from, and unaffected by, such an action within the container environment <NUM>.

As indicated, in some instances, it can be desirable to allow processes executing within the container environment <NUM> to modify aspects of the container operating system <NUM>, including aspects that can be established during early portions of the boot process of the container operating system <NUM>. However, the layered file system presented by the container operating system <NUM> can be established at a much later point during the boot process of the container operating system <NUM>. For example, the container file system drivers <NUM> and virtual file system drivers <NUM> may not establish the layered file systems <NUM> and <NUM> until after the kernel of the container operating system <NUM> has executed. Indeed, in some instances, the kernel of the operating system <NUM> can be responsible for executing the relevant drivers <NUM> and <NUM>. Accordingly, if, for example, an application or process executing within the container <NUM> changed an aspect of the container operating system <NUM>, such a change could be stored in the container file system <NUM>, such as in the manner detailed previously. However, since the container file system <NUM> may not be accessible until after the kernel of the operating system <NUM> has executed the relevant drivers <NUM>, the operating system <NUM> will not be able to access such a change early in its boot process, since such a change will not be accessible until after the operating system kernel has already been loaded into memory. Accordingly, not having access to the container file system <NUM>, early portions of the booting of the container operating system <NUM> can only utilize data from the host computing device, which, as indicated previously, is unaffected by, and isolated from, changes made within the container environment <NUM>.

To enable processes executing within the container environment <NUM> to modify aspects of the container operating system <NUM>, including aspects that can be established during the early portions of the boot process of the container operating system <NUM>, a layered composite boot device and file system can be utilized during the booting of the container operating system <NUM>. Turning to <FIG>, the exemplary system <NUM> shown therein illustrates an exemplary initiation of a booting of an operating system in a container file system virtualization environment, such as the exemplary container <NUM>. More specifically, the exemplary host computing environment <NUM> can comprise a hypervisor, such as the exemplary hypervisor <NUM>, or other like hardware and/or software capabilities. The exemplary hypervisor <NUM> can utilize data from a container operating system image, such as the exemplary container operating system image <NUM>, or other like container data stored on the storage media <NUM> of the exemplary host computing environment <NUM>, to start booting an operating system within the exemplary container <NUM>. For example, as illustrated by the exemplary system <NUM>, the hypervisor <NUM> can cause container firmware <NUM> to execute within the container environment <NUM>, in the form of the executing container firmware <NUM>, as illustrated by the action <NUM>. The information and executable instructions stored in a container operating system image, such as the exemplary container operating system image <NUM>, can be static, such that they are not affected by changes to the host computing environment <NUM>, or they can be dynamic in that changes to the host computing environment <NUM> can result in changes to some or all of the container operating system image <NUM>. A dynamic container operating system image <NUM>, for example, can utilize the same executable instructions for some or all of the relevant portions of the container operating system image <NUM> as are utilized to boot the operating system of the host computing device <NUM> itself. In such an instance, changes to the executable instructions that boot the operating system of the host computing device <NUM> can necessarily result in changes to the container operating system image <NUM>.

As indicated previously, a virtualized network connection, such as the exemplary container host connection <NUM>, can be utilized for processes executing within the container <NUM> to access information from the host computing environment <NUM>. Accordingly, one aspect of the execution of the container firmware <NUM> within the container <NUM> can be the execution of drivers for the relevant devices that the container <NUM> may need to access during the boot process, such as graphics drivers, user input device drivers, such as keyboard drivers and mouse drivers, and, of relevance to the descriptions provided herein, network drivers that can enable the executing firmware <NUM> to access information from the host computing environment <NUM>.

Turning to <FIG>, the exemplary system <NUM> shown therein illustrates the execution of the container firmware <NUM> utilizing the container host connection <NUM> to access the container boot manager <NUM> as stored on the storage media <NUM> of the host computing environment <NUM>. As illustrated, the exemplary container boot manager <NUM> can be part of the exemplary container operating system image <NUM>. The location of the exemplary container boot manager <NUM> can be identified, to the firmware <NUM>, by the hypervisor <NUM> as part of the action <NUM> illustrated in <FIG>. Once accessed, the container boot manager <NUM> can be executed within the container, as the executing container boot manager <NUM>, by the container firmware <NUM>. The location and execution of the container boot manager <NUM>, by the firmware <NUM>, is illustrated in the exemplary system <NUM> by the action <NUM>.

Like the firmware <NUM>, the execution of the container boot manager <NUM> can include the execution of drivers, or other like computer-executable instructions, that can enable the container boot manager <NUM> to access relevant portions of the host operating system environment <NUM>, including, for example, graphics drivers, input peripheral drivers, network drivers, and the like. Such drivers can be the same drivers as utilized by the container firmware <NUM>, and, thus, such drivers can remain in memory for utilization by the container boot manager <NUM>, or the drivers can be different drivers, which can require the container boot manager <NUM> to clear the memory previously utilized by the container firmware <NUM>, and load its own drivers into the memory of the container environment <NUM>.

As part of the action <NUM>, the firmware <NUM> can identify, to the container boot manager <NUM>, a boot device to be utilized for the booting of an operating system into the container environment <NUM>. In the system <NUM> illustrated in <FIG>, such a boot device can be the container host connection <NUM>. Subsequently, the container boot manager <NUM> can access such a boot device to locate and obtain container boot configuration data. Turning to <FIG>, the exemplary system <NUM> illustrates an action <NUM> by which the container boot manager <NUM>, executing within the container environment <NUM>, utilizes the container host connection <NUM>, that was previously specified, to the container boot manager <NUM>, as the boot device, to locate and read the container boot configuration data <NUM>. As illustrated, the container boot configuration data <NUM> can also be part of the exemplary container operating system image <NUM>.

According to one aspect, to facilitate the utilization of modifications occurring within the container environment <NUM> in the subsequent booting of an operating system within the container environment <NUM>, the container boot configuration data <NUM> can specify a new boot device. More specifically, as illustrated by the specification <NUM> in the system <NUM> shown in <FIG>, the container boot configuration data <NUM> can specify a composite device, such as the exemplary composite device <NUM>, as a new boot device. The exemplary composite device <NUM> can implement a layering, analogous to that described above, having a primary layer and a secondary layer. Although described within the context of two layers, namely a primary layer and a secondary layer, a composite device, such as the exemplary composite device <NUM>, can comprise any number of layers, including a tertiary layer below the secondary layer, and so on.

According to the invention, as will be detailed further below, a composite device, such as the composite device <NUM>, accesses multiple devices in a hierarchical order. Thus, a primary device is accessed first, and, if the information sought is unavailable from the primary device, the secondary device is accessed, and the information sought and be provided therefrom. Conversely, again, if the information sought is available from the primary device, the secondary device may not need to be accessed. For purposes of implementing the ability for processes executing within a container environment <NUM> to make changes that can affect the subsequent boot of an operating system within the container environment <NUM>, the composite device <NUM> according to the invention is a composite of a primary device <NUM>, which, in the present example, can be the sandbox <NUM>, and a secondary device <NUM>, which, in the present example, can be the container host connection <NUM>.

A composite device, such as the exemplary composite device <NUM>, can enable the utilization of a composite file system, such as the exemplary composite file system <NUM>. As described above, a composite file system can layer a primary file system over a secondary file system such that, if a file is found in the primary file system, it can be utilized even if the same file exists in the secondary file system, while, if a file is not found in the primary file system, the secondary file system can be checked and, if found, the file can be sourced therefrom. As with the composite device <NUM>, the composite file system <NUM> can comprise multiple layers beyond just a primary and secondary file system. For example, the composite file system <NUM> can comprise a tertiary file system, and so on. Again, for purposes of implementing the ability for processes executing within a container environment <NUM> to make changes that can affect the subsequent boot of an operating system within the container environment <NUM>, the exemplary composite file system <NUM> can comprise a primary file system <NUM>, which, in the present example, can be the container file system <NUM>, and a secondary file system <NUM>, which, in the present example, can be the host file system <NUM>.

According to one aspect, a container boot manager, such as the container boot manager <NUM> from the container operating system image <NUM> in the host computing environment <NUM>, instantiated into, and executing within, the container environment <NUM> as the executing container boot manager <NUM>, can comprise computer-executable instructions that can operate with a composite device, such as the exemplary composite device <NUM>, and implement a composite file system, such as the exemplary composite file system <NUM>.

For example, the executing container boot manager <NUM> can execute drivers, filters, mini-filters, or other like computer executable instructions that can access the sandbox <NUM>. Such computer executable instructions are illustrated in the exemplary system <NUM>, shown in <FIG>, as the exemplary drivers <NUM>. According to one aspect, the exemplary drivers <NUM> can be similar to, or even equivalent to, the drivers ultimately utilized by the container operating system to implement the container file system <NUM>, such as the exemplary drivers <NUM> illustrated in <FIG> and described above. According to another aspect, the exemplary drivers <NUM> can differ from the exemplary drivers <NUM> in that they can implement a more rudimentary, simpler functionality that can comprise the device and file system functionality utilized by the container boot manager <NUM>, but can lack support for more complex functions. Again, the exemplary drivers <NUM> need not be driver code in a traditional sense, but rather can be computer-executable instructions capable of implementing the mechanisms described herein, in whatever form such computer executable instructions are packaged for purposes of convenience and/or interoperation with other aspects of the container environment <NUM> and/or the host computing environment <NUM>.

Analogously, the executing container boot manager <NUM> can execute drivers, filters, mini-filters, or other like computer executable instructions that can access the data of the host file system over the container host connection <NUM>. Such computer executable instructions are illustrated in the exemplary system <NUM>, shown in <FIG>, as the exemplary drivers <NUM>. According to one aspect, the exemplary drivers <NUM> can be similar to, or even equivalent to, the drivers ultimately utilized by the container operating system to implement the host file system <NUM> in the container <NUM>, such as the exemplary drivers <NUM> illustrated in <FIG> and described above. According to another aspect, the exemplary drivers <NUM> can differ from the exemplary drivers <NUM> in that they can implement a more rudimentary, simpler functionality that can comprise the device and file system functionality utilized by the container boot manager <NUM>, but can lack support for more complex functions. As before, the exemplary drivers <NUM> need not be driver code in a traditional sense, but rather can be computer-executable instructions capable of implementing the mechanisms described herein, in whatever form such computer executable instructions are packaged for purposes of convenience and/or interoperation with other aspects of the container environment <NUM> and/or the host computing environment <NUM>.

According to one aspect, the container boot manager <NUM> can further comprise computer-executable instructions, such as the exemplary drivers <NUM>, that can provide the device and/or file system aggregation described in further detail below. For example, the computer-executable instructions illustrated in <FIG> as the exemplary drivers <NUM> can comprise a device driver or other like computer-executable instructions that can receive device-centric requests, such as requests to mount a device, open a device, enumerate a device, and other like device-centric requests, and can direct such requests to the computer-executable instructions implementing the interfaces with the devices that are part of the composite device <NUM>, such as the primary device <NUM> and the secondary device <NUM>. Thus, for example, the drivers <NUM> can receive a request to mount the composite device <NUM>, and, in response, can direct mount device requests to both the drivers <NUM>, in order to mount the sandbox <NUM> as a device accessible from within the container environment <NUM>, and the drivers <NUM>, in order to mount the container host connection <NUM> as a device accessible from within the container environment <NUM>. The drivers <NUM> can then respond to the mount device request as if a single device was mounted, with the mounting of the individual primary and secondary devices being abstracted in the manner described. In an analogous manner, the computer-executable instructions illustrated in <FIG> as the exemplary drivers <NUM> can comprise one or more file system drivers, filter drivers, mini-filters or other like computer-executable instructions that can receive file-system-centric requests, such as requests to open a file, read from a file write to a file, enumerate the files within a folder, and other like file-system-centric requests, and can direct such requests to the computer-executable instructions implementing the interfaces with the file systems that are part of the composite file system <NUM>, such as the primary file system <NUM> and the secondary file system <NUM>. Thus, for example, the drivers <NUM> can receive to open a file in the composite device <NUM>, and, in response, can direct the file open request first to the drivers <NUM>, if the file is part of the container file system <NUM>, or the drivers <NUM> if the file is not part of the container file system <NUM>, but rather is part of the host file system <NUM>. The drivers <NUM> can then respond to the open file request as if a single file system was accessed, with the underlying interactions with the primary and secondary file systems being abstracted in the manner described.

Utilizing the composite device <NUM> as a new boot device, as specified by the container boot configuration data <NUM>, as illustrated by the specification <NUM>, the container boot manager <NUM> can locate container operating system loader computer executable instructions, such as the exemplary container operating system loader <NUM>, which can be part of the exemplary container operating system image <NUM>, and can execute such a container operating system loader <NUM> in the container environment <NUM>. Turning to <FIG>, the exemplary system <NUM> shown therein illustrates an action <NUM> by which the container boot manager <NUM> executes the container operating system loader <NUM> in the container environment <NUM> as the executing container operating system loader <NUM>.

According to one aspect, the container boot manager <NUM> can utilize the composite device <NUM> and the composite file system <NUM> to locate the container operating system loader <NUM>. In conformance with the specific aspect illustrated by the exemplary system <NUM>, the container operating system loader <NUM> can have been left unchanged by processes executing within the container <NUM> and, consequently, can be part of the host file system <NUM>, and not the container file system <NUM>, such as in accordance with the mechanisms described with reference to <FIG> above. Accordingly, requests by the container boot manager <NUM> to open and execute the container operating system loader <NUM> can be passed through the drivers <NUM> and the container operating system loader <NUM> can be obtained, through the container host connection <NUM>, from the container operating system image <NUM> in the host computing environment <NUM>.

Turning to <FIG>, the exemplary system <NUM> shown therein illustrates an exemplary operation of the container operating system loader <NUM> as executing within the container environment <NUM>. According to one aspect, the container operating system loader <NUM> can open the composite device <NUM>, and access the composite file system <NUM> in order to and locate configuration data for the container operating system that the container operating system loader <NUM> is to boot into the container environment <NUM>, such as the exemplary container operating system <NUM>. The exemplary system <NUM> shown in <FIG> illustrates an exemplary operation if the container operating system configuration was not changed by processes executing within the container <NUM>. More specifically, if the container operating system configuration was not previously changed by processes executing within the container <NUM>, then, when the container operating system loader <NUM> attempts to access the container operating system configuration data, no such modified data can be found in the primary file system <NUM>, such as, for example the container file system <NUM> that contains files that have been changed by processes executing within the container environment <NUM>, such as in the manner described above with reference to <FIG>. Accordingly, the container operating system loader <NUM> can, instead, find the container operating system configuration data through the secondary file system <NUM>, such as, for example, the host file system <NUM>. As shown in the exemplary system <NUM>, therefore, action <NUM>, by the container operating system loader <NUM>, obtains the container operating system configuration data <NUM>, from the operating system container image <NUM> in the host computing environment <NUM>, and utilizes such container operating system configuration data <NUM> to configure the booting of the container operating system <NUM>. Because the container operating system configuration data <NUM> was obtained from the host file system <NUM>, the composite device <NUM> can have provided the relevant access via the secondary device <NUM>, the composite file system <NUM> can have provided the relevant access via the secondary file system <NUM>, and the drivers <NUM>, responsible for the secondary device <NUM> and the secondary file system <NUM>, can have been utilized by the container operating system loader <NUM> to obtain the container operating system configuration data <NUM> from the host file system <NUM> and utilize the same to boot the container operating system <NUM> in the container environment <NUM>, as illustrated by the black highlighting.

By contrast, if one or more processes executing within the container environment <NUM> had previously changed part of the operating system configuration data, a copy of the container operating system configuration data <NUM> can be part of the container file system <NUM>, such as can be stored, or otherwise retained, in the sandbox <NUM>. As indicated previously, existing mechanisms would not have provided access to the container file system <NUM> until after the container operating system <NUM> had completed booting, because the relevant computer-executable instructions, such as the relevant drivers, can be loaded by the booting of the container operating system <NUM> and can be unavailable prior to such booting, rendering the container file system <NUM> inaccessible until after the container operating system <NUM> has completed booting. However, utilizing the mechanisms described herein, access to the container file system <NUM>, such as for purposes of accessing a modified copy of the container operating system configuration data <NUM>, reflecting modifications made by processes executing within the container environment <NUM>, can be provided at an earlier stage of the boot process, more specifically, prior to the container operating system loader <NUM> utilizing such operations and configuration data to boot and configure the container operating system <NUM>.

More specifically, and with reference to <FIG>, the exemplary system <NUM> shown therein illustrates an exemplary operation by which container operating system configuration data <NUM>, reflecting changes previously made by processes executing within the container environment <NUM>, can be utilized to control the configuration and booting of the container operating system <NUM>. The container operating system <NUM> is labeled with a different identifier than the container operating system <NUM> of the system <NUM> shown in <FIG> to indicate that the executing operating systems can have different configurations. The container operating system configuration data <NUM>, having been changed by processes executing within the container environment <NUM>, can be stored within the container file system <NUM>, as persisted in the sandbox <NUM>. Accordingly, the container operating system loader <NUM> can find the container operating system configuration data <NUM> through the primary file system <NUM>, such as, for example, the container file system <NUM>. As shown in the exemplary system <NUM>, therefore, action <NUM>, by the container operating system loader <NUM>, can obtain the container operating system configuration data <NUM>, from the sandbox <NUM>, and can utilize such container operating system configuration data <NUM> to configure the booting of the container operating system <NUM>. Because the container operating system configuration data <NUM> was obtained from the container file system <NUM>, the composite device <NUM> can have provided the relevant access via the primary device <NUM>, the composite file system <NUM> can have provided the relevant access via the primary file system <NUM>, and the drivers <NUM>, responsible for the secondary device <NUM> and the secondary file system <NUM>, can have been utilized by the container operating system loader <NUM> to obtain the container operating system configuration data <NUM> from the sandbox <NUM> and utilize the same to boot the container operating system <NUM> in the container environment <NUM>, as illustrated by the black highlighting in <FIG>.

In such a manner, changes made by processes executing within the container <NUM> can be accessible sufficiently early in the boot process to change the configuration of the container operating system booted within the container environment <NUM>. For example, and with reference back to the exemplary system <NUM> of <FIG>, if the application <NUM>, executing within the container environment <NUM>, was, for example, and anti-malware program, such an application <NUM> may need to have computer-executable instructions executed during an early stage of the operating system boot in order to detect potential malware threats occurring during the operating system boot. Accordingly, the application <NUM> can modify a file <NUM>, such as illustrated by the action <NUM>, where, in the present example, such a file <NUM> can be an operating system configuration file, and the modification <NUM> can be the insertion of a path to the computer-executable instructions that are to be executed during an early stage of the operating system boot, together with the insertion of an instruction to execute such computer-executable instructions, for example. The resulting modification to the file <NUM> can result in a copy of the file <NUM>, with such modifications, existing within the container file system <NUM>, such as, for example, persisted within the sandbox <NUM>, while the file <NUM> on the host file system <NUM> remains unchanged. With reference back to the exemplary system <NUM> shown in <FIG>, the exemplary container operating system configuration data <NUM> can remain unchanged within the host computing environment <NUM>, and the modifications described in the above example can, instead, be reflected in the container operating system configuration data <NUM>, which can be part of the container file system <NUM>, and can be persisted in the sandbox <NUM>.

Upon execution, having been informed of the composite device <NUM> to utilize as the boot device by the container boot manager, such as in the manner detailed previously, the container operating system loader <NUM> can access the composite device <NUM> and search the composite device <NUM> for container operating system configuration data. As will be detailed further below, the primary device <NUM>, such as the sandbox <NUM> in the present example, and the corresponding primary file system <NUM>, such as the container file system <NUM> in the present example, can be referenced first to determine whether the container operating system configuration data exists therein. Such a referencing can identify the container operating system configuration data <NUM>, comprising the modifications made by the application within the container process <NUM>, such as those detailed above in the present example. Consequently, the container operating system loader <NUM> can boot the container operating system <NUM>, as illustrated by the action <NUM>, utilizing the modified container operating system configuration data <NUM>.

According to one aspect, a slight modification can accommodate changes, not just to the container operating system configuration data, but to the container boot configuration data itself. More specifically, and turning back to the exemplary system <NUM> shown in <FIG>, as indicated previously, the container boot configuration data <NUM> can comprise an indicator <NUM> of the composite device <NUM> to be utilized as the new boot device, since the container boot manager process <NUM> can initially have been executed utilizing the container host connection <NUM> as the boot device, such as detailed previously. According to the presently described aspect, the container boot manager <NUM> can detect when the boot device specified by the container boot configuration data <NUM> differs from the boot device utilized by the container boot manager <NUM> to read the container boot configuration data <NUM> in the first place. For example, the container boot manager <NUM> can compare identifiers of the boot device specified by the container boot configuration data <NUM> and the boot device utilized by the container boot manager <NUM> to read the container boot configuration data <NUM> in the first place. Such identifiers can be based on the device type and/or the entire device descriptor. In the case of composite devices, such as the exemplary composite device <NUM>, the identifier of such composite devices can be based on the device types and descriptors of the devices abstracted by the composite device. According to one aspect, device descriptors can, in turn, include metadata of the corresponding devices, such as interface identifiers, channel identifiers, disk identifiers, partition identifiers and the like. When such a difference between devices is detected, the container boot manager <NUM> can utilize the newly specified device, such as, in the present example, the composite device <NUM>, to locate and read the container boot configuration data again.

More specifically, and turning to the exemplary system <NUM> shown in <FIG>, upon detecting that the container boot configuration data <NUM> specifies a different boot device, namely the composite device <NUM>, the container boot manager <NUM> can utilize the composite device <NUM> to again open the container boot configuration data. If the container boot configuration data had been modified, then, in accordance with the mechanisms detailed previously, a new copy of the container boot configuration data, such as the exemplary copy of the container boot configuration data <NUM>, reflecting such modifications, can be part of the container file system <NUM> such as persisted in the sandbox <NUM>. The container boot manager <NUM> can be provided with an enumeration of the container boot configuration data <NUM> from the primary file system <NUM>, rather than the container boot configuration data <NUM> from the secondary file system <NUM>. Accordingly, the container boot manager <NUM> can read the container boot configuration data <NUM>, from the container file system <NUM>, is illustrated by the action <NUM>. In the exemplary system <NUM> shown in <FIG>, the container boot configuration data <NUM> can also specify the composite device <NUM> as the boot device, as illustrated by the specification <NUM>. Accordingly, since the boot device specified by the container boot configuration data <NUM> is the same, namely the composite device <NUM>, as that utilized by the container boot manager <NUM> to read the container boot configuration data, the container boot manager <NUM> can proceed, such as in the manner detailed above, except that the configuration utilized by the container boot manager <NUM> can be in accordance with the modifications made from within the container environment <NUM>, as reflected in the container boot configuration data <NUM>, as opposed to the container boot configuration data <NUM>, which can remain unchanged on the host computing environment <NUM>, thereby maintaining the isolation of the host computing environment <NUM> from the container environment <NUM>.

Turning to <FIG>, exemplary flow diagram <NUM> shown therein illustrates an exemplary series of steps that can enable processes executing within a container file system virtualization environment to make modifications that can be implemented during early stages of the operating system boot within such a container file system virtualization environment. Initially, at step <NUM>, a container instance can be created on a host computing device. Such a creation of a container instance can include the reservation of memory, the establishment of underlying hardware and communication functionality, and the like. Subsequently, at step <NUM>, a hypervisor, such as based on instructions and/or parameters provided by container manager processes, can instantiate firmware to execute within the container instance. As indicated in <FIG>, step <NUM> can correspond to the exemplary system <NUM> shown in <FIG> and described in detail above.

At step <NUM>, the firmware executing within the container instance created at step <NUM> can initialize and open a container host connection that can have been identified as part of the instantiation of the firmware, by the hypervisor, at step <NUM>, as the boot device to be utilized by the firmware. At step <NUM>, the firmware can utilize the boot device, namely the container host connection, to locate a container boot manager, such as from the host computing environment, and instantiate the container boot manager into the container instance created at step <NUM>. The instantiation of the container boot manager, by the firmware, at step <NUM>, can include the passing of parameters from the firmware to the container boot manager. For example, the parameters can be provided from the firmware to the container boot manager in the form of pointers to values, command line parameters provided as part of an execution instruction implemented by the container boot manager, or other like passing of parameters into a process being instantiated. Such parameters can include a specification of the boot device, which, in the present example, can still be the container host connection. As indicated in <FIG>, step <NUM> can correspond to the exemplary system <NUM> shown in <FIG> and described in detail above.

At step <NUM>, the container boot manager can utilize the identified boot device, passed in as a parameter by the firmware at step <NUM>, to find, open and read container boot configuration data. In the present example, at step <NUM>, such container boot configuration data will be read through the container host connection. At step <NUM>, as part of the reading of the container boot configuration data, the container boot configuration data can include a specification of a new boot device, namely the composite device detailed above. As indicated in <FIG>, steps <NUM> and <NUM> can correspond to the exemplary system <NUM> shown in <FIG> and described in detail above. Optionally, at step <NUM>, the container boot manager can detect that the boot device specified by the container boot configuration data read at step <NUM> differs from the boot device utilized by the container boot manager to read the container boot configuration data in the first place at step <NUM>. Step <NUM> is illustrated in <FIG> utilizing dashed lines to indicate that it is an optional step. Also optionally, at step <NUM>, upon determining, at step <NUM>, that the boot device specified by the container boot configuration data differs from that utilized to read the container boot configuration data in the first place, the container boot manager can utilize the newly specified boot device and again read the container boot configuration data, at step <NUM>, this time from such a newly specified boot device. As detailed above, such a step <NUM> can enable container boot configuration data that can have been modified by processes executing within the container to be utilized during the boot process. Like step <NUM>, step <NUM> is also illustrated in <FIG> utilizing dashed lines to indicate that it is optional. As indicated in <FIG>, steps <NUM> and <NUM> can correspond to the exemplary system <NUM> shown in <FIG> and described in detail above.

Subsequent to step <NUM>, if performed, or step <NUM>, if the aspect represented by steps <NUM> and <NUM> is not utilized, processing can proceed to step <NUM> and the container boot manager can utilize the new boot device specified at step <NUM>, namely the composite device in the present example, to locate and instantiate a container operating system loader into the container instance created in step <NUM>. At step <NUM>, the container operating system loader can be obtained from the host computing device, if no changes were made to the operating system loader within the container environment. More specifically, the utilization of the new boot device, being a composite device, can first check a primary device, and a primary file system, for the container operating system loader. As detailed above, such a primary device, and primary file system, can be associated with the container environment in order to find any modified copies of the container operating system loader. If the container operating system loader was unmodified, then a secondary device and a secondary file system, such as associated with the host computing device, can be utilized and the container operating system loader can be instantiated, at step <NUM>, from there. As indicated in <FIG>, step <NUM> can correspond to the exemplary system <NUM> shown in <FIG> and described in detail above.

Once instantiated into the container instance, and executing therefrom, the operating system loader can proceed to find and read operating system configuration data from the boot device, which, in the present example, can be the composite device specified by the container boot configuration data at step <NUM>. At step <NUM>, therefore, if the operating system configuration data was modified, a primary device, and a primary file system, associated with the container environment, can be utilized to provide access to the modified operating system configuration data, and the operating system loader can utilize such modified operating system configuration data to boot the operating system. In such a manner, processes executing within a container environment can modify the booting of the operating system of such a container environment. Conversely, if the operating system configuration data was not modified, a secondary device, and a secondary file system, associated with the host computing environment, can be utilized to provide access to the unmodified operating system configuration data and, at step <NUM>, the operating system loader can utilize such unmodified operating system configuration data to boot the operating system. The booting of an operating system into the container environment can then complete at step <NUM>.

Turning to <FIG>, the exemplary flow diagram <NUM> shown therein illustrates an exemplary series of steps by which the composite device and the composite file system can be implemented to provide the above described layering. Initially, at step <NUM>, an access request can be received. Such an access request can be a file-based access request, a folder-based or directory-based access request, or a device-based access request. Although only select access requests are illustrated and described, other corresponding access requests can proceed among the multiple layers of the composite device and composite file system in an analogous manner to those illustrated and described. For example, device-based access requests that can be directed to a composite device include traditional device-based requests, such as requests to open the device, initialize the device, read from the device, write to the device, mount the device, unmount the device, and the like. Analogous files-based or directory-based requests can be directed to files and/or directories of the composite file system.

Turning back to the exemplary flow diagram <NUM>, if the access request is a file open request, then, at step <NUM>, the requested file can be read from the highest layer where the file exists. Thus, for example, if the file exists in the primary file system, that file can be provided in response to the request received at step <NUM>. Conversely, if the file does not exist in the primary file system, the secondary file system can be checked for the file, and if the file is located in the secondary file system, it can be provided from there in response to the request. According to one aspect, metadata, such as in the form of a flag or other like indicator, can be generated to identify the file system where the file was located, such that subsequent requests for the same file can be directed to the identified file system, with the other file systems being skipped. Such metadata can be cached in one or more tables, such as file tables implemented by the composite file system, or other analogous databases or data structures. Subsequently, the relevant processing can end at step <NUM>.

If the access request is a file write request, then, if the file is not already in the primary file system, a copy-on-write can be performed to copy the file from the secondary file system into the primary file system and then persist the changes being written to the file in the primary file system, thereby maintaining isolation between the container environment, having access to the primary file system, and the host computing environment, having access to the secondary file system, where the unchanged file can remain. Subsequently, the relevant processing can end at step <NUM>.

If the access request is a file deletion request, received from processes executing within the container environment, then a deletion marker can be placed in the primary file system, at step <NUM>. According to one aspect, when a file deletion marker is encountered at any layer, the composite file system returns an indication that the file is no longer available. Subsequently, the relevant processing can end at step <NUM>.

If the access request is a device-based request, such as a request to mount a volume, processing can proceed to step <NUM> and the composite device can send such a request to each layer's file system. In such a manner, the relevant volume can be mounted at each layer such that subsequent directory enumerations, or file access requests can encompass both an underlying base layer provided by the host computing environment, which is not changeable from the container environment, and a primary, or overlay, layer accessible from within the container environment and persisting changes made within the container environment. According to one aspect, if, at step <NUM>, it is determined that one or more layers do not have the requested volume, then metadata, such as in the form of a flag or other like indicator, can be generated to identify the layers where the volume is accessible, or, conversely, to identify the layers were the volume is not accessible. Subsequent performance of the step <NUM> can then send the request only to those layers at which the volume is accessible, based on existing, previously generated markings. Subsequently, the relevant processing can end at step <NUM>.

If the access request is a directory-based access request, such as a request to enumerate the files within a folder, or otherwise open a folder, processing can proceed with step <NUM>, at which point the request can be sent to each layer's file system. As before, if, at step <NUM>, it is determined that one or more layers do not have the requested folder, then metadata, such as in the form of a flag or other like indicator, can be generated to identify the layers that do not have such a folder, or, conversely, to identify the layers having such a folder. Subsequent iterations of step <NUM> can then direct the open folder request, for example, only to those layers that have the folder, based on the existing markings, such as from prior performance of step <NUM>. Once the files in the folder are enumerated at every layer, or, more specifically, every layer having such a folder, the presented listing of files can be in accordance with the priority of the layers. More specifically, and as detailed above, if the file exists in a higher layer, the same file from a lower layer is not shown. Thus, for example, if a modified copy of the file, modified from within a container environment, and therefore saved in the primary file system, namely the container file system, exists, then the unmodified file, from the host environment, they can be part of the secondary file system, can be not shown, and an enumeration of the files in the folder can include only the modified copy of the file from the primary file system. Such an aggregation can be performed at step <NUM>. Additionally, such as at step <NUM>, any file corresponding to a deletion marker at a higher layer can be indicated as no longer available, and, therefore, not presented as part of the enumeration of files in a folder. Deletion markers can also be utilized for folders, with a folder corresponding to a deletion marker at a higher layer causing the aggregation, at step <NUM>, to not present any files that are present in that folder at lower layers. Upon completion of steps <NUM> and <NUM>, the relevant processing can end at step <NUM>.

Turning to <FIG>, an exemplary computing device <NUM> is illustrated which can perform some or all of the mechanisms and actions described above. The exemplary computing device <NUM> can include, but is not limited to, one or more central processing units (CPUs) <NUM>, a system memory <NUM>, and a system bus <NUM> that couples various system components including the system memory to the processing unit <NUM>. The system bus <NUM> may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. The computing device <NUM> can optionally include graphics hardware, including, but not limited to, a graphics hardware interface <NUM> and a display device <NUM>, which can include display devices capable of receiving touch-based user input, such as a touch-sensitive, or multi-touch capable, display device. Depending on the specific physical implementation, one or more of the CPUs <NUM>, the system memory <NUM> and other components of the computing device <NUM> can be physically co-located, such as on a single chip. In such a case, some or all of the system bus <NUM> can be nothing more than silicon pathways within a single chip structure and its illustration in <FIG> can be nothing more than notational convenience for the purpose of illustration.

The computing device <NUM> also typically includes computer readable media, which can include any available media that can be accessed by computing device <NUM> and includes both volatile and nonvolatile media and removable and non-removable media. Computer storage media includes media implemented in any method or technology for storage of content such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired content and which can be accessed by the computing device <NUM>. Computer storage media, however, does not include communication media. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any content delivery media. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media.

A basic input/output system <NUM> (BIOS), containing the basic routines that help to transfer content between elements within computing device <NUM>, such as during start-up, is typically stored in ROM <NUM>. By way of example, and not limitation, <FIG> illustrates operating system <NUM>, other program modules <NUM>, and program data <NUM>.

The computing device <NUM> may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only, <FIG> illustrates a hard disk drive <NUM> that reads from or writes to non-removable, nonvolatile magnetic media. Other removable/non-removable, volatile/nonvolatile computer storage media that can be used with the exemplary computing device include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and other computer storage media as defined and delineated above. The hard disk drive <NUM> is typically connected to the system bus <NUM> through a non-volatile memory interface such as interface <NUM>.

The drives and their associated computer storage media discussed above and illustrated in <FIG>, provide storage of computer readable instructions, data structures, program modules and other data for the computing device <NUM>. In <FIG>, for example, hard disk drive <NUM> is illustrated as storing operating system <NUM>, other program modules <NUM>, and program data <NUM>. Note that these components can either be the same as or different from operating system <NUM>, other program modules <NUM> and program data <NUM>. Operating system <NUM>, other program modules <NUM> and program data <NUM> are given different numbers hereto illustrate that, at a minimum, they are different copies.

The computing device <NUM> may operate in a networked environment using logical connections to one or more remote computers. The computing device <NUM> is illustrated as being connected to the general network connection <NUM> (to the network <NUM>) through a network interface or adapter <NUM>, which is, in turn, connected to the system bus <NUM>. In a networked environment, program modules depicted relative to the computing device <NUM>, or portions or peripherals thereof, may be stored in the memory of one or more other computing devices that are communicatively coupled to the computing device <NUM> through the general network connection <NUM>. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between computing devices may be used.

Although described as a single physical device, the exemplary computing device <NUM> can be a virtual computing device, in which case the functionality of the above-described physical components, such as the CPU <NUM>, the system memory <NUM>, the network interface <NUM>, and other like components can be provided by computer-executable instructions. Such computer-executable instructions can execute on a single physical computing device, or can be distributed across multiple physical computing devices, including being distributed across multiple physical computing devices in a dynamic manner such that the specific, physical computing devices hosting such computer-executable instructions can dynamically change over time depending upon need and availability. In the situation where the exemplary computing device <NUM> is a virtualized device, the underlying physical computing devices hosting such a virtualized computing device can, themselves, comprise physical components analogous to those described above, and operating in a like manner. Furthermore, virtual computing devices can be utilized in multiple layers with one virtual computing device executing within the construct of another virtual computing device. The term "computing device", therefore, as utilized herein, means either a physical computing device or a virtualized computing environment, including a virtual computing device, within which computer-executable instructions can be executed in a manner consistent with their execution by a physical computing device. Similarly, terms referring to physical components of the computing device, as utilized herein, mean either those physical components or virtualizations thereof performing the same or equivalent functions.

Claim 1:
A method of booting an operating system in a container (<NUM>) providing a file system virtualization environment isolated from a host computing environment (<NUM>) hosting the container, the method comprising:
receiving a specification of a composite device (<NUM>) as a boot device, the composite device abstracting a first device (<NUM>) as a primary layer of the composite device and a second device (<NUM>) as a secondary layer of the composite device, wherein the first device (<NUM>) and the second device (<NUM>) of the composite device are accessed as the boot device in an hierarchical order such that the first device is accessed first, and the second device is accessed if information from the first device (<NUM>) is unavailable;
reading, in response to the receipt of the specification of the composite device as the boot device, operating system configuration data from a composite file system (<NUM>) associated with the composite device (<NUM>), the composite file system abstracting, as a primary layer of the composite file system, a first file system (<NUM>) providing access to data persisted on the first device (<NUM>) and, as a secondary layer of the composite file system, a second file system (<NUM>) providing access to data persisted on the second device; and
utilizing the operating system configuration data to boot the operating system in the container (<NUM>), wherein
the operating system configuration data is read from the first file system (<NUM>) if the operating system configuration data is found in the first file system; and
the operating system configuration data is read from the second file system based upon the operating system configuration data not being found in the first file system.