Isolating operating system in-memory modules using error injection

A continuously operating system is provided and includes a processor and a computer readable medium to which the processor is operatively coupled, the computer readable medium having executable instructions stored thereon which, when executed, cause the processor to continuously load an operating system and to simultaneously operate as at least first and second mechanisms. The first mechanism loads a new module, which is a new version of an in-memory module of the operating system, into the operating system, and the second mechanism isolates and interrupts current access to the in-memory module such that subsequent access is to the new module.

BACKGROUND

Aspects of the present invention are directed to a system and a method of testing operating system isolation using error injection.

Modern enterprise systems are employed by large entities, such as corporations and universities, small entities and individuals for computing services. In that way, the enterprise systems are formed of significant hardware resources and software, including an operating system, installed on the hardware. The hardware and software are then accessed and used by individuals on their own or within the entities for their computing needs.

Since computing services are so often integral to the proper functioning of individuals within the entities, uninterrupted availability of the enterprise systems that are being used by the entities is one of the key features in modern enterprise systems. Unfortunately, these systems still need to be brought down on occasion for maintenance. With that said, tools are available for mitigating the need for bringing down systems for maintenance purposes. These tools include dynamic software updates with which software used by the systems is updated without the systems on the whole having to be brought down although the software being updated must be restarted to incorporate any new changes.

For kernel modules that are system critical, however, it is understood that such kernel modules cannot be refreshed without causing systems failures. In these cases, dynamic software updates cannot be used.

SUMMARY

In accordance with an aspect of the invention, a continuously operating system is provided and includes a processor and a computer readable medium to which the processor is operatively coupled, the computer readable medium having executable instructions stored thereon which, when executed, cause the processor to continuously load an operating system and to simultaneously operate as at least first and second mechanisms. The first mechanism loads a new module, which is a new version of an in-memory module of the operating system, into the operating system, and the second mechanism isolates and interrupts current access to the in-memory module such that subsequent access is to the new module.

In accordance with another aspect of the invention, a computer readable medium is provided, to which a processor of a continuously operating system is operatively coupled, having executable instructions stored thereon which, when executed, cause the processor to continuously load an operating system and to simultaneously to operate as at least first and second mechanisms, wherein the first mechanism loads a new module, which is a new version of an in-memory module of the operating system, into the operating system, and the second mechanism isolates and interrupts current access to the in-memory module such that subsequent access is to the new module.

In accordance with yet another aspect of the invention, a method of managing a continuously operating system in accordance with executable instructions stored in a computer readable medium for execution by a processor of the system, which continuously loads an operating system, is provided and includes assigning a version number to a new module to be loaded into the operating system, determining, from the version number, whether the new module is a new version of an in-memory module, in an event the new module is a new version of the in-memory module, loading the new module at an unused location of system memory and updating a module table therein with an address of the unused location, and, in an event of current access to the in-memory module, interrupting the current access to initiate a retry, including a reading of the address from the module table and subsequent access to the new module at the address.

DETAILED DESCRIPTION

With reference toFIGS. 1 and 2, a continuously operating computing system10is provided. The system10may include one or more computing devices20that communicate with each other via connections with a network11. Where the system10includes a set of computing devices20, such as where the system10is used by a corporate entity, the computing devices20each operate in accordance with an enterprise class operating system (OS) which is widely distributed. In this case, in order to promote efficiency within the entity, the operating system is to be maintained with the least possible downtime without failure.

Each of the computing devices20includes multiple components and each of the multiple components has multiple functions. A system bus21is provided to allow each of the components to interact with others. A microprocessor22(i.e., a central processing unit) is provided to perform computing operations and calculations. Random Access Memory (RAM)23and Read-Only Memory (ROM)24, along with additional types of memory, act as computer readable media and provide storage space for the storage of information and instructions for use by the microprocessor22. A port adapter25, to which a data port26is coupled, is coupled to the system bus21and allows for the computing device20to communicate with the network11. A mass storage device28and a removable storage device29provide additional storage space for information and are coupled to the system bus21by way of an input/output (I/O) adapter27. User interface devices31and32, such as a mouse and a keyboard, allow a user of the computing device20to issue commands and are coupled to the system bus21by way of the user interface adapter30. Finally, a display device34, which is coupled to the system bus21by way of the display adapter33, allows for the display of information to the user.

In accordance with embodiments of the invention, the microprocessor22and the RAM23and the ROM24, along with the other components described above, are associated with the computing device20and the system10as a whole. In that way, the microprocessor22and the RAM23and the ROM24are operatively coupled to one another as described above with the computer readable media having executable instructions stored thereon. These executable instructions, when executed, cause the microprocessor22to continuously load the operating system by continuously executing the operating system instructions and to simultaneously operate as at least first and second mechanisms. The first mechanism, which may be regarded as a compiler, serves to translate code of a source program, which is usually written in a programming language, into machine language code of a new module, which, in some cases, is a new version of an in-memory kernel module of the operating system. The second mechanism, which may be regarded as a loader, loads executable instructions into memory and isolates and interrupts current access to the in-memory module such that subsequent access is to the new module. Here, although the first and second mechanisms have been described as relating to a compiler and a loader, respectively, it is understood that this convention is merely exemplary and that functions normally associated with compilers may be undertaken by the second mechanism and functions normally associated with loaders may be undertaken by the first mechanism.

Since the microprocessor22continuously executes the operating system instructions while the new module is loaded, while the previously loaded version of the same module is isolated and while the current and future access to the module is redirected, the need for shutting down the computing device20and/or the system10, as a whole, is mitigated. Thus, downtime for the system10is decreased even though maintenance may be performed on the kernel modules of the operating system with the operating system being otherwise available for use on an uninterrupted basis.

The new and in-memory modules refer to kernel modules of the operating system. A kernel module, in this context, may refer to a single executable object which encapsulates some logical unit of kernel functionality. That is, as an example, multiple drivers for software or some other logical grouping of functionality might be separate individual source files. These source files are compiled by being transformed from high level source code into single logical binary units. Each logical binary unit is a kernel module. Typical kernel modules have names that end in an extension like .ko, such as kernel objects in Linux, though other operating systems may not have specialized extensions, yet implement similar concepts under differing names.

The operating system is configured to recognize when the new module is to be loaded (see operation200ofFIG. 2). The compiler compiles the source code of the new module during a compilation time thereof and assigns a version number to the new module.

Upon new module compilation completion, the loader determines whether the new module is a version of the in-memory module, which is already loaded into the memory. This operation is shown schematically in operation210ofFIG. 2in which the question of whether the module (i.e., the new module) has a non-zero entry in the module table is considered. The module table refers to data that describes a location in system memory at which a module is located. Thus, if the new module is a version of the in-memory module, the loader will recognize that the module table indicates that a module that is substantially similar to the new module is already stored at an address within the system memory. In many cases, each module typically has a name associated with it. The loader can compare the name of the new module and that of the in-memory module to decide if the new module is a version of the in-memory module. The loader will then proceed to operation220. On the other hand, if no instance of the module already exists in the memory, the loader loads the new module to an unused location of the memory (operation230).

At operation220, the loader determines whether the new module is or is not a new version of the preloaded module by comparing the version number of the new module with that of the in-memory module. Here, it will generally be the case that the newer module will have the greater version number, as shown inFIG. 2, although this is not required and embodiments exist in which this is not necessarily the case. In any event, if the comparison reveals that the in-memory module is the newer version, the loading of the new module is refused (operation240) and the module load request is regarded as complete (operation280). Conversely, if the new module is the newer version, the new module is loaded into the memory in accordance with operation230at the unused location.

Once the new module is loaded, the loader updates the module table with the address of the memory at which the new module is loaded (operation240).

The loader then determines whether the in-memory module is currently being accessed by a program of the operating system. This determination is accomplished by the operating system, which marks module entry and exit points during normal operations as shown inFIG. 2in which it is determined whether the in-memory module has a reference count which is greater than zero. If the in-memory module is not being currently accessed, the module load request is regarded as being complete (operation280). In this case, subsequent access by a program to the module is made to the new module following a read of the address of the new module in the module table. If, on the other hand, the in-memory module is being currently accessed by a program, control proceeds to operation260.

At operation260, the in-memory module will be found to have current execution threads and, at this point, the loader will artificially inject an error into the running or execution of the in-memory module. This error injection will initiate retry mechanisms on the module which involve a reading of the address of the new module from the module table such that subsequent access by the accessing program is only made to the new module. Subsequently, the reference count for the module (i.e., the new module) is reset (operation270). In this way, the in-memory module may be regarded as being isolated from the operating system programs such that future access to the in-memory module by these programs is prevented.

In a further embodiment, it is possible for the module into which the error is injected to percolate the recovery process to its caller. The caller's recovery routine can then initiate a retry, which will call the module again using the new address.

In accordance with further embodiments of the invention, the in-memory module, having been isolated, may also be invalidated. Invalidation may include marking memory pages of the per-loaded module as being invalid and/or turning memory pages of a non-executable (NX) bit of the in-memory module on. Additionally or alternately, the in-memory module may be deleted entirely from the module table, the memory or both. At this point the module table should already have no reference to the old module (the entry should now contain the address for the new module). So the old module can't really be deleted from the module table.

This is how application binary isolation (ABI) can be tested. At this point, any attempt to access the module in a non-standard way (e.g. using a hard coded address rather than reading from the module table) would trigger a segment fault since the pages are now invalid.

In accordance with another aspect of the invention, a computer readable medium is provided, to which a microprocessor22of a continuously operating system10is operatively coupled, having executable instructions stored thereon which, when executed, cause the microprocessor22to continuously load an operating system and to simultaneously to operate as at least first and second mechanisms. As above, the first mechanism loads a new module, which is a new version of an in-memory module of the operating system, into the operating system, and the second mechanism isolates and interrupts current access to the in-memory module such that subsequent access is to the new module.

In accordance with another aspect of the invention, a method of managing a continuously operating system10in accordance with executable instructions stored in a computer readable medium for execution by a microprocessor22of the system10, which continuously loads an operating system, is provided. The method includes assigning a version number to a new module to be loaded into the operating system during compilation of the new module, and determining, from the version number, whether the new module is a new version of an in-memory module. In an event the new module is a new version of the in-memory module, the method further includes loading the new module at an unused location of system memory and updating a module table therein with an address of the unused location. Also, in an event of current access to the in-memory module, the method still further includes interrupting the current access to initiate a retry, including a reading of the address from the module table and subsequent access to the new module at the address.