Patent Description:
In a virtual computing system, a hypervisor may be configured to virtualize the interrupts. For example, the hypervisor may provide a virtual interrupt to a virtual processor. Once the virtual interrupt is delivered to the virtual processor, the virtual processor may execute the virtual interrupt.

It is with respect to these and other general considerations that examples have been described. Also, although relatively specific problems have been discussed, it should be understood that the examples should not be limited to solving the specific problems identified in the background.

<CIT> discloses processing virtual interrupts in a logically partitioned system. An intelligent virtual global interrupt queue (virtual GIQ) that may be associated with a plurality of virtual processors running in a logical partition may be utilized. Upon receiving a virtual interrupt, the virtual GIQ may examine the operating states of the associated virtual processors. In an effort to ensure the virtual interrupt is processed as quickly as possible.

<CIT> discloses binding interrupts to central processing units (CPUs). An interrupt controller receives an interrupt that is generated by a device coupled to the computer system. The interrupt controller identifies a preferred CPU associated with the device based on a predetermined binding.

<CIT> discloses an interrupt controller for controlling the routing and handling of interrupts received at a data processing apparatus.

This disclosure generally relates to enabling a hypervisor to provide one or more virtual interrupts to one or more virtual processors. More specifically, the hypervisor may be configured to provide the interrupt to a particular virtual processor in a group of processors once the particular virtual processor becomes available. For example, the hypervisor context may be used when any one of a specified set of virtual processor interrupt priorities drops below a hypervisor -specified priority.

In another example, the hypervisor may be configured to provide a particular virtual interrupt to the hardware of a host machine on which the hypervisor is executing. Once the hardware has the virtual interrupt, the hardware may provide the virtual interrupt to a particular virtual processor even if the virtual processor is not currently executing. However, when the virtual processor enters an operational state, the virtual interrupt may be immediately handled by the particular virtual processor.

Accordingly, described herein is a method that includes, among other features, monitoring, by a hypervisor of a host machine, a workload of a plurality of virtual processors. When an interrupt is received, the hypervisor determines a first virtual processor that can execute the interrupt. In some instances, this determination is based, at least in part, on the workload of each of the plurality of virtual processors and on a time frame that the first virtual processor can execute the interrupt. Once the first virtual processor is identified, the interrupt is provided to the first virtual processor but not to the other virtual processors in the plurality of virtual processors.

Also described is a method that includes determining a current priority of at least one virtual processor of the plurality of virtual processors. When an interrupt is received, a priority of the interrupt is determined. When it is determined that the priority of the interrupt is higher than the priority of the at least one virtual processor, the interrupt is provided to the at least one virtual processor. However, when it is determined that the priority of the interrupt is not higher than the priority of the at least one virtual processor, a determination is made as to whether one or more other virtual processors of the plurality of virtual processors has a lower priority. The interrupt is then delivered to one of the one or more other virtual processors of the plurality of virtual processors that has a lower priority.

The present application also describes a host machine that includes at least one processor and a memory coupled to the at least one processor. The memory stores instructions that, when executed by the at least one processor, perform a method that includes, among other features, causing a hypervisor to monitor a workload of each virtual processor in a plurality of virtual processors. When an interrupt is received, the hypervisor determines which virtual processor of the plurality of virtual processors is to receive the interrupt. In some cases, the determination is based, at least in part, on the workload of each virtual processor of the plurality of virtual processors. The interrupt may then be provided to the determined virtual processor.

Non-limiting and non-exhaustive examples are described with reference to the following Figures.

In the following detailed description, references are made to the accompanying drawings that form a part hereof, and in which are shown by way of illustrations specific embodiments or examples. These aspects may be combined, other aspects may be utilized, and structural changes may be made without departing from the present disclosure. Examples may be practiced as methods, systems or devices. Accordingly, examples may take the form of a hardware implementation, an entirely software implementation, or an implementation combining software and hardware aspects. The following detailed description is therefore not to be taken in a limiting sense.

According to the invention, a hypervisor handles or otherwise provide various interrupts that are received to various virtual processors. An interrupt is delivered to a virtual processor in order to signal to the virtual processor that work is to be done. An interrupt is associated with a priority. As such, interrupts with a higher priority are handled by a virtual processor before interrupts with a lower priority.

In a system in which multiple virtual processors have been created and/or are executing, a hypervisor of the system may be able to queue various interrupts for each virtual processor. They hypervisor and/or the hardware associated with the hypervisor is also aware of the current priority of interrupts each virtual processor is executing. As such, interrupts are assigned to a virtual processor based, at least in part, on the priority of the interrupt that is to be queued and/or the current priority of interrupts the virtual processor is currently handling and/or are queued for the virtual processor.

These and other examples will be explained in more detail below with respect to <FIG>.

<FIG> illustrates an example host machine <NUM> on which a hypervisor <NUM> may be securely launched according to an example. In some implementations, the host machine <NUM> may be any computing device capable of launching one or more virtual machines, such as, for example, virtual machine <NUM>. The host machine <NUM> may be a desktop computer, a laptop computer, a mobile computing device, a tablet computing device, a wearable computing device, a gaming device and so on.

As shown in <FIG>, the host machine <NUM> may include hardware <NUM>. The hardware <NUM> may include one or more processors, one or more storage devices, one or more memory devices and so on.

In the example shown in <FIG>, the host machine <NUM> also includes a hypervisor <NUM>. In some cases, the hypervisor <NUM> may be software, hardware, firmware or a combination thereof. As will be explained in more detail below, the hypervisor <NUM> is configured to create, run and/or manage one or more virtual machines <NUM>.

In the example shown in <FIG>, the hypervisor <NUM> is configured to communicate directly with the hardware <NUM> of the host machine <NUM>. In such cases, the hypervisor <NUM> may be viewed as having the highest privilege level among the various other software, firmware and/or other hardware components of the host machine <NUM>. Thus, for example, when the host machine <NUM> boots up, the hypervisor <NUM> may be the first item or component that is created, instantiated or otherwise executed on the host machine <NUM>.

Once the hypervisor <NUM> is initialized, it may create one or more virtual machines <NUM>. Each virtual machine <NUM> may emulate a computer system and, as such, may provide the functionality of a physical computing device. In some examples, the virtual machine <NUM> may include a privileged kernel <NUM> and a normal kernel <NUM>.

The privileged kernel <NUM> may be configured to execute a secure operating system. As such, the privileged kernel <NUM> can run one or more secure programs that contain various secrets utilized by the virtual machine <NUM>, the hypervisor <NUM>, and/or the normal kernel <NUM>. For example, the privileged kernel <NUM> may store various credentials, encryption keys and the like.

The normal kernel <NUM> may be configured to execute various "normal" programs and applications, such as, for example, word processing applications, browser applications, spreadsheet applications and so on. However, due to the less secure security configuration (e.g., when compared to the security configuration of the privileged kernel <NUM>) of the normal kernel <NUM>, the normal kernel <NUM> may not store any credentials, encryption keys, or other secrets that may be utilized by the virtual machine <NUM> and/or the hypervisor <NUM>. As such, when various secrets are needed by the various applications running on the normal kernel <NUM>, the normal kernel <NUM> may request those secrets from the privileged kernel <NUM>. In another implementation, the normal kernel <NUM> may request that the privileged kernel <NUM> perform one or more actions, using one or more of the stored secrets, on behalf of the normal kernel <NUM> and/or one or more applications executing on the normal kernel.

In some instances and due to the hypervisor <NUM> allowing the virtual machine to execute both the privileged kernel <NUM> and the normal kernel <NUM>, the hypervisor <NUM> may execute, or may cause the virtual machine <NUM> to execute, in a privileged context. The privileged context enables the hypervisor <NUM> to switch between the privileged kernel <NUM> and the normal kernel <NUM> and/or various user modes.

As the hypervisor <NUM> is responsible for various virtual machines <NUM> and each virtual machine's respective kernels, it is important that the hypervisor <NUM> be one of the most, if not the most secure component on the host machine <NUM>. For example, if the hypervisor <NUM> is software, the hypervisor <NUM> may have the highest privilege level when compared to other software that may be executing on the host machine <NUM>. In some cases, the hardware <NUM> provides the hypervisor <NUM> with privilege level architecture that enables the hypervisor <NUM> to run and to exert authority over every virtual machine <NUM> the hypervisor <NUM> creates. As will be explained in more detail below with respect to <FIG>, the host machine <NUM> may include nested hypervisors. In such cases, the primary hypervisor may have authority over the secondary hypervisor.

<FIG> illustrates another example host machine <NUM> on which a software layer <NUM> exists between the hypervisor <NUM> and the hardware <NUM> of the host machine <NUM> according to an example. In this example, the hardware <NUM>, the hypervisor <NUM> and virtual machine <NUM>, the normal kernel <NUM> and the privileged kernel <NUM> may function in a similar manner such as was described above with respect to <FIG>. However, in this example, the host machine <NUM> includes a software layer <NUM> positioned between the hypervisor <NUM> and the hardware <NUM>.

In some cases, the software layer <NUM> may be responsible for certain aspects of the hardware <NUM>. For example, the software layer <NUM> may be responsible for putting the host machine <NUM> in a sleep state, resuming programs or applications when the host machine <NUM> awakens from a sleep state and so on.

It is also possible that the software layer <NUM> has a higher privilege level than the hypervisor <NUM>. In such cases, the hypervisor <NUM> should be configured to communicate directly with the software layer <NUM>. That is, any communication between the software layer <NUM> and any of the other components (e.g., the privileged kernel <NUM>, the normal kernel <NUM> etc.) of the host machine <NUM> should be routed through or otherwise mediated by the hypervisor <NUM>. For example, any communication that occurs between the normal kernel <NUM> and the software layer <NUM> should be handled by the hypervisor <NUM>. However, it is also possible that certain communication channels could be allowed directly between lower privilege software and the software layer <NUM> without each individual message having to go through the hypervisor <NUM>.

In some cases when the software layer <NUM> is present, it may be desirable for the hypervisor <NUM> to be able to turn off or deactivate the software layer <NUM>. For example, once the hypervisor <NUM> has been initialized, the hypervisor <NUM> may be configured to turn off the software layer <NUM>, suspend operations performed by the software layer <NUM>, intercept commands provided by or sent to the software layer <NUM> and so on. In this way, the hypervisor <NUM> may have the highest privilege level within the host machine <NUM>. As such, security features of the host machine <NUM> may be improved as the hypervisor <NUM> controls communications between the various components of the host machine <NUM>. As will also be described below, the host machine <NUM> may be able to determine that the hypervisor <NUM> was securely launched thereby preventing any attacks that may be brought to the host machine <NUM>.

<FIG> illustrates an example host machine <NUM> having nested hypervisors that support nested virtualization according to an example. As shown in <FIG>, the host machine <NUM> may include hardware <NUM> and a hypervisor <NUM>. In some cases, the hardware <NUM> and the hypervisor <NUM> may function in a similar manner such as described above. For example, the hypervisor <NUM> may communicate with the hardware <NUM> as well as with a normal kernel <NUM> and a privileged kernel <NUM> of a virtual machine <NUM>.

Additionally, the hypervisor <NUM>, and/or the hardware <NUM>, may be able to create, run, and/or command another hypervisor (shown in <FIG> as hypervisor <NUM><NUM>) and another virtual machine (shown in <FIG> as virtual machine <NUM><NUM>). As with the virtual machine <NUM>, the virtual machine <NUM><NUM> may include a privileged kernel (shown in <FIG> as privileged kernel <NUM><NUM>) and a normal kernel (shown in <FIG> as normal kernel <NUM><NUM>). Each of these kernels may function in a similar manner to the normal kernel <NUM> and the privileged kernel <NUM> described above.

In some instances, each virtual machine may have one or more virtual processors. Further, each virtual machine may be associated with an identifier that enables a hypervisor <NUM> (or hypervisor <NUM><NUM>) to target a specific virtual machine and its associated processors. The hypervisor <NUM> may also be configured to target a specific virtual processor associated with a virtual machine or a set of virtual processors associated with one or more virtual machines. This information may also be provided to the hardware <NUM>. As such, a specific virtual processor may be targeted without enumerating through each virtual machine in order to find specific virtual processors.

The hypervisor <NUM><NUM> may communicate with and run the privileged kernel <NUM><NUM> and the normal kernel <NUM><NUM> in a similar manner as described above. For example, the hypervisor <NUM><NUM> of the virtual machine <NUM><NUM> may run in a privileged context, which enables the hypervisor <NUM><NUM> to switch between the privileged kernel <NUM><NUM> and the normal kernel <NUM><NUM>.

The hypervisor <NUM><NUM> may believe that it is the only hypervisor in the host machine <NUM>. However, the hypervisor <NUM><NUM> may be subject to and commanded by the hypervisor <NUM>. That is, any communications between the hypervisor <NUM><NUM> and the hardware <NUM> may be passed through the hypervisor <NUM>.

Although not shown in <FIG>, the host machine <NUM> may also include a software layer, such as, for example, software layer <NUM> (<FIG>). When the software layer is present, the hypervisor <NUM><NUM> should only be configured to communicate the hypervisor <NUM>. In some cases, the hypervisor <NUM><NUM> will not be launched until a verification is received that the hypervisor <NUM> has been launched securely.

Regardless of the configuration of the host machine, it is imperative that the hypervisor be launched securely. The options to securely launch the hypervisor may differ depending on the configuration of the host machine. In some implementations, the options described below may be performed separately. In other implementations, the options described below are mutually exclusive. In yet other implementations, the options described below may be performed sequentially, simultaneously or substantially simultaneously.

The first option to ensure that the hypervisor is securely launched is to ensure that that the hardware (e.g., hardware <NUM>) launches the hypervisor <NUM> once the host machine <NUM> boots. For example, the hardware <NUM> may have knowledge of where the hypervisor <NUM> binary is located and may be configured to immediately cause the hypervisor <NUM> to execute or establish a privilege level for the hypervisor upon booting up. Stated another way, the hypervisor <NUM>, or a secure hypervisor loader associated with the hardware <NUM>, can be authenticated and start executing before any non-secure code is executed. In some cases, the non-secure code may be part of the software layer <NUM> (<FIG>).

A second option may be to include or otherwise provide access to a special boot loader. In some cases, the special boot loader may be able to leverage a specialized secure launch mechanism (e.g., an instruction or command) that causes the hardware <NUM> to launch the hypervisor <NUM> and ensure the hypervisor <NUM> is securely executed. In some cases, the second option may be used when the software layer <NUM> is present in the host machine and/or when a unified extensible firmware interface (UEFI) (or a basic input/output system (BIOS)) is executed prior to the hypervisor being launched.

In some cases, and regardless of which option above is used to launch the hypervisor <NUM>, the hardware <NUM> may validate that the hypervisor <NUM> is in a secure state. If not, the hardware <NUM> may be configured to place the hypervisor <NUM> in the secure state. Once the hypervisor <NUM> is in the secure state, the hypervisor <NUM> may begin creating one or more virtual machines <NUM>.

As discussed above, the hypervisor <NUM> may be configured to provide (via software and hardware architectural mechanisms) various different privilege levels. For example, the hypervisor <NUM> may allow the virtual machine <NUM> to execute in a "privileged" level and a "normal" level or "less privileged" level. Although two specific levels are mentioned, the hypervisor <NUM> may allow one or more virtual machines to execute in various different privilege levels. Because of this configuration, the hypervisor <NUM> may be able to switch between the privileged kernel <NUM> and the normal kernel <NUM>.

When the hypervisor <NUM> is in the privileged level, various platform details associated with the host machine <NUM> may be obtained by the hypervisor <NUM>. In some cases, the platform details may be conveyed to the hypervisor <NUM> using one or more Advanced Configuration and Power Interface (ACPI) tables. In other cases, the hypervisor <NUM> may be instructed to search or otherwise obtain these platform details from various other software or hardware components associated with the host machine <NUM>.

For example, in some cases, platform details may be hard-coded or discovered via a non-architectural interface. In this example, a highly privileged software module executing on host machine may be responsible for boot-strapping the system and providing these details.

In some cases, the details may include a location of one or more IOMMU that the hypervisor <NUM> may use to protect itself from direct memory access (DMA) attacks, how to zero some or all of the memory (e.g., on shutdown or reboot), how to power the host machine <NUM> down, how to reset the host machine <NUM>, what the memory maps look like (e.g., what ranges include the MMIO, RAM, persistent memory, etc.), how to start additional processors, and so on.

<FIG> illustrates a host machine <NUM> having a logical processor <NUM> and multiple virtual processors (e.g., virtual processor <NUM><NUM>, virtual processor <NUM><NUM> and virtual processor <NUM><NUM>) according to an example. Host machine <NUM> may be similar to host machine <NUM> (<FIG>), host machine <NUM> (<FIG>) and/or host machine <NUM> (<FIG>).

As discussed above, the host machine <NUM> may include hardware <NUM> and a hypervisor <NUM>. In some instances, the hypervisor <NUM> may be securely launched such as described above. The host machine <NUM> also includes at least one processor <NUM>. The processor <NUM> controls or otherwise is associated with one or more virtual processors such as, for example, virtual processor <NUM><NUM>, virtual processor <NUM><NUM> and virtual processor <NUM><NUM>. Each of these virtual processors may be associated with a single virtual machine. In other examples, each of these virtual processors may be associated with different virtual machines. For example, virtual processor <NUM><NUM> and virtual processor <NUM><NUM> may be associated with a first virtual machine (e.g., virtual machine <NUM> (<FIG>)) and virtual processor <NUM><NUM> may be associated with a second virtual machine (e.g., virtual machine <NUM><NUM> (<FIG>)).

In some cases, the processor <NUM> is configured, by the hypervisor <NUM> to divide processing time between each virtual processor. That is, when one processor is active and executing, the other virtual processors may be inactive. For example, the processor <NUM> may cause virtual processor <NUM><NUM> to execute for a first amount of time. When that first amount of time ends, the processor <NUM> may cause virtual processor <NUM><NUM> to execute for a second amount of time (that may be the same as, or different than, the first amount of time). When the second amount of time ends, the processor <NUM> may cause virtual processor <NUM><NUM> to execute for a third amount of time (that may be the same as, or different than, the first amount of time and the second amount of time). In some cases, the processor <NUM> may split time between each virtual processor based on instructions from the hypervisor <NUM>.

In addition, and as shown, each virtual processor is also in communication with the hypervisor <NUM>. As will be explained in more detail below, the hypervisor <NUM> is configured to provide one or more interrupts <NUM> to each virtual processor and/or to the processor <NUM>. In another implementation, the hardware <NUM> may be configured to provide the interrupts <NUM> to one or more of the virtual processors and/or the processor <NUM>.

The hypervisor <NUM> may also be configured to virtualize various interrupts that are received from various software components executing on the host machine (and/or one or more software components executing on one or more virtual machines such as, for example, virtual machine <NUM> (<FIG>)) and/or interrupts that are received from the hardware <NUM>.

Typically, each interrupt <NUM> is associated with a priority. In some instances, when a processor, such as virtual processor <NUM><NUM> is executing an interrupt with a particular priority, that virtual processor is viewed as having that particular priority. In some instances, each virtual processor may have or otherwise be associated with different priorities. Accordingly, the hardware <NUM> and/or the hypervisor <NUM> is configured to deliver interrupts with various priorities to each different virtual processor based, at least in part, on its current priority.

For example, if an interrupt with a priority of one is queued for virtual processor <NUM><NUM>, and an interrupt with a priority of two is queued for virtual processor <NUM><NUM> and a new interrupt with a priority of one is received, the new interrupt with the priority of one is delivered to virtual processor <NUM><NUM> and it would take priority over the virtual interrupt with the priority of two and would be executed first. It is also possible that virtual processor <NUM><NUM> is idle. In such a scenario, the hypervisor <NUM> causes the new interrupt to be sent to virtual processor <NUM><NUM>.

In other examples, various interrupts having different priorities may be queued for different virtual processors. For example, virtual processor <NUM><NUM> may have a number of interrupts in its queue and two or more of the interrupts may have different priorities. Likewise, virtual processor <NUM><NUM> and virtual processor <NUM><NUM> may also have any number of different interrupts in a queue with two or more having different priorities.

In such cases, the hardware <NUM> is configured to monitor the priority of each virtual processor and inject or otherwise provide an interrupt with a corresponding priority to the virtual processor. In some example, the hardware <NUM> or the hypervisor <NUM> may be configured to query each virtual processor for its current priority so that the interrupts can be delivered accordingly.

For example, when a processor, such as virtual processor <NUM>, has its priority drop to or below a priority associated with the interrupt <NUM>, the hypervisor <NUM> or the hardware <NUM> provides the interrupt <NUM> to that virtual processor.

In some cases, the hardware <NUM> and/or the hypervisor <NUM> is also configured to track the load of each virtual processor. Thus, if one virtual processor is less busy than another virtual processor, more interrupts are provided to the less busy virtual processor. The hardware <NUM> delivers lower priority interrupts to the various virtual processors based on their current load.

In other cases, the hypervisor <NUM> may be configured to monitor the current load of each virtual processor. When a virtual processor is available (e.g., not executing an interrupt or its priority drops to or below a priority of a particular interrupt <NUM>) the hypervisor pends (e.g., virtually deliver) a particular interrupt <NUM> to that virtual processor. The hardware <NUM> or the hypervisor <NUM> is configured to deliver a lower priority interrupt to a particular virtual processor upon determining that the virtual processor has finished its higher priority work. The virtual processor may be configured to notify the hypervisor <NUM> that it has finished its work and is ready for additional work.

This is in contrast to current configurations in which a hypervisor may send a request to each virtual processor in a group of virtual processors. In that case, the first virtual processor that becomes available may execute the interrupt. However, the other virtual processors that were also notified of the interrupt, when available, may still request the interrupt from the hypervisor and/or notify the hypervisor that they are now available to handle the interrupt - even after the interrupt has already been handled by another virtual processor. This wastes valuable processing time.

However, the current implementation solves the above problem. As described above, the hypervisor <NUM> may hold one or more interrupts <NUM> and monitor virtual processor <NUM><NUM>, virtual processor <NUM><NUM> and virtual processor <NUM><NUM>. Once one of the virtual processors becomes available (e.g., their priority drops below the priority of the interrupt) the newly available virtual processor may notify the hypervisor <NUM> of its status (or priority). Once the notification is received and there are interrupts to execute, the hypervisor <NUM> provides the interrupt <NUM> to the newly available virtual processor without notifying the other virtual processors.

In some examples, the hypervisor <NUM> may be configured to order the pending interrupts <NUM> in a particular order. In some examples, the interrupts <NUM> may be ordered based on the amount of time the interrupt has been pending, the priority of the interrupt, a schedule of when the virtual processors will be active and so on.

Continuing with the example above, the hypervisor <NUM> receives an interrupt <NUM>. The hypervisor <NUM> monitors the status of each virtual processor and determine that virtual processor <NUM><NUM> is currently available or, based on the current load of the virtual processors, that virtual processor <NUM> will be available first. As such, the hypervisor <NUM> provides the interrupt <NUM> to virtual processor <NUM><NUM> as soon as it becomes available (e.g., a priority of virtual processor <NUM><NUM> drops to or below a priority associated with the interrupt <NUM>).

In some cases, an interrupt <NUM> may specify that a particular virtual processor is to handle the interrupt. In such cases, the hypervisor <NUM> may be required to provide the interrupt <NUM> to the specified virtual processor. However, when the virtual processor is not specified, the hypervisor <NUM> may select any one of the virtual processors to handle the interrupt <NUM>.

In some examples, the hypervisor <NUM> may be configured to provide the interrupt to a virtual processor regardless of which virtual machine (or logical processor) the virtual processor is associated with. For example, if virtual machine <NUM> (<FIG>) and virtual machine <NUM><NUM> (<FIG>) each have virtual processors that are receiving interrupts, the hypervisor <NUM> may be configured to deliver interrupts to any of the virtual processors that are present on the host machine <NUM>.

As describe above, the processor <NUM> may cause each virtual processor to be active at different times. As such, the hardware <NUM> may be configured to pend an interrupt <NUM> to a virtual processor even when the virtual processor is not currently executing. For example, if the hypervisor <NUM> determines that virtual processor <NUM><NUM> should receive a particular interrupt <NUM>, but that interrupt is not currently executing, the hypervisor <NUM> may provide the interrupt <NUM> to the hardware <NUM>.

When the hardware <NUM> receives the interrupt, the hypervisor <NUM> considers the interrupt <NUM> delivered. As such, the hypervisor <NUM> can continue to deliver other interrupts <NUM> as needed and/or perform other tasks. When the hardware <NUM> receives the interrupt <NUM>, the hardware <NUM> may be configured to deliver or pend the interrupt <NUM> to the non-running virtual processor (e.g., virtual processor <NUM><NUM> in this example). However, the next time the virtual processor <NUM><NUM> is active, the interrupt <NUM> has already been delivered to (or is marked for delivery to) the virtual processor and may be executed.

In some cases, the hardware <NUM> doesn't need to track or otherwise monitor where the particular virtual processor is running (e.g., which logical processor is controlling the virtual processors). Instead, the hypervisor <NUM> may provide a data structure or other such memory device that specifies where the interrupt <NUM> should be delivered. The hardware <NUM> may have access to this data structure when it is determining where to deliver the interrupt <NUM>.

The hypervisor <NUM> and/or the hardware <NUM> may also be configured to block various interrupts from being delivered to the virtual processors even if the interrupt has a higher priority than the virtual processor. In some example, the interrupts may be blocked because the virtual processor is executing critical activities. Once the critical activities are complete, the virtual processor may resume receiving interrupts.

<FIG> illustrates a method <NUM> for delivering a virtual interrupt to a virtual processor according to an example. The method <NUM> may be used by a virtual processor associated with a host machine such as described above.

Method <NUM> begins at operation <NUM> in which an interrupt is received by a hypervisor. According to the invention, the interrupt is associated with a priority. It is also contemplated that the hypervisor may receive multiple interrupts with each interrupt having its own priority. In such cases, the hypervisor may be configured to order the interrupts based on the priority (or based on some other factor such as, for example, length of time the interrupt has been pending).

Once the interrupt has been received, flow proceeds to operation <NUM> and the hypervisor monitors the status of each virtual processor. The hypervisor monitors the status of each virtual processor to determine which virtual processor should receive the interrupt. According to the invention, the monitoring includes determining a workload of each virtual processor, a priority of each virtual processor, whether the virtual processor is executing critical tasks, whether the virtual processor is idle and so on.

Although method <NUM> shows operation <NUM> as being sequential with respect to operation <NUM>, the hypervisor may be continuously monitoring the status of each virtual processor. In other cases, the virtual processor may be configured to report its status to the hypervisor and/or hardware associated with the hypervisor.

In some cases, a host machine may have multiple virtual machines and multiple hypervisors. In such cases, the main hypervisor may monitor each virtual processor (either by itself or with the help of child hypervisors that are associated with a nested virtual machine). Additionally, each child hypervisor may be configured to monitor its own virtual processors and/or any virtual processors associated with another nested virtual machine (e.g., a second nested virtual machine). Additionally, each virtual processor may be configured to report its status to one or more of the hypervisors (e.g., parent hypervisor and/or child hypervisor) associated with the host machine and/or the guest machine.

Flow then proceeds to operation <NUM> and the interrupt is delivered to a selected virtual processor or a set of virtual processors. In some instances, the hypervisor delivers the interrupt to the first available virtual processor or the virtual processor that will be able to execute the interrupt first regardless of the current workload of the virtual processor and/or the status of virtual processor. For example, the interrupt may be delivered to the virtual processor whose priority is lower than the priority of the interrupt. In some additional examples, the interrupt may also be delivered to a set of virtual processors.

Method <NUM> begins at operation <NUM> in which an interrupt is received by a hypervisor. According to the invention, the interrupt is associated with a priority. It is also contemplated that the hypervisor may receive multiple interrupts and with each interrupt having its own priority. In such cases, the hypervisor may be configured to order the interrupts based on the priority (or based on some other factor such as, for example, length of time the interrupt has been pending).

Once the interrupt has been received, flow proceeds to operation <NUM> and the hypervisor monitors the status of each virtual processor to determine which virtual processor should receive the interrupt. Similar to what was described above, although method <NUM> shows operation <NUM> as being sequential with respect to operation <NUM>, the hypervisor may be continuously monitoring the status of each virtual processor. In some cases, a host machine may have multiple virtual machines and multiple hypervisors. In such cases, the main hypervisor may monitor each virtual processor (either by itself or with the help of child hypervisors that are associated with a nested virtual machine). Additionally, each child hypervisor may be configured to monitory its own virtual processors and/or any virtual processors associated with another nested virtual machine (e.g., a second nested virtual machine).

Once the interrupt is received, flow may proceed to operation <NUM> in which the interrupt is delivered to the hardware of the host machine. As part of the delivery, the hardware may also receive the priority of the interrupt as well as an identification as to which virtual processor the interrupt is to be delivered to. The hardware may then store the interrupt as well as the associated information.

In some instances, the hypervisor may be configured to provide or otherwise store the interrupt in a data structure that the hardware has access to. The hypervisor may then notify the hardware of the pending interrupt and where it is located in the data structure. In such cases, the hardware may not need to track which processor is in charge of each virtual processor or where each virtual processor is located.

For example, the hardware includes a virtual interrupt controller that assists the hardware in storing and/or delivering received interrupts. For example, the virtual interrupt controller may be configured to notify the hardware or otherwise store an identifier of the virtual processor the virtual interrupt controller is associated with. In some cases, an interrupt also has an associated identifier that identifies a particular virtual interrupt controller and/or a particular virtual processor. Once the identifier of the interrupt is determined, the hardware or the virtual interrupt controller delivers the interrupt to the identified virtual processor (e.g., the virtual processor having a corresponding identifier). In some cases, the interrupt may be delivered to the virtual processor regardless of where the virtual processor is executing and regardless of which virtual machine (or host machine) the virtual processor is currently associated with. In some cases, the hardware may have access to another data structure that stores information about each logical processor in the system as well as each virtual processor associated with each logical processor. For example, the data structure may be a bit map in which one axis has an entry for each virtual processor and the other axis has an entry for each logical processor. A bit in the bit map is set for each virtual processor that is associated with each logical processor. That is, when a particular virtual processor is running on a particular logical processor, the intersection in the bit map between that virtual processor and the logical processor is set. Using the other information described above (e.g., an identifier and/or a priority associated with the interrupt) the hardware and/or the hypervisor may be able to deliver an interrupt to a specified virtual processor while reducing the burden on silicon.

Flow then proceeds to operation <NUM> and the hardware delivers the interrupt to the identified virtual processor. According to the invention, the hardware delivers the interrupt to a virtual processor even if the virtual processor is not currently active. In other cases, the hardware may mark the interrupt as intended for the virtual processor. In either case, once the virtual processor is active, it may already have or otherwise be aware of the interrupt and execute the interrupt such as described above. In some cases, a hypervisor may be configured to move a virtual processor from the host machine to a guest/virtual machine and vice versa.

In some instances, a virtual machine, such as, for example, virtual machine <NUM> and virtual machine <NUM><NUM> (<FIG>) may be associated with or assigned an identifier. <FIG> illustrates a method <NUM> for assigning an identifier to a particular virtual machine and using the identifier to target a specific virtual processor. In some instances, the identifier may be used in conjunction with a data structure, such as the bit map described above, in order to quickly and efficiently determine which logical processor is hosting which virtual processor(s).

In some instances, a virtual machine may broadcast an inter-processor-interrupt (IPI) to all of its virtual processors. In doing so, the virtual machine may write one or more instructions to its associated virtual interrupt controller. The hypervisor associated with the virtual machine may track these instructions and see that the virtual machine wants to broadcast to all of its virtual processors or a subset of its virtual processors. In response, the hypervisor may enumerate all of the virtual processors and determine which logical processor the virtual processors operate on. Once that is discovered, the hypervisor may send a physical broadcast interrupt to the logical processor and inject a virtual interrupt. In some cases, this may be expensive in terms of processing power. As such, method <NUM> illustrates how the hardware can assist in identifying specific virtual machines and their associated virtual processors.

As such, in operation <NUM>, each virtual machine associated with a particular host system may be assigned an identifier. In some instances, the identifier may also include information or some other identifier about each virtual processor associated with that virtual machine.

Once the identifier is assigned to each virtual machine, the hypervisor may be able to target a specific virtual machine and its associated virtual processors when providing an interrupt to the virtual processors. For example, in operation <NUM>, the identifier of a specific virtual machine may be provided to the hardware. That is, the hypervisor may provide the identifier of a specific virtual machine to the hardware.

Flow then proceeds to operation <NUM> and the hardware targets or otherwise causes the virtual processors associated with the identified virtual machine to awaken or otherwise be active. The hardware may also provide an interrupt to one or more of the available virtual processors such as described above. In one example, if one of the virtual processors isn't active, the hardware may pend the interrupt to the inactive virtual processor such as described above.

<FIG> and their associated descriptions provide a discussion of a variety of operating environments in which aspects of the disclosure may be practiced. However, the devices and systems illustrated and discussed with respect to <FIG> are for purposes of example and illustration and are not limiting of a vast number of electronic device configurations that may be utilized for practicing aspects of the disclosure, as described herein.

<FIG> is a block diagram illustrating physical components (e.g., hardware) of a computing device <NUM> with which aspects of the disclosure may be practiced. The computing device <NUM> may be similar to the host machine <NUM> described above with respect to <FIG>.

In a basic configuration, the computing device <NUM> may include at least one processing unit <NUM> and a system memory <NUM>. Depending on the configuration and type of computing device <NUM>, the system memory <NUM> may comprise, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories. The system memory <NUM> may include an operating system <NUM> and one or more program modules <NUM> or components suitable for identifying various objects contained within captured images such as described herein.

The operating system <NUM>, for example, may be suitable for controlling the operation of the computing device <NUM>. Furthermore, examples of the disclosure may be practiced in conjunction with a graphics library, other operating systems, or any other application program and is not limited to any particular application or system. This basic configuration is illustrated in <FIG> by those components within a dashed line <NUM>.

The computing device <NUM> may have additional features or functionality. For example, the computing device <NUM> may also include additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated in <FIG> by a removable storage device <NUM> and a non-removable storage device <NUM>.

As stated above, a number of program modules and data files may be stored in the system memory <NUM>. While executing on the processing unit <NUM>, the program modules <NUM> (e.g., a hypervisor <NUM>) may perform processes including, but not limited to, the aspects, as described herein.

Furthermore, examples of the disclosure may be practiced in an electrical circuit comprising discrete electronic elements, packaged or integrated electronic chips containing logic gates, a circuit utilizing a microprocessor, or on a single chip containing electronic elements or microprocessors. For example, examples of the disclosure may be practiced via a system-on-a-chip (SOC) where each or many of the components illustrated in <FIG> may be integrated onto a single integrated circuit. Such an SOC device may include one or more processing units, graphics units, communications units, system virtualization units and various application functionality all of which are integrated (or "burned") onto the chip substrate as a single integrated circuit.

When operating via an SOC, the functionality, described herein, with respect to the capability of client to switch protocols may be operated via application-specific logic integrated with other components of the computing device <NUM> on the single integrated circuit (chip). Examples of the disclosure may also be practiced using other technologies capable of performing logical operations such as, for example, AND, OR, and NOT, including but not limited to mechanical, optical, fluidic, and quantum technologies. In addition, examples of the disclosure may be practiced within a general purpose computer or in any other circuits or systems.

The computing device <NUM> may also have one or more input device(s) <NUM> such as a keyboard, a trackpad, a mouse, a pen, a sound or voice input device, a touch, force and/or swipe input device, etc. The output device(s) <NUM> such as a display, speakers, a printer, etc. may also be included. The aforementioned devices are examples and others may be used. The electronic device <NUM> may include one or more communication connections <NUM> allowing communications with other computing devices <NUM>. Examples of suitable communication connections <NUM> include, but are not limited to, radio frequency (RF) transmitter, receiver, and/or transceiver circuitry; universal serial bus (USB), parallel, and/or serial ports.

The term computer-readable media as used herein may include computer storage media.

<FIG> and <FIG> illustrate a mobile computing device <NUM>, for example, a mobile telephone, a smart phone, wearable computer (such as a smart watch), a tablet computer, a laptop computer, and the like, with which examples of the disclosure may be practiced. With reference to <FIG>, one aspect of a mobile computing device <NUM> for implementing the aspects is illustrated.

In a basic configuration, the mobile computing device <NUM> is a handheld computer having both input elements and output elements. The mobile computing device <NUM> typically includes a display <NUM> and one or more input buttons <NUM> that allow an individual to enter information into the mobile computing device <NUM>. The display <NUM> of the mobile computing device <NUM> may also function as an input device (e.g., a display that accepts touch and/or force input).

If included, an optional side input element <NUM> allows further input. The side input element <NUM> may be a rotary switch, a button, or any other type of manual input element. In alternative aspects, mobile electronic device <NUM> may incorporate more or less input elements. For example, the display <NUM> may not be a touch screen in some examples. In yet another alternative embodiment, the mobile computing device <NUM> is a portable phone system, such as a cellular phone. The mobile computing device <NUM> may also include an optional keypad <NUM>. Optional keypad <NUM> may be a physical keypad or a "soft" keypad generated on the touch screen display.

In various examples, the output elements include the display <NUM> for showing a graphical user interface (GUI) (such as the one described above that provides visual representation of a determined pronunciation and may receive feedback or other such input, a visual indicator <NUM> (e.g., a light emitting diode), and/or an audio transducer <NUM> (e.g., a speaker). In some aspects, the mobile computing device <NUM> incorporates a vibration transducer for providing an individual with tactile feedback. In yet another aspect, the mobile computing device <NUM> incorporates input and/or output ports, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., a HDMI port) for sending signals to or receiving signals from an external device.

<FIG> is a block diagram illustrating the architecture of one aspect of a mobile computing device <NUM>. That is, the mobile computing device <NUM> can incorporate a system (e.g., an architecture) <NUM> to implement some aspects. In one embodiment, the system <NUM> is implemented as a "smart phone" capable of running one or more applications (e.g., browser, e-mail, calendaring, contact managers, messaging clients, games, media clients/players, content selection and sharing applications and so on). In some aspects, the system <NUM> is integrated as an electronic device, such as an integrated personal digital assistant (PDA) and wireless phone.

One or more application programs <NUM> may be loaded into the memory <NUM> and run on or in association with the operating system <NUM>. Examples of the application programs include phone dialer programs, e-mail programs, personal information management (PIM) programs, word processing programs, spreadsheet programs, Internet browser programs, messaging programs, and so forth.

The system <NUM> also includes a non-volatile storage area <NUM> within the memory <NUM>. The non-volatile storage area <NUM> may be used to store persistent information that should not be lost if the system <NUM> is powered down.

The application programs <NUM> may use and store information in the non-volatile storage area <NUM>, such as email or other messages used by an email application, and the like. A synchronization application (not shown) also resides on the system <NUM> and is programmed to interact with a corresponding synchronization application resident on a host computer to keep the information stored in the non-volatile storage area <NUM> synchronized with corresponding information stored at the host computer.

The power supply <NUM> may further include an external power source, such as an AC adapter or a powered docking cradle that supplements or recharges the batteries.

The visual indicator <NUM> may be used to provide visual notifications, and/or an audio interface <NUM> may be used for producing audible notifications via an audio transducer (e.g., audio transducer <NUM> illustrated in <FIG>). In the illustrated embodiment, the visual indicator <NUM> is a light emitting diode (LED) and the audio transducer <NUM> may be a speaker. These devices may be directly coupled to the power supply <NUM> so that when activated, they remain on for a duration dictated by the notification mechanism even though the processor <NUM> and other components might shut down for conserving battery power. The LED may be programmed to remain on indefinitely until the individual takes action to indicate the powered-on status of the device.

The audio interface <NUM> is used to provide audible signals to and receive audible signals from the individual (e.g., voice input such as described above). For example, in addition to being coupled to the audio transducer <NUM>, the audio interface <NUM> may also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation. In accordance with examples of the present disclosure, the microphone may also serve as an audio sensor to facilitate control of notifications, as will be described below.

The system <NUM> may further include a video interface <NUM> that enables an operation of peripheral device <NUM> (e.g., on-board camera) to record still images, video stream, and the like.

Data/information generated or captured by the mobile computing device <NUM> and stored via the system <NUM> may be stored locally on the mobile computing device <NUM>, as described above, or the data may be stored on any number of storage media that may be accessed by the device via the radio interface layer <NUM> or via a wired connection between the mobile electronic device <NUM> and a separate electronic device associated with the mobile computing device <NUM>, for example, a server computer in a distributed computing network, such as the Internet. Similarly, such data/information may be readily transferred between electronic devices for storage and use according to well-known data/information transfer and storage means, including electronic mail and collaborative data/information sharing systems.

As should be appreciated, <FIG> and <FIG> are described for purposes of illustrating the present methods and systems and is not intended to limit the disclosure to a particular sequence of steps or a particular combination of hardware or software components.

<FIG> illustrates one aspect of the architecture of a system <NUM> for providing virtualization using a plurality of computing devices. The system <NUM> may include a general computing device <NUM> (e.g., personal computer), tablet computing device <NUM>, or mobile computing device <NUM>, as described above. Each of these devices may include a hypervisor <NUM> such as described herein.

In some aspects, each of the general computing device <NUM> (e.g., personal computer), tablet computing device <NUM>, or mobile computing device <NUM> may receive various other types of information or content that is stored by or transmitted from a directory service <NUM>, a web portal <NUM>, mailbox services <NUM>, instant messaging stores <NUM>, or social networking services <NUM>.

In aspects, and as described above, each computing device may have access to a virtual machine data store <NUM> that is provided on a server <NUM>, the cloud or some other remote computing device.

By way of example, the aspects described above may be embodied in a general computing device <NUM>, a tablet computing device <NUM> and/or a mobile computing device <NUM>. Any of these examples of the electronic devices may obtain content from or provide data to the store <NUM>.

As should be appreciated, <FIG> is described for purposes of illustrating the present methods and systems and is not intended to limit the disclosure to a particular sequence of steps or a particular combination of hardware or software components.

<FIG> illustrates an example tablet computing device <NUM> that may execute one or more aspects disclosed herein. In addition, the aspects and functionalities described herein may operate over distributed systems (e.g., cloud-based computing systems), where application functionality, memory, data storage and retrieval and various processing functions may be operated remotely from each other over a distributed computing network, such as the Internet or an intranet. User interfaces and information of various types may be displayed via on-board electronic device displays or via remote display units associated with one or more electronic devices. For example, user interfaces and information of various types may be displayed and interacted with on a wall surface onto which user interfaces and information of various types are projected. Interaction with the multitude of computing systems with which examples of the invention may be practiced include, keystroke entry, touch screen entry, voice or other audio entry, gesture entry where an associated electronic device is equipped with detection (e.g., camera) functionality for capturing and interpreting gestures for controlling the functionality of the electronic device, and the like.

As should be appreciated, the figures herein <FIG> is described for purposes of illustrating the present methods and systems and is not intended to limit the disclosure to a particular sequence of steps or a particular combination of hardware or software components.

Examples of the present disclosure provide a method, comprising: monitoring, by a hypervisor of a host machine, a workload of a plurality of virtual processors; receiving an interrupt; determining, by the hypervisor and based on the workload of each of the plurality of virtual processors, a first virtual processor that can execute the interrupt, wherein the first virtual processor is determined based on a time frame that the first virtual processor can execute the interrupt; and providing the interrupt to the first virtual processor but not to the other virtual processors in the plurality of virtual processors. In other examples, each of the plurality of virtual processors is associated with a virtual machine. In some examples, the method also includes assigning an identifier to the virtual machine. In some examples, the method also includes providing the interrupt to hardware associated with the host machine when the first virtual processor is not active. In some examples, the hardware pends the interrupt to the first virtual processor. In some examples, the interrupt is associated with a priority. In some examples, each of the plurality of virtual processors is associated with a priority.

The present disclosure also describes a method, comprising: determining a current priority of at least one virtual processor of the plurality of virtual processors; receiving an interrupt; determining a priority of the interrupt; when it is determined that the priority of the interrupt is higher than the priority of the at least one virtual processor, providing the interrupt to the at least one virtual processor; when it is determined that the priority of the interrupt is not higher than the priority of the at least one virtual processor, determining whether one or more other virtual processors of the plurality of virtual processors has a lower priority; and delivering the interrupt to one of the one or more other virtual processors of the plurality of virtual processors that has a lower priority. In some examples, each of the plurality of virtual processors is associated with a logical processor. In some examples, the method also includes determining whether the priority of the interrupt is higher than the priority of the virtual processor. In some examples, the method also includes delivering the interrupt to the one of the one or more other virtual processors of the plurality of virtual processors that has a corresponding priority when it is determined that the priority of interrupt is not higher than the priority of the virtual processor. In some examples, the one of the one or more other virtual processors has a priority that is lower than the priority of the interrupt. In some examples, the method also includes determining a workload of each virtual processor. In some examples, the method also includes determining whether the virtual processor is executing critical activities. In some examples, the method also includes blocking a delivery of interrupts to the virtual processor when it is determined that the virtual processor is executing critical activities.

The present application also describes a host machine, comprising: at least one processor; and a memory coupled to the at least one processor and storing instructions that, when executed by the at least one processor, perform a method, comprising: causing a hypervisor to monitor a workload of each virtual processor in a plurality of virtual processors; receiving an interrupt; causing the hypervisor to determine which virtual processor of the plurality of virtual processors is to receive the interrupt, wherein the determination is based, at least in part, on the workload of each virtual processor of the plurality of virtual processors; and providing the interrupt to the determined virtual processor. In some examples, the memory also includes instructions for assigning an identifier to the virtual processor. In some examples, the memory also includes instructions for providing the interrupt to hardware associated with the host machine when the virtual processor is not active. In some examples, the interrupt is associated with a priority. In some examples, each of the plurality of virtual processors is associated with a priority.

The description and illustration of one or more aspects provided in this application are not intended to limit or restrict the scope of the disclosure as claimed in any way. The aspects, examples, and details provided in this application are considered sufficient to convey possession and enable others to make and use the best mode of claimed disclosure. The claimed disclosure should not be construed as being limited to any aspect, example, or detail provided in this application. Regardless of whether shown and described in combination or separately, the various features (both structural and methodological) are intended to be selectively included or omitted to produce an embodiment with a particular set of features. Additionally, each operation in the described methods may be performed in different orders and/or concurrently, simultaneously or substantially simultaneously with other operations.

Claim 1:
A method, comprising:
monitoring, by a hypervisor (<NUM>) of a host machine (<NUM>), a workload of a plurality of virtual processors (<NUM>, <NUM>, <NUM>) of the host machine (<NUM>);
receiving, at the hypervisor (<NUM>), an interrupt (<NUM>), the interrupt (<NUM>) being associated with a priority;
wherein, when a virtual processor (<NUM>, <NUM>, <NUM>) is executing an interrupt (<NUM>), the virtual processor (<NUM>, <NUM>, <NUM>) is associated with a priority which is the priority of the interrupt (<NUM>) that the virtual processor (<NUM>, <NUM>, <NUM>) is executing;
determining, by the hypervisor (<NUM>) and based, at least in part, on the workload of each of the plurality of virtual processors (<NUM>, <NUM>, <NUM>), the priority associated with the interrupt (<NUM>) and the current priority of the virtual processors (<NUM>, <NUM>, <NUM>), a virtual processor (<NUM>, <NUM>, <NUM>) that can execute the interrupt (<NUM>) first; and
providing, by the hypervisor (<NUM>) to a virtual interrupt controller of hardware (<NUM>) of the host machine (<NUM>), an identification of said determined virtual processor (<NUM>, <NUM>, <NUM>) that can execute the interrupt (<NUM>) first and to which the interrupt (<NUM>) is to be delivered, when said virtual processor (<NUM>, <NUM>, <NUM>) is not active such that the hardware (<NUM>) pends the interrupt (<NUM>) to said virtual processor (<NUM>, <NUM>, <NUM>), thereby causing the virtual interrupt controller to deliver the interrupt (<NUM>) to said virtual processor (<NUM>, <NUM>, <NUM>) that can execute the interrupt (<NUM>) first but not to the other virtual processors (<NUM>, <NUM>, <NUM>) in the plurality of virtual processors (<NUM>, <NUM>, <NUM>).