Patent Publication Number: US-8984199-B2

Title: Inter-processor interrupts

Description:
FIELD 
     An embodiment of the invention relates to computer operation in general, and more specifically to inter-processor interrupts. 
     BACKGROUND 
     A computer may include multiple processors, which may include physical and logical processors. Operating systems may utilize inter-processor interrupts (IPIs) to transfer requests between processors in a system. An operating system may use an inter-processor interrupts in order to have one processor initiate specific actions for one or more other processors. Such actions may include a TLB (translation look-aside buffer) shootdown interrupt, in which a processor sends an interrupt to other processor to request invalidation of a TLB entry. Cache flushing may be initiated by receiving processors in response to a global change made by a sending processor, such as changes in the linear address mappings or changes in the memory caching attributes for a particular memory range. 
     However, inter-processor interrupt signals may require a large overhead for both the sending processor side and the receiving processor side. The sending processor needs to perform memory accesses to send an interrupt through a programmable interrupt controller, such as a local advanced programmable interrupt controller (APIC). In turn, the receiving processor may absorb considerable overhead in the process of receiving an interrupt. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The invention may be best understood by referring to the following description and accompanying drawings that are used to illustrate embodiments of the invention. In the drawings: 
         FIG. 1  illustrates a sending processor sequence; 
         FIG. 2  illustrates a receiving processor sequence; 
         FIG. 3  is illustrates an embodiment of a sending processor sequence; 
         FIG. 4  is illustrates an embodiment of a receiving processor sequence; 
         FIG. 5  demonstrates an embodiment of an inter-processor interrupt sequence; and 
         FIG. 6  illustrates an embodiment of a multi-processor computer. 
     
    
    
     DETAILED DESCRIPTION 
     A method and apparatus are described for inter-processor interrupts in a multi-processor system. 
     Under an embodiment of the invention, an inter-processor interrupt function is performed using an instruction for calling the interrupt. The instruction is referred to herein as an Mcall instruction, although the instruction can have any designation. In the embodiment, the operational cost of the function to the sending processor side is a store to a writeback memory location, with the cost to the receiving side being a forced call to a function. An embodiment of the invention may greatly reduce the operational cost of inter-processor interrupts, thereby improving system performance. 
     According to an embodiment of the invention, an interrupt function is performed by a signal that is sent through the memory system. The sending processor performs a store to a writeback memory location. The store thereby triggers a function call on the receiving side. The operation may be contrasted to a conventional interrupt that is sent through the APIC. The embodiment may allow improved operating system performance in multi-processor and multi-threaded environments by reducing the cost of sending inter-processor interrupts. Under an embodiment of the invention, an inter-processor interrupt function may be performed without an APIC or in systems with alternative signal operations. 
     A conventional mechanism for sending an inter-processor interrupt is illustrated in  FIG. 1 . In this illustration, a first initiating or sending processor provides an interrupt to a second receiving or target processor. The example provided in  FIG. 1  concerns a 64-bit command written in two 32-bit write operations. In this sequence, the first processor performs the following tasks:
         1. Raise interrupt request level (IRQL) via write to the processor local APIC task priority register  105 . This is a write to a non-cached location.   2. Create (in a memory location or register) a command to write to the processor local APIC interrupt command register  110 . Among other parameters, this command specifies the target processors and the interrupt vector V to be used for interrupting the target processor. Interrupt vector V would correspond to the interrupt service routine that would be executed on the target processor in response to sending the inter-processor interrupt.   3. Write the command to the processor local APIC interrupt command register (ICR)  115 . This is a write to a non-cached location. The process for writing the command may include:
           a. Disable operation of interrupts  120 .   b. Wait for the local APIC to be not busy  125 . This may be done by polling on the BUSY bit in the interrupt command register.   c. Write the upper 32 bits of the command to the APIC interrupt command register High word  130 .   d. Write the lower 32 bits of the command to the APIC interrupt command register Low word  135 .   e. Wait for the local APIC to be not busy  140 .   f. Re-enable interrupt operation  145 .   
           4. Wait for the target processor to acknowledge receipt of the inter-processor interrupt via a write of a particular data value to a particular memory location  150 . This write would occur as part of servicing of the interrupt just sent on the target processors.   5. Resume normal operation  155 .       

     On the receiving processor, the interrupt is conventionally latched and delivered to the processor core via logic incorporated in the local APIC interrupt delivery mechanism. The illustrated interrupt mechanism takes into account the interrupt priority under which the processor core is operating (as is reflected in an APIC task priority register), other pending interrupts that may have higher priority, and the interruptibility state of the processor&#39;s core. When the processor core has interrupts enabled and the vector corresponding to the inter-processor interrupt is the highest priority interrupt vector pending, then the local APIC dispatches the vector to the core. 
     For a receiving processor, a conventional sequence of events is illustrated in  FIG. 2 . The inter-processor interrupt process for the receiving processor may include:
         1. The processor&#39;s local APIC dispatches the interrupt vector V  205 , which corresponds to the interrupt service routine (ISR) to the processor core. At boot time, the OS would have programmed the interrupt descriptor table entry corresponding to the vector V to contain an interrupt gate with the interrupt service routine.   2. Raises the task priority register level to a level corresponding to the vector V  210 .   3. The processor core dispatches the vector V via the interrupt descriptor table  215 .   4. The interrupt service routine corresponding to the inter-processor interrupt gains control with interrupts disabled  220 .   5. The interrupt service routine writes to a memory location to signal to the sending processor the receipt of the inter-processor interrupt  225 .   6. Perform the action for the inter-processor interrupt  230 .   7. Resume normal operation  235 .       

     Under an embodiment of the invention, the use of an instruction (a Mcall instruction in this description) for the operation of an interrupt may simplify the operational sequence for the sending processor and the receiving processor. At boot time, each processor in a multi-processor system may register a function, the function corresponding to an interrupt service routine that would have executed in kernel mode on receipt of an interrupt service routine, such as the inter-processor interrupt function via a Mcall instruction. However, this operation may alternately be accomplished by other mechanisms, including the use of model specific registers. 
       FIG. 3  is an illustration of an embodiment of an inter-processor interrupt sequence for a sending processor. A process for a sending processor may comprise:
         1. Performing a memory write of the inter-processor interrupt request to a linear address X  305 .   2. Waiting for the receiving processor to acknowledge receipt of the inter-processor interrupt by polling a particular memory location to determine whether the value changes  310 . The value change occurs via a write as part of servicing of the interrupt on the receiving processor. This operation is not needed for transmission of an inter-processor interrupt, and in some embodiments operations may resume without polling a memory location or receiving acknowledgement of receipt of the interrupt.   3. When the memory location has changes values, resuming normal operation  315 .       

     The example shown in  FIG. 3  illustrates an example in which an interrupt is sent to one receiving processor. An inter-processor interrupt may be sent to multiple processors. In one embodiment, multiple processors monitor a single memory location to detect inter-processor interrupts. In another embodiment, each processor may monitor separate memory locations. If an inter-processor interrupt is sent to multiple target processors, then the sending processor may perform a write to a memory location monitored by the processors, or may perform multiple writes, writing to each of the addresses that the target processors are monitoring. Under an embodiment of the invention, each write performed is to a cached memory location, thus being significantly faster than writes to uncached task priority register addresses in a conventional sequence. 
       FIG. 4  is an illustration for a receiving processor sequence. Under an embodiment of the invention, the receiving processor operation for inter-processor interrupts using the Mcall operation may comprise:
         1. In kernel mode, establish a state for enabling ring transition on receipt of inter-processor interrupt  405 .   2. Monitor memory location &lt;Linear Address X&gt;  410 . A write to the memory location signifies an inter-processor interrupt request.   3. Upon detection of inter-processor interrupt in either user mode or in kernel mode  415 , save the current state  420 .   4. Perform the interrupt, Mcall &lt;EPI ISR Linear Address&gt;  425 .   5. The performance of the function may include writing to a memory location being polled by the sending processor  430 .   6. Resume normal operation  435 .       

     Under this embodiment, the Mcall instruction puts the receiving processor in a state in which the processor monitors the linear address X for writes and, upon detection of a write operation, the receiving processor transfers execution control to the IPI ISR linear address. A ring transition is performed as needed, with the appropriate state established on the stack and the processor priority level raised to the appropriate priority. 
       FIG. 5  illustrates an embodiment of a sequence between a first sending processor and a second receiving processor. In this illustration, the sending processor  505  is sending an inter-processor interrupt to the receiving processor  510 . In other illustrations, an interrupt may be sent to multiple processors. The sending processor  505  writes an inter-processor interrupt request  515  to an address designated in the figure as linear address X  520 . Linear address X  520  is monitored  535  by the receiving processor  510 . The sending processor  505  then may poll  525  a memory location designated in the illustration as Y  530 . A change in value in memory location Y  530  indicates an acknowledgment of receipt of the interrupt request by receiving processor  510 . However, receiving acknowledgement is not necessary for transmission of the inter-processor interrupt request, and in some embodiments the sending processor  505  may commence normal operations without polling a memory location or receiving an acknowledgement. 
     The receiving processor  510  monitoring  535  the linear address X  520  is notified of the interrupt request when a write to linear address X  520  occurs. In kernel mode, the receiving processor will have established a state for enabling ring transition on receipt of an inter-processor interrupt. When the interrupt is received, the current state of the receiving processor is saved  540 . The linear processor performs the interrupt, with the call for the interrupt being shown as Mcall &lt;IPI ISR Linear Address&gt;  545 . The performance of the function may include writing  550  to the memory location Y  530  being polled  525  by the sending processor  505 . Upon detecting a change in value in memory location Y  530 , the sending processor may resume normal operation. Upon completing the inter-processor interrupt, the receiving processor may resume normal operation. 
     Techniques described here may be used in many different environments.  FIG. 6  is block diagram of an exemplary computer that can be used in conjunction with an embodiment of the invention. Under an embodiment of the invention, the computer may comprise an embedded system or other special purpose computer. An embedded system or other special purpose computer may operate without certain of the components and features described herein. 
     Under an embodiment of the invention, a computer  600  comprises a bus  605  or other communication means for communicating information, and a processing means such as one or more processors  610  (shown as  611 ,  612  and continuing through  613 ) coupled with the first bus  605  for processing information. Any of the processors  610  may provide an inter-processor interrupt to one or more of the other processors. Each processor may comprise an execution unit and logic for inter-processor interrupt operation. 
     The computer  600  further comprises a random access memory (RAM) or other dynamic storage device as a main memory  615  for storing information and instructions to be executed by the processors  610 . Main memory  615  also may be used for storing temporary variables or other intermediate information during execution of instructions by the processors  610 . The computer  600  also may comprise a read only memory (ROM)  620  and/or other static storage device for storing static information and instructions for the processor  610 . 
     A data storage device  625  may also be coupled to the bus  605  of the computer  600  for storing information and instructions. The data storage device  625  may include a magnetic disk or optical disc and its corresponding drive, flash memory or other nonvolatile memory, or other memory device. Such elements may be combined together or may be separate components, and utilize parts of other elements of the computer  600 . 
     The computer  600  may also be coupled via the bus  605  to a display device  630 , such as a liquid crystal display (LCD) or other display technology, for displaying information to an end user. In some environments, the display device may be a touch-screen that is also utilized as at least a part of an input device. In some environments, display device  630  may be or may include an auditory device, such as a speaker for providing auditory information. An input device  640  may be coupled to the bus  605  for communicating information and/or command selections to the processor  610 . In various implementations, input device  640  may be a keyboard, a keypad, a touch-screen and stylus, a voice-activated system, or other input device, or combinations of such devices. Another type of user input device that may be included is a cursor control device  645 , such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor  610  and for controlling cursor movement on display device  630 . 
     A communication device  650  may also be coupled to the bus  605 . Depending upon the particular implementation, the communication device  650  may include a transceiver, a wireless modem, a network interface card, or other interface device. The computer  600  may be linked to a network or to other devices using the communication device  650 , which may include links to the Internet, a local area network, or another environment. 
     In the description above, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form. 
     The present invention includes various steps. The steps of the present invention may be performed by hardware components or may be embodied in machine-executable instructions, which may be used to cause a general-purpose or special-purpose processor or logic circuits programmed with the instructions to perform the steps. Alternatively, the steps may be performed by a combination of hardware and software. 
     Portions of the present invention may be provided as a computer program product, which may include a machine-readable medium having stored thereon instructions, which may be used to program a computer (or other electronic devices) to perform a process according to the present invention. The machine-readable medium may include, but is not limited to, floppy diskettes, optical disks, CD-ROMs, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, magnet or optical cards, flash memory, or other type of media/machine-readable medium suitable for storing electronic instructions. Moreover, the present invention may also be downloaded as a computer program product, wherein the program may be transferred from a remote computer to a requesting computer by way of data signals embodied in a carrier wave or other propagation medium via a communication link (e.g., a modem or network connection). 
     Many of the methods are described in their most basic form, but steps can be added to or deleted from any of the methods and information can be added or subtracted from any of the described messages without departing from the basic scope of the present invention. It will be apparent to those skilled in the art that many further modifications and adaptations can be made. The particular embodiments are not provided to limit the invention but to illustrate it. The scope of the present invention is not to be determined by the specific examples provided above but only by the claims below. 
     It should also be appreciated that reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature may be included in the practice of the invention. Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims are hereby expressly incorporated into this description, with each claim standing on its own as a separate embodiment of this invention.