Abstract:
We are proposing a low latency mechanism for asynchronously running a code segment on a processor in a remote computer by transmitting a special network packet or datalink frame that causes a hardware interrupt. Preferably, the mechanism for invoking the interrupt uses MSI or MSI-X. The information required for interrupting a processor in a remote system is sent in a special network packet or datalink frame.

Description:
BACKGROUND OF THE INVENTION 
       [0001]    Existing mechanisms for executing code segments on remote processors are not efficient for highly latency sensitive applications. Existing mechanisms for invoking a code on remote systems involve sending a message to a remote server application, which in turn invokes the desired code. 
         [0002]    Existing mechanisms are inefficient for highly latency sensitive applications such as code segments or programs that should get invoked during emergencies such as a computer virus attack, an incoming missile, etc. where even a delay of microseconds can result in a disaster. Today many mission critical applications require special ASICs (Application Specific Integrated Circuits) to handle latency requirements. 
         [0003]    It is desirable to have a low latency mechanism for execution of code on a remote system for latency sensitive distributed applications which invoke a code segment on a remote systems. 
         [0004]    It is also important that latency sensitive networking applications process incoming RDMA read or write completions quickly. If a latency sensitive application can invoke a code in a remote system processor while sending latency sensitive data or completions, it can improve response times. 
       BRIEF SUMMARY OF THE INVENTION 
       [0005]    The present invention relates to a method of asynchronously interrupting a remote system processor and invoking a code segment in the remote system directly or indirectly. 
         [0006]    The parameter definitions and parameter values required for interrupting a remote processor and invoking a code segment is passed to a remote application using one or more network packets. A parameter definition will define the type of the parameter, for example a function pointer or a variable to be passed to a function. The parameter value will contain the actual function pointer or value for the variable etc. We refer to the parameter definition and parameter value pair as a Parameter. If the parameter definition is implicit, the Parameter may optionally contain only parameter value. 
         [0007]    The remote application uses the Parameters to create a special network packet or datalink frame and transmits the packet/frame to a switch or a Network Interface Card in another system. The special network packet or datalink frame contains Parameters for invoking an interrupt in the remote system that receives the packet or datalink frame. The Parameters for invoking an interrupt preferably contain an MSI/MSI-X vector and MSI/MSI-X datum. The special packet or datalink frame is referred to as “Code Segment Invoking Network Packet or Datalink Frame” or CSINPDF. 
         [0008]    The interrupt service routine (ISR) that processes that CSINPDF uses the Parameters contained in the CSINPDF to invoke one or more code segments directly or indirectly. 
         [0009]    The direct invocation of a code segment may be by invoking the code or calling a function specified in one or more Parameters. The control goes back to the ISR after the code segment is executed. The code segment or the function called by an ISR is should not block and must be resident in memory. 
         [0010]    Indirect invocation of a code segment may be done by doing a wakeup of a thread using a value or address specified in a Parameter; Another way of indirect invocation of a code segment is by causing an event specified in a Parameter. The indirect invocation uses interfaces provided by operating systems. The interfaces used by an ISR for indirect invocation of code segment should not block. 
         [0011]    Some of the Parameters in the CSINPDF could be used by the code that is invoked. Based on the parameter definition in the Parameter, such parameter values may be pushed to the stack, or stored in an absolute or relative address or in a location specified in the parameter definition. 
     
    
     
       BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 
         [0012]      FIG. 1  illustrates an example of a state machine of a Network Interface Card which processes the incoming CSINPDFs and generates an interrupt. 
           [0013]      FIG. 2A  illustrates an example for transmission of a network packet which is used for sending Parameters to be used for invoking a code segment in a computer by a remote application. 
           [0014]      FIG. 2B  illustrates transmission of a CSINPDF from an application to a network interface card (NIC) which can handle CSINPDF. 
           [0015]      FIG. 3A  illustrates an example of a CSINPDF containing 3 Parameters. 
           [0016]      FIG. 3B  illustrates an examples of the contents of the Parameters. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0017]      FIG. 1  illustrates an example for a state machine of a Network Interface Card (NIC) that can process the CSINPDF. Only the receive side of the state machine is shown as the transmit side is not impacted by the CSINPDF. When a new valid datalink frame (a valid datalink frame is a datalink frame with a valid checksum or CRC etc.) is received  101 , the NIC checks whether the frame format corresponds to CSINPDF  102 . If the frame format does not corresponds to CSINPDF  105 , the frame is treated as a normal datalink frame. If a CSINPDF is received  103 , the application that sent the CSINPDF is authenticated  103 . If authentication fails, the CSINPDF is discarded. If authentication is successful  104 , the CSINPDF is DMAed into the computer memory  104  and the MSI/MSI-X vector and data specified in a CSINPDF Parameter (not shown) is used to interrupt  104  a CPU on the computer. The interrupt causes an ISR (not shown) to be invoked which processes the CSINPDF and Parameters contained in the CSINPDF. The ISR directly or indirectly invokes code segments requested in the Parameters in the CSINPDF. 
         [0018]    The format of the CSINPDF is implementation specific and must be different from the format of other supported datalink frames and network packets. The format of the CSINPDF should be easily identifiable by a NIC or switch. For example, the first data byte in an Ethernet packet (after length field) or the Ethernet packet source address, etc. could be used to differentiate a CSINPDF from other datalink frames or network packets. 
         [0019]      FIG. 2A  illustrates an example of how the Parameters required for creating a CSINPDF is transmitted to a remote application. The Computer A  201  and the computer B  202  are connected through a network. The network interface card (NIC)  203  connecting Computer A to the network is capable of handling CSINPDFs. An application (not shown) in Computer A establishes a connection with an application  205  which runs on Computer B and is capable of generating CSINPDFS. The connection is used for sending  204  Parameters to the application  205 . 
         [0020]    The Parameters can be sent to different applications on different computers. 
         [0021]    In  FIG. 2B  illustrates an example of how a code segment can be invoked on a remote computer. When execution of the application  205  running on the Computer B requires invoking a particular code segment on the Computer A  201 , the application can send a CSINPDF to invoke that code segment, provided the required Parameters are available with the application  205 . The application  205  invokes a code segment on the Computer A by sending a CSINPDF  211  to the NIC  203  on Computer A. 
         [0022]    The code invoked on a remote computer may be a function or a code segment. A code segment or a function directly invoked by an ISR must be resident in the computer memory. If the code segment or the function called by the ISR is not resident, the call to the code segment or the function will block and ISR is not allowed to block. Preferably, such code and functions must be part of the operating system kernel. 
         [0023]    If the code invoked is a function, the caller must setup arguments and return pointers for the function to return to the caller. 
         [0024]    The code segment invoked indirectly may be a kernel or a user space thread. The thread may be invoked by performing a wakeup on an address in a Parameter or on an address derived from a Parameter, where the Parameter is sent in a CSINPDF. A thread may be invoked by making an event occur. The method used for invoking a thread indirectly is operating system and implementation specific. 
         [0025]    Some Parameters in a CSINPDF may be created by the application creating the CSINPDF. 
         [0026]    Preferably, the Parameters required for interrupting a remote processor is an MSI/MSI-X vector and data, or an index, or an identifier. If an index or identifier is used, the NIC processing the CSINPDF must be capable of deriving MSI/MSI-X vectors and data using the index or the identifier. 
         [0027]    A NIC or a switch processing CSINPDFs must be capable of generating a hardware interrupt to the computer to which it is connected or attached. 
         [0028]      FIG. 3A  illustrates an example for the format of a CSINPDF. The frame contains layer  1  header  301 , layer  2  header  302 , layer  2  trailer  308  and layer  1  trailer  309 . The first field after the headers  303  identify the frame as a CSINPDF. The next field  304  gives the number of parameters in the CSINPDF. There are three Parameters  305   306   307 . 
         [0029]      FIG. 3B  illustrates an example for the contents of Parameters. The first parameter  305  is an MSI-X vector and supports a 64 bit address. The length of the Parameter # 1  is 18 bytes and contains an MSI-X vector and data. This parameter is used by a NIC or switch to generate an interrupt. The Parameter #2  306  is an argument to be used with a function call. The length of the Parameter # 2  is 8 bytes. The argument is to be pushed to the stack as an argument for a function call specified in the next parameter. The Parameter # 3   307  is an address to a function to be invoked. The length of the Parameter # 3  is 10 bytes. The Parameter # 3  contains the address of the function to be used for making a function call by an ISR. 
         [0030]    Preferably, a CSINPDF contains a sequence number and an ISR can reject duplicate CSINPDFs. This will enable the retransmission of CSINPDFs. 
         [0031]    Optionally, a frame containing a CSINPDF may also contain data; The data contained in a CSINPDF frame may be an RDMA packet; The NIC in this case should be capable of doing DMA for the data part and another DMA for the CSINPDF. This will allow consumption of an RDMA packet as soon as it arrives.