Patent Publication Number: US-7917908-B2

Title: Flow lookahead in an ordered semaphore management subsystem

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     Under 35 USC §120, this application is a continuation application and claims the benefit of priority to application Ser. No. 10/247,602, filed Sep. 19, 2002, now U.S. Pat. No. 7,406,690, issued Jul. 29, 2008 entitled Flow Lookahead in an Ordered Semaphore Management Subsystem, all of which is incorporated herein by reference. 
     This application claims the benefit of the filing date of provisional application Ser. No. 60/325,069, filed Sep. 26, 2001 for “Flow Lookahead In Ordered Semaphore Management”. 
    
    
     FIELD OF THE INVENTION 
     This invention relates to ordered semaphore management systems and more particularly to a flow look ahead method and apparatus for bypassing the existing head of an ordered semaphore queue to permit a subsequent thread in the ordered semaphore queue to contemporaneously access a different semaphore value. 
     BACKGROUND 
     While the invention is generic in nature and capable of use with a large variety of multi-threaded processor systems, it will be described in conjunction with a multi-threaded processor system such as the IBM Part No. IBM32NPR161EPXCAE133 Network Processor which employs a plurality of processors and threads each of which concurrently process data frames which may be from the same or different data flows. The individual threads/processors share common resources in the network processor. Semaphores defined to be associated with specific resources are used to allocate the specific resources to the individual threads as requested. 
     Within such a network processor several data frames are processed at the same time. Each data frame is processed by one processor/thread. Each processor/thread operates independently from all the other processors/threads. Thus, as the software (picocode) processes a data frame, the software has no knowledge of other frames which have been, are being, or will be processed. As data frames are processed, a thread may need access to a shared resource. This shared resource is shared among all threads. To allow a thread access to the resource without interference from other threads, semaphores are used. A semaphore is a mechanism which allows a processor/thread to use a resource without interference from another processor/thread. Semaphores exist in almost every multi-processor environment where multiple processors can access common resources. A semaphore is used to ensure that one and only one processor/thread has “ownership” or use of a given resource at any given time. 
     A network processor is a multi-processor environment with resources which can be accessed by all processors/threads. Thus, semaphores are an intricate part of network processors. As discussed above, network processors process data frames which belong to one or more data flows. Traditionally, semaphores are implemented in software using “read modify write” or “test and set” instructions. When these instructions are used as a basis to create and allocate semaphores, valuable system resources must be used. To implement a semaphore, system memory must be used. To access a semaphore, several lines of code must be executed. If these system resources were not used for semaphore implementation, they could be used for other functions or provide a performance increase by not executing extra line(s) of code. 
     When semaphores are implemented in software, several lines of code must be executed to access and lock the semaphore, impacting performance. If the semaphore is unavailable (locked by another thread/processor), the software would need to poll on the semaphore. This would waste valuable bandwidth on the arbitrated memory holding semaphore locks to be accessed by all threads/processors. To implement a fair semaphore access in software requires more system memory and lines of code. For example, if a semaphore is locked, the thread/processor would need to put itself in a queue waiting for access. This queue would be implemented in system memory and require software management, impacting performance. This allows threads/processors to have fair access to resources. 
     In a software semaphore environment, multiple threads/processors cannot unlock their respective semaphores at the same time. Typically, all the semaphores are in the same system memory. Each thread/processor must arbitrate to access the memory to unlock their semaphore. This may add to the processing time of other threads/processors waiting to access the same memory to access the semaphore locks. The same is true for locking semaphores. When semaphores are implemented in software, only one semaphore can be unlocked/locked at a time since all the semaphores reside in a common area of system memory. 
     In the IBM Network Processor System identified above a device termed Completion Unit monitors the order in which frames or packets in a flow are processed by the threads or Dyactic Protocol Processor Units (DPPUs) and generates information used by a semaphore sub-system to control the order in which semaphores are assigned. Such systems require ordered semaphores which must perform two functions. First, the well known semaphore function, ensure that one and only one processor/thread has access to a single resource at any time. And second, ordered semaphores must ensure that the processors/threads which are processing data frames of the same data flow access the common resource in frame order, for example, an e-mail message which must be encrypted using an encryption co-processor shared among all of the processors/threads. The encryption of the data frames must occur in order to properly encrypt the message. The software would use an ordered semaphore mechanism to access the encryption co-processor. 
     This would ensure two things. First, only one processor/thread accesses the co-processor at a time. And second, the encryption of the data frames of the data flow (e-mail message) occurs in order. Ordered Semaphores are needed since processing time for each data frame can be different. Data frames from the same data flow may take different amounts of time to process. For example, tree searches for each data frame can take different amounts of time. Threads which share a common ALU may stall occasionally to allow the other thread to process data. Thus, frames in the same data flow being processed by different threads will attempt to access a shared resource at different times and not necessarily in data flow order. Thus, ordered semaphores are required to ensure the shared resource is accessed in data flow order. 
     The Completion Unit logic block contains all the information required to put processed data frames (received from processors/threads) back in the correct order for each data flow. U.S. Pat. No. 6,633,920, issued on Oct. 14, 2003, and U.S. Pat. No. 6,977,928, issued on Dec. 20, 2005, incorporated herein by reference describe how the Completion Unit performs this function. Within the completion unit, linked lists of the data frames assigned to processors/threads represent the data frame order of the data flows. One linked list exists for each data flow which currently has a data frame being processed by a processor/thread. The head of the linked list is associated with a processor/thread. It is from this processor/thread that the next processed data frame is to be taken from and sent out onto the network. When the processed data frame is sent, the head of the linked list is removed and the next element of the linked list is examined; see the referenced applications for details. 
     U.S. Pat. No. 7,089,555, issued on Aug. 8, 2006, incorporated herein by reference, describes a generic Ordered Semaphore Management Subsystem (herein after referred to as the OSMS system) which employs an ordered semaphore queue. The ordered semaphore queue mirrors the flow queue maintained by the network processor system. It includes an ordered semaphore field (OSF) for each processor/thread in the flow. Each processor/thread resides in one of four states. Only one processor/thread resides in the Semaphore Head (SH) state and is entitled to access a semaphore. The other processors/threads in the ordered semaphore queue must wait until they enter the SH state before they can gain access to a semaphore (for more information concerning the operation of the OSMS system refer to the incorporated application). In those situations where a processor behind the current SH processor/thread wants a different semaphore it must nevertheless wait until it enters the SH state before it can access this different (non-conflicting) semaphore (this condition is often referred to as head of line blocking). This application addresses that situation and provides a solution thereto. 
     SUMMARY OF THE INVENTION 
     The invention contemplates an application system which includes one or more shared resources each controlled by a unique semaphore, a plurality of threads adapted to perform ordered and unordered tasks on assigned segments of a continuous data flow, using one or more shared resources controlled by semaphores. The system employs an ordered queue of threads having one semaphore head (SH) thread identifying the next thread in the queue which is entitled to lock a semaphore and in which the next eligible thread in the queue behind the SH thread becomes the SH thread when the current SH thread locks a semaphore. 
     A storage means is associated with each thread for storing the unique semaphore requested, a valid indicia indicating when set that the semaphore is locked and a pending field for indicating when set that the thread is entitled to the identified semaphore when it becomes unlocked. A first logic circuit in response to a semaphore lock request from an SH thread examines the storage means and locks the requested semaphore if it is not locked by another thread. If the requested semaphore is locked by another thread, it provides an indication that it is locked. A second logic circuit responsive to the indication that the requested semaphore is locked examines the pending fields in the storage means and sets the pending field associated with the SH thread if no other pending field for the requested semaphore is set. The next eligible thread in the ordered semaphore queue is made the SH thread when the pending field is set and the current thread goes to the SHB state. A third logic circuit monitors the status of the requested semaphore and locks the requested semaphore for the pending thread and resets the associated pending field when the requested semaphore becomes unlocked. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram of an application system which incorporates a semaphore manager subsystem according to the invention; 
         FIG. 2  is a block diagram of a semaphore manager subsystem according to the invention; 
         FIG. 3  is a diagram of the semaphore value storage  21  of  FIG. 2 ; 
         FIGS. 4A and 4B  are flow diagrams defining (on a per thread basis) the lock command logic  22  of  FIG. 2 ; and, 
         FIG. 5  is a flow diagram defining the error and exit logic  24  of  FIG. 2 , which is executed upon a thread exit. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     In  FIG. 1  a network processor such as the IBM processor identified above includes an input/output data storage unit  11  which stores a plurality or stream of data frames which require processing. A dispatching unit  12  transfers individual data frames to a plurality of processors  13 - 1 - 12 - n  which process the individual frames received from the dispatching unit  12 . The processors  13 - 1 - 13 - n  pass the processed data frames on to a completion unit  14  which reorders the data frames before passing the ordered stream of data frames on to a second input/output unit  15 . 
     Each of the processors  13 - 1 - 13 - n  include a semaphore coprocessor  13   p  which interfaces a hardware semaphore manager subsystem  16  constructed according to the invention. The semaphore subsystem  16  is implemented in hardware and interfaces with, for example, the Dyatic Protocol Processor Unit (DPPU) of the using processing system. Each DPPU contains multiple threads (4, in the case of the IBM Network Processor) which can each process one data frame. Each DPPU has one Semaphore Co-Processor associated with it. The threads in a given DPPU interface to the one Semaphore Co-Processor within the DPPU. The multiple Semaphore Co-Processors all communicate with the central Semaphore Manager subsystem. The Semaphore Manager subsystem  16  contains all of the tables and control logic to lock, unlock, and arbitrate for semaphores. The semaphore manager  16  communicates with the completion unit  14  over a bus  17 . 
     In  FIG. 2  the semaphore coprocessors  13   p −1- 13   p - n  communicate with the Semaphore Manager subsystem  16  via a bus  20 . The subsystem  16  includes a semaphore value storage  21 , semaphore lock command logic  22 , semaphore unlock command logic  23  and semaphore exit and error detection logic  24 . It also includes reservation release command logic  25  and completion unit interface logic  26 . Except for the semaphore value storage  21 , the lock command logic  22  and exit and error detection logic  24 , the remaining components of  FIG. 2  operate as described in the incorporated OSMS system application. 
       FIG. 3  is a block diagram of the semaphore value storage  21 . The storage can be based upon a RAM, CAM, or discrete latches. For each thread there exists exactly three registers. The first register (Semaphore_Value) is a 32 bit register which holds the 32 bit Semaphore Value (Sem_Val) that can be locked by the associated thread. The second register (Semaphore_Lock) is a 1 bit register which indicates if the Sem_Val stored in the associated Semaphore_Value register is locked or unlocked. When the Semaphore_Lock register is set to 1′b, the Sem_Val in the Semaphore_Value register is locked. When the Semaphore_Lock register is reset to ‘0’b, no Sem_Val is locked by the associated software thread. 
     The third register (not disclosed in the OSMS system application) is a Semaphore_Pending register. The register has one or more bits Sem_Pending which when set indicate that the associated thread is to be granted the identified and currently locked semaphore when it is unlocked regardless of the fact that it is not in the SH state. If more than one pending level (two or more bits) is used the threads must be prioritized according to their order in the ordered data flow sequence. The use of the pending state will be described below. 
     A semaphore can be locked when a software thread issues a single command “Semaphore Lock” (Sem_Lock) with two parameters. The first parameter is the “Semaphore Value” (Sem_Val). This is, for example, a 32 bit value which the thread wishes to lock. The second parameter is the “Timeout Enable” (Timeout_Enable) bit. When the Timeout Enable bit is set and the requested semaphore is already locked by a different thread, the Semaphore Lock command will terminate without locking the semaphore. 
     In the preferred embodiment, each thread has an assigned register in the semaphore value storage and is thus identified as the source of the semaphore value requested. Alternatively, the requested semaphore value could be placed in any available register along with a thread ID. The association could be made dynamically by providing thread ID information with the Sem_Val. 
     The OSMS system application generates an Ordered Semaphore Field (OSF) for each thread in the data flow. The fields are arranged in an ordered queue that mirrors the data flow queue in the system completion unit. Each individual OSF is assigned one of four states (Semaphore Head (SH), Behind Semaphore Head (BSH), Semaphore Head Behind (SHB) or Skip). The SH state is assigned to only one thread in a data flow at any given time and only the SH thread is eligible to acquire a semaphore. When the SH thread locks the requested semaphore, the SH state moves down the queue to the next eligible thread in the ordered semaphore queue. 
     If the thread behind the SH thread is requesting a different (available) semaphore than the SH thread, it must wait until it becomes the SH thread before it can gain access to the available different semaphore. The present invention in certain circumstances eliminates this waiting period and thus improves overall system performance. 
     This is accomplished by providing a new (pending) state in the semaphore value storage  21 . When a thread N in the SH state issues a semaphore lock command for a currently locked semaphore, the semaphore manager places the thread N in the pending state by setting the corresponding pending register in storage  21  provided no other thread is pending on that semaphore. At the same time the semaphore manager removes thread N from the SH state. This action advances the SH state to the next eligible thread N+1 in the ordered semaphore queue. If thread N+1 is requesting a different non-conflicting semaphore it will be granted while thread N waits in the pending state for the previously requested semaphore to become available. The sequence can advance as described until an SH thread requests a locked semaphore value which also has a requesting thread in the pending state for that semaphore. In the interests of simplicity of explanation, the invention will be described with a single level for the pending state. However, by adding additional levels to the pending state, the SH state could obviously skip over two or more consecutive threads pending on the same semaphore value. 
       FIGS. 4A and 4B  are similar to  FIGS. 4A and 4B  of the OSMS system, however they include additional logic to implement the new functionality provided by the new pending state. The figures illustrate the function of the lock command logic as modified by the pending state. To the extent possible identical reference numerals will be used for identical logic functions. 
     Directional blocks A, B, C and D indicate the connections between the two figures. The process starts when a thread N issues a semaphore lock command which includes a 32 bit semaphore value and a 1 bit time out enable value  400 . If the request is an ordered semaphore request and thread N is not enabled for ordered semaphores  400 -A, a queue not enabled error is generated  40  land the request is changed to an unordered request  402 . If it is not the former, it is examined to determine if it is an unordered request or an ordered request at the head of a queue  403 . If the request is either of the above it enters a round robin  404  for selection and exits  FIG. 4A  at A. If the answer in block  403  is “no” (in which case it is an ordered request which is not the head of a queue) it is examined to determine if the timeout enable bit is set. If the timeout enable bit is not set  405  the request loops back to  403 . If the bit is set it exits  FIG. 4A  at D. 
     The non-winning threads at block  406  loop back to block  404  while the semaphore request from the winning thread is examined  407  in the semaphore value storage to determine if that semaphore value is currently locked. If the semaphore value is not locked it is examined in block  408  to determine if the request is ordered. 
     If the request is ordered, the thread is removed from the ordered queue by sending a Pop signal  409  and the semaphore value requested is locked  410  for thread N. If the request was not ordered, the requested semaphore value is locked for thread N  410  and in either case the operation for thread N is complete  411 . 
     If the locked semaphore value requested by thread N is already locked by thread N  412 , a lock same semaphore value error for thread N is generated  413  and the operation completes at  411 . If the requested semaphore value  412  was not locked by thread N the Sem_Pending registers in storage  21  are examined  420  to determine if the requested Sem_Value is in the pending state for another thread; if it is, time out enable bit is examined  414 . If the bit is set an indication that the semaphore could not be locked is generated  415  and the operation completes  411 . If the timeout enable bit is not set the request loops back through C to block  404 . 
     If the semaphore value requested by thread N is not pending for another thread, thread N is removed from the ordered semaphore queue by sending a Pop signal  421  to the completion unit interface  26  which also moves the SH state to the next qualified thread in the ordered semaphore queue (for the details of this operation see the OSMS system application). Thread N waits for the requested semaphore to be unlocked  422 . When the requested semaphore value is unlocked, the pending state for thread N is reset  423  and the requested semaphore value is locked for thread N  410 . 
     With a single level for the pending state a thread desiring a non-conflicting semaphore can bypass only one thread vying for a locked semaphore. Threads desiring a non-conflicting semaphore will continue to bypass the single thread in the pending state until the first thread desiring a conflicting semaphore is reached. By expanding the number of levels of the pending state, a thread can bypass as many consecutive threads vying for the same locked semaphore. This can be accomplished by increasing the number of bits in the Sem_Pending field. For example, two bits would allow up to three consecutive threads vying for the same locked semaphore to be bypassed by a thread seeking to lock a non-conflicting available semaphore. All that is required is to assign a priority to the different values of the Sem_Pending field so that the order or sequence of the threads is not violated. 
     If two bits are used 00 can be assigned to the reset state of the pending field and 01, 10 and 11 assigned to priority levels 1, 2 and 3, respectively. When a locked semaphore is unlocked the 01 pending thread will be granted the requested semaphore and the threads at the 10 and 11 pending levels (if any) will be elevated to levels 01 and 10, respectively. Resetting the pending level of the thread granted the requested semaphore and the adjustment of the priority levels of the other threads can be easily accomplished by subtracting 1 from their current values. 
       FIG. 5  illustrates the steps needed to handle errors that occur on thread exit. When a thread N exits  500 ,  FIG. 5 , the logic checks the semaphore value storage associated with thread N. If thread N has not unlocked its semaphore  501 , it is unlocked  502  and a semaphore locked at exit error  503  is generated for thread N. The logic also checks the Sem_Pending field  504 . If the field is set the logic resets the Sem_Pending field  505  and signals an error at exit condition  503 . 
     The foregoing is illustrative of the present invention and is not to be construed as limiting the invention. While several embodiments of this invention have been described in detail, those skilled in this art will readily appreciate that many modifications are possible without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention as defined by the claims. In the claims, means-plus-function clauses are intended to cover the structures described herein as performing the recited function and structural functional equivalents thereof. Therefore, it is to be understood that the foregoing is illustrative of the present invention and is not to be construed as limited to the specific embodiments disclosed, and that modifications to the disclosed embodiments, as well as other embodiments, are intended to be included within the scope of the claims appended hereto.