Abstract:
A system and method for employing multiple hardware contexts and programming engines in a functional pipeline partitioned to facilitate high performance data processing. The system and method includes a parallel processor that assigns system functions for processing data including programming engines that support multiple contexts arranged to provide a functional pipeline by a functional pipeline control unit that passes functional data among the programming engines.

Description:
BACKGROUND  
         [0001]    This invention relates to functional pipelines.  
           [0002]    Parallel processing is an efficient form of information processing of concurrent events of a computing system. Parallel processing demands concurrent execution of many programs, in contrast to sequential processing. In the context of parallel processing, parallelism involves doing more than one thing at the same time. Unlike a serial paradigm where all tasks are performed sequentially at a single station or a pipelined machine where tasks are performed at specialized stations, with parallel processing, many stations are provided, each capable of performing and carrying out various tasks and functions simultaneously. A number of stations work simultaneously and independently on the same or common elements of a computing task. Accordingly, parallel processing solves various types of computing tasks and certain problems are suitable for solution by applying several instruction processing units and several data streams. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0003]    [0003]FIG. 1 is a block diagram of a processing system.  
         [0004]    [0004]FIG. 2 is a detailed block diagram of the processing system of FIG. 1.  
         [0005]    [0005]FIG. 3 is a block diagram of a programming engine of the processing system of FIG. 1.  
         [0006]    [0006]FIG. 4 is a block diagram of a functional pipeline unit of the processing system of FIG. 1.  
         [0007]    [0007]FIG. 5 is a block diagram illustrating details of the processing system of FIG. 1. 
     
    
     DESCRIPTION  
       [0008]    Architecture:  
         [0009]    Referring to FIG. 1, a computer processing system  10  includes a parallel, hardware-based multithreaded network processor  12 . The hardware-based multithreaded processor  12  is coupled to a memory system or memory resource  14 . Memory system  14  includes dynamic random access memory (DRAM)  14   a  and static random access memory  14   b  (SRAM). The processing system  10  is especially useful for tasks that can be broken into parallel subtasks or functions. Specifically, the hardware-based multithreaded processor  12  is useful for tasks that are bandwidth oriented rather than latency oriented. The hardware-based multithreaded processor  12  has multiple functional microengines or programming engines  16  each with multiple hardware controlled threads that are simultaneously active and independently work on a specific task.  
         [0010]    The programming engines  16  each maintain program counters in hardware and states associated with the program counters. Effectively, corresponding sets of context or threads can be simultaneously active on each of the programming engines  16  while only one is actually operating at any one time.  
         [0011]    In this example, eight programming engines  16   a - 16   h  are illustrated in FIG. 1. Each engine from the programming engines  16   a - 16   h  processes eight hardware threads or contexts. The eight programming engines  16   a - 16   h  operate with shared resources including memory resource  14  and bus interfaces (not shown). The hardware-based multithreaded processor  12  includes a dynamic random access memory (DRAM) controller  18   a  and a static random access memory (SRAM) controller  18   b . The DRAM memory  14   a  and DRAM controller  18   a  are typically used for processing large volumes of data, e.g., processing of network payloads from network packets. The SRAM memory  14   b  and SRAM controller  18   b  are used in a networking implementation for low latency, fast access tasks, e.g., accessing look-up tables, memory for the core processor  20 , and the like.  
         [0012]    The eight programming engines  16   a - 16   h  access either the DRAM memory  14   a  or SRAM memory  14   b  based on characteristics of the data. Thus, low latency, low bandwidth data is stored in and fetched from SRAM memory  14   b , whereas higher bandwidth data for which latency is not as important, is stored in and fetched from DRAM memory  14   a . The programming engines  16   a - 16   h  can execute memory reference instructions to either the DRAM controller  18   a  or SRAM controller  18   b.    
         [0013]    The hardware-based multithreaded processor  12  also includes a processor core  20  for loading microcode control for the programming engines  16   a - 16   h . In this example, the processor core  20  is an XScale™ based architecture.  
         [0014]    The processor core  20  performs general purpose computer type functions such as handling protocols, exceptions, and extra support for packet processing where the programming engines  16  pass the packets off for more detailed processing such as in boundary conditions.  
         [0015]    The processor core  20  has an operating system (not shown). Through the operating system (OS), the processor core  20  can call functions to operate on the programming engines  16   a - 16   h . The processor core  20  can use any supported OS, in particular, a real time OS. For the core processor  20  implemented as an XScale™ architecture, operating systems such as Microsoft NT real-time, VXWorks and μCOS, or a freeware OS available over the Internet can be used.  
         [0016]    Advantages of hardware multithreading can be explained by SRAM or DRAM memory accesses. As an example, an SRAM access requested by a context (e.g., Thread — 0), from one of the programming engines  16  will cause the SRAM controller  18   b  to initiate an access to the SRAM memory  14   b . The SRAM controller  18   b  accesses the SRAM memory  14   b , fetches the data from the SRAM memory  14   b , and returns data to a requesting programming engine  16 .  
         [0017]    During an SRAM access, if one of the programming engines  16   a - 16   h  had only a single thread that could operate, that programming engine would be dormant until data was returned from the SRAM memory  14   b.    
         [0018]    By employing hardware context swapping within each of the programming engines  16   a - 16   h , the hardware context swapping enables other contexts with unique program counters to execute in that same programming engine. Thus, another thread e.g., Thread — 1 can function while the first thread, Thread — 0, is awaiting the read data to return. During execution, Thread — 1 may access the DRAM memory  14   a . While Thread — 1 operates on the DRAM unit, and Thread — 0 is operating on the SRAM unit, a new thread, e.g., Thread — 2 can now operate in the programming engine  16 . Thread — 2 can operate for a certain amount of time until it needs to access memory or perform some other long latency operation, such as making an access to a bus interface. Therefore, simultaneously, the multi-threaded processor  12  can have a bus operation, an SRAM operation, and a DRAM operation all being completed or operated upon by one of the programming engines  16  and have one more threads or contexts available to process more work.  
         [0019]    The hardware context swapping also synchronizes the completion of tasks. For example, two threads can access the shared memory resource, e.g., the SRAM memory  14   b . Each one of the separate functional units, e.g., the SRAM controller  18   b , and the DRAM controller  18   a , when they complete a requested task from one of the programming engine threads or contexts reports back a flag signaling completion of an operation. When the programming engines  16   a - 16   h  receive the flag, the programming engines  16   a - 16   h  can determine which thread to turn on.  
         [0020]    One example of an application for the hardware-based multithreaded processor  12  is as a network processor. As a network processor, the hardware-based multithreaded processor  12  interfaces to network devices such as a Media Access Controller (MAC) device, e.g., a 10/100BaseT Octal MAC  13   a  or a Gigabit Ethernet device  13   b . In general, as a network processor, the hardware-based multithreaded processor  12  can interface to any type of communication device or interface that receives or sends large amount of data. The computer processing system  10  functioning in a networking application can receive network packets and process those packets in a parallel manner.  
         [0021]    Programming Engine Contexts:  
         [0022]    Referring to FIG. 2, one exemplary programming engine  16   a  from the programming engines  16   a - 16   h , is shown. The programming engine  16   a  includes a control store  30 , which in one example includes a RAM of 4096 instructions, each of which is 40-bits wide. The RAM stores a microprogram that the programming engine  16   a  executes. The microprogram in the control store  30  is loadable by the processor core  20  (FIG. 1).  
         [0023]    In addition to event signals that are local to an executing thread, the programming engine  16   a  employs signaling states that are global. With signaling states, an executing thread can broadcast a signal state to all programming engines  16   a - 16   h . Any and all threads in the programming engines can branch on these signaling states. These signaling states can be used to determine availability of a resource or whether a resource is due for servicing. The context event logic has arbitration for the eight (8) threads. In one example, the arbitration is a round robin mechanism. Other techniques could be used including priority queuing or weighted fair queuing.  
         [0024]    As described above, the programming engine  16   a  supports multi-threaded execution of eight contexts. This allows one thread to start executing just after another thread issues a memory reference and must wait until that reference completes before doing more work. Multi-threaded execution is critical to maintaining efficient hardware execution of the programming engine  16   a  because memory latency is significant. Multi-threaded execution allows the programming engines  16  to hide memory latency by performing useful independent work across several threads.  
         [0025]    The programming engine  16   a , to allow for efficient context swapping, has its own register set, program counter, and context specific local registers. Having a copy per context eliminates the need to move context specific information to and from shared memory and programming engine registers for each context swap. Fast context swapping allows a context to do computation while other contexts wait for input-output (I/O), typically, external memory accesses to complete or for a signal from another context or hardware unit.  
         [0026]    For example, the programming engine  16   a  executes the eight contexts by maintaining eight program counters and eight context relative sets of registers. There can be six different types of context relative registers, namely, general purpose registers (GPRs)  32 , inter-programming agent registers (not shown), Static Random Access Memory (SRAM) input transfer registers  34 , Dynamic Random Access Memory (DRAM) input transfer registers  36 , SRAM output transfer registers  38 , DRAM output transfer registers  40 .  
         [0027]    The GPRs  32  are used for general programming purposes. The GPRs  32  are read and written exclusively under program control. The GPRs  32 , when used as a source in an instruction, supply operands to an execution datapath  44 . When used as a destination in an instruction, the GPRs  32  are written with the result of the execution datapath  44 . The programming engine  16   a  also includes I/O transfer registers  34 ,  36 ,  38  and  40  which are used for transferring data to and from the programming engine  16   a  and locations external to the programming engines  16   a , e.g., the DRAM memory  14   a , the SRAM memory  14   b , and etc.  
         [0028]    A local memory  42  is also used. The local memory  42  is addressable storage located in the programming engine  16   a . The local memory  42  is read and written exclusively under program control. The local memory  42  also includes variables shared by all the programming engines  16   a - 16   h . Shared variables are modified in various assigned tasks during functional pipeline stages by the programming engines  16   a - 16   h , which are described next. The shared variables include a critical section, defining the read-modify-write times. The implementation and use of the critical section in the computing processing system  10  is also described below.  
         [0029]    Functional Pipelining and Pipeline Stages:  
         [0030]    Referring to FIG. 3, the programming engine  16   a  is shown in a functional pipeline unit  50 . The functional pipeline unit  50  includes the programming engine  16   a  and a data unit  52  that includes data, operated on by the programming engine, e.g., network packets  54 . The programming engine  16   a  is shown having a local register unit  56 . The local register unit  56  stores information from the data packets  54 .  
         [0031]    In the functional pipeline unit  50 , the contexts  58  of the programming engines  16   a , namely, Programming Engine0.1 (PE0.1) through Programming Engine0.n (PE0.n), remain with the programming engine  16   a  while different functions are performed on the data packets  54  as time  66  progresses from time=0 to time=t. A programming execution time is divided into “m” functional pipeline stages or pipe-stages  60   a - 60   m . Each pipeline stage of the pipeline stages  60   a - 60   m  performs different pipeline functions  62   a ,  64 , or  62   p  on data in the pipeline.  
         [0032]    The pipeline stage  60   a  is, for example, a regular time interval within which a particular processing function, e.g., the function  62  is applied to one of the data packets  54 . A processing function  62  can last one or more pipelines stages  62 .  
         [0033]    The function  64 , for example, lasts two pipeline stages, namely pipeline stages  60   b  and  60   c.    
         [0034]    A single programming engine such as the programming engine  16   a  can constitute a functional pipeline unit  50 . In the functional pipeline unit  50 , the functions  62   a ,  64 , and  62   p  move through the functional pipeline unit  50  from one programming engine  16  to another programming engine  16 , as will be described next.  
         [0035]    Referring to FIG. 4, the data packets  54  are assigned to programming engine contexts  58  in order. Thus, if “n” threads or contexts  58  execute in the programming engine  16   a , the first context  58 , “PE0.1” completes processing of the data packet  54  before the data packets  54  from the “PE0.n” context arrives. With this approach the programming engine  16   b  can begin processing the “n+1” packet.  
         [0036]    Dividing the execution time of the programming engines  16  into functional pipeline stages  60   a - 60   c  results in more than one programming engine  16  executing an equivalent functional pipeline unit  70  in parallel. The functional pipeline stage  60   a  is distributed across two programming engines  16   a  and  16   b , with each of the programming engines  16   a  and  16   b  executing eight contexts each.  
         [0037]    In operation, each of the data packets  54  remains with one of the contexts  58  for a longer period of time as more programming engines  16  are added to the functional pipeline units  50  and  70 . In this example, the data packet  54  remains with a context sixteen data packet arrival times (8 contexts×2 programming engines) because context PE0.1 is not required to accept another data packet  58  until the other contexts  58  have received their data packets.  
         [0038]    In this example, the function  62  of the functional pipeline stage  60   a  can be passed from the programming engine  16   a  to the programming engine  16   b . Passing of the function  62  is accomplished by passing the processing functions from one programming engine to another, as illustrated by dotted lines  80   a - 80   c  in FIG. 4.  
         [0039]    The number of functional pipeline stages  60   a - 60   m  is equal to the number of the programming engines  16   a  and  16   b  in the functional pipeline units  50  and  70 . This ensures that a particular pipeline stage executes in only one programming engine  16  at any one time.  
         [0040]    Referring to FIG. 5, functional pipeline units  50 ,  70 , and  90  are shown to include the programming engines  16   a  (PE0),  16   b  (PE1), and  16   c  (PE2), respectively, in addition to the data units  52   a - 52   c . Between the programming engines  16   a - 16   c , critical sections  82   a - 82   c  and  84   a - 84   c  are provided. The critical sections  82   a - 82   c  and  84   a - 84   c  are used in conjunction with shared data  86   a - 86   c  and  88   a - 88   c.    
         [0041]    In the critical sections  82   a - 82   c  and  84   a - 84   c , the programming engine contexts  58   a - 58   c  are given exclusive access to the shared data  86   a - 86   c  (e.g., cyclic redundancy check residue (CRC), reassembly context, or a statistic) in external memory.  
         [0042]    In operation, functions can be distributed across one or more functional pipeline stages  60   a - 60   d . For example, the critical section  82   a  represents a section of code where only one programming engine context, in this case, the context  58   a  of the programming engine  16   a , has exclusive modification privileges for a global resource (i.e., shared data  86   a ), such as a location in memory, at any one time. Thus, the critical section  82   a  provides exclusive modification privileges to a particular functional pipeline stage of the programming engine  16   a . The critical section  82   a  also provides support for exclusive access between the contexts  58   a - 58   c  in the programming engines  16   a - 16   c.    
         [0043]    In certain implementations only one function modifies the shared data  86   a - 86   c  between the programming engines  16   a - 16   c , to ensure exclusive modification privileges between the programming engines  16   a - 16   c . The function that modifies the shared data  86   a - 86   c  executes, e.g., in a single functional pipeline stage  60   a , and the functional pipeline unit  50  is designed so that only one programming engine from all the programming engines  16   a - 16   c  executes the functional pipeline stage  60   a  at any one time.  
         [0044]    Still referring to FIG. 5, each of the programming engines  16   a - 16   c  is assigned exclusive modification privileges to the non-shared data  86   a - 86   c  and  88   a - 88   c , satisfying the requirement that only one function modifies the non-shared data  86   a - 86   c  and  88   a - 88   c  between the programming engines  16   a - 16   c.    
         [0045]    In this example, by optimizing the control flow through the functional pipeline units  50 ,  70 , and  90 , the architectural solution described above presents greater network processing power to the hardware-based multithreaded network processor  12 .  
         [0046]    In the functional pipeline unit  50 , the programming engine  16   a  transitions into the critical section  82   a  of the functional pipeline stage  60   a  unless it can be assured that its “next” programming engine  16   b  has transitioned out of the critical section  82   a . The programming engine  16   b  uses inter-thread signaling where the signaling states can be used to determine availability of a memory resource or whether a memory resource is due for servicing. There are four ways to signal a next context using inter-thread signaling:  
                                                   Thread Signaling   Mechanism                           1. Signal next thread in the same PE   Local Control and Status               Registers (CSR) write           2. Signal a specific thread in the   Local CSR write           same PE           3. Signal the thread in the next PE   Local CSR write           4. Signal any thread in an PE   CSR write                      
 
         [0047]    Critical sections such as CRC calculations are performed in the order of incoming data packets  54   a - 54   c  because inter-thread signaling is performed in order.  
         [0048]    When the functional pipeline stage  60   a  transition occurs between the programming engines  16   a  and  16   b , the elasticity buffer  92 , implemented as a ring, is used. Each functional pipeline stage  60   a  is implemented to execute within the time allocated to the functional pipeline stage  60   a . However, the elasticity buffer  92  accommodates jitter in the execution of the functional pipeline unit  50 . Thus, if a functional pipeline stage  60   a  falls behind in execution due to system anomalies such as high utilization of memory units over a short period of time, the elasticity buffer  92  allows the context  58   a  of the functional pipeline stage  60   a  to be buffered so that the previous functional pipeline stages will not be stalled waiting for the next pipeline stage to complete. The elasticity buffer  92  as shown in FIG. 5 also allows different heartbeats to the different functional pipelines units  70  and  90 .  
         [0049]    By using the functional pipeline stages, as described above, the functional pipeline units  50 ,  70 , and  90  cover memory latency and provides sufficient computation cycles for data packets  54  arriving faster than a single stream computation stage. By providing a mechanism for fast synchronization from one programming engine to the next programming engine which performs the same set of functions on a new set of data packets  54 , parallel processing capability of the programming engines are greatly enhanced. Multiple means for passing functional state and control is thus provided.  
         [0050]    Other Embodiments:  
         [0051]    In the example described above in conjunction with FIGS.  1 - 5 , the computer processing system  10  may implement programming engines  16  using a family of network processors, namely, Intel Network Processor Family chips designed by Intel® Corporation, of Santa Clara, Calif.  
         [0052]    It is to be understood that while the invention has been described in conjunction with the detailed description thereof, the foregoing description is intended to illustrate and not limit the scope of the invention, which is defined by the scope of the appended claims. Other aspects, advantages, and modifications are within the scope of the following claims.