Abstract:
Using a lockless protocol, readers and writers exchange data of arbitrary size without using operating system services other than to initially establish a region of global shared memory. The readers and writers may be in interrupt context, process context and/or thread context. Multiple readers and writers are permitted, on the same or on separate processors sharing a global memory. Writers own a set of buffers in global shared memory. The buffers are re-used by their owner using an LRU algorithm. New data is made available to readers by atomically writing the buffer ID (and sequence number) of the most recently written buffer into a shared location. Readers use this shared location to find the most recently written data. If a reader does not have sufficient priority to read the data in the buffer before a writer must re-use the buffer for subsequent data, the reader restarts its read. Buffers contain sequence numbers maintained by the writers to allow the readers to detect this “slow read” situation and to restart its read using the most recently written buffer. Provisions are provided for data time stamps and for resolving ambiguity in the execution order of multiple writers that could cause time stamps to retrogress.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS  
       [0001]    This application is a continuation-in-part of U.S. application Ser. No. 09/642,041, filed Aug. 18, 2000, and claims benefit and priority of U.S. Provisional Application No. 60/149,831, filed Aug. 19, 1999, and of U.S. application Ser. No. 09/642,041, both of which are incorporated herein by reference. 
     
    
     
       FIELD OF THE INVENTION  
         [0002]    The present invention is related to data exchange between execution contexts, and in particular to a deterministic, lockless protocol for data exchange.  
         BACKGROUND OF THE INVENTION  
         [0003]    The exchange of data among processes within general purpose and real-time operating systems is a basic mechanism that is needed by all complex software applications, and various mechanisms are widely available. For simple data, that occupies no more than the native word length of the CPU, the exchange of data can be trivial, consisting of a mailbox that is written and read by single instructions. But for more complex data, which cannot be stored in a single word, the exchange of data is more complex, owing to the existence of races between reader and writer (or among multiple writers) that can cause the data read to be an inconsistent mixture of the data from multiple writes. The races come in two forms:  
           [0004]    Between readers and writers running simultaneously on separate processors sharing the mailbox;  
           [0005]    Between readers and writers running on the same processor but where one execution context is preempted (or interrupted) by the operating system and the other context is allowed to run.  
           [0006]    In both cases, the corruption can be avoided by preventing more than one execution context from executing a region of code called the critical section. This is accomplished on uniprocessor systems by either 1) disabling preemption during critical sections; or by 2) allowing preemption of critical sections, detecting when another execution context tries to enter the preempted critical section and arranging for the critical section to be vacated before another execution context is allowed to enter. On multiprocessor systems, similar techniques are used to control preemption. In addition, simultaneous execution of a critical section by multiple processors is avoided ultimately by spin locks, which make use of special instructions provided by the processor.  
           [0007]    Disabling preemption during a critical section is usually considered a privileged operation by many operating systems and may or may not be provided to some execution contexts as a service of the operating system. If provided as an operating system service, the overhead of calling the service is usually high when compared to the overhead in exchanging the data (at least for small data exchanges). Disabling preemption during a critical section also has the undesirable side effect on real-time systems of increasing the preemption latency. For large transfers, and therefore long critical sections, the increase in the maximum preemption latency can be substantial.  
           [0008]    Allowing critical sections to be preempted but entered by only one execution context at a time is the preferred method on real-time systems, since this does not lead to increases in the maximum preemption latency. This technique requires operating system support, and is therefore dependent on the operating system in use. It also has the disadvantage of adding high overhead to exchanges of small amount of data, as already discussed.  
           [0009]    Locks and critical sections are generally not robust with respect to application failures. If an execution context were to fail while holding the lock or critical section, other execution contexts would be denied access to the data. While recovery techniques exist, these techniques take time and are not compatible with time critical systems.  
           [0010]    All of the above systems are lacking in one or more of the following desirable features:  
           [0011]    Determinism. For execution environments that are deterministic, the reading and writing of data should be deterministic, without a possibility of a priority inversion requiring operating system intervention. Determinism allows a system to be used in real-time operating systems. Even in general-purpose operating systems, there may be contexts which need to be deterministic, such as interrupt service routines that interact within the timing constraints imposed by physical devices.  
           [0012]    Operating System Independence. It is desirable to use as few operating system services as possible for data exchange to create the most portable system. Reducing the use of operating system services also minimizes overhead when exchanging small amounts of data. Further, an operating system independent system can be used for data exchange between execution environments that are running in different operating system environments on the same system (e.g., when a real-time operating system environment is added to a general-purpose operating system environment, or when data is exchanged between interrupt context and process context within a general-purpose operating system).  
           [0013]    Robustness. The failure of a single reader or writer should not impair the performance of other readers and writers.  
           [0014]    Fully preemptive/interruptible. Preemption and interrupts are preferably never disabled so latencies do not suffer as a consequence of exchanging data. Without fully preemptive data exchanges, severe scheduling latencies may occur with large exchanges.  
           [0015]    Scales efficiently to a large number of concurrent readers.  
           [0016]    Applicable to multiprocessor systems as well as uniprocessor systems.  
         SUMMARY OF THE INVENTION  
         [0017]    It is an object of the present invention to supply data exchange systems and methods that provide some or all of the above-mentioned features. A system according to the invention comprises various control structures manipulated by a lockless protocol to give unrestricted access to reading and writing data within shared buffers. The various control structures and pool of shared buffers implement a data channel between readers and writers. More than one data channel can exist, and these data channels can be named. The data written to the data channel can be arbitrarily large, although an upper bound must be known prior to use so that buffers may be pre-allocated, avoiding the indeterminism and operating system involvement of dynamic buffer allocation during the exchange of data. Readers and writers of the data channel are never blocked by the system of the invention.  
           [0018]    The buffers contain data written at various times. When a reader requests access to data, it is given access to the buffer containing the most recent data at the time of the request. After the reader accesses the data within the buffer, the reader dismisses the buffer. Since writers are not blocked and the pool of buffers is finite, the buffer accessed by the reader may have been reused by a writer and overwritten with more recent data. This case is detectable by the reader at the time of dismissal and it is then up to the reader to repeat the read access to obtain new data.  
           [0019]    Each writer has its own pool of buffers. These buffers are in memory shared with processes that are reading the data. Buffers may be reused for writing in least recently used (LRU) order to maximize the time available for a reader to complete its access to the data in a buffer before the writer that owns the buffer must reuse it for a subsequent write. When a writer requests a buffer to write, it may be given the LRU buffer from its pool of buffers. After the writer writes the data into the buffer, the writer releases the buffer. Once the writer successfully releases the buffer, it becomes the buffer with the most recent data that is available to readers. Alternatively, other algorithms for reusing buffers for writing may be used.  
           [0020]    At any moment in time, several versions of the data may exist in buffers and each buffer may be in the process of being read by zero, one, or more readers. There is, however, always a most recently written buffer that is maintained by the invention. The availability of more recently written data is not necessarily cause for readers to abort their access to the buffer that they started to read. It is only when a writer must reuse one of its buffers that the readers of that buffer must restart.  
           [0021]    An optional timestamp can be specified at the time that a write buffer is released. In such embodiments, the timestamp is available to readers of the buffer and the invention guarantees that timestamps will never decrease even when multiple processes are writing a data channel. If a writer does not have sufficient processor priority to dismiss its buffer before another writer with a later timestamp succeeds in dismissing its buffer, the buffer with the earlier timestamp is ignored so as to preserve time ordering. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWING  
       [0022]    The invention is described with reference to the several figures of the drawing, in which,  
         [0023]    [0023]FIG. 1 is a block diagram showing the various execution contexts (readers and writers) within a computer system that may use the invention to exchange data;  
         [0024]    [0024]FIG. 2 is a block diagram of the data structures shared among readers and writers;  
         [0025]    [0025]FIG. 3 is a flow chart describing the use of the invention by an execution context that is reading a data channel;  
         [0026]    [0026]FIG. 4 is a flow chart describing the use of the invention by an execution context that is writing a data channel;  
         [0027]    [0027]FIG. 5 is a block diagram of data structures maintained by writers for managing the reuse of buffers for one particular embodiment of the invention; and  
         [0028]    [0028]FIG. 6 is a flow chart describing the algorithm for managing the reuse of buffers for one particular embodiment of the invention. 
     
    
     DETAILED DESCRIPTION  
       [0029]    [0029]FIG. 1 depicts the various execution contexts  101  within a computer system that may use the invention to exchange data. The invention does not make use of operating system services to exchange data and assumes that preemption and/or interruption can occur at anytime, so an execution context may be an interrupt service routine  103  or a privileged real-time/kernel thread/process  106  or a general-purpose thread/process  109 . The execution contexts may reside on a single processor or may be distributed among the processors of a multiprocessor with a global memory shared among the processors. If used on a multiprocessor system, execution contexts may freely migrate among the processors as is supported by some multiprocessor operating systems.  
         [0030]    The exchange of data is through buffers allocated in global shared memory  115  along with control structures used by the invention. The portion of global shared memory used by the invention is mapped into the address space of the execution contexts. The allocation of global shared memory and the mapping of this memory into the address space of the execution contexts is operating system dependent and typically is not deterministic. The embodiment of the invention on a particular operating system would make use of whatever API that is provided for this purpose and perform the allocation and mapping prior to the exchange of data so that the exchange of data is deterministic.  
         [0031]    For the purposes of explaining the invention, execution contexts are categorized as either readers or writers. In practice, an execution context can be both a reader and a writer. An execution context that will write data is assigned a pool of buffers to manage in global shared memory. The number of buffers assigned to a writer is a configurable of the invention.  
         [0032]    The invention implements a data channel  112  in software for the exchange of data. Upon a request for read access, a reader is given access to the buffer in global shared memory that contains the most recently written data at the time of the request. The reader may access the buffer provided to the reader for an unbounded length of time. But the reader cannot make any assumptions about the consistency of the buffer until read access to the buffer is relinquished and consequently a check is made to be sure the buffer was not reused by a subsequent write during the interval that read access was taking place. If upon relinquishing read access the reader determines that a writer has reused the buffer, the reader repeats its request for read access.  
         [0033]    The reader should not modify a buffer provided for read access. In a preferred embodiment of the invention, providing readers with read-only mapping of the control structures and buffer pool can enforce this.  
         [0034]    Upon receiving a request for a write buffer, in certain embodiments of the invention a writer is given access to the least recently used buffer from the writer&#39;s own pool of buffers residing in global shared memory. The writer may change the buffer in whatever fashion desired. Once the buffer has been updated, write access to the buffer is relinquished and the buffer subsequently becomes available to readers as the most recently written data, unless more current data, as determined from time stamps associated with the data, is already available to readers. If the buffer is associated with a numerically smaller time stamp than what is already available to readers, the write to the data channel is ignored (i.e., the contents of the buffer is changed, but the buffer is not made available to readers). Writers of the data channel are never blocked. In certain embodiments of the invention, rather than giving the writer access to the least recently used buffer from its own pool of buffers, other algorithms for reusing buffers for writing may be employed, provided the buffer given to a writer upon the writer&#39;s request for a buffer is not the most recently written buffer from that writer&#39;s assigned pool of buffers.  
         [0035]    While a buffer is the most recently written buffer, writers are not permitted to change its data. Subsequent writes to the data channel are accomplished by modifying the contents of other buffers from the pool of buffers and then designating these buffers, in turn, as the most recently written buffer. Simply requiring the pool of buffers assigned to each writer to contain at least two buffers enforces this.  
         [0036]    No restriction is placed on the data that is exchanged, other than that it fit in the buffers that are allocated from global shared memory. Writers may specify a time stamp to be associated with the data written. The interpretation of the time stamp is left as a contract between readers and writers of the data but must never retrogress in its numerical value.  
         [0037]    In one embodiment of the invention, an Application Programming Interface (API) provides the ability to read and write to the data channel. This API may have a binding to the various programming languages that are in common use. The API of an illustrative embodiment of the invention is depicted in Table 1.  
                   TABLE 1                       API   Description                   OpenForWriting   Identify the caller as a writer of the data           channel and perform initializations.       AcquireBufferForWriting   Return a reference to a buffer to be filled           with new data to be written to the data           channel.       ReleaseWrittenBuffer   Release the buffer, making the buffer           available to readers as the last written buffer.       CloseForWriting   Disassociate the caller as a writer to the data           channel.       OpenForReading   Identify the caller as a reader of the data           channel and perform initializations.       AccessBufferForReading   Return a reference to the buffer that has the           latest data written to the data channel.       DismissBufferForReading   Relinquish read access to the buffer and           determine if the data in the buffer has           changed during access.       CloseForReading   Disassociate the caller as a reader of the data           channel.                  
 
         [0038]    Table 2 shows data types that are relevant to the invention.  
                   TABLE 2                       Type   Description                   seq_t   A value, preferably 32-bit or larger, that is used to version a           data structure associated with it       time_t   A timestamp, with whatever granularity of time required by           the application.       buffer_t   A buffer containing control structures specific to the           invention and the application data read from and written to           the data channel.                  
 
         [0039]    [0039]FIG. 2 is a block diagram of the data structures shared among readers and writers for the purpose of implementing a data channel. Only a single data channel is illustrated in the examples described below, but those skilled in the art will recognize that multiple data channels can be created. A data channel is composed of the data structures of Table 3, which reside in global shared memory:  
                       TABLE 3                       Variable   Type   Description                   Buffer[]   Array of buffer_t   A pool of N buffers used for the           (See text).   exchange of data.       Write Ticket   seq_t   Encodes the buffer index of the most               recently written buffer and the value of               the buffer sequence number of the               most recently written buffer.                  
 
         [0040]    A buffer index, an integer from 0 . . . N-1, identifies each buffer within the buffer pool. These N buffers are partitioned among the M writers to the data channel. In certain preferred embodiments of the invention each writer to the data channel manages its own subset of the buffer pool in a LRU fashion. The LRU algorithm may use locks without compromising robustness since failure of the writer does not jeopardize the ability of other readers or writers in the system. Writers need not be provided with the same number of buffers from the pool.  
         [0041]    The initial allocation of buffers in global memory and the assignment of buffers to writers are illustrated in the following example of an embodiment of the invention. In this example, readers and writers are processes. Prior to or upon running the first process that may read or write the data channel, the Write Ticket and pool of N buffers are allocated from global shared memory. From this global pool, mutually exclusive subsets of the pool will be assigned to each writer. Processes indicate their intention to write to the data channel by calling the OpenForWriting API, passing a count of buffers to claim from the pool of N buffers. The OpenForWriting API will allocate the data structures of FIG. 5 in process private memory. If there are enough unassigned buffers in shared memory to satisfy the request, the requested number of unassigned buffers are assigned to the writer. The simplest approach is to make such assignments as a consecutive sequence of buffer IDs. The first buffer ID of the sequence is stored in Base Buffer Index and the length of the sequence is stored in Write Buffer Count. The caller of the OpenForWriting API now has write ownership of the buffers of the sequence until the process calls the CloseForWriting API or the process exits. The AcquireBufferForWriting API uses Next Buffer Index to cycle buffer IDs in LRU fashion from the sequence of buffer IDs defined by Base Buffer Index and Write Buffer Count. FIG. 6 depicts an algorithm to be used by AcquireBufferForWriting to pick a buffer for reuse.  
         [0042]    In this particular example, the write buffers are assigned to writing processes and not to writing threads (that is the execution context is a process and not a thread). Consequently, it is not valid for multiple threads within the same process to be writing simultaneously to the data channel. This can be enforced by the AcquireBufferForWriting API, which can return an error if a buffer ID is already outstanding. A buffer ID is outstanding from the time that it is returned by AcquireBufferForWriting until the ReleaseWrittenBuffer API is called.  
         [0043]    Bits within the Write Ticket encode both the buffer index of the most recently written buffer and the value of the sequence number of the most recently written buffer. Various methods of encoding may be used. An illustrative embodiment of the invention is provided as follows. Given T as the value of the Write Ticket, N as the number of buffers within the buffer pool, B as the buffer index of the last write to the data channel and S as the value of the sequence number of the last write to buffer B, the following relationships hold:  
                                                   B = T % N           S = T/N           T = S * N + B                      
 
         [0044]    Each buffer in the buffer pool comprises the elements listed in Table 4.  
                       TABLE 4                       Member   Type   Description                   Buffer   seq_t   A sequence number incremented by each       Sequence       writer before writing to the buffer.       Number       Time Stamp   time_t   An application-supplied timestamp               associated with the data written to the buffer.       Data   Application   The data that has been written to the buffer.           defined.                  
 
         [0045]    The Buffer Sequence Number for the buffer is incremented when write access to a buffer is provided. (As used herein, “incremented” need not mean simply adding 1 to a value, but comprises any change to the value). The Buffer Sequence Number is used to determine if Data and Time Stamp have changed since read access to a buffer has been provided. Upon providing read access, the value of Buffer Sequence Number is decoded from the Write Ticket and stored by each reader. After reading the buffer, the current value of the Buffer Sequence Number is compared with the value that was provided with the read access. If there is a mismatch, the integrity of the data read is in question and the reader must repeat its request for the most recently written buffer. On uniprocessor systems, a repeated read can only take place if a writer to the same data channel preempts/interrupts the reader. The effect of the repeated read on performance can be viewed as a lengthening of the effective context switch/interrupt service time. This allows the invention to be used with existing real-time scheduling theories that account for the latency to switch contexts.  
         [0046]    The interpretation of Time Stamp is application defined. It may represent the time that the data was acquired, the time that the data was written to the data channel or may be an expiration date beyond which time the data is invalid. Applications not using time stamps can effectively disable this aspect of the invention by setting Time Stamp to 0 for all writes.  
         [0047]    [0047]FIG. 3 is a flow chart describing the use of the invention by an execution context that is reading a data channel. The most recently written buffer is determined by reading the Write Ticket  301 . The Current Buffer Index, which is the index of the most recently written buffer, is encoded in the Write Ticket along with the Current Buffer Sequence Number, which is the sequence number of the most recently written buffer at the time that it was written. The bits encoding the Current Buffer Index and Current Buffer Sequence Number may straddle word boundaries, so the Write Ticket must be read atomically (i.e., as an uninterruptible operation) to insure its integrity in the presence of preemption or simultaneous access by multiple processors.  
         [0048]    The reader can now access the data and timestamp  307 . The data within the buffer can be read but the reader should not act upon the data until the Buffer Sequence Number is checked to be sure that its value has not changed  310 , indicating that a writer has reused the buffer. If the Buffer Sequence Number has changed from underneath the reader  313 , the reader repeats—reading the Write Ticket again to determine the new most recently written buffer (and buffer sequence number).  
         [0049]    [0049]FIG. 4 is a flow chart describing the use of the invention by an execution context that is writing a data channel. The least recently used buffer from the writer&#39;s pool of buffers is picked for reuse  401 . The LRU algorithm provides maximum opportunity for slow readers to read the data before a writer must reuse a buffer however, as discussed above, other algorithms may be used. Prior to changing the data in the buffer, the writer increments the Buffer Sequence Number within the buffer  404  and creates a new value for the Write Ticket. Buffer Sequence Numbers must be atomically modified and read to insure integrity in the presence of preemption or simultaneous access by multiple processors.  
         [0050]    The new value, T2, for the Write Ticket is constructed from the Buffer Index and the Buffer Sequence Number  405 . The combination of Buffer Index and Buffer Sequence Number will be used to uniquely describe the new state of the data channel as a consequence of the write.  
         [0051]    Once the Buffer Sequence Number is incremented, the writer modifies the Data and Time Stamp within the buffer  407 . The buffer is now ready to be released to readers. To release the buffer, the Write Ticket is read to determine the Current Buffer Index  410 . The Time Stamp of the new buffer is then compared with the current buffer  413 . If the new buffer has an earlier Time Stamp, the new buffer is assumed to be late and is silently rejected  419 . If the new buffer has a later (or same) Time Stamp, the writer attempts to update the value of the Ticket to reflect the new Current Buffer Index and new Buffer Sequence Number  422 . The update must be done atomically since another writer may be updating the Write Ticket simultaneously. The update is easily implemented as a Compare and Swap operation, which is implemented as an instruction on most processor architectures. If the update is successful, the writer returns  428 . Otherwise, the writer must repeat its update of the Ticket.  
         [0052]    In certain embodiments of the invention it is preferred that the Write Ticket not merely encode the Current Buffer Index, but also encode the Buffer Sequence Number of the current buffer. To understand why, consider a design where the detection of slow readers is left entirely to monitoring the Buffer Sequence Number contained within the buffers. Suppose that Reader A has just read the Write Ticket and determined the current buffer index to be X but is preempted before referencing buffer X. While Reader A is preempted, any manner of activity can take place, including the reuse of the buffer X by Writer B. If Reader A resumed execution after Writer B had incremented the buffer sequence number of buffer X but before it had completed updating the data within the buffer, Reader A would not observe a change in the buffer sequence number even though the data was in the process of being modified. By recording the expected value of the Buffer Sequence Number in the Write Ticket, any change to a buffer since it was released as the most recently written data can be detected by readers.  
       Sequence Number Rollover  
       [0053]    Sequence numbers are stored in the Buffer Sequence Number and encoded within the Write Ticket. These sequence numbers can rollover, depending on the size of the seq_t type. In this section, we discuss the implications of rollover and how rollover can be avoided by an appropriately large size of seq_t. In the following discussion, MAXSEQ-1 is the maximum sequence number that can be stored (or encoded) in the variable in question.  
         [0054]    Buffer Sequence Number rollover, whether in the Write Ticket or in the buffers, introduces the possibility that a reader will not detect that writes have corrupted the buffer being read. The probability that a rollover will prevent this reader from detecting a buffer overwrite is exceedingly small, however, since the number of writes that must take place to escape detection must be an exact integral multiple of MAXSEQ.  
         [0055]    Sequence number rollover can be avoided entirely be using a large seq_t type. For 64-bit seq_t types, MAXSEQ is approximately 16·10 18 . Assuming a write takes place every 1 microsecond, it would take approximately 5·10 5  years of continuous operation for rollover to occur.  
         [0056]    Sequence number rollover in the Write Ticket is more frequent since fewer bits are available to encode the sequence number and is therefore the limiting factor. But even if there were as many as 1,000 buffers in the pool of the data channel (requiring 10 of the 64 bits to encode), it would take approximately 500 years of continuous operation for rollover to occur.  
         [0057]    Other embodiments of the invention will be apparent to those skilled in the art from a consideration of the specification or practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims.