A Priority Synchronization Algorithm for Multiprocessors Krishna Harathi Theodore Johnson krishna@cis.ufl.edu ted@cis.ufl.edu Computer and Information Sciences Department University of Florida Gainesville , FL 32611 January 21 , 1993 Abstract In this paper , we propose a prioritized spin-lock mutual exclusion algorithm , the PR - Lock . Our priority synchronization algorithm is characterized by a constant time release lock phase , which means that not only does a higher priority process acquire a lock faster , but also does not waste its time to release the lock for a lower priority process . All the work is done in the acquire lock phase during which the process has to wait for the lock to be free anyway . The busy-wait spinning is on globally accessible local variables only , thus avoiding net contention . The PR-Lock requires only a constant amount of space per process per lock . Furthermore , priority inheritance protocols can easily be implemented to avoid the priority inversion problem . We also present simulation results that demonstrate the prioritized acquisition of locks . 1 Introduction Mutual exclusion is a fundamental synchronization primitive for exclusive access to critical sections or shared resources on multiprocessors -LSB- 9 -RSB- . The spin-lock is one of the mechanisms that can be used to provide mutual exclusion on shared memory multiprocessors -LSB- 2 -RSB- . Typically , a spin-lock is implemented using atomic instructions like Test & Set or Compare & Swap , which are available on most shared-memory multiprocessors -LSB- 8 -RSB- . Busy waiting is effective when the critical section is small and the processor resources are not needed by other processes in the interim . However , a spin-lock is usually not fair , and will severely limit performance -LSB- 1 -RSB- -LSB- 4 -RSB- . By careful design of the algorithms one can avoid the high bus contention associated with busy-wait spinning in the algorithm -LSB- 11 -RSB- . Most of the work done in synchronization is not based on priorities , and thus is not suitable for real time systems . In real time systems , each process has strict timing constraints and is associated with a priority indicating the urgency of that process -LSB- 15 -RSB- . This priority is used by the operating system to order the rendering of services among competing processes . Normally , the higher the priority of a process , the faster it 's request for services gets honored . When the synchronization primitives disregard the priorities , lower priority processes may block the execution of a process with a higher priority and a stricter timing constraint -LSB- 13 -RSB- . This priority inversion may cause the higher priority process to miss its deadline , leading to the failure of a real time system . In this paper , we propose a prioritized spin-lock algorithm , the PR-Lock , in which processes acquire locks in the order of their priorities . Our PR-Lock algorithm is suitable for use in real time systems which assign either static priorities for processes , or dynamic priorities depending on the timing constraints of the processes and the scheduling mechanism used in the system . The processes are correctly enqueued in a parallelized priority queue during the lock acquiring phase of the algorithm . The advantage of this approach is two fold . First , all the work is done by the acquiring process that has to wait for the busy lock to be free anyway . Second , the release lock phase is executed in constant time , which means that not only does a waiting higher priority process acquire a lock faster , but also a higher priority process does not waste its time to release the lock for a waiting lower priority process . Also , our PR-Lock implementation has a handle on the process in the critical section through the lock variable , which facilitates implementing priority inheritance protocols to avoid the priority inversion problem . We organized this paper as follows . In section 1.1 , we describe previous work in this area and in section 2 , we present our algorithm . In section 3 , we argue the correctness of our algorithm . In section 4 , we show the simulation results emphasizing the acquisition of the lock in a priority order . And in section 5 , we conclude this paper by suggesting some applications and future extensions to the PR-Lock algorithm . 1.1 Previous Work Our PR-Lock algorithm is based on MCS-Lock algorithm , which is a spin-lock mutual exclusion algorithm for shared-memory multiprocessors -LSB- 11 -RSB- . The MCS-Lock algorithm guarantees FIFO ordering of lock acquisitions and spins on locally accessible flag variables only , thus avoiding contention usually associated with busy-waiting . The MCS-Lock algorithm uses a distributed queue data-structure to implement protocols for busy-wait synchronization . For hardware support the algorithm uses the atomic Swap and Compare & Swap instructions . This algorithm performs well on machines independent of the presence of coherent caches , requiring only a small amount of space per lock . Markatos -LSB- 10 -RSB- has developed a priority spin-lock algorithm based on the MCS-Lock algorithm . Their priority spin-lock algorithm takes the priority of the process waiting to acquire the lock into account - the process having the highest priority among a set of waiting processes is given the lock . Markatos assumes that the priority of the process can change dynamically , every process is guaranteed to have access to the communication medium within a finite amount of time , and the processes are non-preemptable . Our algorithm differs from Markatos 's algorithm in some important ways . Markatos 's algorithm maintains a FIFO order queue in the acquire lock phase just as in the MCS-Lock algorithm , whereas PR-Lock maintains a priority ordered queue . Thus , the release lock phase does most of the work to identify the highest priority process currently enqueued in the Markatos 's algorithm . As there can be a delay between identifying the highest priority process in a FIFO queue and handing over the lock , a still higher priority process may get enqueued in the mean time . This leads to an additional one process execution time delay between a highest priority process getting enqueued in the queue and that process actually acquiring the lock . In PR-Lock algorithm , the lock acquisition is well-defined : a highest priority enqueued process gets to use the lock , as soon as the process currently holding the lock releases it . Another difference is that Markatos 's algorithm maintains the tail of the FIFO queue in the lock whereas the PR-Lock algorithm maintains the process currently using the lock . These differences lead to the advantages of using the PR-Lock algorithm , as mentioned earlier . Goscinski -LSB- 5 -RSB- developed two algorithms for mutual exclusion to be used in real time distributed systems . The algorithms are based on token passing . A process requests the critical section by broadcasting its intention to all other processes in the system . One algorithm grants the token based on the priorities of the processes , whereas the other algorithm grants the token to processes based on remaining time to run the processes . The holder of the token enters the critical section . In -LSB- 13 -RSB- , Rajkumar et . al. , address the issue of priority inversion and propose a class of priority inheritance protocols for synchronizing tasks executing in parallel on multiprocessors . Their multiprocessor priority ceiling protocol bounds the time a task has to wait for access to global data and/or resources and also prevents deadlocks . 2 PR-Lock Algorithm Our PR-Lock algorithm is similar to the MCS-Lock algorithm in that both maintain queues of blocked processes using the Compare & Swap instruction . However , while the MCS-Lock -LRB- and Markatos 's Lock -RRB- maintain a global pointer to the tail of the queue , PR-Lock algorithm maintains a global pointer to the head of the queue . In both the MCS-Lock and the Markatos 's Lock , the processes are queued in a FIFO order , whereas in the PR-Lock , the queue is maintained in the priority order of the processes . The advantage to this approach is two fold : All processes release the lock in lesser time thereby a higher priority task does not spend its useful time to wake up a lower priority task . Secondly , a process entering a non-empty queue spends its otherwise wasteful busy-wait time in enqueuing itself at the proper place in the queue . 2.1 Assumptions The following assumptions are made about the environment under which PR-Lock algorithm functions correctly . ffl The underlying multiprocessor architecture supports an atomic Compare & Swap instruction . ffl The multiprocessor has shared memory with or without coherent caches . ffl Each processor has a record to place in the queue for each lock . This record is allocated in the local , but globally accessible , memory . This record is not used for any other purpose for the lifetime of the queue . ffl The higher the actual number assigned for priority , the higher the priority of a process . It should be noted that each process pi participating in the synchronization using spin-lock can be associated with an unique processor Pi . We expect that the queued processes will not be preempted , though this is not a requirement for correctness . 2.2 Implementation The PR-Lock algorithm consists of two operations : the acquire lock procedure which acquires a designated lock and a release lock procedure which releases the lock . Each process uses the acquire lock and release lock operations to synchronize access to a resource : acquire lock -LRB- L , r -RRB- critical section release lock -LRB- L -RRB- The following sub-sections present version of the Compare & Swap , the data structures needed and the acquire lock and release lock procedures . 2.2.1 The Compare & Swap We assume the availability of the double Compare & Swap operation for convenience and to avoid the A-B-A problem -LSB- 7 -RSB- . A Compare & Swap -LRB- A , B , C -RRB- is used to modify a shared variable C by first making a private copy of the contents of the variable in A. Then , the Compare & Swap puts a new value from B in C only if C contains the same value as in A . The Compare & Swap operation is often used on pointers to records . C is a pointer to a record , A is a previously sampled value of C , and B is a pointer to a record that we would like to substitute for * A -LRB- the record pointed to by A -RRB- . We compute the record * B based on * A -LRB- or decide to perform the swap based on * A -RRB- , so we want to set C equal to B if C points to the record * A. However , even if C contains A , it might point to a different object than the one originally read . This will occur if * A is removed from the data structure , then re-inserted as C. Procedure CAS -LRB- structure Pointer * Current , * Old , * New -RRB- / * Assume CAS operates on double words * / f if -LRB- * Current == * Old -RRB- f * Current = * New ; return -LRB- TRUE -RRB- ; gelse f * Old = * Current ; return -LRB- FALSE -RRB- ; g g Figure 1 : CAS used in the PR-Lock Algorithm This sequence of events can not be detected by the Compare & Swap and is known as the A-B-A problem . To avoid this problem , a counter is appended to C which is treated as a part of C. Thus C consists of two parts : the value part of C and the counter part of C . This counter is incremented every time a modification is made to C. Now , all the variables A , B , C are twice their original size . This approach reduces the probability of occurrence of the A-B-A problem to acceptable levels for practical applications . We use a version of the Compare & Swap operation in which the current value of the target location is stored in old . The semantics of the Compare & Swap used is given in Figure 1 . Other versions of the Compare & Swap operation can also be used by suitable modification to the presented algorithm . 2.2.2 Data Structures The basic data structure used in the PR-Lock algorithm is a priority queue . The lock L contains a pointer to the first record of the queue . The first record of the queue belongs to the process currently using the lock . If there is no such process , then L contains nil . Each process allocates a locally-allocated but globally accessible record , which is the queuing element , to participate in the lock synchronization . If process p inserts record q into the queue , we say that q is p 's record and p is q 's process . The record contains the process priority , the next record pointer and a boolean flag Locked on which the process owning the element busy-waits if the lock is not free . The next record pointer is a double sized variable : one half is the actual pointer and the other half is a counter by the use of which the A-B-A problem is avoided . The counter portion of the pointer itself is apportioned into two parts : a one bit of the counter called the Dq bit is used to indicate whether the queuing element is in the queue . The rest of the bits are used as the actual counter . This technique is similar to the one used by Prakash , Lee and Johnson -LSB- 17 -RSB- . In their usage , the counter refers to the record referenced by the pointer . In our algorithm , the counter refers to the record that contains the pointer , not the record that is pointed to . The counter contains the number of times the record is used to acquire/release the lock . If the Dq bit of a record q is FALSE , then the record is in the queue for a lock L . If the Dq bit is TRUE , then the record is probably not in the queue . The Dq bit is used to avoid garbage accesses . Each process maintains the node address of its record in a local variable . In addition , each process requires two local pointer variables to hold the previous and the next queue element for navigating the queue during the enqueue operation . The data structures used are shown in Figure 2 . A typical queue formed by the PR-Lock algorithm is shown in Figure 3 below . Here L points to the record q0 of the current process holding the lock . The record q0 has a pointer to the record q1 of the next process having the highest priority among the processes waiting to acquire the lock L. Record q1 points to record q2 of the next higher priority waiting process and so on . The record qn belongs to the process with the least priority among waiting processes . 2.2.3 Acquire Lock Operation The acquire lock operation is called by a process ^ p before using the critical section or resource guarded by a lock L . The parameters of the acquire lock operation are the lock L and the record ^ q of the process . The strategy is , starting from L , to find the correct position for the record ^ q within the queue structure Counter f integer Count ; boolean Dq ; g structure Pointer f structure Object * Ptr ; structure Counter Ctr ; boolean Dq ; g structure Record f structure structure of data Data ; boolean Locked ; integer Priority ; structure Pointer Next ; g Shared Variable structure Pointer L ; Private Variables structure Pointer Self Node , Prev Node , Next node ; boolean Success , Failure ; constant TRUE , FALSE , NULL , MAX PRIORITY ; Data PriorityNext.CtrNext.PtrLocked Next.Dq Record Structure Figure 2 : Data Structures used in the PR-Lock Algorithm L q0 q1 qnq2 Figure 3 : Queue data structure used in PR-Lock algorithm q NLP q0 q1 qi qi +1 qn Start L q0 q1 qi qi +1 qn P N Position Prepare L q0 q1 qi qnqi +1 q ^ P N L q0 q1 qi qnqi +1 q ^ P N Commit L q0 q1 qi ^ qnqi +1 Result Figure 4 : Stages in the acquire lock operation by the help of two local pointer variables Prev Node and Next Node , designated as P and N in Figure 4 . The records pointed by P and N are qi and qi +1 . Let qi and qi +1 records belong to processes pi and pi +1 . The positioning is done such that the Pr -LRB- pi -RRB- > = Pr -LRB- ^ p -RRB- > Pr -LRB- pi +1 -RRB- . Once such a position is found , ^ q is prepared for insertion by making ^ q to point to qi +1 . Then the insertion is committed by making qi to point to ^ q by using the Compare & Swap instruction . The various stages and final result are shown in Figure 4 . The acquire lock algorithm is given in Figure 5 . The acquire lock operation begins by assuming that the lock is currently free , i.e. , the PR-Lock pointer L is nil . It attempts to change L to point to its own record by the use of the Compare & Swap instruction . If the swap is successful , the lock is indeed free , so the current process enqueuing its record will acquire the lock without busy-waiting . If the swap is unsuccessful , then the acquiring process traverses the queue to position itself between a higher or equal priority process record and a lower priority process record . Once such a junction point is found , Prev Node will point to the record of the higher priority process and Next Node will point to the record of the lower priority process . The process first sets its link to Next Node . Then , it attempts to change the previous record 's link to its own record by the atomic Compare & Swap . If successful , the process sets the Dq flag in its record to FALSE indicating its presence in the queue . The process then busy-waits until its Locked bit is set to FALSE , indicating that it has been admitted to the critical section . There are three cases for an unsuccessful attempt . Problems are detected by examining the returned value by the failed Compare & Swap marked as F in the algorithm . Note that the returned value is in the Next Node . When we read Next Node , the contents of the record pointed to by Prev Node are fixed because the record 's counter is read into Next Node . 1 . A concurrent acquire lock operation may overtake the acquire lock operation and insert its own record immediately after Prev Node , as shown in Figure 10 . In this case the Compare & Swap will fail at the position marked F in Figure 5 . The correctness of this operation 's position is not affected , so the operation continues from its current position Procedure acquire lock -LRB- Current Locker , Self Node -RRB- f Success = FALSE ; do f Prev Node = NULL ; Next Node = NULL if -LRB- CAS -LRB- & Current Locker , & Next Node , & Self Node -RRB- -RRB- / * No Lock * / f Success = TRUE ; Failure = FALSE ; / * Use Lock * / gelse / * Lock in Use * / f Failure = FALSE ; Self Node ? > Locked = TRUE ; do f Prev Node = Next Node ; Next Node = Prev Node.Ptr ? > Next ; if -LRB- Next Node.Dq == TRUE -RRB- / * Deque , Try Again * / ii or Prev Node.Ptr ? > Priority < Self Node ? > Priority -RRB- -RRB- iii f Failure = TRUE ; gelse f if -LRB- Next Node.Ptr == NULL or -LRB- Next Node.Ptr ! = NULL and Next Node.Ptr ? > Priority < Self Node ? > Priority -RRB- -RRB- f Self Node ? > Next = Next Node.Ptr if -LRB- CAS -LRB- Prev Node.Ptr.Next , & Next Node , & Self Node -RRB- -RRB- F f Self Node ? > Dq = FALSE ; while -LRB- Self Node ? > Locked -RRB- ; / * Busy Wait * / Success = TRUE ; / * Then , use lock * / gelse f if -LRB- Next Node.Dq == TRUE -RRB- / * Deque , Try Again * / ii or Prev Node.Ptr ? > Priority < Self Node ? > Priority -RRB- -RRB- iii f Failure = TRUE ; gelse Next Node = Prev Node ; i g g g gwhile -LRB- ! Success and ! Failure -RRB- ; g gwhile -LRB- ! Success -RRB- ; g Figure 5 : The acquire lock operation procedure -LRB- line marked by i in Figure 5 -RRB- . 2 . A concurrent release lock operation may overtake the acquire lock operation and removes the record pointed to by Prev Node , as shown in Figure 11 . In this case , the Dq bit in the link pointer of this record will be TRUE . The algorithm checks for this condition when it scans through the queue and when it tries to commit its modifications . The algorithm detects the situation in the two places marked by ii in the Figure 5 . Every time a new record is accessed -LRB- by Prev Node -RRB- , its link pointer is read into Next Node and the Dq bit is checked . In addition , if the Compare & Swap fails , the link pointer is saved in Next Node and the Dq bit is tested . If the Dq bit is TRUE , the algorithm starts from the beginning . 3 . A concurrent release lock operation may overtake the acquire lock operation and remove the record pointed to by Prev Node , and then the record is put back into the queue , as shown in Figure 12 . If the record returns with a priority higher than or equal to Self Node 's priority , then the position is still correct and the operation can continue . Otherwise , the operation can not find the correct insertion point , so it has to start from the beginning . This condition is tested at the lines marked iii in Figure 5 . The spin-lock busy waiting of a process is broken by the eventual release of the lock by the process which is immediately ahead of the waiting process . 2.2.4 Release Lock Operation The release lock operation is straight forward and the algorithm is given in Figure 6 . The process p releasing the lock sets the Dq bit in its record 's next link pointer to TRUE , indicating that the record is no longer in the queue . Setting the Dq bit prevents any acquire lock operation from modifying the link . The releasing process copies the address of the record of the process waiting next , if any , to the lock L . The process then releases the lock by setting the Locked boolean variable in the record of the next process waiting to be FALSE . To avoid testing special cases in the acquire lock operation , the priority of the head record is set to the Procedure release lock -LRB- Current Locker , Self Node -RRB- f Self Node.Dq = TRUE ; Current Locker = Self Node.Ptr - > Next ; / * Release Lock * / if -LRB- Self Node.Ptr - > Next ! = NULL -RRB- f Current Locker.Ptr - > Priority = MAX PRIORITY ; Current Locker.Ptr - > Locked = FALSE ; g g Figure 6 : The release lock operation procedure highest possible . The release lock operation is illustrated in Figure 7 . 3 Correctness of PR-Lock Algorithm In this section , we present an informal argument for the correctness properties of our PR-Lock algorithm . We prove that the PR-Lock algorithm is correct by showing that it maintains a priority queue , and the head of the priority queue is the process that holds the lock . The PR-Lock is decisive-instruction serializable -LSB- 14 -RSB- . Both the operations of the PR-Lock algorithm have a single decisive instruction . The decisive instruction for the acquire lock operation is the successful Compare & Swap and the decisive instruction for the release lock operation is setting the Dq bit . Corresponding to a concurrent execution C of the queue operations , there is an equivalent -LRB- with respect to return values and final states -RRB- serial execution Sd such that if operation O1 executes its decisive instruction before operation O2 does in C , then O1 < O2 in Sd . Thus , the equivalent priority queue of a PR-Lock is in a single state at any instant , simplifying the correctness proof . In contrast , a concurrent data structure that is linearizable but not serializable might be in several states simultaneously -LSB- 6 -RSB- . The following notation is used for the discussion : Let there be N processes p1 , p2 , : : : , pN that participate in the lock synchronization for a priority lock L , using the PR-Lock algorithm . As mentioned earlier , each process pi allocates a record qi to enqueue and dequeue . Thus , each process pi participating in the lock usage synchronization is associated with a queuing element qi . Lock L points to the head element of the queue of process elements waiting for the lock to become free , the head element being the element associated with the process currently holding the lock . Let Pr -LRB- pi -RRB- be a function which maps a process to its priority , a number between 1 and N . We also define another function Pr -LRB- qi -RRB- which maps a record belonging to a process pi to its priority . A priority queue is an abstract data type that consists of : ffl A finite set Q of elements qi , i = 1 : : : N ffl A function Pr : qi ! ni where ni 2 N . For simplicity , we assume that every ni is unique . ffl Two operations enqueue and dequeue At any instant , the state of the queue can be defined as Q = -LRB- q0 , q1 , q2 , : : : , qn -RRB- where q1 < Q q2 < Q : : : < Q qn and qi < Q qj iff Pr -LRB- qi -RRB- > Pr -LRB- qj -RRB- . Thus , the elements of the queue Q are totally ordered . Here , q0 is called the head record of the priority queue Q and q0 's process is the current holder of the lock . The enqueue operation is defined as enqueue = -LRB- -LRB- q0 , q1 , q2 , : : : , qn -RRB- , ^ q -RRB- ! -LRB- q0 , q1 , q2 , : : : , qi , ^ q , qi +1 , : : : , qn -RRB- where Pr -LRB- qi -RRB- > Pr -LRB- ^ q -RRB- > Pr -LRB- qi +1 -RRB- The dequeue operation on a non-empty queue is defined as dequeue -LRB- -LRB- q0 , q1 , q2 , : : : , qn -RRB- -RRB- ! -LRB- q1 , q2 , : : : , qn -RRB- where the return value is q0 . A dequeue operation on an empty queue is undefined . For every PR-Lock L , there is an abstract priority queue Q. Initially , both L and Q are empty . When a process ^ p with a record ^ q performs the decisive instruction for the acquire lock operation , Q changes state to enqueue -LRB- Q , ^ q -RRB- . Similarly , when a process executes the decisive instruction for a release lock operation , Q changes state to dequeue -LRB- Q -RRB- . We will show that a PR-Lock L corresponds to an abstract priority queue Q. Theorem 1 The representative abstract priority queue Q is equivalent to the observed queue of the PR-Lock L. q1 qnq2 L q0 q1 qnq2 L Before After Figure 7 : Observed queue L before and after a release lock Proof : We prove the theorem by induction using the following two lemmas . To observe the queue , we start at the lock L and observe the records following the linked list . If the head record has its Dq bit set and the process has exited the acquire lock operation , then we discard it from our observation . Lemma 1 If Q , L before a release lock operation , then Q , L after the release lock operation . Proof : Let Q = -LRB- q0 , q1 , q2 , : : : , qn -RRB- before a release lock operation . A release lock operation is equivalent to a dequeue operation on the abstract queue . By definition , dequeue -LRB- -LRB- -LRB- q0 , q1 , q2 , : : : , qn -RRB- -RRB- ! -LRB- q1 , q2 , : : : , qn -RRB- The before and after states of L are shown in Figure 7 . If L points to the record q0 before the release lock decisive instruction , the release lock decisive instruction sets the Dq bit in q0 to TRUE , removing q0 from the observable queue . Thus , Q , L after the release lock operation . Note that L will point to q1 before the next release lock decisive instruction . k Lemma 2 If Q , L before an acquire lock decisive instruction , then Q , L after the acquire lock operation . Proof : There are two different cases to consider : Case 1 : Q = -LRB- -RRB- before the acquire lock operation . The equivalent operation on the abstract queue Q is the enqueue operation . Thus , enqueue -LRB- -LRB- -RRB- , ^ q -RRB- ! -LRB- ^ q -RRB- In the case of the PR-Lock L with no lock holder , ^ q 's process executes a successful decisive Compare & Swap operation to make L to point to ^ q and acquires the lock -LRB- Figure 8 -RRB- . L L Before After ^ q Figure 8 : Observed queue L before and after an acquire lock q L q0 q1 qi qi +1 qn P N Before L q0 q1 qi ^ qnqi +1 After Figure 9 : Observed queue L before and after an acquire lock Clearly , Q , L after the acquire lock operation . Case 2 : Q = -LRB- q0 , q1 , q2 , : : : , qn -RRB- before the acquire lock operation . The state of the queue Q after the acquire lock is given by enqueue = -LRB- -LRB- q0 , q1 , q2 , : : : , qn -RRB- , ^ q -RRB- ! -LRB- q0 , q1 , q2 , : : : , qi , ^ q , qi +1 , : : : , qn -RRB- The corresponding L before and after the acquire lock is shown in Figure 9 . P , N are the Prev Node and Next Node pointers by which the acquire lock operation of the ^ q 's process positions its record such that the process observes Pr -LRB- qi -RRB- > Pr -LRB- ^ q -RRB- > Pr -LRB- qi +1 -RRB- . Then , the next pointer in ^ q is made to point to qi +1 . The Compare & Swap instruction , marked F in Figure 5 , attempts to set the next pointer in qi to point to ^ q . If the Compare & Swap instruction succeeds , then it is the decisive instruction of ^ q 's process and the resulting queue L is illustrated in the Figure 9 . This is equivalent to Q after the enqueue operation . If the Compare & Swap succeeds only when qi is in the queue , qi +1 is the successor record , and Pr -LRB- qi -RRB- < = Pr -LRB- ^ q -RRB- < Pr -LRB- i +1 -RRB- , Q , L . If there are no concurrent operations on the queue , we can observe that the P and N are positioned correctly and the Compare & Swap succeeds . If there are other concurrent operations , they can interfere with the execution of an acquire lock operation , A . There are three possibilities : Case a : Another acquire lock A ' enqueued its record q ' between qi and qi +1 . If Pr -LRB- ^ q -RRB- < = Pr -LRB- qi +1 -RRB- , then A will continue searching from qi +1 . Since A ' inserted q ' before qi +1 , then Pr -LRB- q ' -RRB- > Pr -LRB- qi +1 -RRB- > = Pr -LRB- ^ q -RRB- , so P and N are still correctly positioned . If Pr -LRB- ^ q -RRB- > Pr -LRB- qi +1 -RRB- , then the Compare & Swap instruction is executed , which fails as qi no longer points to qi +1 . Operation A continues searching from qi . This is shown in Figure 10 . Case b : A release lock operation R overtakes A and removes qi from the queue . Based on its observations of qi and qi + 1 , A may continue searching the queue or commit its operation . In either case A sees the Dq bit set and fails . Operation A starts again from the beginning of the queue . This is shown in Figure 11 . Case c : A release lock operation R overtakes A and removes qi from the queue , and then qi is put back in the queue by another acquire lock A ' . If A tries to commit its operation , then the pointer in qi is changed , so the Compare & Swap fails . If A tries to continue searching , then there are two possibilities based on the new value of Pr -LRB- qi -RRB- . If Pr -LRB- ^ q -RRB- > Pr -LRB- qi -RRB- , then A can not find the right place to insert ^ q past qi . This condition is detected when the priority of qi is examined , at which point the operation A starts from the beginning again . If Pr -LRB- ^ q -RRB- < = Pr -LRB- qi -RRB- , then A can find a correct place to insert ^ q past qi , and A continues searching . This is shown in Figure 12 . Since no matter what interference occurs , A always takes the right action . Therefore , Q , L. k To prove the theorem we use induction . Initially , Q = -LRB- -RRB- and L points to nil . So , Q , L is trivially true . Suppose that the theorem is true before the ith decisive instruction . If the ith decisive instruction is an acquire lock operation , lemma 2 -RRB- Q , L after the ith decisive instruction . If the ith decisive instruction is for a release lock operation , lemma 1 -RRB- Q , L after the ith decisive instruction . Therefore , the inductive step holds , and hence , Q , L. k L q0 q1 qi qi +1 qn L q0 q1 qi qnqi +1 q ' P L q0 q1 qi qnqi +1 q ' P N N NP Before A ' After A ' Continue A Figure 10 : A concurrent acquire lock A ' succeeds before A L T F qi +1 qn qi qi +1 qn P N qi qi +1 qn P N N PL Before R After R Restart A Figure 11 : A concurrent release lock R succeeds before A q0 q1 qi qi +1 qn qi P P N qi +1 N L L qi P N L qiL N P r0 r1 ri +1 rm r0 r0 r1 r1 ri +1 ri +1 rm rm Restart A if Pr -LRB- q ^ -RRB- > Pr -LRB- qi -RRB- Continue A if Pr -LRB- q ^ -RRB- < = Pr -LRB- qi -RRB- After R and A ' Before R Figure 12 : Release lock R and acquire lock it A ' succeed before it A 4 Simulation Results We simulated the execution of the PR-Lock algorithm in PROTEUS , which is a configurable multiprocessor simulator -LSB- 3 -RSB- . We also implemented the MCS-Lock algorithm to demonstrate the difference in the acquisition of locks in the PR-Lock and MCS-Lock algorithms . In the simulation , we use a multiprocessor model with eight processors and a global shared memory . Each processor has a local cache memory of 2048 bytes size . Each process executes for a uniformly randomly distributed time before it issues an acquire-lock request . After acquiring the lock , the process stays in the critical section for a fixed amount of time plus another uniformly randomly distributed amount of time before releasing the lock . This cycle is repeated a twenty times . Figures 13 and 14 depict the average time taken for a process to acquire a lock using the MCS-Lock algorithm and the PR-Lock algorithm , respectively . It clearly demonstrates that the average acquisition time for a lock using PR-Lock is proportional to the process priorities in case of PR-Lock algorithm , whereas the average acquisition time is randomly distributed in the case of MCS-Lock algorithm . This feature makes the PR-Lock algorithm attractive for use in real time systems . Note that the priority of a process is set equal to the process/processor number and the lower the number , the higher the priority of a process . Figure 15 shows the cache hit ratio at each instance of time on all the processors . Most of the time the cache-hit ratio is 95 % or higher on each of the processors indicating the very small amount of bus contention generated , in spite of the processes using busy-waiting . This is because all the processes are spinning on locally accessible variables only . 5 Conclusion In this paper , we have presented a priority spin-lock synchronization algorithm , PR-Lock , which is suitable for real-time shared-memory multiprocessors . The PR-Lock algorithm is characterized by a prioritized lock acquisition , low release overhead , very little bus-contention , and well-defined semantics . Simulation results show that the PR-Lock algorithm performs well in practice . This priority lock algorithm can be used as presented for mutually exclusive access to a critical section or can be used to provide higher level synchronization constructs such as prioritized semaphores and monitors . The PR-Lock algorithm uses a global prioritized queue . Thus , the PR-Lock algorithm can be adapted wherever a priority queue needs to be used on a shared-memory multiprocessor , with the restriction that there can be only a single dequeue process active at a time , whereas multiple enqueuing processes can be operating concurrently . Also , priority inheritance protocols can be implemented easily using the PR-Lock algorithm to avoid the priority inversion problem . The ideas and the mechanisms used in the PR-Lock algorithm can be extended in several directions . Instead of maintaining a single head pointer to the priority queue , multiple pointers , one for each priority can be maintained . This leads to the issue of maintaining coherency among the pointers , but there is the advantage of faster enqueuing and hence faster lock acquisition . Another direction of extension is to allow multiple dequeue processes that is more suitable for some specific applications . Yet another area of interest is to prioritize operations on data structures other than queues . Also , we are developing a more rigorous and formal proof for the PR-Lock algorithm , and are extending it to permit the records to be shared among multiple PR-Locks . References -LSB- 1 -RSB- T. E. Anderson , The Performance of Spin Lock Alternatives for Shared Memory Multiprocessors , IEEE Transactions on Parallel and Distributed Systems , Vol . 1 , No. 1 , 1990 , pp 6-16 . -LSB- 2 -RSB- Gregory R. Andrews , Concurrent Programming Principles and Practice , The Beajamin/Cummings Publishing Company , Inc. , 1991 . -LSB- 3 -RSB- Eric A. Brewer and Chrysanthos N. Dellarocas , PROTEUS User Documentation , MIT Laboratory for Computer Science , October 1991 . -LSB- 4 -RSB- R. R. Glenn , D. V. Pryor , J. M. Conroy and T. Johnson , Characterizing memory Hotspots in a Shared Memory MIMD Machine , Supercomputing ' 91 , IEEE and ACM SIGARCH , 1991 . -LSB- 5 -RSB- A. Goscinski , Two Algorithms for Mutual Exclusion in Real-time Distributed Computer Systems , The Journal of Parallel and Distributed Computing , No. 9 , 1990 , pp 77-82 . -LSB- 6 -RSB- M. Herlihy and J. Wing , Linearizability : A Correctness Condition for Concurrent Objects , ACM Transactions on Programming Languages and Systems , Vol . 12 , No. 3 , 1990 , pp 463-492 . -LSB- 7 -RSB- IBM T. J. Watson Research Center , System/370 Principles of Operations , 1983 , pp. 7.13 - 14 . -LSB- 8 -RSB- C. P. Kruskal , L. Rudolph , and M. Snir , Efficient Synchronization on Multiprocessors with Shared Memory , Proceedings of the 5th Annual ACM Symposium on Principles of Distributed Computing , 1986 , pp 218-228 . -LSB- 9 -RSB- Maekawa , Oldehoeft and Oldehoeft , Operating Systems Advanced Concepts , The Beajamin/Cummings Publishing Company , Inc. , 1987 . -LSB- 10 -RSB- Evangelos P. Markatos , Multiprocessor Synchronization Primitives with Priorities , Technical Report , University of Rochester , 1991 . -LSB- 11 -RSB- John M. Mellor-Crummey and Michael L. Scott , Synchronization Without Contention , ACM Symposium on Programming Languages and Operating Systems -LRB- ASPLOS -RRB- 1991 , pp 269-278 . -LSB- 12 -RSB- John M. Mellor-Crummey and Michael L. Scott , Algorithms for Scalable Synchronization on Shared-Memory Multiprocessors , ACM Transactions on Computer Systems , Vol . 9 , No. 1 , 1991 , pp 21-65 . -LSB- 13 -RSB- Ragunathan Rajkumar , Lui Sha and John P. Lehoczky , Real-Time Synchronization Protocols for Multiprocessors , IEEE Real-Time Systems Symposium , 1988 , pp 259-269 . -LSB- 14 -RSB- Dennis Shasha , Nathan Goodman , Concurrent Search Structure Algorithms , ACM Transactions on Database Systems , Vol . 13 , No. 1 , March 1988 , pp 53-90 . -LSB- 15 -RSB- John A. Stankovic and Krithi Ramamritham , Tuturial Hard Real-Time Systems , IEEE Computer Society Press , 1986 . -LSB- 16 -RSB- J. M. Stone , A Simple and Correct Shared-Queue Algorithm Using Compare-and-Swap , Proceedings of the IEEE Computer Society and ACM SIGARCH Supercomputing '90 Conference , 1990 . -LSB- 17 -RSB- Sundeep Prakash , Yann-Hang Lee and Theodore Johnson , A Non-Blocking Algorithm for Shared Queues Using Compare-and-Swap , International Conference on Parallel Processing , 1991 , Vol II , pp 68-75 . Average Time x 10000 Processor -LRB- Priority -RRB- 4 8 12 16 20 24 1 2 3 4 5 6 7 8 9 Figure 13 : Lock acquisition time for the MCS-Lock Algorithm Average Time x 10000 Processor -LRB- Priority -RRB- 4 8 12 16 20 24 28 1 2 3 4 5 6 7 8 9 Figure 14 : Lock acquisition time for the PR-Lock Algorithm Time x 100000 Processor 0 - 9495 - 100 2 4 6 8 10 12 14 16 123456789 Figure 15 : Cache hit ratio for the PR-Lock Algorithm

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

data structures
mutual exclusion
parallel processing