Centralized management of resources shared by multiple processing units

Resource allocation logic for a computer system including a plurality of processors which share access to, and control of, a plurality of resources, such as disk drive units or busses. The resource allocation logic coordinates the execution of requests received from the processors to avoid resource sharing inefficiencies and deadlock situations. The allocation logic maintains a "request" queue for each processor, seeking to satisfy all requests quickly and fairly. The queues contain an entry corresponding to each request received from its corresponding processor and an identification of resources that are required by the entry's corresponding request. The allocation logic also maintains a "resources available" status array of resources which are not currently in use by any processors, or are not reserved for future use by any processors. The logic repeatedly compares each entry in the request queues with the entries in the resources available status array to detect an entry in the request queue identifying resources all of which are contained in the resources available status array. Once the allocation logic can satisfy a particular request, it signals a grant to the requesting processor for the resources requested. The requested resources are removed from the resources available status array. Upon conclusion of execution of the granted request, the resources are again released to the resource allocation logic for utilization by other resource requests. Additionally, each request queue contains a list age indicating the relative age of each request queue with respect to the other request queues, and each entry in the request queues includes a request age indicating the relative age of each entry in a request queue with respect to other entries in the request queue. In examining the request queues to identify I/O requests for execution, priority is awarded to entries based on the relative ages of the request queues and request queue entries.

The present invention relates to disk array storage systems and, more 
particularly, to a method for managing the operations of multiple disk 
array controllers which share access to the disk drive units within the 
array. 
BACKGROUND OF THE INVENTION 
Disk array storage devices comprising a multiplicity of small inexpensive 
disk drives, such as the 51/4 or 31/2 inch disk drives currently used in 
personal computers and workstations, connected in parallel are finding 
increased usage for non-volatile storage of information within computer 
systems. The disk array appears as a single large fast disk to the host 
system but offers improvements in performance, reliability, power 
consumption and scalability over a single large magnetic disk. 
Most popular RAID (Redundant Array of Inexpensive Disks) disk array storage 
systems include several drives for the storage of data and an additional 
disk drive for the storage of parity information. Thus, should one of the 
data or parity drives fail, the lost data or parity can be reconstructed. 
In order to coordinate the operation of the multitude of drives to perform 
read and write functions, parity generation and checking, and data 
restoration and reconstruction, many RAID disk array storage systems 
include a dedicated hardware controller, thereby relieving the host system 
from the burdens of managing array operations. An additional or redundant 
disk array controller (RDAC) can be provided to reduce the possibility of 
loss of access to data due to a controller failure. 
FIG. 1 is a block diagram representation of a disk array storage system 
including dual disk array controllers 11 and 13. Array controller is 
connected through a SCSI host bus 15 to host system 17. Array controller 
13 is likewise connected through a SCSI host bus 19 to a host system 21. 
Host systems 17 and 21 may be different processors in a multiple processor 
computer system. Each array controller 11 has access to ten disk drives, 
identified by reference numerals 31 through 35 and 41 through 45, via five 
SCSI busses 51 through 55. Two disk drives reside on each one of busses 51 
through 55. Disk array controllers 11 and 13 may operate in one of the 
following arrangements: 
(1) Active/Passive RDAC 
All array operations are controlled by one array controller, designated the 
active controller. The second, or passive, controller is provided as a hot 
spare, assuming array operations upon a failure of the first controller. 
(2) Active/Active RDAC--Non Concurrent Access of Array Drives 
One controller has primary responsibility for a first group of shared 
resources (disk drives, shared busses), and stand-by responsibility for a 
second group of resources. The second controller has primary 
responsibility for the second group of resources and stand-by 
responsibility for the first group of resources. For example, disk array 
controller 11 may have primary responsibility for disk drives 31 through 
35, while disk array controller has primary responsibility for disk drives 
41 through 45. 
(3) Active/Active RDAC--Concurrent Access of Array Drives 
Each array controller has equal access to and control over all resources 
within the array. 
Providing each array controller with equal access to and control over 
shared resources may lead to resource sharing inefficiencies or deadlock 
scenarios. For example, certain modes of operation require that subgroups 
of the channel resources be owned by one of the array controllers. Failure 
to possess all required resources concurrently leads to blockage of the 
controller until all resources have been acquired. In a multiple 
controller environment obtaining some but not all the required resources 
for a given transaction may lead to resource inefficiencies or deadlock in 
shared resource acquisition. 
Likewise, an array controller that provides hardware assist in generating 
data redundancy requires simultaneous data transfer from more than one 
drive at a time. As data is received from the drives or the host, it is 
passed through a RAID striping ASIC to generate data redundancy 
information that is either stored in controller buffers or passed 
immediately to a drive for storage. So that the data may be passed through 
the RAID striping ASIC from the multiple data sources concurrently, each 
controller must have access to multiple selected drive channels 
concurrently. Deadlock can occur if no means to coordinate access to the 
drive channels exists. 
Two examples are given below to illustrate the deadlock situation in a two 
disk array controller environment. 
Deadlock Condition 1: 
Referring to FIG. 1, disk array controllers 11 and 13 are seen to share 
five SCSI buses 51 through 55 and the ten drives that are connected to the 
SCSI buses. Disk array controller 11 is requested to perform an I/O 
operation to transfer data from drives disk drive 31 and 33. 
Simultaneously, disk array controller 13 is requested to perform an I/O 
operation to transfer data from disk drives 41 and 43. Both disk 
controllers attempt to access the drives they need concurrently as 
follows: 
Array controller 11 acquires bus 5 1 and disk drive 3 1 and is blocked from 
acquiring bus 53 and disk drive 33. 
Array controller 13 acquires bus 53 and disk drive 43 and continues 
arbitrating for bus 5 1 and disk drive 41. 
Controller 1 now has SCSI bus 51 in use, and is waiting for disk drive 33 
on SCSI bus 53 (owned by Controller 13). Controller 13 now has SCSI bus 53 
in use, and is waiting for disk drive 41 on SCSI bus 51 (owned by 
Controller 11). 
Deadlock Condition 2: 
Deadlock can occur when multiple controllers are attached to the same host 
bus. This may occur when host SCSI bus 15 and host SCSI bus 19 are the 
same physical SCSI bus, identified as bus 27 in FIG. 2. Controller 11 is 
requested to perform an I/O operation requiring a transfer of data from 
disk drive 31 on SCSI bus 51 to host 17. Simultaneously, controller 13 is 
requested to perform an I/O operation requiring a transfer of data from 
disk drive 41 on SCSI bus 51 to host 21. Both controllers attempt access 
of the resources they need concurrently as follows: 
Array controller 11 acquires the single Host SCSI bus, identified by 
reference numeral 27 and is blocked from acquiring SCSI bus 51 and disk 
drive 31. 
Array controller 13 acquires SCSI bus 51 and disk drive 41, and is blocked 
from acquiring the host SCSI bus 15. 
Controller 11 now has the host SCSI bus 27 in use, and is waiting for 
access to SCSI bus 51 (owned by Controller 13.) so that it can connect to 
disk drive 31. Controller 13 now has SCSI bus 51 in use, and is waiting 
for access to the host SCSI bus 27 (owned by Controller 1.). 
A method and structure for coordinating the operation of multiple 
controllers which share access to and control over common resources is 
required to eliminate resource sharing inefficiencies and deadlock 
situations. 
OBJECTS OF THE INVENTION 
It is therefore an object of the present invention to provide a new and 
useful method and structure for coordinating the operation of multiple 
controllers which share access to and control over common resources. 
It is another object of the present invention to provide such a method and 
structure which reduces or eliminates resource sharing inefficiencies and 
deadlock situations which arise in systems which include shared resources. 
It is yet another object of the present invention to provide a new and 
useful disk array storage system including multiple active array 
controllers. 
It is still a further object of the present invention to provide a method 
for coordinating the operation of multiple active controllers within a 
disk array which share access to and control over common resources. 
It is an additional object of the present invention to provide a new and 
useful method for avoiding contention between controllers in a disk array 
system including multiple active controllers. 
SUMMARY OF THE INVENTION 
There is provided, in accordance with the present invention, a method for 
coordinating the execution of requests received from multiple requesting 
agents which share access to and control over common resources within a 
computer system in order to avoid resource sharing inefficiencies and 
deadlock situations. The method includes the steps of: (A) establishing a 
"request" queue, said request queue including an entry corresponding to 
each request received from the requesting agents, each entry including an 
identification of resources that are required by said entry's 
corresponding request; (B) maintaining a "resources available" status 
array, said resources available status array including an entry for each 
resource which is not currently in use by any requesting agent and is not 
currently reserved for future use by any requesting agent; (C) 
systematically comparing each entry in said request queue with the entries 
in said resources available status array to detect an entry in said 
request queue identifying resources all of which are contained in said 
resources available status array; (D) granting control of the resources 
associated with said entry detected in step C to the requesting agent 
providing the request corresponding to the entry detected in step C; and 
(E) executing the request corresponding to the entry identified in step C. 
The resources associated with the granted request are removed from the 
resources available status array during the execution of step (E). Upon 
conclusion of execution of the granted request, the resources are again 
placed in the resources available status array for utilization by other 
resource requests. 
The described embodiment is incorporated into a disk array subsystem 
including multiple array controllers which share access to, and control 
over, multiple disk drives and control, address and data busses within the 
disk array. A request queue containing entries for I/O requests received 
from the host computer system is maintained for each array controller, the 
method of the present invention alternately examining entries in each 
request queue to detect an entry in either request queue identifying 
resources all of which are contained in the resources available status 
array. Additionally, each request queue contains a list age indicating the 
relative age of each request queue with respect to the other request 
queues, and each entry in the request queues includes a request age 
indicating the relative age of each entry in a request queue with respect 
to other entries in the request queue. In examining the request queues to 
identify I/O requests for execution, priority is awarded to entries based 
on the relative ages of the request queues and request queue entries. 
The above and other objects, features, and advantages of the present 
invention will become apparent from the following description and the 
attached drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
A disk array system including dual active controllers constructed in 
accordance with a preferred embodiment of the present invention is shown 
in block diagram form in FIG. 3. In addition to the structure shown in the 
disk array system of FIG. 1, the system of FIG. 3 includes a dedicated 
communication link 57 connected between the array controllers 11 and 13, 
and an Inter-Controller Communication Chip application specific integrated 
circuit incorporated into each of the controllers, identified by reference 
numerals 61 and 63, respectively. 
The communication link 57 and Inter-Controller Communication Chips provide 
communication between, and resource arbitration and allocation for the 
dual disk array controllers. 
The Inter-Controller Communication Chip 
FIG. 4 is a block diagram of the Inter-Controller Communication Chip 
incorporated into each of the dual active array controllers 11 and 13 
included within the disk array system shown in FIG. 3. The 
Inter-Controller Communication Chip (hereafter referred to as the ICON 
chip) contains all functions necessary to provide high speed serial 
communication and resource arbitration/allocation between two Disk Array 
controllers. The primary application for the ICON chip is in Disk Array 
systems utilizing redundant disk array controllers. Because the redundant 
controller configuration shares resources (disk drives and SCSI buses) 
between two controllers, a method of arbitrating for these common 
resources must be utilized in order to prevent deadlocks and to maximize 
system performance. The ICON chip contains a hardware implementation of a 
resource allocation algorithm which will prevent deadlocks and which 
strives to maximize system performance. In addition to performing resource 
arbitration/allocation, the ICON chip also provides a means of 
sending/receiving generic multiple byte messages between Disk Array 
controllers. The ICON chip includes the following logic modules: 
Microprocessor Interface Control Logic 100 
The microprocessor interface block allows an external microprocessor to 
configure and monitor the state of the ICON chip. Configuration and status 
information are maintained in registers within the ICON chip. The 
configuration, control, and status registers are designed to provide 
operating software with a wide range of functionality and diagnostic 
operations. Interrupt masking and control are also included in this 
functional block. 
Inter-controller Communication Logic 200 
The Inter-controller Communication block contains all structures and logic 
required to implement the inter-controller communication interface. This 
block includes the following structures/logic: Send State Sequencer 201, 
Receive State Sequencer 203, Message Send Buffer 205, Message Receive 
Buffer 207, Status Send Register 209, and Status Receive Buffer 211. These 
modules work together to form two independent unidirectional communication 
channels. Serialization and Deserialization of data packets occurs in Send 
State Sequencer 201 and Receive State Sequencer 203 modules. Serial data 
output from the Send State Sequencer 201 may be fed into the Receiver 
State Sequencer 203 module for a full diagnostic data turnaround. 
The Inter-controller Communication Block 200 is used to send generic 
messages and status or to send specific request/grant/release resource 
messages between two Disk Array controllers. 
Communication between pairs of ICON chips is provided by 6 signals. These 
signals are defined as follows: 
TABLE 1 
______________________________________ 
Communication Signal Descriptions 
Name Type Deseription 
______________________________________ 
ARDY/ OUT `A` Port ready. This output is controlled 
by the ICON Ready bit in the Control 
Register and is monitored by the alternate 
controller. 
BRDY/ IN `B` Port ready. This input is used to 
monitor the Ready/Not Ready status of 
the alternate controller. 
AREQ.DAT/ OUT `A` Port Request/Serial Data. This output 
signal is used to request data transfer and 
then send serial data to the alternate 
controller in response to the `A` Port 
Acknowledge signal. 
BREQ.DAT/ IN `B` Port Request/Serial Data. This input is 
used to receive serial data from the 
alternate controller. 
AACK/ IN `A` Port Acknowledge. This signal is 
received from the alternate controller as 
the handshake for a single data bit 
transfer. 
BACK/ OUT `B` Port Acknowledge. This output signal 
is sent to the alternate controller to 
control a serial receive data transfer 
operation. 
______________________________________ 
Resource Allocation Logic 300 
The Resource Allocation block 300 contains all structures and logic 
required to manage up to 8 shared resources between two Disk Array 
controllers, referred to as the master and slave disk array controllers. 
These structures/logic include the Resource Allocator 301, two sets of 
Resource Request Lists (Master/Slave) 303 and 305, two sets of Release 
Resource FIFOs (Master/Slave) 307 and 309, two sets of Resources Granted 
FIFOs (Master/Slave) 311 and 313, and the Resource Scoreboard comprising 
resources allocated and resources available blocks 315 and 317, 
respectively. 
The key element in this block is Resource Allocator 301. This block 
consists of a hardware implementation of an intelligent resource 
allocation algorithm. All other data structures in this block are directly 
controlled and monitored by the Resource Allocator 301. The Resource 
Allocator 301 present in the ICON chip for the master controller 
continually monitors the state of the Resource Request Lists 303 and 305, 
the Release Resource FIFOs 307 and 309, and Resource Scoreboard to 
determine how and when to allocate resources to either controller. The 
Resource Allocator 301 present in the ICON chip for the slave controller 
is not active except during diagnostic testing. 
Controller Functions 400 
The Controller Functions logic 400 provides several board-level logic 
functions in order to increase the level of integration present on the 
disk array controller design. 
Communication Link and Protocol 
This invention encompasses the establishment of a simple communication link 
and protocol between devices sharing resources, and a unique arbitration 
algorithm which is used for the management of the shared resources. 
The communication link and protocol are used to request, grant, and release 
resources to or from the resource arbiter. The protocol requires the 
establishment among the devices sharing resources of a single master 
device, and one or more slave devices. The master/slave distinction is 
used only for the purposes of locating the active resource allocation 
logic 300. Although each controller includes resource allocation logic, 
this logic is only active in the master controller. In the discussion 
which follows, references to the resource allocation logic 300 and its 
components will refer to the active resource allocation logic and its 
components. Both master and slave devices retain their peer to peer 
relationship for system operations. 
The active resource allocator 301 is implemented in the master device. A 
device formulates a resource request by compiling a list of resources that 
are required for a given operation. The resource request is then passed to 
the resource allocation logic 300. The resource allocation logic 300 
maintains a list of requests for each device in the system, seeking to 
satisfy all requests quickly and fairly. Once the allocation logic can 
satisfy a particular request, it signals a grant to the requesting device 
for the resources requested. The device with the granted resource requests 
has access to the granted resources until it releases them. The release is 
then performed by sending a release message to the resource allocator to 
free the resources for consumption by other resource requests. 
All resource requests, request granting, and request freeing involving a 
slave device is performed by sending inter-device messages, which include 
message type and data fields, between the master (where the active 
resource allocation logic is located) and the slave devices using the 
interface described above. All resource requests, request grants, and 
request freeing involving only the master device may be done within the 
local to the master device. 
Shared Resource Management Algorithm 
The resource allocation logic 300 located in the arbitrarily assigned 
master device includes a resource allocation algorithm and associated data 
structures for the management of an arbitrary number of shared resources 
between an arbitrary number of devices. The data structures and algorithm 
for sharing resources are discussed below. 
Data Structures 
For each device which requires shared resource management, a request queue, 
or list of resource requests, of arbitrary depth is maintained by the 
master device (master and slave request lists 303 and 305). Associated 
with each of the device request queues are two count values, a list age 
(which indicates the relative age of a device request queue with respect 
to the other request queues) and a request age (which indicates the 
relative age of the oldest entry in a single device's request queue with 
respect to other entries in the same request queue). In addition to the 
count values associated with each device request queue, two boolean flags 
are also maintained; a Request Stagnation flag and a List Stagnation flag. 
Request Stagnation TRUE indicates that the relative age of a device's 
oldest resource request has exceeded a programmable threshold value. List 
Stagnation TRUE indicates that the relative age of a device's request 
queue with respect to other devices' request queues has exceeded a 
programmable threshold value. Stagnation (Request or List) is mutually 
exclusive between all devices, only one device can be in the Stagnant 
state at any given time. 
The master device also maintains the current state of resource allocation 
and reservation by tracking "Resources Available" and "Resources 
Reserved". "Resources Available" indicates to the resource allocation 
algorithm which resources are not currently in use by any device and are 
not currently reserved for future allocation. Any resources contained 
within the "Resources Available" structure (Resources Available block 317) 
are therefore available for allocation. "Resources Reserved" indicates to 
the resource allocation algorithm which resources have been reserved for 
future allocation due to one of the devices having entered the Stagnant 
state (Request Stagnation or List Stagnation TRUE). Once a device enters 
the Stagnant state, resources included in the stagnant request are placed 
into the "Reserved Resources" structure (Resource Reserved block 315) 
either by immediate removal from the "Resources Available" structure, or 
for resources currently allocated, at the time they are released or 
returned to the resource pool) and kept there until all resources included 
in the stagnant request are available for granting. Stagnation (Request or 
List) is mutually exclusive between all devices; only one device can be in 
the Stagnant state at any given time. The last two data structures used by 
the resource allocation algorithm are pointers to the currently selected 
device (generically termed TURN and LISTSELECT) which is having it's 
resource request queue being searched for a match with available 
resources. 
Algorithm 
Resource allocation fairness is provided using the above-defined data 
structures. The Request Stagnation flag as previously described is used to 
ensure fairness in granting resource requests within a single device. For 
example, assuming random availability of resources, a device which 
requests most resources in groupings of two could starve it's own requests 
for groupings of five resources from the same resource pool unless a 
mechanism for detecting and correcting this situation exists. The request 
age counts with their associated thresholds ensure that resource requests 
within a single device will not be starved or indefinitely blocked. 
The List Stagnation flag is used to ensure fairness in granting resource 
requests between devices. For example, a device which requests resources 
in groupings of two could starve another device in the system requesting 
groupings of five resources from the same resource pool. The list age 
counts with their associated thresholds ensure that all devices' requests 
will be serviced more fairly and that a particular device will not become 
starved waiting for resource requests. 
Two modes of operation are defined for the resource allocation algorithm: 
Normal mode and Stagnant mode. Under Normal mode of operation, no devices 
have entered the Stagnant state and the algorithm uses the TURN pointer in 
a round-robin manner to systematically examine each of the device's 
request queues seeking to grant any resources which it can (based on 
resource availability) with priority within a device request queue based 
on the relative ages of the request entries. Upon transition to the 
Stagnant mode (a device has enter the Stagnant state), the TURN pointer is 
set to the Stagnant device and the resource allocation algorithm will 
favor granting of the request which caused the Stagnant state by reserving 
the resources included in the stagnant request such that no other device 
may be granted those resources. Although the TURN pointer is effectively 
frozen to the Stagnant device, other device request queues and other 
entries within the Stagnant device's request queue will continue to search 
for resource matches based on what is currently available and not reserved 
using the secondary list pointer (LISTSELECT). 
The actual resource grant operation includes the removal of granted 
resources from the "Resources Available" structure along with the clearing 
of "Resources Reserved" structure (if the resource grant was for a 
Stagnant request). Resource freeing or release operations are accomplished 
simply by updating the "Resources Available" structure. 
A Specific Resource Algorithm implementation 
The following is an implementation of the algorithm using the "C" 
programming language for a sample case of a master and a single slave 
device with the following characteristics: 
Resource Request Queue depth for both devices=4 
Number of Shared Resources between the devices=8 
As stated earlier, the number of devices, number of shared resources, and 
queue depth are strictly arbitrary. The functionality contained and 
implied by this algorithm is implemented in the device sharing the 
resources designated the master. The description describes the service 
poll used to look for a resource request to be granted from any 
controller. The release operation is simply provided by allocating the 
resources to be released to the channels available variable. 
Although this example implementation uses the "C" programming language, the 
implementation may take any form, such as other programming languages, 
hardware state machine implementations, etc. 
__________________________________________________________________________ 
void resource.sub.-- allocation.sub.-- algorithm(void) /* begin 
resource 
allocation algorithm */ 
int service.sub.-- loops; 
resource.sub.-- operation *stagnant.sub.-- operation; 
/* while ((q.sub.-- head.sub.-- is.sub.-- not.sub.-- empty(slave.sub.-- 
list)) && (q.sub.-- head.sub.-- is.sub.-- not.sub.-- empty(master.sub.-- 
list))) 
*/ 
for (service.sub.-- loops = 0; service.sub.-- loops &lt; 4: service.sub.-- 
loops++) 
{ 
if (service.sub.-- loops == 0) 
{ 
if ((!master.sub.-- request.sub.-- stagnation) && (!master.sub.-- 
list.sub.-- stagnation) && 
(!slave.sub.-- request.sub.-- stagnation) && (!!slave.sub.-- list.sub.- 
- stagnation)) 
{ 
if (last.sub.-- serviced == MASTER) 
turn = SLAVE; 
else 
turn = MASTER; 
} 
} 
if (turn == MASTER) 
{ 
if (!(q.sub.-- head.sub.-- is.sub.-- not.sub.-- empty(master.sub.-- 
list))) 
{ 
master.sub.-- list.sub.-- age = 0; 
turn = SLAVE; 
continue; 
} 
if ((!master.sub.-- list.sub.-- stagnation) && (!master.sub.-- request.s 
ub.-- stagnation)) 
{ 
if (acquire.sub.-- from.sub.-- master()) 
{ 
if (oldest.sub.-- master.sub.-- serviced) 
{ 
turn = SLAVE; 
master.sub.-- request.sub.-- age = 0; 
} 
else 
{ 
master.sub.-- request.sub.-- age++; 
if (master.sub.-- request.sub.-- age &gt;= request.sub.-- threshold) 
{ 
num.sub.-- master.sub.-- req.sub.-- stagnation++; 
master.sub.-- request.sub.-- stagnation = TRUE; 
} 
else 
{ 
turn = SLAVE; 
} 
} 
/* a request from the master queue was serviced */ 
master.sub.-- list.sub.-- age = 0; 
slave.sub.-- list.sub.-- age++; 
if ((slave.sub.-- list.sub.-- age &gt;= 
list.sub.-- thresho1d)&&(!master.sub.-- request.sub.-- stagnation)) 
{ 
if (q.sub.-- head.sub.-- is.sub.-- not.sub.-- empty(slave.sub.-- 
list)) 
{ 
num.sub.-- slave.sub.-- list.sub.-- stagnation++; 
slave.sub.-- list.sub.-- stagnation = TRUE; 
turn = SLAVE; 
} 
} 
} 
else 
{ 
/* no master queue request was serviced */ 
turn = SLAVE; 
} 
} 
else 
{ 
/* master.sub.-- list.sub.-- stagnation or master.sub.-- request.sub.-- 
stagnation */ 
stagnant.sub.-- operation = (resource.sub.-- operation*)master.sub.-- 
list-&gt;head; 
slave.sub.-- list.sub.-- age = 0; 
if (acquire.sub.-- from.sub.-- master()) 
{ 
if (oldest.sub.-- master.sub.-- serviced) 
{ 
turn = SLAVE; 
master.sub.-- list.sub.-- stagnation = FALSE; 
master.sub.-- request.sub.-- stagnation = FALSE; 
slave.sub.-- list.sub.-- age++; 
master.sub.-- request.sub.-- age = 0; 
master.sub.-- list.sub.-- age = 0; 
} 
} 
else 
{ 
if (acquire.sub.-- from.sub.-- slave()) 
{ 
slave.sub.-- list.sub.-- age = 0; 
if (oldest.sub.-- slave.sub.-- serviced) 
slave.sub.-- request.sub.-- age = 0; 
} 
} 
} 
} 
else 
{ 
/* turn = SLAVE */ 
if (!(q.sub.-- head.sub.-- is.sub.-- not.sub.-- empty(slave.sub.-- 
list))) 
{ 
slave.sub.-- list.sub.-- age = 0; 
turn = MASTER; 
continue; 
} 
if ((!slave.sub.-- list.sub.-- stagnation) && (!slave.sub.-- request.sub 
.-- stagnation)) 
{ 
if (acquire.sub.-- from.sub.-- slave()) 
{ 
if (oldest.sub.-- slave.sub.-- serviced) 
{ 
turn = MASTER; 
slave.sub.-- request.sub.-- age = 0; 
} 
else 
{ 
slave.sub.-- request.sub.-- age++; 
if (slave.sub.-- request.sub.-- age &gt;= request.sub.-- threshold) 
{ 
num.sub.-- slave.sub.-- req.sub.-- stagnation++; 
slave.sub.-- request.sub.-- stagnation = TRUE; 
} 
else 
{ 
turn = MASTER; 
} 
} 
/* a request from the slave queue was serviced */ 
slave.sub.-- list.sub.-- age = 0; 
master.sub.-- list.sub.-- age++; 
if ((master.sub.-- list.sub.-- age &gt;= list.sub.-- threshold) && 
(!slave.sub.-- request.sub.-- stagnation)) 
{ 
if (q.sub.-- head.sub.-- is.sub.-- not.sub.-- empty(master.sub.-- 
list)) 
{ 
num.sub.-- master.sub.-- list.sub.-- stagnation++; 
master.sub.-- list.sub.-- stagnation = TRUE; 
turn = MASTER; 
} 
} 
} 
else 
{ 
/* no slave queue request was serviced */ 
turn = MASTER; 
} 
} 
else 
{ 
/* slave.sub.-- list.sub.-- stagnation or slave.sub.-- request.sub.-- 
stagnation */ 
stagnant.sub.-- operation = (resource.sub.-- operation *)slave.sub.-- 
list-&gt;head; 
master.sub.-- list.sub.-- age = 0; 
if (acquire.sub.-- from.sub.-- slave()) 
{ 
if (oldest.sub.-- slave.sub.-- serviced) 
{ 
turn = MASTER; 
slave.sub.-- list.sub.-- stagnation = FALSE; 
slave.sub.-- request.sub.-- stagnation = FALSE; 
master.sub.-- list.sub.-- age++; 
slave.sub.-- request.sub.-- age = 0; 
slave.sub.-- list.sub.-- age = 0; 
} 
} 
else 
{ 
if (acquire.sub.-- from.sub.-- master()) 
{ 
master.sub.-- list--age = 0; 
if (oldest.sub.-- master.sub.-- serviced) 
master.sub.-- request.sub.-- age = 0; 
} 
} 
} 
} 
} 
} 
/************************************************************ 
***/ 
status acquire.sub.-- from.sub.-- master() 
/************************************************************ 
***/ 
{ 
resource.sub.-- operation *list.sub.-- end, * operation, *first.sub.-- 
operation; 
node *operation.sub.-- node; 
int temp.sub.-- channels.sub.-- available; 
int first.sub.-- op.sub.-- channels.sub.-- available, other.sub.-- 
op.sub.-- channels.sub.-- available; 
oldest.sub.-- master.sub.-- serviced = FALSE; 
if (q.sub.-- head.sub.-- is.sub.-- not.sub.-- empty(master.sub.-- 
list)) 
{ 
list.sub.-- end = (resource.sub.-- operation *)master.sub.-- list; 
first.sub.-- operation = operation = (resource.sub.-- operation 
*)master.sub.-- list-&gt;head; 
first.sub.-- op.sub.-- channels.sub.-- available = channels.sub.-- 
available; 
other.sub.-- op.sub.-- channels.sub.-- available = channels.sub.-- 
available; 
if (master.sub.-- list.sub.-- stagnation .parallel. master.sub.-- 
request.sub.-- stagnation) 
{ 
other.sub.-- op.sub.-- channels.sub.-- available = 
channels.sub.-- available (channels.sub.-- available & 
operation-&gt;channel.sub.-- map); 
} 
if (slave.sub.-- list.sub.-- stagnation .parallel. slave.sub.-- 
request.sub.-- stagnation) 
{ 
first.sub.-- op.sub.-- channels.sub.-- available = 
other.sub.-- op.sub.-- channels.sub.-- available = 
channels.sub.-- available (channels.sub.-- available & 
operation-&gt;channel.sub.-- map); 
} 
do 
{ 
operation.sub.-- node = (node *)operation; 
if (operation == first.sub.-- operation) 
temp.sub.-- channels.sub.-- available = first.sub.-- op.sub.-- 
channels.sub.-- available; 
else 
temp.sub.-- channels.sub.-- available = other.sub.-- op.sub.-- 
channels.sub.-- available; 
if (operation-&gt;channel.sub.-- map == (operation-&gt;channel.sub.-- map & 
temp.sub.-- channels.sub.-- available)) 
{ 
/* channels are available for this operation; grant it */ 
unlink.sub.-- node(operation.sub.-- node); 
link.sub.-- q.sub.-- tail(master.sub.-- granted.sub.-- list, 
operation.sub.-- node); 
channels.sub.-- available = operation-&gt;channel.sub.-- map; 
if (operation == first.sub.-- operation) 
oldest.sub.-- master.sub.-- serviced = TRUE; 
last.sub.-- serviced = MASTER; 
age.sub.-- request.sub.-- age(master.sub.-- list); 
check.sub.-- channel.sub.-- use(); 
return(TRUE); 
} 
operation = (resource.sub.-- operation *)operation.sub.-- node-&gt;next; 
} 
while (operation != list.sub.-- end); 
return(FALSE); 
} 
else 
{ 
return(FALSE); 
} 
} 
/************************************************************ 
***/ 
status acquire.sub.-- from.sub.-- slave() 
/************************************************************ 
***/ 
{ 
resource.sub.-- operation *list.sub.-- end, *operation, *first.sub.-- 
operation; 
node *operation.sub.-- node; 
int temp.sub.-- channels.sub.-- available; 
int first.sub.-- op.sub.-- channels.sub.-- available, other.sub.-- 
op.sub.-- channels.sub.-- available; 
oldest.sub.-- slave.sub.-- serviced = FALSE; 
if (q.sub.-- head.sub.-- is.sub.-- not.sub.-- empty(slave.sub.-- 
list)) 
{ 
list.sub.-- end = (resource.sub.-- operation *)slave.sub.-- list; 
first.sub.-- operation = operation = (resource.sub.-- operation 
*)slave.sub.-- list-&gt;head; 
first.sub.-- op.sub.-- channels.sub.-- available = channels.sub.-- 
available; 
other.sub.-- op.sub.-- channels.sub.-- available = channels.sub.-- 
available; 
if (slave.sub.-- list.sub.-- stagnation .parallel. slave.sub.-- 
request.sub.-- stagnation) 
{ 
other.sub.-- op.sub.-- channels.sub.-- available = 
channels.sub.-- available (channels.sub.-- available & 
operation-&gt;channel.sub.-- map); 
} 
if (master.sub.-- list.sub.-- stagnation .parallel. master.sub.-- 
request.sub.-- stagnation) 
{ 
first.sub.-- op.sub.-- channels.sub.-- available = 
other.sub.-- op.sub.-- channels.sub.-- available = 
channels.sub.-- available (channels.sub.-- available & 
operation-&gt;channel.sub.-- map); 
} 
do 
{ 
operation.sub.-- node = (node *)operation; 
if (operation == first.sub.-- operation) 
temp.sub.-- channels.sub.-- available = first.sub.-- op.sub.-- 
channels.sub.-- available; 
else 
temp.sub.-- channels.sub.-- available = other.sub.-- op.sub.-- 
channels.sub.-- available; 
if (operation-&gt;channel.sub.-- map == (operation-&gt;channel.sub.-- map & 
temp.sub.-- channels.sub.-- available)) 
{ 
/* channels are available for this operation; grant it */ 
unlink.sub.-- node(operation.sub.-- node); 
link.sub.-- q.sub.-- tail(slave.sub.-- granted.sub.-- list, 
operation.sub.-- node); 
channels.sub.-- available = operation-&gt;channel.sub.-- map; 
if (operation == first.sub.-- operation) 
oldest.sub.-- slave.sub.-- serviced = TRUE; 
last.sub.-- serviced = SLAVE; 
age.sub.-- request.sub.-- age(slave.sub.-- list); 
check.sub.-- channel.sub.-- use(); 
return(TRUE); 
} 
operation = (resource.sub.-- operation *)operation.sub.-- node-&gt;next; 
} 
while (operation != list.sub.-- end); 
return(FALSE); 
} 
else 
{ 
return(FALSE); 
} 
} /* end resource allocation algorithm */ 
__________________________________________________________________________ 
Explanations and definitions for terms used in the above algorithm are 
provided below: 
service.sub.-- loops--the number of requests that can be outstanding at any 
one time. 
master.sub.-- request.sub.-- stagnation--the state entered when the master 
ICON chip has serviced the slave icon requests too many times without 
servicing a master ICON's request. (Inter-ICON fairness parameter) 
master.sub.-- list.sub.-- stagnation--the state entered when a request on 
the master ICON's request list is `aged` beyond a configurable threshold 
relative to other requests being serviced in the master ICON's request 
queue. (This is used to promote Intra-ICON list request fairness to ensure 
starvation within the master ICON's list is avoided because of a request 
requiring a large number of resources waiting behind many requests 
requiring only small numbers of resources.) 
slave.sub.-- request.sub.-- stagnation--the state entered when the master 
ICON chip has serviced the master ICON's requests too many times without 
servicing a slave ICON's request. (Inter-ICON fairness parameter) 
slave.sub.-- list.sub.-- stagnation--the state entered when a request on 
the slave ICON's request list is `aged` beyond a configurable threshold 
relative to other requests being serviced in the slave ICON's request 
queue. (This is used to promote Intra-ICON list request fairness to ensure 
starvation within the slave ICON's request list is avoided because of a 
request requiring a large number of resources waiting behind lots of 
requests requiring only small numbers of resources.) 
last.sub.-- serviced--a mechanism for providing fairness in servicing the 
least recently serviced controller. 
turn--indicates which list will be looked at first when servicing requests. 
master.sub.-- list.sub.-- age--the relative age of the request list for the 
master as compared to the number of requests serviced from the slave's 
list. It is used to ensure that the master is serviced at worst case, 
after some number of requests have been serviced from the slave. When the 
master list age exceeds a threshold, the master.sub.-- request.sub.-- 
stagnation state is entered into. 
master.sub.-- request.sub.-- age--the relative age of the oldest member of 
the master list when compared to the number of requests serviced from the 
master's list. It is used to ensure that the oldest request on the 
master's list is serviced at worst case, after some number of other 
requests have been serviced within the master list. When the master 
request age exceeds a threshold, the master.sub.-- list.sub.-- stagnation 
state is entered into. 
slave.sub.-- list.sub.-- age--the relative age of the request list for the 
slave as compared to the number of requests serviced from the master's 
list. It is used to ensure that the slave is serviced at worst case, after 
some number of requests have been serviced from the master. When the slave 
list age exceeds a threshold, the slave.sub.-- request.sub.-- stagnation 
state is entered into. 
slave.sub.-- request.sub.-- age--the relative age of the oldest member of 
the slave list when compared to the number of requests serviced from the 
slave's list. It is used to ensure that the oldest request on the slave's 
list is serviced at worst case, after some number of other requests have 
been serviced within the slave list. When the slave request age exceeds a 
threshold, the slave.sub.-- list.sub.-- stagnation state is entered into. 
The algorithm presented above, together with the description of the 
invention provided earlier, should be readily understood by those skilled 
in the art as providing a method for managing the operations of multiple 
disk array controllers which share access to the disk drive units, busses, 
and other resources within the array. 
Although the presently preferred embodiment of the invention has been 
described, it will be understood that various changes may be made within 
the scope of the appended claims.