Abstract:
A heterogeneous computer system has multiple interconnected cells, each cell has multiple primary processors of the same Instruction Set Architecture (ISA) type, but different cells may have processors of different ISA types. Each cell has a cell type register readable by a processor external to the cell. The cell type register of each cell is used at system startup time to ensure that all processors of a system partition have compatible ISA types.

Description:
FIELD OF THE INVENTION 
     The apparatus and method relate to the field of computer systems and computer system firmware. In particular, they relate to heterogeneous computer systems having dynamically allocated elements 
     BACKGROUND OF THE INVENTION 
     Modern, high performance, computer systems typically have multiple processors. It is known that some computer systems have primary processors of multiple instruction set types, multiple processor systems having primary processors of multiple instruction set architectures (ISAs) are known herein as heterogeneous computer systems. 
     Heterogeneous computer systems offer advantages in that they may run application code written for a variety of processor types and operating systems. 
     In addition to primary processors, upon which operating system and user programs run, there are typically additional embedded processors of additional types. Embedded processors are typically provided for control of specific hardware devices, such as disk drives, in the system. In a computer system, embedded processors may also perform system management functions as monitoring of primary processor voltages and temperatures, control of cooling subsystems, as well as boot-time configuration of various system components. 
     Machine-language operating system code, including low level system code and BIOS (basic input-output system) code, is ISA specific. For example, machine-level code for a PA8800 will not run correctly on an Intel Itanium processor. In a heterogeneous computer system, each low-level operating system code module typically exists in a separately-compiled module for each primary processor type. 
     A family of high performance heterogeneous computer systems from Hewlett-Packard can be configured to use primary processors of two or more ISA types, including the Intel Itanium and PA8800 instruction set architectures. 
     In this family of computer systems, a field replaceable “cell” has several primary processor circuits of the same type, together with memory, circuitry for communicating with other cells over a backplane bus, input output (I/O) bus interface circuitry, JTAG (Joint Test Action Group) scan circuitry, and other circuitry. There may be one or more additional embedded processors in each cell to perform system management functions. 
     One or more cells, which may, but need not, be of the same type, are installed into a backplane. A heterogeneous computer system is formed when cells having two or more types of processors are inserted into the backplane. 
     This family of heterogeneous computer systems supports simultaneous execution of multiple operating systems, including multiprocessor variants of Windows-NT, Unix, VMS, and Linux. Multiple instances of each system are also supported. Each operating system instance operates in a partition of the computer system. 
     At system boot time, a group of processors of a particular type are assigned to operate in each partition. These processors may belong to more than one cell, but must all be of the same ISA. As the operating system instance running in the partition boots, or reinitializes; processors of the partition become aware of each other and appropriate task routing and assignment datastructures built in system memory. The process of processors becoming aware of each other and task routing and assignment datastructures being built in system memory is known herein as a Rendezvous of the processors. 
     It is known that nonvolatile memory circuits having board identification and timing information may be designed into modules of a computer system. Many Synchronous Dynamic Random Access Memory (SDRAM) modules contain serial memory devices having interface timing information recorded therein. Information in these memory devices is used to configure memory interface circuitry of the computer system such that the system will properly communicate with those memory modules actually installed in the system. The Peripheral Component Interconnect (PCI) bus specification provides for machine-readable identification registers within each peripheral device attached to a PCI bus, information read from these identification registers is typically used by an operating system to allocate bus address space and to determine appropriate drivers for each peripheral device. 
     Some prior heterogeneous computer systems have assigned processors to partitions according to the physical location of the processors in the system. In these systems, processors on cells installed in particular slots of the backplane are assigned to one partition, those in other slots are assigned to a second partition. Should cells be moved in the backplane, assignment of processors to partitions based on physical location may result in incompatible processors being assigned to a partition. 
     It is desirable to assign processors to system partitions in a simple, reliable, way. It is desirable to assign processors to partitions in a manner that ensures that each partition includes only compatible processors. 
     SUMMARY OF THE INVENTION 
     A machine-readable identification register is provided on each cell of a heterogeneous computer system. This identification register is read during system startup to identify an instruction set architecture (ISA) associated with each cell. The ISA information is used by the system management subsystem to ensure that mutually compatible processors are assigned to each partition of the system. 
     In a particular embodiment, the identification register is read by, and initial assignment of processors to partitions is performed by, a master processor of each partition. In another embodiment, the identification register is read by, and assignment of processors to partitions is performed by, a system management subsystem. 
     In an embodiment, the identification register is part of a field programmable gate array (FPGA) installed on each cell of the computer system. 
     In another embodiment, the identification register is a serially addressable nonvolatile memory. 
    
    
     
       BRIEF DESCRIPTION OF THE FIGURES 
         FIG. 1  is a block diagram of a heterogeneous computing system; 
         FIG. 2 , a block diagram of a cell for a heterogeneous computing system; 
         FIG. 3 , a block diagram of an alternate embodiment of a cell for a heterogeneous computing system. 
         FIG. 4 , a flowchart of processor allocation by the management processor at boot time; and 
         FIG. 5 , a flowchart of processor compatibility verification during partition rendezvous. 
     
    
    
     DETAILED DESCRIPTION OF THE EMBODIMENTS 
     A heterogeneous computer system  100  ( FIG. 1 ) has a system management processor  102 , and two or more processor cells  104 ,  106 , and  108 . Processor cells  104 ,  106 , and  108  are of two or more ISA types.  FIG. 2  is a detailed block diagram of a cell, such as cells  104 ,  106 , and  108  of  FIG. 1 . With reference to  FIG. 1  and  FIG. 2 , processor cells  104 ,  106 , and  108  each have primary processors  202 . Processor cells  104  embody primary processors of a first ISA type, while processor cells  106  embody processors of a second ISA type. In a particular embodiment, processor cells  104  embody processors  202  of the PA8800 type, while processor cells  106  embody processors  202  of the Intel Itanium type. Additional processor cells  108  may exist in the system  100 , having additional types of processors, including in an embodiment processors of earlier Intel ISA types. In an embodiment, each cell has four primary processors, in another embodiment each cell has sixteen processors. 
     Cells of the system  100  are interconnected through high-speed interconnect  110 . High-speed interconnect  110  provides for communications between cells. Some cells of the system  100  may also be coupled to I/O (Input/Output) interconnect  112 . I/O interconnect  112  provides a path for communication between cells of the system, such as cell  108 , and I/O devices  114 . I/O devices  114  may include disk drives and network interface devices, as well as other peripherals. 
     Cells  104 ,  106 , and  108  of the system  100  are connected to management processor  102  over a management interconnect  116 . Management processor  102  is also coupled to control power supplies and fans  118 . 
     In addition to primary processors  202 , each cell ( FIG. 2 ) also has a memory system  204 , and a high-speed interconnect interface device  206 . In a particular embodiment, high speed interconnect interface device  206  includes crossbar switching circuits, bus bridging circuits, and memory control circuits. In an embodiment, high speed interconnect interface device  206  includes a protection domain register  207  for specifying a protection domain to which the cell is assigned. High speed interconnect interface device  206  interfaces processor busses, such as processor bus  208 , from primary processors  202  to memory system  204 , to at least one high-speed system interconnect  110 , and I/O interconnect  112 . 
     Memory  204  of each cell is accessible from processors  202  of that cell, and from high speed interconnect  110 . The high speed interconnect interface device  206  is capable of using protection domain register  207  to limit access to memory  204  by high speed interconnect  110  to references originating at other cells assigned to the protection domain specified in protection domain register  207 . 
     A portion of system memory, which in an embodiment is memory  120  separate from the cells, may be configured as Globally Shared Memory (GSM). In an alternative embodiment, a portion of memory  204  of one or more cells is configurable to serve as GSM memory. GSM memory  120  includes Coherency Set (CS) registers  122 . The CS registers include protection domain information whereby protection domains may be specified for each region of GSM memory. References to regions of GSM memory are rejected if they originate in processors of a protection domain other than those specifically permitted to access the region according to associated CS registers. 
     In one embodiment, primary processors  202  of each cell each are large integrated circuits each having multiple CPUs (Central Processor Units) together with multiple levels of cache memory. In one version of this embodiment, each processor  202  has four CPUs. It is anticipated that the number of effective CPUs per processor  202  may be greater than four. 
     Each cell also has a small management subprocessor  210 , which in one embodiment is a microcontroller of the Intel 80251 type. It is anticipated that management subprocessor  210  may be a microcontroller of the Intel 8096, Motorola 6811 or 6805 type, or of another type as known in the art. Management subprocessor  210  is adapted for communication over management interconnect  116 . In a particular embodiment, management subprocessor  210  controls cell-level cooling devices  212 , and is capable of monitoring temperatures of the cells primary processors  202 . 
     In a particular embodiment, management subprocessor  210  communicates to cooling devices  212  and other devices (not shown) through an FPGA (Field Programmable Gate Array)  214 . In this embodiment, FPGA  214  includes a cell type register  216 . In an alternative embodiment, adapatable to embodiments wherein management subprocessor  210  connects with cooling devices  210  without an FPGA  214 , cell type register  218  is incorporated into firmware code of management subprocessor  210 . In another embodiment, cell type register contents is readable to primary processors  202  of the cell. 
     In an alternative embodiment, as illustrated in  FIG. 3 , there is a separate processor type register  254  associated with each processor integrated circuit of each cell; each processor integrated circuit includes one or more processor of primary processors  252 . This arrangement is particularly adapted to embodiments having processors mounted on daughter cards, or for embodiments where the processor type register  254  is implemented within each processor integrated circuit. 
     As with the embodiment of  FIG. 2 , the embodiment of  FIG. 4  also has a high speed interconnect interface  256 , memory  258 , a protection domain register  260 , a management processor  262 , an FPGA  264 , and cooling devices  266 . 
     At system boot time, a particular primary processor  202  of primary processors  202  reads  302  a cell type register  302  from each cell  104 ,  106 ,  108 . In one embodiment, a read request is transferred over management interconnect  116  to the cell. This read request is answered by management subprocessor  210  of the cell with information read from the cell type register  216  or  218 . Management processor  102  uses the cell type information to determine  304  an ISA type of the processors of each cell. In one embodiment, determining an ISA type of each cell is performed by extracting an ISA type field from the cell type  302 . Cells are then allocated  306  to partitions according to desired system configuration information and in such manner that the management processor ensures  308  that all processors of each partition have compatible ISAs. 
     In an alternative embodiment, at system boot time a particular primary processor  252  (which becomes a master processor of the cell) of primary processors  252  or  252  reads  302  a cell type register  254  associated with each processor  252  of the cell. The master processor of the processors  252  of the cell uses the processor type information to determine  304  an ISA type of the processors of each cell. If the processors of the cell are not all compatible with the ISA type of the master, processor, an error message is generated and those processors having ISA type differing from the master processor ISA type are disabled. Cells are then allocated  306  to partitions according to desired system configuration information and in such manner that all processors of each partition have compatible ISAs. 
     Partitions are then setup  310  such that each processor of each partition can correctly rendezvous  312  with processors of the partition as each operating system boots. 
     Setup  310  of partitions includes configuring the protection domain registers  307  of each cell to contain a protection domain code associated with the partition to which processors of that cell are assigned. Region of GSM memory are also assigned to each partition, and to each group of partitions permitted to communicate with each other. Setup  310  of partitions also includes configuration of CS registers of each assigned region of GSM memory to prevent unauthorized access of each region. 
     Once processors are assigned to partitions, processors of those partitions are permitted to rendezvous. 
     During rendezvous, a master processor of each partition polis  402  ( FIG. 5 ) other cells of the partition to determine the cell type of that partition, and to determine  404  the ISA of each processor of the partition. These determined ISAs are compared with the cell type of the master processor to ensure  406  that all processor ISAs of a partition match. In the event that processor ISAs do not match, an error is declared and operation of incompatible processors of the partition is suspended. Operating system boot continues with the remaining, compatible, processors. 
     While the forgoing has been particularly shown and described with reference to particular embodiments thereof, it will be understood by those skilled in the art that various other changes in the form and details may be made without departing from the spirit and hereof. It is to be understood that various changes may be made in adapting the description to different embodiments without departing from the broader concepts disclosed herein and comprehended by the claims that follow.