ADDRESS TRANSFORMATION IN A CLUSTER COMPUTER SYSTEM

Abstract: To integrate a plurality of processors (11-14), each capable of directly addressing a limited internal storage range, with a large external memory, processors are organized into clusters, each having a plurality of processors and a common secondary cache (7). An address translator (18) is provided to transform internal main memory space (8) addresses to external memory space addresses. External memory space is divided into private and shared areas. An internal address indicator bit, in conjuction with the cluster number form a requesting processor primary cache, is employed to set up the transformation either to the private external space of the cluster or the shared external space. In reverse external-to-internal transformation, a pair of indicator bits are used to generate the internal address and define the shared and private space of the designated cluster. A cluster member number assigned to each processor is used by the secondary cache to track with processor sends/receives information to/from the external memory.

Description: ADDRESS TRANSFORMATION IN A CLUSTER COMPUTER SYSTEM Field of the Invention This invention i elates to the art of data processing and, more particularly, to addressing in a cluster computer system employing multiple clusters, each cluster having multiple processors, each processor being incapable of directly addressing all inforniation stored in the total mam memory' BackmOiind of the Invention In the design of central processing units (CPUs) used in modem computing systems, improvements must take into account the command stnictuie and the woid length w hich often must remain the same as used in the predecessors to a "new" CPU This is particularly true in the case of systems running under proprietary software operating systems The fundamental reason for this constraint is the often enormous investment in software which a user may have In some instances, there may be represented three decades of software development in a given user system, and a user typically wishes to merely load its old software on a new, improved hardware system and experience immediate higher performance j results As a result, CPU designers go to great lengths to maintain 2 compatibility with earlier CPUs in a given operatmg system family. 3 Thus, m CPU design based on existing systems, even limited 4 modification of the address range capability (I e., allocation of address fields 5 m a CPU instruction word) of each CPU would have a major impact on 6 hardware design and, more especially, on the software which can or must be 7 reused This dilemma is faced in view of the ongoing increase in density and 8 capacity of memory chips and other technologies that have resulted in the 9 availability of mass memories capable of storing information which is one to io two times greater than was contemplated at the time the address field i i lengths/word lengths were oiiginally established m the operating system/CPU 12 family 13 Another factor involved in the design of CPUs, particularly CPUs 14 intended to function under a long-established proprietary operating system, is 15 the fact that ever-new technology allows an existing CPU design to be 16 physically implemented into a much smaller space. However, again, the 17 advantages of adhering to the fundamental specifications of existing operating 18 system software (in order to permit the reuse of existing application software) 19 directs the retention of the operating system, notwithstanding its limiting 1 effects on CPU design. Finally, and perhaps most importantly, those skilled 2 in the art are aware of inefficiencies which occur when too many CPUs are 3 operated in a single operating system environment 4 All these factor are aggravated in a multiprocessor cluster environment 5 in which multiple groups of multiple CPUs are incoφorated into a large 6 computer system which includes a large main memory containing stored 7 information at more addresses than can be directly addressed by the 8 individual CPUs, with the addressing constraint being imposed because of the 9 address field/word length characteristics which are the natural heritage of the io histoπcal hardware and operating system/application software of a given π computer system family, lt is to the solution of this dilemma that the subject 12 invention is directed 13 Objects of the Invention 14 It is therefore a broad object of this invention to provide an improved 15 multi-processor computer system incoφorating extended memory addressing 16 with total system memory beyond that seen by a single CPU or operating π system. 18 It is a more specific object of this invention to provide an improved 19 multiprocessor system employing a cluster architecture in which an external memory space, larger than that inherently directly addressable by the processors employed, is divided into private spaces for each multiprocessor cluster and shared spaces directly accessible by all the clusters according to an address transformation which incoφorates cluster member number information for identifying each clustei and each member thereof. Summary of the Invention Briefly, these and other objects of the invention are achieved, in an exemplary system which includes an external memory space (e.g., mam memory) a power of two larger than the internal memory space which is inherently directly addressable by an address field generated/inteφreted by the individual processors, by dividing the pi ocessors into multiple clusters with multiple processors in each. Each CPU in a cluster of processors, four on a single multiprocessor board in the example, is assigned a cluster number which is set, for example, by a two-bit number (thus identifying four clusters). In addition, each CPU is assigned a two-bit number (thus identifying four CPUs) setting which CPU it is m its cluster. In the example, the two two-bit numbers are concatenated to provide each CPU with a unique four-bit identification niunber in the system 1 In an internal-to-external address transformation mode, the cluster 2 number is sent, along with an internal address which includes an indicator bit 3 specifying private/shared classification, from a requesting cluster to an 4 address translator, ln the address translator, the indicator bit is used to 5 control the set up of the transformation 6 In the exemplaiy system in which each CPU can inherently directly 7 address one gigabyte (30 bit address field identified as bits 0-29 with bit 0 8 being the most significant), the available extemal space is two gigabytes and 9 the length of the extemal space address field is 32 bits (bits 0-31 , capable of io specifying four gigabytes), bit 0 of the intemal address is used as the i i indicator. If bit 0 is a "0", the cluster membei is requesting inforniation from 12 its private area of extemal space; conversely, if it is a " 1 ", the cluster member 13 is addressing information in the shared regions of extemal space. 14 If the indicator bit is a "0", the cluster number is copied into bits 1 , 2 of 15 the extemal address, bits 1 , 2 of the internal address are copied into bits 3, 4 16 of the extemal address and the lower order address bits 3-29 of the internal 17 address are copied directly to bits 5-31 of the extemal address. The resulting 18 extemal address points to the specified private region m exte al space. 1 If, however, the indicator bit 0 in the internal address is a "1" 2 indicating a call to a shared region of external space, then a different 3 transformation takes place. CNO, CN I are ignored. Bit I of the internal 4 address is copied to bit 1 of the extemal address while bits 2, 3 ofthe extemal 5 address are each forced to logic "1 ". The resulting extemal address points to 6 the specified shared region in extemal space. 7 In an external-to-internal address transformation mode, bits 1 , 2 of the 8 external address are copied into positions CNO, CNI of the intemal address 9 The external address indicator bits are bits 3, 4. If one or the other or both io are "0", then an address from the pπvate space of one of the clusters is i i indicated. Bit 0 of the intemal address is forced to "0", and bits 3, 4 of the 12 extemal address are copied to bits 1 , of the intemal address. The lower order 13 bits 5 - 31 of the external address are copied directly to bits 3 - 29 of the 14 intemal address. Thus, the developed internal address fully identifies an 15 inforniation block in external memory as that of the pπvate space reserved to 16 the cluster identified by CNO, CN 1 . 17 If both bits 3, 4 of the extemal address are "1 ", then the extemal space 18 is known to be shared. Bit 0 of the intemal address is forced to "1 ", bits 1 , 2 19 of the exte al address are copied to bits 1 , 2 of the internal address, and the 1 lower order address bits are copied as before. The resulting intemal address 2 identifies an information block in external memory that is available to all the 3 clusters. 4 Description of the Drawing 5 The subject matter of the invention is particularly pointed out and 6 distinctly claimed in the concluding portion of the specification. The 7 invention, however, both as to organization and method of operation, may 8 best be understood by reference to the following descπption taken in 9 conjimction with the subjoined claims and the accompanying drawing of io which: i i FIG. 1 is a high level block diagram of a cluster computer system in 12 which the present invention may be practiced; 13 FIG. 2 is a block diagram of the system shown in FIG. 1 illustrating 14 more detail of the individual clusters and CPUs; 15 FIG. 3 is an exemplary representation of the allocation of intemal 16 address space as allocated to the individual multi-processor clusters and 17 extemal space representing the address space of a main memory device; 18 FIG. 4 is an exemplary transformation diagram illustrating an 19 embodiment of the invention in which each CPU is inherently capable of 1 addressing one gigabyte of mfoπiiation, and a ma memory device in 2 communication with all the CPUs is capable of stoπng two gigabytes of 3 inforniation, 4 FIG 5 is a representation of the two types of addresses, internal and 5 extemal, which are transfoπned according to the mvention, 6 FIG 6 is a simplified logic diagram illustrating the apparatus within an 7 address translator component of each CPU operatmg in an internal-to-external 8 mode, and 9 FIG 7 is a simplified logic diagram of an address translator operating io in an exteπial-to-intemal mode i i Description of the Preferred Embodiment s) 12 Referπng first to the exemplary cluster computer system of FIG 1 , it 13 will be observed that the system includes first 1 , second 2, third 3 and fourth 14 4 multiprocessor circuit boards, each multiprocessor board including four 15 CPUs effecting a cluster, each multipiocessor board bemg connected to a 16 system bus 5 via an integral secondary cache 7 The system bus 5 couples 17 the multiprocessor boards 1 , 2, 3, 4 to one another and to various other 18 system components (including a service processor "SP") represented by the 19 block 6 In addition, system bus 5 couples the multiprocessor boards 1 , 2, 3, 4 to a memory controller 10 which, in turn, interfaces via a memory bus 9 with a main memory 8 representative of one or more main memory devices. As illustrated for multiprocessor board 1 , each multiprocessor board 1 , 2, 3, 4 includes four CPUs 1 1 , 12, 13, 14. The four CPUs 1 1 , 12, 13, 14 are provided with respective primary caches 1 1 C, 12C, 13C, 14C. In the exemplary system, the primary caches on each multiprocessor board are each disposed between its directly associated CPU and a common secondary cache 7 and utilize a the classical "store-into-cache" configuration. The secondary cache 7 interfaces with the system bus 5. Inteφosed intennediate each of the primary caches 1 1 C, 12C, 13C, 14C and the common (to the cluster) secondary cache 7 are address translators 1 1 A, 12A, 13 A, 14A which are also constituents of their respective processors 1 1 , 12, 13, 14. In accordance with a preferred aspect of the invention, each of the multiprocessor boards 1 , 2, 3, 4 constitutes a cluster, and each cluster mns under its own operating system to improve system efficiency. Attention is now directed to FIG. 2 which illustrates the environment of the subject invention in more detail. Thus, it will be seen that CPU 1 1 on multiprocessor board 1 (board "0") communicates with the system bus 5 via an address register 15, address translator 18 and secondary cache 7. Register 1 15 is merely a convenient representation of address interface circuitry in the 2 primary cache of the CPU 1 1 by wliich an address generated by the CPU 1 1 3 may be transmitted, transformed in the address translator 18, as a request to 4 the secondary cache 7. This condition occurs when information required by 5 the CPU 1 1 is not resident in at least one of the primary caches of the CPUs 6 1 1 , 12, 13, 14 on the multiprocessor board 1 . (Those skilled in the art will 7 understand that, in many such multiprocessor configurations, it is possible for 8 one CPU to "siphon" information from another CPU's primary cache.) 9 Corresponding CPUs 2 1 , 22, 23, 24 are found on multiprocessor board io 2 (board "I "). Similarly, corresponding CPUs 31 , 32, 33, 34 and 41 , 42, 43, i i 44 are found on multiprocessor boards 3 (board " 2"), 4 (board "3"), 12 respectively. In addition, registers 25, 35, 45, corresponding to register 15, 13 are respectively found on multiprocessor boards 2, 3, 4. Also, address 14 translators 28, 38, 48, corresponding to address translator 18, are respectively 15 found CPUO on each of multiprocessor boards 2, 3, 4. 16 Also shown incoφorated in the primary cache of CPU 1 1 on the 17 multiprocessor board 1 are first and second two-bit identification registers, 18 16, 17, respectively. (It will be understood that the registers 16, 17 are 1 representative of any convenient means for establishing a two-bit cluster identification for the multiprocessor board 1 and a two-bit cluster member identification for the CPU 1 1. For example, a cluster/cluster member identification could be hard-wired in or exist as four switches which can be set to a complete cluster member number ) In the case of CPU 1 1 in multiprocessor 1 , the four stages of the registers 16, 17 are each set to the logic "0" position, thereby providing a cluster identification of "00" (board "0") and a cluster member identification of "00" (CPU "0"). As a practical matter, the registers 16, 17 may be concatenated and may be set upon system initialization under control of the service processor SP as represented by the dashed line 20. Corresponding register pairs, 26, 27; 36, 37; and 46, 47 are provided on multiprocessor boards 2, 3, 4, for respective CPUOs 21 , 31 , 41 and are respectively set to provide cluster identifications of "01 ", "10" and " 1 1 " and, m each case, cluster member identification of "00" In the succeeding discussion, the descπption will be directed to multiprocessor board 1 (board "0"), it being understood that multiprocessor boards 2, 3, 4 operate identically, taking into account their diffeπng cluster number identifications as stored in the registers 26, 36, 46. 1 e., the cluster member number for CPU2 33 on multiprocessor board 3 is "1010" (board "2'7CPU "2"). The 30-bit register 15 in the pnmary cache of the CPU 1 1 can specify one gigabyte of address locations which is the family limit of the exemplary CPUs. There is value in reserving a portion of the addressable space in main memory 8 to the private use of the CPUs on each multiprocessor board for, among other reasons, permitting each multiprocessor board to operate under an independent operating system. (These independent operating systems may be the same or different operating systems.) On the other hand, there is a need for establishing direct communication among the clusters and CPUs operating under the different operating systems in order that they can function with the full power of a cluster system architecture. One way in which this feature can be achieved is by providing shared memory space in main memory 8. The present invention provides for the enjoyment of all these features while also peπnitting the collective direct and indirect address of more memory spaces in main memory than the CPUs in the system are individually inherently capable. In the followmg discussion, the temi "internal space" means the memory addressable by the CPUs on each multiprocessor board while the teπn "external space" means the total memory addressable, I e , the memory space available in mam memory 8 These two types of spaces exist at two interfaces of, and are reconciled by, the address translator 18 which, in the exemplary system, is disposed in the primary cache of each CPU The concept of internal and extemal spaces and pπvate and shared memory in the present context may be better understood with reference to FIG 3 In this example, it may be assumed that each CPU has a 30-bit address field and can therefoie directly address one gigabyte It may further be assumed that the available external space in the exemplary main memory is two gigabytes and that the exte al address field is 32-bιts long, l e , capable of addressing four gigabytes A key aspect of the present invention is the manner m which addiessable memory for each multiprocessor board is allocated For the extemal memory space to be accessible, as a result of address transformation, by each multiprocessor board 1 , 2, 3, 4, three-eighths of the inherently addressable internal space is pπvate and one-half is shared Comment on the apparently remaining one-eighth is reserved for discussion below 1 Under these conditions, the memory allocation may be appreciated 2 from an examination of FIG. 3. Multiprocessor board 1 enjoys 3/8th gigabyte 3 private memory space identified as 1 /8th gigabyte segments Pl l , P12, P13. 4 In addition, Multiprocessor board 1 has access to 1/2 gigabyte shared 5 memory space in l /8th gigabyte segments SA, SB, SC, SD. Similarly, 6 Multiprocessor board 2 enjoys 3/8th gigabyte private memory space 7 identified as l /8th gigabyte segments P21 , P22, P23. In addition, 8 Multiprocessor board 2 has access to 1/2 gigabyte shared memory space in 9 l/8th gigabyte segments SA, SB, SC, SD The internal space for io multiprocessor boards 3 and 4 are correspondingly assigned as shown in FIG. i i 3. 12 This intemal space address inforniation is provided to the address 13 translation block which, as previously described, is present in the primary 14 cache of each CPU, but is shown separately as a single block in FIG. 3 for 15 clarity. In the address translation block, this address information is 16 transformable, according to the invention, to obtain exte al space address 17 segments distributed as shown schematically in FIG. 3. 18 Consider now an implementation of this example with reference to 19 FIGs. 2, 4, 5, 6 and 7. FIG 4 shows the higher order address bit distribution for the four multiprocessor boards for both the internal and extemal space addresses as they are transfoπned by the address translation block (e.g., the address translator 18 in FIG. 2). In each case, there are three columns representing, respectively, the cluster number (already set by the registers 16, 26, 36, 46; FIG. 2), the three most significant bits of the 30-bit internal address and the second, third, fourth and fifth most significant bits of the 32- bit transfonned address. As will be discussed in more detail below, the most significant bit of the 32-bit transformed address is always a "0" in the example. Referring also to FIG. 5 and examining first the intemal address pattem for multiprocessor board 1 , the cluster number is preset to "00", the private space three most significant bits are "000", "001 " and "010" and the shared space three most significant bits are " 100", " 101 ", " 1 10" and " 1 1 1 ". It will be noted that, in this example, the combination "01 1 " for the three most significant bits is reserved. The three highest order internal address bits may be deemed the higher order segment of the processor address field and the remaining bits 3-29 may be deemed the lower order segment thereof. Because the address field for external addresses has 32-bits and the main memory stores only two gigabytes, the most significant bit in the external addresses is always "0" in the example, (lt may be noted, however, that this most significant bit is available to denote one of two complete systems which is to be granted access to a shared main memory.) Thus, the corresponding addresses, transfomied into the five most significant bits of the private external space addresses to be transformed (i.e., bits 1 , 2, 3, 4), are, respectively, "OOOOO", "00001 ", "00010" (oi Pl 1 , P12, P13) and those for the shared external space addresses are, respectively, "0001 1 ", "00011" , "0101 1 " and "01 1 1 1 " (or SA, SB, SC, SD). The five highest order internal address bits may be deemed the higher order segment of the memory address field and the remaining bits 5-31 may be deemed the lower order segment thereof. For multiprocessor board 2, the cluster number is preset to "01", the private space three most significant bits are, again, "000", "001" and "010" and the shared space three most significant bits are, again, "100", "101 ", "1 10" and " 1 1 1 ". (Except for the cluster number, there is no difference, and this is also true for multiprocessor boards 3, 4 which have respective cluster numbers of " 10" and 1 1 ".) The conesponding addresses, transfomied into bits 1 , 2, 3, 4 of the private external space addresses are, respectively, "00100", "00101 ", "001 10" (or P21 , P22, P23) and those for the shared 1 external space addresses aie, respectively, "OOOl 1 ", "00011", "0101 1 " and 2 "01 1 1 1 " (or, again, SA, SB, SC, SD) 3 Similarly, tlie intemal addresses of multiprocessor board 3 and 4 are 4 respectively transformed to extemal addresses "01000" (P31), "01001" 5 (P32), "01010" (P33) "00011" (SA), "0001 1" (SB), "01011" (SC), "01111" 6 (SD) and C '01100"(P41), "01101" (P42), "OH IO" (P43) "00011" (SA), 7 "00011" (SB), "0101 1 " (SC), "01111" (SD). 8 The basis of this transfoπnation, which is different for the pπvate and 9 shared addresses, is not readily evident and requires fiirther explanation io First, it will be noted that the most significant bit of each internal pπvate i i address is "0" while the most significant bit of each internal shared address is 12 "1 " In the case of the pπvate addresses, the CN of each microprocessor 13 board (cluster) is utilized as bits I and 2 of the extemal address Bit 1 , 14 known to be a "0", is not used directly in the transfoπnation Bits 2 and 3 of 15 the internal address are. respectively, used as bits 3 and 4 of the extemal 16 address 17 In the case of the shared addresses, bit 1 of the internal address, known 18 to be a "1", is not used directly in the transfoπnation Bits 1 and 2 of the internal address are used as bits 1 and 2 of the external address. Finally, bits 3 and 4 of the external address are set to "1 1". In the discussion above, the term "used as" should only be understood as representative of the mteπ al-to-extemal transformation process. The external-to-intemal transfoπnation, of course, reverses the process. It will now be understood why the "01 1 " internal address is not used with this particular addressing scheme; it would result in duplicate extemal address ranges for private and shared space accordmg to the transformation process. Exemplary transfomiation apparatus for the address translator 18 of CPUO 1 1 is shown in FIGs. 6 and 7. FIG 6 illustrates this apparatus operating in the intemal-to-extemal transformation mode in which an intemal address, includmg a cluster number, is sent to the address translator 18 as a request from the pπmaiy cache directed to the secondary cache 7 (which, of course, will forward the request to main memory if the requested information is not resident in the secondary cache at the time of the request). The intemal address is thus present in register 15 to serve as an input to die address translator in conjunction with the cluster number m register 16. 1 For this transfoπnation, bit 0 of the internal address is key. If it is a 2 "0", the address is identified as pπvate to the requesting multiprocessor board 3 which itself is identified by CNO, CN I in register 16. Bit 0 is inverted to a 4 "1 " by inverter 55, and the output of inverter 55 therefore enables one input 5 each of AND-gates 61 , 63, 65, 67 The other input to AND-gate 61 is CNO 6 which is therefore copied through OR-gate 56 into bit 1 of the external 7 address in exemplary address translator output register 54. The other input to 8 AND-gate 63 is CNI which is therefore copied through OR-gate 67 into bit 2 9 of the extemal address. The other input to AND-gate 65 is bit 1 of the io intemal address which is therefore copied, via OR-gate 58, to bit 3 of the i i extemal address in register 54. Similarly, the other input to AND-gate 67 is 12 bit 2 of the intemal address which is therefore copied, via OR-gate 59, to bit 13 4 of the extemal address in register 54. The lower order address bits 3 - 29 14 of the internal address in register 18 are directly copied to bits 5 - 31 of the 15 exte al address in register 54. Bit 0 of the extemal address in register 54 is 16 forced to logic "0" because extemal space is limited, in the example, to two 17 gigabytes. As previously mentioned, this bit is available to accommodate two 18 separate systems having access to the main memory. i Assuming by way of example that the address bemg transformed has 2 been supplied by the primary cache of CPUO 1 1 , i e., CNO, CN I are set to 3 "00" and that bits 0,1 ,2 of the internal address are "010" In this example, 4 the extemal space addressed after the transfoπnation by the higher order 5 address bits "00010" will be the 1 /8 gigabyte pπvate block identified FIG 6 3 as P13 7 In another example, assume that bit 0 of an internal address sent to the 8 address translator 18 is a logic "1 " indicating a call to shared space. Since bit 0, m this instance, will be inverted to a logic "0" by the inverter 55, the AND- 10 gates 61 , 63, 65, 67 will be disabled Instead, AND-gates 60, 62, 64, 66 will i i each have one input enabled as a result of the logic "1 " present in bit 0 of 12 register 53 The other input to AND-gate 60 comes from bit 1 of register 53 13 such that the logic level of bit 1 in the internal address is copied, via OR-gate 14 56, mto bit 1 of the extemal address m register 54 The other input to AND- 15 gate 62 comes from bit 2 of register 53 such that the logic level of bit 2 in the 16 intemal address is copied, via OR-gate 57, into bit 2 ofthe external address in 17 register 54. 18 The other inputs to each of the AND-gates 64, 66 are directly tied to 19 logic "1 " such that bits 3 and 4 of the external address in register 54 are set to "1 " via OR-gates 58, 59, respectively Again, bit 0 of the extemal address in register 54 is forced to "0" because of the two gigabyte limitation on extemal space m the example. Lower order address bits 3 - 29 of the internal address are directly copied into bits 5 - 31 of the external address as before. Assume now that bits 0, 1 ,2 of the internal address supplied by the multiprocessor board 1 are " 1 1 1 " In this example, the extemal space addressed after the tran sfonnat ion by the higher order address bits "011 1 1" will be the 1/8 gigabyte shared block identified in FIG. 3 as SD. In the example, the CPU on a given multiprocessor board making a request to the secondary cache 7 is conventionally tracked by the secondary cache. Thus, the contents of the register 17 containing the cluster member number is sent to the secondary cache for that puφose. FIG. 7 illustrates the address translator 18 operating in the extemal-to- intemal address transformation mode An extemal address from the secondary cache is placed in register 68, representative of any appropriate logic for the puφose. Bit 0 of the external address is not used because of the two gigabyte capacity of the main memory. Bits 1 and 2 are applied to corresponding inputs to multiplexer pair 84, 85 which selectively drive inputs to register 86 which is representative of any appropriate logic for interfacing 1 the primary cache with the address translator, ln addition, bits 1 and 2 in the 2 register 68 are applied as respective inputs to comparators 81 , 82 which have 3 CNO and CNI in register 16 as their respective second inputs. If there is a 4 match in both comparators 81 , 82, this state confirms that cluster number 5 "00" is the destination of the external address identified as private space 6 reserved for one of mirltiprocessui board 1 (board "0") as determined by the 7 status of bits 3 and 4 of the external address; i.e., if one or both of bits 3 and 8 4 are logic "0". With both comparators 81 , 82 indicatmg a compare state, 9 two of three inputs to AND-gate 83 are enabled. The third input to AND- 10 gate 83 is driven by the output fiom AND-gate 89 which, in turn, is driven by i i the two outputs from comparators 87, 88. First respective inputs to the 12 comparators 87, 88 are CMO, C 1 from register 17 while the second 13 respective inputs thereto are the cluster member number received from the 14 secondary cache which has conventionally previously recorded that CPUO in 15 the local cluster made the original request for access to the memory block 16 identified by the external address. Thus, if there is a compare condition all 17 the comparators 81 , 82, 87, 88, AND-gate 83 is fully enabled to enable the 18 multiplexer 84 to transfer the transfoπned extemal-to-intemal address to 19 register 86. Since private space is indicated in the example, one or the other of bits 3 and 4 in register 68 are "0", and AND-gate 70 is disabled such that its output is "0" and the output of inverter 71 is "1 ". This condition enables AND-gate 72 to transfer a logic "0", present at one of its inputs, through OR- gate 78, through multiplexer 84 and into bit 0 of register 86 Similarly, the output of inverter 71 enables AND-gate 74 to transfer bit 3 in register 68 through OR-gate 79, multiplexer 84 and into bit 1 of register 86. The output of inverter 71 also enables AND-gate 76 to transfer bit 4 in register 68 through OR-gate 80, multiplexer 84 and into bit 2 of register 86. Finally, bits 5-31 in register 68 are transfened through multiplexer 84 to bits 3-29 register 86 to complete the extemal-to-intemal transfoπnation of the address infoπnation and the selection of the correct CPU in the correct cluster to receive requested information in the private storage area of the mass memory. If both bits 3 and 4 of the external address are "1 ", a shared address is indicated, and AND-gate 70 is fully enabled The resulting "1 " at the output of AND-gate 70 enables one input each of AND-gates 73, 75, 77, 90 Since the indication is that the infoπnation is not private, there is no need to consider the cluster number. Thus, if the cluster member number from the secondary cache identifies CPUO as previously described, the output from i AND-gate 89 fully enables lhe AND-gate 90 to enable multiplexer 85 to 2 transfer inputs thereto into the register 86. 3 Because the other input to AND-gate 73 is driven by a logic "1 ", a "1 " 4 is copied into bit 0 of register 86, via OR-gate 78 and multiplexer 85, to 5 provide the internal address indicator for shared space. The other inputs to 6 AND-gates 75, 76 are, respectively, from bits 1 , 2 in register 68. As a result, 7 the contents of bits 1 , 2 in register 68 are copied into bits 1 , 2 of register 86 8 via OR-gates 79, 80, respectively, and multiplexer 85. Bits 5-31 of tl e 9 extemal address in register 68 are copied through multiplexer 85 into bits 3- 10 29 of the internal address in register 86 as the lower order address bits i i common to the intemal and external addresses. 12 lt will now be understood, in the exemplary embodiment of the 13 invention shown in FIGs. 6 and 7, that each multiprocessor board (cluster) 14 loses the capacity to address 1/8 gigabyte of the one gigabyte inherently is addressable by a 30 bit address field. However, what is gained is the ability 16 to directly communicate among all the multiprocessor boards via the shared 17 space. Further, there is achieved the ability of any multiprocessor board to 18 indirectly address the private space of another multiprocessor board by 19 sending a suitable request via shared space which results in the desired 1 inforniation being moved from pnvate space mto shared space In addition, 2 the previously discussed advantage of permitting each multiprocessor board 3 to operate under its own operating system is achieved Still further, the 4 capability to enjoy the benefits of 1 mining separate operating systems in each 5 cluster is facilitated and enhanced 6 The example of FIGs 3 - 7, inclusive is only one of many address 7 transfoπnations which may be established according to the present invention 8 For example, consider bπefly a system in which each multiprocessor board 9 can, as before, inherently directly address one gigabyte while the main io memory can store four gigabytes Then, bit 0 of the external address would π come mto play, and the logic of FIGs 6 and 7 would be accordingly adjusted 12 and expanded In one simple configuration, the cluster number for each 13 multiprocessor board may be extended to thiee bits with the most sigmficant 14 cluster number bit copied to and from bit 0 of the external address This 15 arrangement will expand the number of foui -multiprocessor clusters which 16 can be accommodated as well as expand the directly and indirectly 17 addressable exte al memory Alternatively, the clusters can be reduced to lb two multiprocessors per cluster (l e , two cluster per multiprocessor board in 19 the exemplary system) The result will be to use more extemal space as 1 private space, each private space block being used by fewer processors. 2 Thus, it will be understood that the address transfoπnations available 3 according to the invention are many and that the foregoing example treated in 4 detail is only representative of one found to be especially applicable to a 5 specific system in wliich the available external space was two gigabytes 6 whereas the processors are limited to directly addressing one gigabyte. It has 7 also been previously mentioned that bit 0 of the extemal address can be used 8 a switch between two separate multi-cluster systems which access common 9 mass memory. io Thus, while the principles of the invention have now been made clear i i in an illustrative embodiment, there will be immediately obvious to those 12 skilled in the art many modifications of stmcture, arrangements, proportions, 13 the elements, matenals, and components, used in the practice of the invention 14 which are particularly adapted for specific environments and operating 15 requirements without departing fiom those pnnciples.

Claims: WHAT IS CLAIMED IS: I . A cluster computer system comprising: A) a plurality of clusters, each said cluster including: 1 ) a plurality of processors, each said processor having inherent limitations on the range of memory which is directly addressable by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including: a) a primary cache; and b) an address translator; and 2) a secondary cache; B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said processor address field, said memory address field being divided into higher order and lower order segments; C) means establishing a cluster number for each said cluster; D) means coupling said plurality of clusters, via said secondary caches, to said memory; and E) each said address translatoi m each said processor in each said cluster being coupled lnteπnediate said primary cache in its said processor and said secondaiy cache in its said cluster, said address translator includmg first address translation logic responsive to processor address field and cluster number infoπnation in said pnmary cache in its said processor said first translation logic compnsmg means responsive to the digital state of a predeteπn ed indicator bit said higher order segment of said processor address field by effecting one of two address transfoπnations, 1 ) if said predeteπ ined indicator bit is a fu st digital value a) forcing the two lowest order bits of said higher order segment of said memory address field to a first logic level, b) copying the two lowest order bits of said higher order segment of said processor address field to two higher order bits of said higher order segment of said memory address field and c) copying the contents of said lower order segment of said processor address field to said lower order segment of said memory address field, thereby specifying a iegion of memory addressable by all said clusters; and 2) if said predeteπnined indicator bit is a second digital value: a) copying the two lowest order bits of said higher order segment of said processor address field to the two lowest order bits of said higher order segment of said memory address field, b) copying the cluster number to two higher order bits of said higher order segment of said memory address field, and c) copying the contents of the lowei order segment of said processor address field to said lower order segment of said memory address field, thereby specifying a region of memory addressable only by the cluster identified by said cluster number 2 The cluster computer system of Claim 1 in which said higher order segment of said processor address field comprises three bits and in which said higher order segment of said memory address field comprises at least five 1 3. The cluster computer system of Claim 2 in which the most significant bit of said higher order segment of said processor address field is employed as said predeteπnined indicator bit. i 4 The cluster computer system of Claim 3 which further includes means for establishing a cluster member number for each said processor in each said 3 cluster and in which said cluster member niunber of a given said processor is sent to said secondary cache in said cluster in which said given said processor 5 resides when a given memory lequest is made by said given said processor 6 such that said secondary cache m said cluster in which said given said 7 processor resides can identify which of said processors in its said cluster a 8 response to said given memory request is to be made. i 5 A cluster computer system comprising 2 A) a plurality of clusters, each said cluster including 3 1 ) a plurahty of processors, each said processor having inherent 4 limitations on the range of memory which is directly addressable 5 by a processor address field, said processor address field being divided into higher order and lower order segments, each said processor including: a) a primary cache; and b) an address translator; and 2) a secondaiy cache; B) a memory having a capacity which exceeds the range each said processor is inherently capable of directly addressing, said memory being addressable by a memory address field, said memory address field exceeding said processor address field, said memory' address field being divided into higher order and lower order segments; C) means establishing a cluster number for each said cluster; D) means coupling said plurality of clusters, via said secondary caches, to said memory; and E) each said address translator in each said processor in each said cluster being coupled inteπnediate said primary cache in its said processor and said secondary cache in its said cluster, said address translator including second address translation logic responsive to memory address field infoπnation, said second translation logic comprising means responsive to the digital states of predetermined indicator bits in said memory address field, which predeteπnined indicator bits are of a lower order than the most significant bit thereof, by effecting one of two address transfonnations, 1 ) if said predeteπnined indicatoi bits are both a first digital value a) forcing a predetennined bit of said higher order segment of said processor address field to a first logic level, b) copying two higher order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field, and c) copying the contents of the lower order segment of said memory address field to said lower order segment of said processor addiess field, thereby specifying a memory region containing infoπnation accessible by all said clusters, and 2) if either of said predeteπnined indicatoi bits are a second digital value a) copying the two lowest order bits of said higher order segment of said memory address field to the two lowest order bits of said higher order segment of said processor address field, b) copying higher bits, representing a cluster number, in said higher order section of said memory address field to said cluster number at said common cluster interface, and c) copying the contents of said lower order segment of said memory address field to said lower order segment of said processor address field thereby specifying a memory iegion containing infoπnation accessible only by the cluster specified by said cluster number. 6 The cluster computer system of Claim 5 in which said higher order segment of said processor address field compnses three bits and in which said higher ordei segment of said memory address field comprises at least five O 97/13191 PC17US96/15937 7 The cluster computer system of Claim 6 which the two least significant bits of said higher order segment of said memory address field are employed as said predeteπnined indicatoi bits 8 The clustei computer system of Claim 7 which further includes means for establishing a cluster member nuinbei for each said processor in each said cluster and which said cluster member number of a given said processor is sent to said secondary cache said cluster in which said given said processor resides when a given memory request is made by said given said processor such that said secondary cache in said cluster in which said given said processor lesides can identify which of said processors in its said cluster a response to said given memory request is to be made 9 A cluster computer system comprising A) a plurahty of clustei s, each said cluster including 1 ) a plurality of processors, each said processor having inherent limitations on the range of memory which is duectly addressable by a processor address field, said processor address field being 6 divided into higher order and lower order segments, each said 7 processor includmg. 8 a) a primary cache, and 9 b) an address translator; and io 2) a secondary cache, i i B) a memory having a capacity which exceeds the range each said 12 processor is inherently capable of directly addressing, said memory 13 being addressable by a memory address field, said memory address 14 field exceeding said processor address field, said memory address field 15 being divided into higher order and lower order segments, 16 C) means establishing a cluster number for each said cluster; 17 D) means coupling said plurahty of clusters, via said secondary caches, 18 to said memory, and 19 E) each said address translator in each said processor in each said 0 cluster being coupled lnleπiiediate said pnmary cache m its said i processor and said secondaiy cache in its said cluster, said address 2 translator including first address translation logic responsive to 3 processor address field and cluster number in said pnmary cache i its 4 said processor, said first translation logic compnsing means responsive r 10 the digital state of a first predeteπnined indicator bit in said higher 6 order segment of said processor address field, which said first 7 predetennined indicator bit is of a lower order than the most significant 8 bit thereof, by effecting one of two address transfoπnations; 9 1 ) if said first predeteπnined indicator bit is a first digital value 0 a) forcing the two lowest order bits of said higher order 1 segment of said memory address field to a first logic level, 2 b) copying the two lowest order bits of said higher order 3 segment of said processor address field to two higher 4 order bits of said higher order segment of said memory 5 address field, and 6 c) copying the contents of said lower order segment of 7 said processor address field to said lower order segment 8 of said memory address field, 9 theieby specifying a region of memory addressable by all said 0 clusters, and i 2) if said first predetermined indicator bit is a second digital 2 value a) copying the two lowest order bits of said higher order segment of said processor address field to the two lowest order bits of said higher order segment of said memory address field; b) copying the cluster number to two higher order bits of said higher order segment of said memory address field; and c) copying the contents of the lower order segment of said processor address field to said lower order segment of said memory address field, thereby specifying a region of memory addressable only by the cluster identified by said cluster number, and F) said address translator further including second address translation logic responsive to memory address field infoπnation, said second translation logic comprising means responsive to the digital states of second and thirds predetennined indicator bits m said memory address field, which second and third predetennined indicator bits are of a lower order than the most significant bit thereof, by effecting one of two address transfonnations, I ) lf said second and thud predetermined indicator bits are both a first digital value a) forcing a predetennined bit of said higher order segment of said processor address field to a first logic level, b) copying two higher order bits of said higher order segment of said memory address field to the two lowest order bits of said higher oi dei segment of said processor address field, and c) copying the contents of the lower order segment of said memory address field to said lower order segment of said processor address field, thereby specifying a memory region contammg infoπnation accessible by all said clusters, and 2) if either of said second and third predetennined indicator bits ai e a second digital value a) copying the two lowest order bits of said higher order segment of said memory address field to the two lowest 0 oidei bus of said highei oi dei segment of said piocessor 81 address field, 82 b) copying higher bits, lepresentmg a cluster number, in 83 said higher order section of said memory address field to 84 said cluster number at said common cluster interface, and 85 c) copying the contents of said lower order segment of 86 said memory addiess field to said lower order segment of 87 said pi ocessoi address field 88 thereby specifying a memory region containing information 89 accessible only by the cluster specified by said cluster number 1 10 The cluster computer system of Claim 9 in which said higher order 2 segment of said processor addi ess field computes three bits and m which said 3 higher order segment of said memory address field compnses at least five 1 1 1 The cluster system of Claim 10 in which the most significant bit of said 2 higher order segment of said pi ocessor address field is employed as said 3 predeteπnined indicator bit and in which the two least significant bits of said higher ordei segment of said memory address field are employed as said predetermined indicatoi bits 12 The cluster computer system of Claim 1 1 which further includes means for establishing a cluster member number for each said processor in each said cluster and in which said cluster member number of a given said processor is sent to said secondary cache in said cluster which said given said processor resides when a given memory lequest is made by said given said processor such that said secondary cache in said cluster in which said given said processor lesides can identify w nch of said processors in its said cluster a response to said given memory request is to be made

Inventors: GUENTHNER RUSSELL W, RABINS LEONARD, GUENTHNER, RUSSELL, W., RABINS, LEONARD
Applicants: BULL HN INFORMATION SYST, BULL HN INFORMATION SYSTEMS INC.
Application year: 1996
Main classification: G06F9
IPC codes: G06F9, G06F12, G06F15