AN ARITHMETIC PROCESSOR HAVING A PARALLEL ACCUMULATE FUNCTION

Abstract: A data processing system includes a multiple floating point arithmetic unit with a putaway and a bypass bus, which includes a new instruction for handling multiple multiply or divide instructions. These instructions are separated by add operations, including passing the results of each multiply/­ divide operation on a bypass bus to the input of an adder along with the inuts from an accumulate bypass bus which is the output from the adder for an automatic add operation on an accumulate multiply or accumulate divide operation. This allows two floating point results to be produced each cycle, one of which can be accumulated without any inter­ vening control by the central decoder. The accumulation is performed under distributed control of the accumulator log­ ic itself.

Description: The present invention relates to an arithmetic processor having a parallel accumulate function. Many computer programs are characterised as having a large number of floating point arithmetic operations. As a consequence, a large number of the instructions executed by the machine are floating-point instructions (floating add, floating subtract, floating multiply, floating compare, etc.). For such programs, the greater the number of floating-point arithmetic instructions that can be executed per cycle, the faster the machine speed of operation. Arithmetic results produced by the floating-point arithmetic unit must be saved, in a register-file, for instance, for later use and for eventual storage into memory. Although many designs for arithmetic units allow multiple arithmetic operations to be in execution at once, most allow only one actual result be produced each cycle. The IBM model 360/91 is an example of a machine with multiple floating point arithmetic units. As arithmetic results exit from one of the arithmetic units, they are placed on a putaway or result bus where they travel to the register file and enter a predetermined register for which they are destined. In addition, the results travel along a special bypass bus which is connected to the waiting stages associated with each arithmetic unit. If any instruction in a waiting stage is waiting for the newly produced result, the result is entered into one of the buffers in the waiting stations. In this way, performance is increased by not requiring waiting instructions to wait while the needed result is first gated into the register file and then gated from the register file into the waiting station. A complete description of this scheme may be found in "An Efficient Algorithm for Exploiting Multiple Arithmetic Units" by R M Tomasulo, IBM Journal, January 1967, pp. 25-33. Since there is only one putaway bus and one bypass in the Tomasulo scheme, only one floating point result nay be produced each cycle. US Patent 4,075,704 to O'Leary describes a floating point data processor for high speed operation, which includes two arithmetic units and two putaway busses. Both busses pass back to both arithmetic units. However, each bus enters only one side of each arithmetic unit. Although the drawings in O'Leary illustrate multiple entries into each side of the arithmetic unit, these are controlled by the decoder (central control). Thus, O'Leary's scheme allows only a single data item, that is a result, to enter a given side of the pipeline during any cycle, and O'Leary's scheme requires central control. US Patent 3,697,734 to Booth et al sets forth a digital computer utilising a plurality of parallel asynchronous arithmetic units, which include a bussing scheme similar to O'Leary, in that only one arithmetic result is produced each cycle. IBM Technical Disclosure Bulletin, Vol 14, No 10, March 1972, pp 2930-2933 in an article by Sofa et al sets forth a floating point arithmetic scheme in which a single arithmetic result is allowed onto the pipeline during each cycle. In an article by Ramamoorthy in "Computing Surveys" Vol 9, No 1, March 1977, pp 61-85, various schemes are set forth for making floating point arithmetic operations, with each of the schemes allowing only one arithmetic result to enter the pipeline during each cycle. The present invention provides and arithmetic processor capable of performing the basic operations of floating point arithmetic and incorporating an adder unit, having first and second inputs and a wait station, and a multiplier unit, operable in parallel with the adder unit, together with a register stack, status latches, interconnecting bus structures and an instruction decoder based control mechanism responsive to a set of machine level instructions including ADD and MULTIPLY, wherein (a) the bus structures comprise a result bus (referred to in the following description as the put-away bus), connected to the outputs of the adder and multiplier units respectively via first and second selectively operable switches and to the register stack, a bypass bus, the bypass bus being directly connected to the result bus thereby connecting the outputs of the adder and multiplier units via the first and second selectively operable switches respectively to all the inputs of the arithmetic units, and an accumulator bypass bus connecting the output of the adder unit via a third selectively operable switch to the first input of the adder unit, the first selectively operable switch being on when an ADD operation is being completed and the second selectively operable switch being on when a MULTIPLY operation is being completed, the first and second selectively operable switches not being on at the same time, (b) once a machine instruction is decoded, it executes under control of the status latches; (c) the machine instruction set includes an ACCUMULATE instruction having a source register and a result target register associated with it, which causes a status latch associated with a result target register of the register stack to indicate "busy" and a partial sum to be stored in the wait station and a STOP ACCUMULATE instruction which causes the partial sum to be directed to the result target register as the final result of the ACCUMULATE instruction; and (d) between the execution of an ACCUMULATE instruction and a STOP ACCUMULATE instruction, the appearance on the bypass bus of a result having as designated destination the source register causes the result on the bypass bus to be added, in the adder unit, to the partial sum and the third selectively operable switch to direct the result of the addition back to the wait station via the accumulator bypass bus, the result of the addition replacing the partial sum and thereby accumulating the final result. It is thus possible to compile a higher level instruction to, say, perform a vector multiplication, into a sequence of machine instructions in which a MULTIPLY instruction forms a stream unbroken by ADD and ACCUMULATE instructions so that it is possible to produce two floating point results each cycle, one of which can be accumulated without intervention by the control mechanism. A mechanism is described for use within the arithmetic unit of a high-speed digital computer. The purpose of the mechanism is to allow for the rapid, efficient accumulation of partial arithmetic results into a single accumulating register. Such operations are commonly and frequently used in programs performing complex arithmetic operations on vectors and matrices. Because these operations involve the accumulation of partial results, a typical sequence of actions might be "operate", "accumulate", "operate", "accumulate", and so on. Since each step of the sequence is usually controlled by a separate machine instruction, maximum machine speeds are usually one step per machine cycle. Consequently, if for example eight partial results are to be created and accumulated, 16 machine cycles would be required. Vector machines use a special vector arithmetic unit to reduce the total cycle time to the number of operations performed, with the accumulations occurring in parallel. Thus 8 partial results can be accumulated in 8 machine cycles as opposed to 16 on non-vector machines. But vector units are expensive and are separate mechanisms from those normally found in computer units. This specification describes a means of reducing the total cycle time with the use of a conventional-type arithmetic unit. Only minimal additional hardware is required, and thus the performance increase is obtained at minimal cost. This provides, in effect, a floating point arithmetic unit which allows two floating point results to be produced each cycle, one of which can be accumulated without any intervening control by the central decoder, and in which the accumulation is performed under distributed control of the accumulator logic itself. The present invention will be described further, by way of example, with reference to an embodiment thereof as illustrated in the accompanying drawings, in which:- Figure 1 is a block diagram of one form of floating point arithmetic unit according to the present invention; Figure 2 is a detailed diagram of the registers illustrated generally in Figure 1; and Figure 3 is a detailed diagram of the adder unit which is shown gener ally in Figure 1. A data processing system is disclosed and illustrated, which includes a multiple floating point arithmetic unit with a putaway and a bypass bus, which includes a new machine instruction for inclusion in the handling of higher level multiple multiply or divide instructions. These higher level instructions include passing the results of each multiply/divide operation on a bypass bus to the input of an adder along with the inputs from an accumulate bypass bus which is the output from the adder for an automatic add operation on an accumulate multiply or accumulate divide operation. This allows two floating point results to be produced each cycle, one of which can be accumulated without any intervening control by the central decoder. The accumulation is performed under distributed control of the accumulator logic itself. A mechanism is described for use within the arithmetic unit of a high-speed digital computer. The purpose of the mechanism is to allow for the rapid, efficient accumulation of partial arithmetic results into a single accumulating register. Such operations are commonly and frequently used in programs performing complex arith metic operations on vectors and matrices. Because these operations involve the accumulation of partial results, a typical sequence of actions might be "operate", "accumulate", "operate", "accumulate", and so on. Since each step of the sequence is usually controlled by a separate machine instruction, maximum machine speeds are usually one step per machine cycle. Consequently, if for example eight partial results are to be created and accumulated, 16 machine cycles would be required. Vector machines use a special vector arithmetic unit to reduce the total cycle time to the number of operations performed, with the accumulations occurring in parallel. Thus 8 partial results can be accumulated in 8 machine cycles as opposed to 16 on non-vector machines. But vector units are expensive and are separate mechanisms from those normally found in computer units. This specification describes a means of reducing the total cycle time with the use of a conventional-type arithmetic unit. Only minimal additional hardware is required, and thus the performance increase is obtained at minimal cost. Consider two vectors of size N called A and B. To compute the inner product of A and B, the following FORTRAN code would suffice: Clearly N multiplications and N additions are required. The existence of at least two separate, independent pipelined arithmetic units, one of which will typically perform additions and subtractions and the other performing multiplications and divisions is assumed. In such organisations, one result can be produced each cycle by either of the two units. However, usually only one of the units is allowed to produce a result during any given cycle since only one result can be stored away each cycle. In this arrangement, under special circumstances, two results will be allowed in some cycles; one of these will be a partially accumulated result. When the above FORTRAN code is executed, a sequence of machine instructions similar to the following is executed: It can be seen that each multiply instruction must wait on the preceding multiply instruction to finish since each uses register R1 to form a product. One product must be produced by the multiplier unit and sent to the adder before a second product can be formed in the same register. Similarly, each add instruction must wait on the preceding add instruction to finish since each uses the partial sum in register R2 to compute the next partial sum. If both pipelines are four stages in length, each stage requiring one machine cycle, then one multiply and one add can be executed every four cycles. For a vector length of N then, 4N cycles might be required to compute the inner product. To overcome this inadequate performance, optimising compilers perform a transformation on the code called "unraveling." Instead of the compiler would produce At the end of execution of this code, the four partial sums would have to be accumulated into a single register, as in the following code: By unravelling the loop, four multiplications and four additions are now performed in each iteration of the loop. And because each of the four multiplications uses different registers, they do not have to wait on each other. Nor do the additions have to wait on each other. Thus, if the pipelines are four stages long, one multiplication and one addition can be computed each cycle. Now, instead of 4N cycles to compute the inner product, only N are theoretically required. However, as previously mentioned, usually only one result is actually allowed to be produced each cycle in most arithmetic units. Consequently, 2N cycles are required. A mechanism is described below by which the inner product can be produced in only N cycles. Two results are produced each cycle, but one is a partial result. Figure 1 is a block diagram of a floating point arithmetic unit according to the present invention. The unit is comprised of an adder pipeline unit 2, a multiplier pipeline unit 4, with the output of the adder pipeline unit 2, and the multiplier pipeline unit 4 being connected via connect switches 3 and 5, respectively, to a putaway bus (PAB) 6 and thence to a register file 8. The adder pipeline unit 2 is also connected via an accumulator connect switch 11 to an accumulator bypass bus (ABB) 9 which is connected to a left adder wait station 10. The putaway bus (PAB) is directly connected to a bypass bus (BPB) 12 which is connected to the left adder wait station 10, a right adder wait station 14, a left multiplier wait station 16 and the right multiplier wait station 18. The register file 8 is connected via a left register bus (LRB) 20 to the left adder wait station and the left multiplier wait stations 10 and 16, respectively, with a right register bus (RRB) 22 being connected to the right adder wait station and right multiplier wait stations 14 and 18, respectively. An instruction register (IR) 24 is connected via a bus 26 to the register file 8. A central processing memory unit 28 provides an input to the instruction register 24 via an instruction fetch bus (IFB) 30, and an input to the register file 8 via a register load bus (RLB) 32, and receives an input from the register file 8 via a register store bus (RSB) 34. The instruction register 24 and memory 28 are included in the overall central processing unit (CPU), not shown, and are operative with, but not part of the arithmetic unit herein described. The instruction register 24 sequentially receives instructions from the central memory 28 and executes them. As the instruction are executed, floating-point numbers are transferred between the register file 8 and memory 28, and certain operations on these numbers are initiated within the arithmetic unit. Instructions are fetched from central memory (MEM) 28 into the instruction register (IR) 24 where they are decoded. Depending on the instruction decoded, control signals are sent to the other components. The register file (REGS) 8 consists of some number of high-speed storage locations called "registers." These are designated R1 through Rn, as shown in detail in Figure 2. Each register may be loaded with data from memory 28 over the register load bus (RLB) 32, and data in the registers may be stored into memory 28 over the register store bus (RSB) 34. Two registers may be added together by sending them to the adder pipeline unit (ADD) 2. One register travels to the adder over the left register bus (LRB) 20, and another register travels to the adder over the right register bus (RRB) 22. When the two registers have been added together, the result emerges from the adder pipeline over the adder result bus (ARB) 7. The result is a triple consisting of a value, register number, and operation type designator, or (value, reg,op). If the adder connect switch (ACS) 3 is on, the result passes from the adder result bus (ARB) 7 onto the putaway bus (PAB) 6 and the bypass bus (BPB) 12. After travelling over the putaway bus (PAB) 6, the result is stored away in a designated register of the register file (REGS) 8. After travelling over the bypass bus (BPB) 12, the result is available for reuse by other, waiting instructions in a manner to be described below. Similarly, two registers can be multiplied together by the multiplier pipeline unit 4. At any one time, only the adder connect switch (ACS) 3 or the multiplier connect switch (MCS) 5 can be active since only one result may be placed on the putaway bus (PAB) 6 each cycle. These connect switches become active depending on the op field of the output of the pipeline units. Several instructions can be in execution at any one time in both of the pipeline units. Although instructions are entered in the order in which they are executed, they may emerge from the pipeline in any order. Each instruction is of the form, OP TR,SR1,SR2, where OP specifies which operation is to be performed, that is, add-type or multiply-type, SR1 and SR2 specify the two source registers, and TR specifies the target register, the register which is to receive the result of the operation. Each register in the register file, as shown in FIG. 2, has a format comprised of a busy bit and a data field. When an instruction is decoded with register Rn as the target register, where Rn designates any of the existing registers, the busy bit for register Rn is turned on, and the instruction is sent to one of the arithmetic pipes along with its two operands, if available. When the instruction completes, the result will return over the putaway bus (PAB) 6 and enter the specified target register, Rn. At this point, the busy bit of Rn is turned off. During the time that the busy bit for Rn is on, no new instruction is allowed to be decoded if that instruction also specifies register Rn as the target register. This is due to the fact that instructions may complete out of order once they have been sent to the pipeline units. The busy bit allows the arithmetic unit to ensure that the results are entered into Rn in the appropriate sequence since only one instruction in execution may ever specify Rn as the target. When an instruction is decoded with a target register whose busy bit is off, the two source registers are gated out of the register file. The register designated by SR1 is gated onto the left register bus (LRB) 20 and SR2 onto the right register bus (RRB) 22. Then, depending on the class of the operation, these values both go to either the adder or the multiplier wait station. Both wait stations consist of a number of wait station slots, but not necessarily the same number. With reference to Figure 3, each slot has the format illustrated. To enter a slot 36, the slot must be presently unoccupied. If the VALID bit of a slot is off, the slot is unoccupied; otherwise, it is occupied. Any unoccupied slot is acceptable. When a slot is selected, the VALID bit for that slot is turned on. Register SR1 enters the SR1 slot field, SR2 enters the SR2 slot field, RDY1 and RDY2 are both set on, and the TR and OP slot fields are set. Each cycle, both pipelines scan their waiting stations for a slot with all three of the VALID, RDY1, and RDY2 bits set. Such slots contain operations ready to be executed. When one is found, the data is entered into the pipeline, and the slot's VALID bit is turned off, allowing it to be reused. Once the instruction enters the pipeline, the OP field controls which operation is performed on the data, and the TR field specifies the target register for the result. The above description describes the actions taken when both source registers are available at the time the instruction is decoded, that is, when neither SR1 nor SR2 has its busy bit on. If either does have its busy bit on, then that register does not presently contain its correct, current value, for there is an instruction in execution that will produce a new value for that register. Instead of holding up the instruction in the decoder until both source registers become valid, the instruction is allowed to fire anyway. However, the instruction must now be marked as not yet ready to execute. If the busy bit of SR1 is on, RDY1 is set to false in the wait station; similarly, if the busy bit for SR2 is on, RDY2 is set to false. Because all three of the VALID, RDY1, and RDY2 bits must be on in a slot for that slot to be selected, slots with unready (BUSY) source registers will be delayed in the waiting station slot to which they have been assigned. If the busy bit for SR1 is on in REGS no value for SR1 is gated out onto the left register bus (LRB) and into the SR1 slot field. Instead, the register number SR1 itself is placed in the NSR1 field and RDY1 is set to false. Similarly for SR2. If RDY1 is on, SR1 holds a value; if RDY1 is off, NSR1 holds a register number. As values exit from a pipeline, they travel over the putaway bus (PAB) 6. THey also travel over the bypass bus (BPB) 12. The bypass bus 12 is connected to every slot in both waiting stations. Each cycle, every slot that is valid but has either RDY1 or RDY2 set to false checks to see if the TR field accompanying the value on the bypass bus is the same as the number in the NSR1 or NSR2 fields. If so, then the value on the bypass bus enters the matching SR1 or SR2 slot field and the corresponding ready bit or bits are turned on. In this way, waiting, valid slots become ready, valid slots. A special instruction of the form, ACC TR,SR2, is provided. It performs an operation similar to ADD TR,TR+SR2. However, once the ACC (accumulate) instruction enters a wait station slot, the ACC bit field of the slot is turned on. For an ACC instruction to decode, the busy bit for register TR must be off. Upon decoding, the busy bit for register TR is turned on, an empty slot in the adder pipe is selected, the slot's valid bit is turned on, the SR1 field is set to zero, RDY1 is set on, RDY2 is set off, NSR2 is set to SR2 the TR and NSR1 are fields set to TR, and the ACC bit is turned on. The slot is now ready to accumulate partial sums. Suppose a multiply instruction is executed with the form MUL TR,SR1 * SR2 where the TR designation is the same as in NSR2 of the ACC instruction. When the result of the multiplication appears on the bypass bus (BPB) 12, the TR number matches the unready NSR2 field of the waiting accumulate instruction, and so it enters the SR2 field of the ACC slot and RDY2 is turned on. Now the ACC instruction is eligible to be selected by the adder pipeline unit for execution (since RDY1, RDY2, and VALID are now all on). When selected, however, the VALID bit is not turned off as is done for other instructions, thereby preventing the slot from being reused. However, both the RDY1 and RDY2 bits are turned off. When the accumulate instruction has finished adding its two values together, the result is placed on the accumulate bypass bus (ABB) 9. The operation code (op) that travels through the pipe with the two register values is determined to be an accumulate operation, so the accumulate control switch (XCS) 11 is turned on by the adder pipe-line unit, thereby placing the new partial sum onto the accumulate bypass bus (ABB) 9. Because this bus is separate from either the putaway bus (PAB) 6 or the bypass bus (BPB) 12, the multiplier pipe-line can also produce a result during the same cycle in which an accumulation result is produced. Each cycle, waiting, valid slots in the adder waiting stations monitor the bypass bus (BPB) 12 and the accumulator bypass bus (ABB) 9. If a valid slot has RDY1 set to false and ACC set to true, then the accumulator bypass bus is monitored for the NSR1 value; otherwise the bypass bus is monitored. The SR2 value can appear on only the bypass bus and not the accumulator bypass bus. To stop an accumulation, an instruction of the form, STAC TR,SR2, is used. STAC means "stop accumulating." To decode, the busy bit for SR2 must be off. This instruction locates the corresponding wait station slot, sets RDY2 to true, sets SR2 to zero, and turns off the ACC bit. When the instruction now fires, the VALID bit will be turned off since now the ACC bit is off. The accumulated sum will pass through the pipe and finally appear on the putaway bus, from where it will enter the target register in the register file. Because the op specifies a STAG instruction instead of an ACC instruction, the adder connect switch (ACS) 3 is activated instead of the accumulate connect switch (XCS) 11. The unravelled loop described before could now be recoded as follows: Now only multiplication instructions appear within the inner loop, no additions. Consequently, one multiplication instruction can be decoded every cycle. Because multiplication results will appear on the putaway bus (PAB) 6 and the bypass bus (BPB) 12 and accumulations will appear on the accumulator bypass bus (ABB) 9, both the adder and multiplier can produce a result each cycle, Consequently, to form an inner product of two N-element vectors, only N cycles are required. No special vector unit is required. The above arrangement provides an improved floating point arithmetic unit which can produce two arithmetic results per cycle of operation, one of which can be accumulated under distributed control of the accumulator logic itself, to accomplish vector and matrix related arithmetic operations.

Claims: An arithmetic processor capable of performing the basic operations of floating point arithmetic and incorporating an adder unit, having first and second inputs and a wait station, and a multiplier unit, operable in parallel with the adder unit, together with a register stack, status latches, interconnecting bus structures and an instruction decoder based control mechanism responsive to a set of machine level instructions including ADD and MULTIPLY, wherein (a) the bus structures comprise a result bus (6) connected to the outputs of the adder (2) and multiplier (4) units respectively via first and second selectively operable switches (3,5) and to the register stack (8), a bypass bus (12), the bypass bus being directly connected to the result bus (6) thereby connecting the outputs of the adder (12) and multiplier (4) units via the first and second selectively operable switches (3, 5) respectively to all the inputs of the arithmetic units (2, 4), and an accumulator bypass bus (9) connecting the output of the adder unit (2) via a third selectively operable switch (11) to the first input of the adder unit, the first selectively operable switch being on when an ADD operation is being completed and the second selectively operable switch being on when a MULTIPLY operation is being completed, the first and second selectively operable switches not being on at the same time; (b) once a machine instruction is decoded, it executes under control of the status latches; (c) the machine instruction set includes (i) an ACCUMULATE instruction having a source register and a result target register associated with it, which causes a status latch associated with a result target register of the register stack to indicate "busy" and a partial sum to be stored in the wait station (ii) a STOP ACCUMULATE instruction which causes the, partial sum to be directed to the result target register as the final result of the ACCUMULATE instruction; and (d) between the execution of an ACCUMULATE instruction and a STOP ACCUMULATE instruction, the appearance on the bypass bus of a result having as designated destination the source register causes the result on the bypass bus to be added, in the adder unit, to the partial sum and the third selectively operable switch to direct the result of the addition back to the wait station via the accumulator bypass bus, the result of the addition replacing the partial sum and thereby accumulating the final result. An arithmetic processor as claimed in Claim 1 wherein: the second and third selectively operable switches are both permitted to be on at the same time. An arithmetic processor as claimed in Claim 1 or Claim 2 wherein: (a) the wait station comprises a plurality of registers, each register including a first field (op) indicative of the arithmetic operation to be performed, a second field (TR) indicative of the target register in the register stack (8), a third field (SR1), a fourth field (SR2) for containing data from source registers associated with the operation, fifth (NSR1) and sixth fields (NSR2) containing the register numbers for the source registers associated with the operation, seventh and eighth fields indicative of whether the third and fourth fields respectively are occupied, a ninth field (ACC) indicative that an ACCUMULATE instruction has been executed and a corresponding STOP ACCUMULATE instruction has not been executed, and a bit (V) to indicate whether the data in the register is valid; (b) the operation may be selected and performed by the adder unit only when the seventh, eighth and valid bits are all on; (c) the execution of the ACCUMULATE instruction causes the third field to be set to the value of the partial sum, the ninth field to be turned on, the valid bit to be turned on, the eighth field to be turned off, the seventh field to be turned on, the second and fifth fields to be loaded with the target registers associated with the ACCUMULATE instruction and the sixth field to be loaded with the source register associated with the ACCUMULATE instruction; (d) the appearance on the bypass bus of a result having as designation destination the source register associated with the ACCUMULATE instruction causes the result on the bypass bus to be added to the partial sum by causing the result to be placed in the fourth field and the eighth to be turned on, thereby enabling the operation to be selected and performed by the adder unit; (e) once the operation has been selected by the adder unit for execution, the seventh and eighth bits are both turned off, the valid bit remaining on; (f) the third selectively operable switch is caused, by the ninth field being on, to direct the output of the adder unit back, via the accumulator bypass bus, to the wait station ; (g) the appearance on the accumulator bypass bus of a result having as designated destination the target register, causes the result to be placed in the third field and tile seventh field to be turned on. An arithmetic processor as claimed in Claim 3 wherein the STOP ACCUMULATE instruction sets the fourth field to zero, causes the eighth field to be turned on and the ninth field to be turned off, thereby enabling the operation to be selected by the adder unit, and once the operation has been selected by the adder unit, the valid bit is turned off.

Inventors: DEGROOT, RICHARD DOUGLAS
Applicants: INTERNATIONAL BUSINESS MACHINES CORPORATION
Application year: 1985
Main classification: G06F7
IPC codes: G06F9, G06F7, G06F17